Write tests that rely on containerised dependencies, and orchestrate them with Tox
Published in · 8 min read · Jan 2, 2023
--
When developing a new application, we often need external services such as databases, message brokers, cache memory, or APIs to other micro-services. Testing the interfaces between these components is a practice that is often neglected. Some completely skip these tests, and others mock responses. This, however, leaves integration tests incomplete, as those interfaces can easily fail due to changes in vendors' APIs, database schema changes, and configuration issues.
One way to conduct these tests is by running the dependencies as containers that communicate with the application in an environment similar to production. In this article, I illustrate how to run these tests in a CI pipeline using GitHub actions. To do this, you must write a small Python application with access to a postgres database to create and select users.
The tests will be written using pytest, one of the leading testing frameworks for Python. The database will be run as a container using Docker. Tests will be coordinated with tox, a testing orchestrator for Python, which originally emerged as a tool to test different Python versions, and has evolved to allow developers to isolate testing from development environments, centralise testing configuration, and even coordinate dependencies using docker containers.
You can follow the instructions and code snippets or check out the whole (but small) project directly on my GitHub repo. The file structure of the project is this one:
.
├── README.md
├── migrations
│ └── schema.sql
├── requirements.txt
├── src
│ └── testing_containers
│ ├── __init__.py
│ ├── db
│ │ ├── __init__.py
│ │ └── db.py
│ └── model
│ ├── __init__.py
│ └── users.py
├── tests
│ ├── __init__.py
│ ├── conftest.py
│ └── db
│ └── test_db.py
└── tox.ini
Since we are using a Postgres database as our dependency, let’s start by writing a small table representing Users
in our application. I’ll call it migrations/schema.sql
. Here’s what it looks like:
CREATE TABLE users (
email varchar(64) primary key,
name varchar(64) not null
);
To represent the entities in the application, I will use pydantic
. This is a great way to represent objects, get type hints, and it is especially useful to perform data validations when we use these entities in APIs. So, create a new Python file, call it users.py
, and paste the following:
from pydantic import BaseModelclass User(BaseModel):
"""Representation of User entity"""
name: str
email: str
Now we can write some boilerplate for our database operations. Let’s start with some functions that wrap our postgres driver ( psycopg2
in this case). As the intent of this article is not to teach how to use this driver, I will not elaborate on them, but I will encourage you to read the PYnative tutorial. We can start our class like this:
from typing import Listimport psycopg2
from psycopg2.extras import execute_values
from testing_containers.model.users import User
class Repo:
# psycopg2 wrapper
def __init__(self) -> None:
self.conn = None
try:
self.connect()
logging.info("db: database ready")
except Exception as err:
logging.error("db: failed to connect to database: %s", str(err))
self.close()
raise err
def connect(self):
"""Stores a connection object `conn` of a postgres database"""
logging.info("db: connecting to database")
conn_str = os.environ.get("DB_DSN")
self.conn = psycopg2.connect(conn_str)
def close(self):
"""Closes the connection object `conn`"""
if self.conn is not None:
logging.info("db: closing database")
self.conn.close()
def execute_select_query(self, query: str, args: tuple = ()) -> List[tuple]:
"""Executes a read query and returns the result
Args:
query (str): the query to execute.
args (tuple, optional): arguments to the select statement. Defaults to ().
Returns:
List[tuple]: result of the select statement. One element per record
"""
with self.conn.cursor() as cur:
cur.execute(query, args)
return list(cur)
def execute_multiple_insert_query(self, query: str, data: List[tuple], page_size: int = 100) -> None:
"""Execute a statement using :query:`VALUES` with a sequence of parameters.
Args:
query (str): the query to execute. It must contain a single ``%s``
placeholder, which will be replaced by a `VALUES list`__.
Example: ``"INSERT INTO table (id, f1, f2) VALUES %s"``.
data (List[tuple]): sequence of sequences or dictionaries with the arguments to send to the query.
page_size (int, optional): maximum number of *data* items to include in every statement.
If there are more items the function will execute more than one statement. Defaults to 100.
"""
with self.conn.cursor() as cur:
execute_values(cur, query, data, page_size=page_size)
Now we write some functions to:
- Retrieve a user by name
- Create new user(s)
def get_user(self, name: str) -> (User | None):
"""Retrieve a User by a name"""
query = "SELECT name, email FROM users WHERE name = %s"
res = self.execute_select_query(query, (name,))
if len(res) == 0:
return None
record = res[0]
return User(name=record[0], email=record[1])def insert_users(self, users: List[User]) -> None:
"""Given a list of Users, insert them in the db"""
query = "INSERT INTO users (name, email) VALUES %s"
data: List[tuple] = [(u.name, u.email) for u in users]
self.execute_multiple_insert_query(query, data)
Awesome! We have some functions that interact without business entities.
Notice that if we don’t test these functions, we are susceptible to errors if the schema changes or if we violate database constraints.
So, let’s start testing!
import pytest
from psycopg2.errors import UniqueViolation@pytest.mark.usefixtures("repo")
class TestRepo:
def test_insert_users(self):
repo: Repo = self.repo # repository instanced passed by the fixture
# define some users
alice = User(name="alice", email="alice@example.com")
bob = User(name="bob", email="bob@example.com")
robert = User(name="robert", email="bob@example.com")
# check that the users are not there at first
result = repo.get_user("alice")
assert result is None
# check that the users are there after inserting
users = [alice, bob]
repo.insert_users(users)
result = repo.get_user("alice")
assert result == alice
result = repo.get_user("bob")
assert result == bob
# check that pk fails
with pytest.raises(UniqueViolation):
repo.insert_users([robert])
The first line is a decorator that specifies that our test class TestRepo
will use a fixture called repo
. Fixtures are baseline functions that allow us to produce consistent and repeatable test results.
Let’s understand this line repo: Repo = self.repo
. We are creating a variable called repo
of type Repo
which is our previously defined repository class. It is being reassigned from self.repo
, which is coming from the fixture, as I will explain later.
We assert a couple of things:
- There are no users at the beginning of the test
Alice
andBob
users are retrieved after insertion- There is a
UniqueViolation
exception, and it is raised when we attempt to create a new user with an already existing email.
Let’s write our fixture in a new conftest.py
file.
@pytest.fixture(scope="class", name="repo")
def repo(request):
"""Instantiates a database object"""
db = Repo()
try:
request.cls.repo = db
yield db
finally:
db.close()
In this fixture, we do the following:
- Instantiate our repository class
db = Repo
. - Within a
try
block (as the test execution might raise an exception), we use pytest’srequest
fixture so our test class can access the reporequest.cls.repo = db
. I encourage you to learn more about therequest
fixture! - Then we yield the instance to be used in the test functions.
Within afinally
block, we make sure we close the database connection.
Great! We have our tests, but if you are eager to run pytest
already, you will see that the tests fail at the fixture, as we cannot instantiate the repository without a postgres database!
Here is where tox comes to the rescue. Create a new file called tox.ini
:
[tox]
envlist = py310[testenv]
setenv =
DB_DSN = postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable
deps =
-r requirements.txt
commands = pytest
This minimal configuration file tells us we will perform testing with a Python 3.10 environment. It sets an environmental variable DB_DSN
, specifies a requirement file to be installed in a virtual environment, and calls pytest. My requirements file looks like this:
psycopg2-binary
pydantic
pytest
By the way, I recommend using pip-toos
to pin dependencies. That is out of the scope of this tutorial, but you can read it here: https://github.com/jazzband/pip-tools.
Running your tests now is as easy as just installing and running tox
:
python -m pip install — user tox
tox
Of course, this fails because I have been promising that tox will coordinate a postgres container, and I haven’t done so.
Tox is a useful tool that can get powerful with its plugins. Tox-docker is one of them, and it’s easily installed by running the following command:
pip install tox-docker
Now we can extend our tox.ini
. Here’s how to do that:
[tox]
envlist = py310[testenv]
setenv =
DB_DSN = postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable
deps =
-r requirements.txt
commands = pytest
docker = postgres
[docker:postgres]
image = postgres:13.4-alpine
environment =
POSTGRES_DB=postgres
PGUSER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_HOST_AUTH_METHOD=trust
ports =
5432:5432/tcp
healthcheck_cmd = pg_isready
healthcheck_timeout = 5
healthcheck_retries = 5
healthcheck_interval = 5
healthcheck_start_period = 1
volumes =
bind:ro:{toxinidir}/migrations/schema.sql:/docker-entrypoint-initdb.d/0.init.sql
Notice that in our testenv
section, we specify we will use a docker container named postgres
which we immediately after define. We set the docker image it should (pull) use, environmental variables, ports to map, health checks (useful to make sure our tests are running only when our containers are healthy), and volumes (notice I refer tomigrations/schema.sql
which contains our SQL table definition). Please check tox-docker documentation if you want more details.
Now, by running tox
we pass our tests!
Notice that tox creates the containers, runs the tests, and then removes the containers for us. Pretty cool, right?
Running our tests locally is a great practice, but to be very thorough, it is better if we also run them in our version control tool when making a pull request. With GitHub Actions, we can create a small CI pipeline to run tox every time a commit is pushed to a pull request. Just create this file, /.github/worflows/pr-test.yaml
:
name: PR Teston:
pull_request:
branches:
- main
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10"]
name: With Python ${{ matrix.python-version }}
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
architecture: x64
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install tox tox-gh-actions
- name: Test with tox
run: tox
Now, every time we create a pull request to our code base, GitHub Actions will run tox and make sure our tests pass.
With the current approach, we can easily scale dependency testing by adding more docker containers in the tox configuration.
[docker:redis]
image = bitnami/redis:latest
environment =
ALLOW_EMPTY_PASSWORD=yes
REDIS_PORT_NUMBER=7000
ports =
7000:7000/tcp
healthcheck_cmd = redis-cli ping
healthcheck_timeout = 5
healthcheck_retries = 5
healthcheck_interval = 5
healthcheck_start_period = 1
External dependencies that can’t be containerized or that represent costs or protected resources, such as a private API, should better be mocked.
If the usefulness does not convince you of tox, a different approach is to set the dependencies in a docker-compose file, create and run the services, wait for them to be healthy, run the pytests, and gracefully stop and remove containers.
- GitHub repo for this project: https://github.com/vrgsdaniel/testing-containers
- Pytest documentation: https://docs.pytest.org/en/7.2.x/
- Pytest fixtures: https://docs.pytest.org/en/6.2.x/fixture.html
- Pytest request fixture: https://docs.pytest.org/en/6.2.x/reference.html#std-fixture-request
- Tox: https://tox.wiki/en/latest/
- Tox-docker: https://tox-docker.readthedocs.io/en/latest/
- GitHub Actions: https://github.com/features/actions
- PYnative postgres tutorial: https://pynative.com/python-postgresql-tutorial/
- Pip-tools: https://github.com/jazzband/pip-tools