Docker Compose for local test environments

When one container is not enough

A month ago, I wrote about how I spin up a single test dependency with Docker. One database, one docker run, one script waiting for readiness. That is enough exactly as long as the application under test has one dependency.

For me, it stopped being enough sooner than I expected. The scenario I wanted to automate looked like this: the application reads from Postgres and stores sessions and the shopping cart in Redis. A regression test for logging out makes no sense without both of them running at the same time. Suddenly, instead of one command, I had three, each with its own set of flags, and a Bash script that was beginning to look like a small operating system: startup order, names, networking, and cleanup after a failure halfway through.

At this point, I stop writing a script and reach for a file. A declarative description of the environment has the advantage that I do not have to remember the order of commands or reconstruct it in my head. I put the file in the test repository, and everyone who clones it gets the same environment.

What I have installed

At the moment, Compose is a separate tool written in Python and run with the docker-compose command, including the hyphen. I do not confuse it with the Engine because they have two independent version numbers:

$ docker version --format '{{.Server.Version}}'
20.10.3

$ docker-compose version --short
1.28.2

Engine 20.10.3 has release notes from the beginning of February. On Windows and macOS, I do not install the Engine separately. I use Docker Desktop instead - version 3.1.0 from January in my case. I installed Compose 1.28.2 from January 26 myself because I am interested in a feature introduced in 1.28.0: service profiles. I still had 1.27.4 in January, and that version does not have profiles. If the examples below are to work for someone, the docker-compose version is the first thing I check when they report that “it does not start for me.”

A minimal file with three services

I start with the shortest thing that does anything useful. The docker-compose.yml file goes into the test directory:

version: "3.8"

services:
  db:
    image: postgres:13.2
    environment:
      POSTGRES_PASSWORD: test
      POSTGRES_DB: shop
    ports:
      - "55432:5432"

  cache:
    image: redis:6.0.10
    ports:
      - "56379:6379"

  api:
    image: rejestr.example/shop-api:2021.02
    environment:
      DATABASE_URL: postgres://postgres:test@db:5432/shop
      REDIS_URL: redis://cache:6379
    ports:
      - "58080:8080"
    depends_on:
      - db
      - cache

Several decisions here are deliberate.

The image tags are pinned. postgres:13.2, not postgres:latest. It is the same rule as with a single docker run: latest is not a version but a moving pointer, and I want a run from February to be reproducible in April.

The ports on the left are high and unusual. Compose creates its own network for the project, and the services see one another there by name, so DATABASE_URL points to db:5432, not localhost. I publish ports to the host only because my test runner operates outside Compose and needs somewhere to connect. If the tests were also a service in this file, I could remove the ports sections for the database and cache entirely.

In this form, depends_on says only what order to start services in. Compose starts db and cache before api, but it does not check whether Postgres has finished initializing. This is the same trap I described with docker run: a running container does not mean a ready application. I will return to it shortly.

Profiles, or why I need a UI for an API test

This is the feature for which I updated Compose. Up to 1.27, docker-compose up started everything in the file. I therefore had two options: keep several files and combine them with the -f flag, or list service names manually on every run. Both variants drifted out of sync.

Starting with 1.28.0, I can assign services to profiles and start only what I need:

version: "3.8"

services:
  db:
    image: postgres:13.2
    environment:
      POSTGRES_PASSWORD: test
      POSTGRES_DB: shop
    ports:
      - "55432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d shop"]
      interval: 5s
      timeout: 3s
      retries: 10

  cache:
    image: redis:6.0.10
    ports:
      - "56379:6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 10

  api:
    image: rejestr.example/shop-api:2021.02
    environment:
      DATABASE_URL: postgres://postgres:test@db:5432/shop
      REDIS_URL: redis://cache:6379
    ports:
      - "58080:8080"
    depends_on:
      - db
      - cache

  adminer:
    image: adminer:4.7.9
    ports:
      - "58081:8080"
    profiles: ["dev"]
    depends_on:
      - db

The rule is simple and easy to miss: a service without a profiles key always starts, while a service with a profile starts only when I explicitly enable that profile. That is why I leave the core of the test environment without profiles and put everything that is a convenience for the person at the keyboard in the dev profile.

For tests, I therefore start three services:

$ docker-compose up -d

When I want to look at the database through a browser myself, I add the profile:

$ docker-compose --profile dev up -d

Alternatively, I set COMPOSE_PROFILE=dev in my local .env so that I do not have to add the flag every time. In 1.28.2 the name is singular. This variable is absent from the CI machine, so by definition CI starts only the core.

Readiness, or why an open port is not the same as health

I added a healthcheck to the database and cache above, and I want to explain why because this is the most common source of flickering tests in such a setup.

Checking whether port 55432 is open returns true very early. Docker maps the host port when the container starts, so a TCP connection can be established before Postgres completes its initial setup and creates the shop database. The test then receives an authentication error or “database does not exist,” and it looks like an application problem rather than a race.

A healthcheck asks for something that only a ready service can answer: pg_isready for Postgres and redis-cli ping for Redis. I can then see the state in the status:

$ docker-compose ps
     Name                   Command                State                Ports
---------------------------------------------------------------------------------------
shop_api_1       /app/entrypoint.sh            Up             0.0.0.0:58080->8080/tcp
shop_cache_1     docker-entrypoint.sh redis    Up (healthy)   0.0.0.0:56379->6379/tcp
shop_db_1        docker-entrypoint.sh postgres Up (healthy)   0.0.0.0:55432->5432/tcp

It is worth knowing that docker-compose up itself does not wait for healthy. On 1.28.2 the longer form of depends_on with condition: service_healthy is honored, and Compose will wait until the service reports healthy. I still prefer a Bash readiness loop, because older Compose on CI does not have that form. It is the same as in the script from January:

#!/usr/bin/env bash
set -euo pipefail

docker-compose up -d

for _ in $(seq 1 30); do
  if docker-compose exec -T db pg_isready -U postgres -d shop > /dev/null 2>&1; then
    echo "Environment ready"
    exit 0
  fi
  sleep 2
done

echo "Database did not start. Logs below:"
docker-compose logs --no-color db
exit 1

The -T flag is important here because without it, exec tries to allocate a terminal and can fail on a CI agent. docker-compose logs without a service name shows the output from all containers interleaved and prefixed. With three services, it is still readable, and it is usually the first thing I paste into a report.

Cleanup

This part determines whether the environment is truly on demand.

$ docker-compose down

down stops and removes the containers and the project network, but it leaves data volumes behind. This can be desirable when I want to inspect the state of the database after a failed test. My default mode, however, is the ruthless version:

$ docker-compose down -v

-v also removes the volumes, so the next up gets an empty database and a fresh data directory. If you have ever seen a test that passes only the first time or only the second time, the answer is very often data that survived the previous run. A clean start is cheaper than debugging such a dependency.

One small detail saves frustration: the project name. By default, Compose derives it from the directory name, so two repositories with a directory named tests will interfere with each other. I set it explicitly with the -p flag or the COMPOSE_PROJECT_NAME variable, especially on a CI agent where several runs may operate in parallel.

What this post does not cover

I am deliberately leaving three things for separate posts.

A browser farm. It can be connected as more services in the same file and even hidden behind a profile, but scaling and stabilizing such a setup is a topic of its own.

An HTTP mock instead of a real external service. In practice, it is simply another service in Compose, but the entire difficulty lies in what to feed it and how to version the responses.

Managing the container lifecycle from the test code so that the test, rather than the file next to it, decides what to start. I want to try it, but I do not have my own results yet, so I will not write about it here.

Compose does not make my test suite better. It only makes the environment stop being something that must be booked and requested and turns it into a file in the repository. That is still a big change, but it is worth remembering what it is for and returning from time to time to the bigger picture of what the entire test process looks like. When I started with a manually prepared environment, half a day went into bringing the machine to a state in which the tests would run at all. Now it is one command and thirty seconds.