Why mock HTTP in a container
In February, when I wrote about how I compose a test environment with Compose, I left one thing for later: an HTTP mock instead of a real external service. I am returning to it today because the problem has resurfaced on its own.
The application I am testing has its own database and cache, but it fetches the product catalog from a partner API. Everything about working with this API is inconvenient. The partner’s sandbox sometimes goes down just when I run a regression. It has a request limit, so a parallel run on CI can get a 429 and the test looks broken, even though it is the limit, not the test, that is broken. The data in the sandbox changes without warning, so a price assertion remains valid for two weeks. And I cannot trigger the scenarios I really care about there on demand: a timeout, a 503, and an empty response. I cannot ask the partner to break their environment for a minute.
I have been using containers to isolate heavyweight dependencies since January, when I described Docker for QA. In April, I went one step further and started a real MSSQL database from the test. This is different, and that is the point of this post: I do not want to run the real partner in a container because I do not have their code and do not want to have it. I want to run something that responds over HTTP exactly as we agreed in the contract and can return an error on demand. For that, I use WireMock, an HTTP server to which I provide pairs in JSON: “when a request like this arrives, respond like this.”
Two paths in May 2021: standalone JAR vs the rodolpheche/wiremock image
WireMock can be started in two ways, and I keep both because they serve different purposes.
The first is a standalone JAR run with java -jar. It starts in a fraction of a second, does not require Docker, and is the most convenient option when I am working on one scenario and changing the response repeatedly. It requires only Java on the machine. The second is a container image. I choose it when the mock is to be part of the environment rather than my private toy: when it goes into docker-compose.yml next to the database and application, and when the same file has to work on a CI agent where Java may not be installed at all.
There is one thing worth saying explicitly because I ran into it myself. There is currently no image published by the WireMock organization. The image I use is rodolpheche/wiremock, and it is a community image maintained outside the project. The practical consequence is that the JAR version and the image tag version do not have to advance at the same pace.
WireMock 2.28.0 (May 11) - artifact com.github.tomakehurst:wiremock-jre8-standalone:2.28.0
The latest version is 2.28.0, published to Maven Central on May 11, 2021, four days ago. The previous line, 2.27.2, is from September 10, 2020 and is still in use.
The artifact I am interested in is the standalone variant com.github.tomakehurst:wiremock-jre8-standalone:2.28.0. The name jre8 can be misleading, so I explain it to the team every time. It is not a requirement for “exactly Java 8.” It is a variant built for Java 8 and newer, as opposed to the older artifact for Java 7. standalone means that the JAR packages all dependencies together with the HTTP server, so I do not need to wrap it in a project.
I download it once and keep it in the test directory:
$ wget -q -O wiremock.jar \
https://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-jre8-standalone/2.28.0/wiremock-jre8-standalone-2.28.0.jar
$ java -jar wiremock.jar --port 8090 --root-dir ./wiremock --verboseThree flags that I always use. --port because the default 8080 is usually occupied by the application under test. --root-dir because it points to the directory where WireMock looks for the mappings/ and __files/ subdirectories, and I want to keep them in the test repository rather than whichever directory I happen to run the command from. --verbose because without it, when a request does not match any stub, I stare at an empty console and guess. With --verbose, the log gives me the full request and a list of the closest matches with information about what did not match.
docker run community image + volumes for mappings/ and __files/
Now the same thing in a container. The directory structure is identical because it is the same application inside:
wiremock/
mappings/
catalog-product-1001.json
catalog-product-unavailable.json
__files/
catalog-page-1.jsonAnd the command:
$ docker run -d \
--name catalog-mock \
-p 58090:8080 \
-v "$(pwd)/wiremock/mappings:/home/wiremock/mappings" \
-v "$(pwd)/wiremock/__files:/home/wiremock/__files" \
rodolpheche/wiremock:2.27.2 \
--verboseSeveral decisions here are deliberate.
The tag is pinned, and it is 2.27.2, not latest. The rule is the same as with Postgres in January, but there is another reason here. As of the day I am writing this post, there is no 2.28.0 tag for this image on the Hub even though the JAR has been on Maven Central for four days, so I take last year’s 2.27.2 line. The image is maintained by the community, and no one guarantees that it will keep up with new versions. I prefer to have that written explicitly in the command rather than discover six months from now that latest arrived in the middle of a sprint.
I mount the volumes under /home/wiremock because that is the working directory in this image. This keeps the stub files in the test repository and makes the container fully disposable. I delete it and start it again without losing anything.
The port on the left is high and unusual, as in the previous posts, because WireMock still listens on 8080 inside the container. Arguments appended after the image name go directly to WireMock, so --verbose works here just as it does with java -jar.
If I absolutely need version 2.28.0 in the container, I do not look for a tag. I write a few lines of Dockerfile and download the JAR directly from Maven Central:
FROM openjdk:8-jre
ARG WIREMOCK_VERSION=2.28.0
WORKDIR /home/wiremock
RUN wget -q -O /wiremock.jar \
https://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-jre8-standalone/${WIREMOCK_VERSION}/wiremock-jre8-standalone-${WIREMOCK_VERSION}.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/wiremock.jar", "--port", "8080", "--root-dir", "/home/wiremock"]This takes a dozen or so seconds of work and removes the dependency on someone else’s publishing schedule. For now, however, I am staying with the pinned community image in Compose because I do not want every clone of the repository to begin by building an image.
A minimal JSON stub and checking GET /__admin/
A stub is one JSON file in mappings/. The structure is always the same: request describes what should match, and response describes what to return.
wiremock/mappings/catalog-product-1001.json:
{
"request": {
"method": "GET",
"urlPath": "/api/products/1001"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"jsonBody": {
"id": 1001,
"name": "400 ml thermal mug",
"price": 59.9,
"currency": "PLN",
"inStock": true
}
}
}Note urlPath, not url. url compares the whole value, including the query string, so /api/products/1001?lang=pl will not match. urlPath compares only the path, and that is usually what I want.
The second file, wiremock/mappings/catalog-product-unavailable.json, is the error path, which is the reason I am doing this at all:
{
"request": {
"method": "GET",
"urlPath": "/api/products/9999"
},
"response": {
"status": 503,
"fixedDelayMilliseconds": 3000
}
}A three-second delay and a 503. I cannot trigger this on the partner’s sandbox, but here it takes ten lines. This also gives me a test that finally makes sense: I check whether my application responds with a reasonable message instead of a blank page. When the response is larger, I do not paste it into the mapping. I move it to __files/ and provide "bodyFileName": "catalog-page-1.json", which lets me open and format the payload file like ordinary JSON.
To check whether it has started at all, I use the administrative API. It is always under /__admin/:
$ curl -s -o /dev/null -w '%{http_code}\n' http://localhost:58090/__admin/
200
$ curl -s http://localhost:58090/__admin/mappings
$ curl -s http://localhost:58090/api/products/1001
{"id":1001,"name":"400 ml thermal mug","price":59.9,"currency":"PLN","inStock":true}The first command is my readiness test. It responds only when the server is actually accepting traffic, so it is suitable for a wait loop. The second tells me whether WireMock saw my files at all, and if the list is empty, nine times out of ten I got the volume path wrong and the container is looking in its own empty directory. The third queries the stub itself.
There are two more administrative endpoints that save me when debugging. GET /__admin/requests shows a journal of the requests that actually arrived, so I can immediately see whether the application made a call at all and to which address. POST /__admin/mappings/reset reloads the stubs from disk, so I do not have to restart the container after editing a file.
Compose: application under test + WireMock on one network
Ultimately, the mock is not a separate entity but another service next to the database and cache. I now have docker-compose version 1.29.2 from May 10 and Engine from the 20.10 line:
$ docker-compose version --short
1.29.2The file extends the one from February:
version: "3.8"
services:
db:
image: postgres:13.2
environment:
POSTGRES_PASSWORD: test
POSTGRES_DB: shop
ports:
- "55432:5432"
cache:
image: redis:6.0.10
ports:
- "56379:6379"
catalog-mock:
image: rodolpheche/wiremock:2.27.2
command: ["--verbose"]
volumes:
- ./wiremock/mappings:/home/wiremock/mappings
- ./wiremock/__files:/home/wiremock/__files
ports:
- "58090:8080"
api:
image: rejestr.example/shop-api:2021.02
environment:
DATABASE_URL: postgres://postgres:test@db:5432/shop
REDIS_URL: redis://cache:6379
CATALOG_API_URL: http://catalog-mock:8080
ports:
- "58080:8080"
depends_on:
- db
- cache
- catalog-mockThe most important line is CATALOG_API_URL: http://catalog-mock:8080. Compose creates a network for the project where services see each other by name, so the application connects to catalog-mock, not localhost. The port is also the internal one, 8080, not 58090. This is a mistake I made on my first attempt: I entered the address I used with curl on my machine in the application configuration and got a connection refused error because for a container, localhost is the container itself.
I leave the ports section on the mock anyway, even though the application does not need it, because I want access to /__admin/requests from my terminal when the test behaves strangely.
One assumption is hidden in all of this: the application under test must have the partner’s address in configuration, not hardcoded in the code. If it does not, switching to the mock is impossible, and the first change is not a test but moving that address to an environment variable. This is usually one line and the greatest value this post can offer someone getting started.
I handle readiness the same way as in February, except that I query /__admin/:
#!/usr/bin/env bash
set -euo pipefail
docker-compose up -d
for _ in $(seq 1 30); do
if curl -sf -o /dev/null http://localhost:58090/__admin/; then
echo "Mock ready on localhost:58090"
exit 0
fi
sleep 1
done
echo "Mock did not start. Logs below:"
docker-compose logs --no-color catalog-mock
exit 1After the run, I clean up ruthlessly with docker-compose down -v. The stubs are in the repository, so the container holds nothing I would regret losing.
What I do not do: Testcontainers lifecycle, official organization image, Pact
I deliberately leave three things outside this post.
I do not start WireMock from the test code. In April, I showed that a container can be controlled from the test, and it made sense there because the MSSQL database had to be clean for a particular test class. An HTTP mock is different: it is part of the environment, it runs as long as the application runs, and I want to see it in the same file as everything else. I do not rule out returning to the test-controlled variant, but first I want to work through this simpler approach.
I do not use an image published by the WireMock organization because no such image exists today. I use the community image with a pinned tag or build my own on openjdk:8-jre. If the project ever starts publishing its own images, switching will be a one-line change in Compose, and I will do it then.
I am not doing contract testing in the style of Pact here. It is a different tool for a different problem: Pact makes sure that the contract between the consumer and provider actually matches and that the provider finds out when it breaks the contract. WireMock does not provide such a guarantee. My stub is my idea of how the partner responds, and if the partner changes the price field to grossPrice, my tests will stay green while production breaks. I therefore keep two rules. The stubs live in the repository and undergo review like code. Alongside the mocked suite, I keep a narrow set of tests that hits the real sandbox and checks only the shape of the response.
A mock does not replace integration. It only means that I stop debugging someone else’s environment instead of my own code and that I can finally write a test for a 503. It is still worth checking from time to time where this fits in the entire test process, because a fast feedback loop is a means, not an end.