Why a recap
It is December 15. A year of posts about tools is behind me, and I am genuinely tempted to write a twelfth tutorial, paste one more YAML file, and stop there. I will not. Anyone can reconstruct the list of tools covered since January from the archive in two minutes and learn nothing. A changelog is not a recap.
A recap makes sense to me only when it asks what a single post cannot because it sits too close to its own tool. Writing about Compose, I think about Compose. Writing about parallelism on an agent, I think about the agent. Only twelve months of distance shows that all these posts circled one thing I could not name at the start of the year.
The question of the year is this: is my test suite isolated, or does it share a machine, database, and WebDriver with someone else?
It is one question answered across several layers at once, which is why it took a year. You can have excellent page objects and still share a database with a teammate. A database can be containerized while the run still shares port 4444 with a previous run that did not clean up. Everything can be clean locally, then fail in CI because the agent is not my machine. These look like different problems until you see the same hole from three sides.
In 2019, I wrote about preparing a test environment, meaning how to install something on your own machine. The instructions made sense, but treated the environment as somewhere you enter, not something created for a test and discarded. I spent this year changing that assumption.
Two pillars already on the blog
Two posts from this year carry this thesis more strongly than the rest. Both have been on the blog for a long time, so I will not rewrite them here.
The first is April’s .NET Testcontainers library, infrastructure for the tests. I started MSSQL from test code through DotNet.Testcontainers. Everything ran on my machine, so the post has no pipeline or agent and a database that lives only for the run. That is when I stopped seeing the database as something beside the test and made it a dependency the test provisions for itself.
The second is July’s Retry policy for methods in terms of integration tests, meaning Polly. I need to say something that sounds contradictory but is not: retry does not cure missing isolation. Polly handles races inherent to the system very well: a service starts a fraction of a second after I query it, a queue is delayed, or a connection drops once in a hundred attempts. Test design cannot remove events on the other side.
If a test is red because another left an order in the database, retry only makes the problem harder to detect. The test goes green, nobody investigates, and state still leaks. My practical rule this year is to retry things that are inherently nondeterministic, never things made nondeterministic because someone failed to clean up.
These two posts are my pillars of the year, but also its beginning and middle, not its end. The remaining months added layers around them.
Containers locally
I started with the simplest possible thing. January’s Docker for QA - on-demand test environments covered one dependency, one docker run, an Engine from the 20.10 line, ports, logs, and cleanup. No orchestration or configuration file. I kept it deliberately narrow because the first hurdle is not technical. It is about no longer treating a database as something installed once a quarter and then maintained.
February brought Docker Compose for local test environments: a file, several services at once, and profiles, introduced in docker-compose 1.28. Profiles mattered more than I expected. They let me keep one environment description and start only what a given test type needs. An API test does not need to wake the front end.
March brought Selenium Grid in Docker - Grid 3 and Grid 4 beta. I used Grid 3 with images pinned to 3.141.59-20210311; Grid 4 was only a beta viewed out of curiosity, not something I put to work then. It completed the local puzzle: the browser is also a dependency that can start on demand instead of living in the system.
Together, these posts form one line: from one dependency, through several described in a file, to one that had seemed impossible to move. They also differ from 2019. That environment setup post said, “install it on your machine and be careful.” These three say, “describe it in the repository and discard it after the run.” The difference is not the tool, but who owns the environment: a person or a repository file.
HTTP or a real service
Midyear, I worked on the test’s other side: whatever responds to requests.
In May, I covered API mocking in containers - WireMock and Docker. I used WireMock 2.28.0 and the community image rodolpheche/wiremock because no official image from the organization existed. Stubs lived in mappings/, mounted as a volume, and the whole setup started with the application from one Compose file.
A year-end note: since 2.31.0 in September, there has been an official wiremock/wiremock image. It does not invalidate the May post because it did not exist then, and I will not rewrite the post because of it. I am recording the fact to check next time I touch that configuration anyway.
At the other end of the same axis is April’s .NET Testcontainers library, infrastructure for the tests: a real service instead of an imitation. The same choice returned all year: a fast, predictable response I invented, or a slow response that is real?
I have no single rule and doubt one exists, but I have partial rules. I use an HTTP stub when the other side is someone else’s, paid, or unstable, and when testing my code against a specific response, including ones the real service will not produce on demand. I use a real containerized service when testing integration with something I control and a stub’s lie could create a false green result. Databases almost always belong here because SQL has too many details to imitate with JSON.
I wrote the honest caveat in May and still stand by it: a stub is my idea of someone else’s response, worth only its latest verification.
Data content, not just infra
Until September, I thought organizing the infrastructure would organize the data. It did not.
September’s Test data in .NET - factories, Respawn, containers explains that a container answers “where,” not “what.” The database starts empty, and someone must choose its contents. I arranged this into four layers: an in-memory factory with a seed, a shared database with a reset, a disposable containerized database, and no database with a stub response.
Two specifics changed my daily work most. First, a seed in a Bogus factory. Random data without one creates occasional failures I cannot reproduce. A seed makes randomness repeatable, so we can discuss it. Second, Checkpoint from Respawn for a shared database. It costs less than starting a container per test while providing state nobody before me touched.
June stands apart: GPT-3 API (beta) - generating test data with Completions. I stand by that post, and six months later see it more clearly. This was an experiment, not a new data strategy layer. I called davinci through Completions with a few-shot prompt for fixture-like JSON. It helped invent data but was useless as CI’s source of truth because identical prompts need not return identical results.
Since November 18, the API has had no waitlist, removing the access barrier. Two other things remain: these are still paid Completions, with text in and text out, not an interface you have a conversation with. Every generated fixture still gets my review before entering the repository. The generator suggests twenty address variants; it does not decide which is correct.
UI code
In August, I put nothing in a container. Page Object refactor in C# - components instead of fat pages covered test code using Selenium 3.141.0 and NUnit 3.13.
Three decisions stayed with me. I model page fragments in Fowler’s sense, not whole pages, because a fat page class grows with the application and after a year nobody reads it. I avoid PageFactory, long marked obsolete in .NET, so I do not build new code on it. The driver lives in test scope, not a static field shared across the run. It is the same isolation story at process level instead of database level.
Compare this with 2019, when I compared App Actions with the Page Object Model. That post is not outdated, and I will not rewrite it in C#. The contexts differ: Cypress lets me go below the UI and set application state from the test; Selenium usually does not. The same pattern name means different things in these worlds, and that is fine.
Selectors remain separate. I wrote about using selectors correctly and the data-cy attribute in 2019 and have not changed my mind in two years. A stable attribute added to markup for tests is cheaper than the world’s best XPath.
CI
The year’s last two months went into CI, and it is no accident they came this late.
October’s Parallel tests in CI - NUnit, VSTest, matrix jobs separates three levels whose confusion had cost me time: framework threads in one process, VSTest processes on one machine, and separate jobs on multiple agents, meaning slices in Azure Pipelines or strategy.matrix in GitHub Actions. Each divides work differently and accidentally shares different things.
November’s Testcontainers on Azure Pipelines completes April. On a hosted ubuntu-20.04 agent, Docker runs on the host, giving the library something to talk to. The trap I encountered and want to remember is a container job. That Azure DevOps mechanism has existed since 2018 and is nothing new, but a step running inside a container has no nested daemon, leaving Testcontainers nothing to connect to.
Their order matters more to me than their content: isolation first, then wall-clock time. Parallelism amplifies what you have by the worker count. Clean state gives a faster suite. Leaking state gives faster noise and a team trained to click “rerun.”
What I am deliberately leaving for later
Three things exist that I did not adopt this year. I am recording them because next year I will not remember this was a decision, not an oversight.
Compose version two, meaning docker compose without the hyphen, had its first releases this year. All my posts this year teach docker-compose with the hyphen in the 1.27 and 1.28 lines, matching my local and agent setup. I will not write a tutorial about an everyday CLI I do not use daily or change a command in team documentation halfway through the year. I will switch when it is obvious, not merely possible.
Selenium 4.0.0 was released on October 13, a real rather than cosmetic change. My suite is still on Selenium.WebDriver 3.141.0, and I am closing the year that way. Client migration is separate work with its own risk, and December is a bad month to replace the foundation under a regression suite.
Third is the language model interface. I have Completions and treat them as the API they are: I call it, receive text, and check it. I am not building a conversation around it, connecting it to test runs, or making it the production suite’s data generation layer. It remains an experiment, and I want that explicit label preserved in the blog archive.
One sentence for 2022
One sentence: next year, I want to measure flake rate and suite run time, not how many tools I can start.
This is an intention, not a report, and I do not know how it will turn out. I do know why these two numbers: both show whether waiting for green is worth someone’s time. Repository tool count says nothing about that but is nicer to show, so it easily replaces the real result.
The second thing I am taking into January comes from 2019, when I described the test process I aim for in projects. This year showed me its other side. A container, stub, factory seed, and CI matrix are tools. Deciding who looks at a red result and what happens within an hour of its appearance is process. I spent 2021 on tools and do not regret it because without isolation there is nothing to discuss. But those tools are organized well enough that adding more is no longer the cheapest improvement.
See you in January.