When I offload the runtime and when I stay with the daemon
It is July 15. For more than three years, my integration tests have rested on one assumption: a Docker daemon runs next to the test process, and the library talks to it through a socket. In April 2021, it was Docker on my laptop and the first MSSQL database started from test code, still through TestcontainersBuilder. In November 2021, it was a daemon on the agent host in Azure Pipelines, and the entire risk came down to whether the job could see that socket. In August 2022, a second job, with Playwright, appeared next to the integration job, and the assumption did not change: Docker lives on the agent host, and the Testcontainers job runs beside it.
Testcontainers Cloud stayed off the blog during that time, deliberately. The public beta started on January 25, 2023, and in my December year-end summary I recorded it as a topic, not an opinion, because for twelve months I did not find a day to test it properly. On December 11, 2023, Docker acquired AtomicJar, the company behind Testcontainers, together with its two products: Testcontainers Desktop and Testcontainers Cloud. Today, a year and a half after the beta and seven months after the acquisition, I am closing this topic with one question: when do I offload the container runtime to the cloud, and when do I stay with the daemon on my laptop and on the hosted agent?
My thesis for July 2024 is narrow. Testcontainers Cloud is mature enough as a service for me to offload the container runtime from my laptop and from a hosted CI agent to the cloud without rewriting tests based on ContainerBuilder. This does not replace the lesson from 2021. A local daemon and Docker on the Azure agent host remain my default, and I start every new suite with them. Cloud is an option for two specific problems: when containers consume the CPU and RAM of the machine running the test, and when the only route to Docker on the agent would be privileged Docker-in-Docker.
“Mature” does not mean “after a GA announcement” here. I did not find a separate general availability announcement, and I am not adding a date that I cannot demonstrate with a source. I assess maturity through three things that can be read today. The Cloud documentation has a ready-made step for Azure Pipelines, an action for GitHub Actions, and instructions for several other CI systems. The billing model is documented, including a free plan for an individual working alone. And the .NET library knows about the service without any plugin, which I will show in the next section using the source code of version 3.9.0.
Here is the state of the workshop, so that a year from now it is clear what I mean. The test project runs on .NET 8, the LTS release from November 14, 2023; last week’s .NET 9 preview is not a pin for anything in this post. The Testcontainers NuGet package is 3.9.0 from mid-June, with release notes dated June 17. Beside it are Npgsql 8.0.3, xUnit 2.9.0, and the postgres:16.3-alpine image, pinned for the same reason that in January 2021 I wrote postgres:13.1, not postgres:latest. Playwright 1.45.1 from July 2 runs in the neighboring job and is background, not the subject of this post.
What will not be here: another introduction to Testcontainers or Azure agent Docker YAML written from scratch, because both are under the links above and still apply. There will be no GA date, prices or minutes from a real account, organization name, or token. There will be a synthetic Orders.IntegrationTests project from the blog examples repository: one class with PostgreSQL, the same test on two runtimes, one changed line, and an excerpt from an Azure Pipelines job.
What it is and what it is not
Testcontainers Cloud consists of three pieces, one of which I have known since 2021. The first is the Testcontainers library, the same one as before, in Java, .NET, Node, and the other languages. The second is a cloud worker, a machine with a Docker daemon allocated for the duration of a run. The third is the agent that connects the two: on a laptop, it is the Testcontainers Desktop application; on CI, it is a separate command-line agent launched with a service account token. According to the documentation, the agent opens an SSH tunnel to the daemon on the worker, the image is pulled and the container starts on the worker side, and after the suite finishes the service waits briefly, closes the connection, and releases the resources.
The most important part is how the library finds that daemon, because this determines whether the test code needs to know anything about the cloud. Both Desktop and the CI agent write their address to ~/.testcontainers.properties as the tc.host property. To avoid relying on the description alone, I looked at the Testcontainers for .NET source code for version 3.9.0. The list of Docker connection providers starts with the one that reads tc.host; only after it come DOCKER_HOST, the Unix socket, and the remaining options. The first provider that has a configuration and responds to a ping wins. The comment above that provider’s class says directly that when a machine has multiple runtimes, Testcontainers Cloud takes precedence as long as it is running.
Three practical consequences follow. First, the switch lives outside the code: in the Desktop menu or a pipeline step, not in ContainerBuilder. Second, DOCKER_HOST is not a runtime selector because a working agent always takes precedence over it. Third, the Cloud documentation describes disabling the service for a single project, but through a testcontainers.properties file on the classpath, which means Java. I did not find such a switch in .NET 3.9.0: every project on a given machine goes wherever the file in the home directory points.
What is it not? It is not a new framework or a new library line that requires a migration, and my tests import nothing from Cloud. It does not replace the open source libraries: they remain free, and when Docker acquired AtomicJar it said that their licenses would not change. The runtime is paid for, not the API. Finally, this is not remote Docker for everything. Desktop creates a Docker context named tcc, through which I can inspect containers with docker ps, but the Cloud documentation warns that running custom Docker commands against this context is unsupported and can allocate a worker that does nothing. Running docker compose against that context is the same unsupported case.
Testcontainers Desktop is worth separating from Cloud because they are two different things. Desktop is a closed-source companion application for macOS, Windows, and Linux, with a free plan for individuals and a required account. It also works without the cloud: it lets me switch between local runtimes, expose a test container on a fixed port so I can inspect the database in a SQL tool, and, for now in beta, pause container cleanup so I can inspect the data after a test. Cloud is one of the options in the Desktop menu.
Now for the code I use to test this. Here is the entire PostgresFixture class from the synthetic project, after the one change that I will return to shortly:
using DotNet.Testcontainers.Builders;
using DotNet.Testcontainers.Containers;
using Npgsql;
using Xunit;
namespace Orders.IntegrationTests;
public sealed class PostgresFixture : IAsyncLifetime
{
private const string Password = "test";
private readonly IContainer _postgres = new ContainerBuilder()
.WithImage("postgres:16.3-alpine")
.WithEnvironment("POSTGRES_PASSWORD", Password)
.WithEnvironment("POSTGRES_DB", "shop")
.WithPortBinding(5432, true)
.WithResourceMapping(new FileInfo("db/init.sql"), "/docker-entrypoint-initdb.d/")
.WithWaitStrategy(Wait.ForUnixContainer().UntilCommandIsCompleted(
new[] { "pg_isready", "--host", "127.0.0.1", "--dbname", "shop", "--username", "postgres" },
o => o.WithTimeout(TimeSpan.FromMinutes(2))))
.Build();
public string ConnectionString => new NpgsqlConnectionStringBuilder
{
Host = _postgres.Hostname,
Port = _postgres.GetMappedPublicPort(5432),
Username = "postgres",
Password = Password,
Database = "shop",
}.ConnectionString;
public Task InitializeAsync() => _postgres.StartAsync();
public Task DisposeAsync() => _postgres.DisposeAsync().AsTask();
}The test uses it through IClassFixture and has one assertion: there are three orders in the orders table created and populated by db/init.sql.
public sealed class OrdersTests : IClassFixture<PostgresFixture>
{
private readonly PostgresFixture _db;
public OrdersTests(PostgresFixture db) => _db = db;
[Fact]
public async Task Seeded_orders_are_readable()
{
await using var connection = new NpgsqlConnection(_db.ConnectionString);
await connection.OpenAsync();
await using var command = new NpgsqlCommand("SELECT count(*) FROM orders", connection);
var count = (long)(await command.ExecuteScalarAsync())!;
Assert.Equal(3, count);
}
}Several details here are deliberate. I use the bare ContainerBuilder, not the Testcontainers.PostgreSql module, because the thesis concerns code that people already have, not code written for the cloud. The library chooses a random host port, and I build the connection string from Hostname and GetMappedPublicPort(5432), asking the runtime where it exposed the container. In January 2021, with a bare docker run, I put localhost:55432 in the configuration, and that was a good choice for one daemon on one machine. Here, a fixed address would stop working as soon as the daemon moved off my machine.
The wait has an explicit two-minute limit because the default wait strategy timeout in 3.9.0 is one hour, and on a worker billed by the minute, an hour spent hanging on a broken container has a concrete price. I query pg_isready through 127.0.0.1, not through a Unix socket, because while running scripts from docker-entrypoint-initdb.d, the Postgres image starts a temporary server that listens only on the socket. Without --host, the readiness check can answer “ready” while init.sql is still loading. I did not have that parameter in the readiness loop from February 2021, and today I would add it.
Desktop vs CI
I work on a Linux laptop with Docker Engine, so a run against the local daemon is my baseline. To see what the library connected to, I run the tests with the console logger at normal verbosity. At the default dotnet test level, it does not show what Testcontainers writes to standard output, which contains the most important line in this post:
$ dotnet test -c Release --logger "console;verbosity=normal"
[testcontainers.org 00:00:00.04] Connected to Docker:
Host: unix:///var/run/docker.sock
...
[testcontainers.org 00:00:00.36] Copy tar archive to container: Content length: 259 byte(s), Docker container: 5489a53300e7
[testcontainers.org 00:00:00.38] Start Docker container 5489a53300e7
[testcontainers.org 00:00:00.49] Execute "pg_isready --host 127.0.0.1 --dbname shop --username postgres" at Docker container 5489a53300e7
[testcontainers.org 00:00:01.56] Execute "pg_isready --host 127.0.0.1 --dbname shop --username postgres" at Docker container 5489a53300e7
[testcontainers.org 00:00:01.64] Docker container 5489a53300e7 ready
Passed Orders.IntegrationTests.OrdersTests.Seeded_orders_are_readable [124 ms]The Connected to Docker block also prints the server version, operating system, and daemon memory, but one line matters in this run: Host: unix:///var/run/docker.sock. This is my daemon, my disk, and my RAM. The log also shows that init.sql reaches the container as a tar archive before the container starts, and that pg_isready answered “not yet” the first time.
Then I select “Run with Testcontainers Cloud” in the Desktop menu. The Desktop documentation promises exactly what I am looking for: containers in the cloud on demand, without consuming local resources and without a local daemon. For people on ARM Macs, there is an additional argument that does not apply to me on Linux: the worker always uses the Linux x86 architecture, usually the same one as production. The free plan for an individual provides 300 cloud minutes per month, without Turbo mode, which is enough for the blog examples repository.
The first version of PostgresFixture would not have survived this switch, and that is the most interesting part of this post. Instead of copying init.sql, it mounted the file from disk:
.WithBindMount(Path.GetFullPath("db/init.sql"), "/docker-entrypoint-initdb.d/init.sql")It works locally because the daemon reads the file from the same disk that contains the project. It cannot work on Cloud because the file is on my disk, while the container starts on a worker that cannot see that disk. The Cloud documentation says this in one sentence: mounting files from the local file system is not implemented, and files must be copied into the container instead. In .NET, that copy is WithResourceMapping. The library packs the file into an archive and sends it through the Docker API between container creation and container start, so it works with every daemon to which it has a connection.
I verified the mechanism without the cloud because I wanted to know exactly what fails and why. I started a second Docker daemon from the docker:26.1-dind image, exposed it over TCP with a forwarded port range, and put its address in tc.host, while leaving DOCKER_HOST set to the local socket. The library connected to the address from tc.host, not DOCKER_HOST, and showed Host: tcp://127.0.0.1:23750/ in the Connected to Docker block. It pulled the database image again on the second daemon, even though the local daemon had it in its cache. The copy variant passed. The mount variant failed while creating the container, before Postgres even started:
Docker.DotNet.DockerApiException : Docker API responded with status code=BadRequest,
response={"message":"invalid mount config for type \"bind\": bind source path does not exist:
.../bin/Release/net8.0/db/init.sql"}That daemon is not Testcontainers Cloud, and I do not pretend that it reproduces Cloud in detail; the message on a worker may be different. It does reproduce the two things that matter here: a daemon that does not share my file system cannot see my init.sql, and tc.host takes precedence over DOCKER_HOST.
The final observation from this experiment is the most important one. When nothing was listening at the address in tc.host, the library returned to the local socket without a warning in the log, and the test passed. Green result, different runtime. I will return to this in the CI discussion.
With the corrected line, the same test passes both on the local daemon and on the second one, and copying is exactly the path the Cloud documentation recommends instead of mounting. On Cloud, the Host: line shows the tcp:// address that the agent wrote to tc.host, while the server fields describe the worker, not my laptop. The example log in the Cloud documentation includes the suffix testcontainerscloud in the server version field, so I can distinguish the cloud from the local daemon using the log alone, without looking at the Desktop menu. The test and assertion did not change at all. This is what I mean by “without rewriting”: not zero lines in the diff, but zero changes to what the test verifies. The line I changed was debt anyway because it assumed that the daemon and the project shared a disk.
On CI, the contrast with 2021 is greater than on a laptop. The November YAML depended on the job seeing the daemon socket on the agent host: that is why the job ran on the host instead of in container:, and why I warned against mapDockerSocket: false. The Cloud agent does not need a socket, and the product page promises that it requires no system privileges on CI. That matters where Docker is absent from the agent, for example on a self-hosted agent running in a container. Until now, there were two routes there, and both required a conversation with an administrator: privileged Docker-in-Docker or the host socket mounted into the agent container. A Microsoft-hosted agent includes Docker, so the benefit is different: containers stop sharing two cores, 7 GB of RAM, and 14 GB of SSD storage with the build and test processes, while the cloud session gives them a separate 8 GB.
My integration job in Azure Pipelines looks like this:
trigger:
- main
pr: none
pool:
vmImage: ubuntu-22.04
variables:
buildConfiguration: Release
steps:
- task: UseDotNet@2
displayName: SDK
inputs:
packageType: sdk
version: 8.0.x
- script: bash ci/tcc-agent.sh
displayName: Testcontainers Cloud agent
timeoutInMinutes: 2
env:
TC_CLOUD_TOKEN: $(TC_CLOUD_TOKEN)
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
command: test
projects: "**/*IntegrationTests.csproj"
arguments: '--configuration $(buildConfiguration) --logger trx --logger "console;verbosity=normal" --results-directory $(Agent.TempDirectory)'The service account token is a secret pipeline variable and goes only to the agent step. Azure does not expose secrets to scripts as environment variables by itself; they must be mapped in env:, which is right because dotnet test does not need that token. The Cloud documentation presents this step as Bash@3 with a one-liner containing $(curl ...). It works because Azure leaves a $( ) expression alone when it is not a variable name, but since August 2022 I have not kept syntax in Azure YAML that is read first by variable substitution and only then by the shell. The installation therefore lives in ci/tcc-agent.sh. The script downloads the agent, starts it in the background with the --terminate flag, and blocks the step with the built-in wait command until the agent connects to Testcontainers Cloud. The documentation describes --terminate specifically for CI: workers end with the agent instead of waiting for the idle timeout.
The agent step has a two-minute limit and no condition based on the presence of the token. If the token is missing or the agent cannot connect to the service, I want a red step, not a green run against the agent’s Docker. From the laptop experiment, I know that the library would silently fall back to the local socket, and a hosted agent has a local socket. Such a run would pass, and nobody would notice where the containers ran.
The console logger at normal verbosity leaves the Connected to Docker block in the run log, and that block replaces two steps that had been in my integration job since 2021. A “What the agent has” step with docker version would now describe the agent’s local daemon, not the runtime on which the test containers start. A “Pull database image” step would download the image to the agent’s Docker, which the test does not use because the pull happens on the worker. I removed both from this job. Image pull time is now visible in the library log at the Docker image postgres:16.3-alpine created line, which appears only when the image was pulled.
The e2e job from August 2022 remains unchanged. It starts the application with docker compose, while Playwright, now at 1.45.1, goes through the UI against Docker on the agent. Cloud applies to containers created by the library, not to Docker commands written in YAML. I also do not start the Cloud agent in jobs without Testcontainers tests because the documentation warns directly that a worker would then be allocated and wait idle.
When I do not enable it
The most common answer is: for a small suite. One Postgres instance and a few dozen integration tests fit comfortably on a hosted agent, where Docker has always been available. Cloud adds a secret, a network hop, and another provider to such a job in exchange for resources that were not scarce. The same applies locally: if the laptop starts the suite without strain, the cloud solves a problem I do not have.
The second reason is the shape of the cost. On CI, the documentation counts worker minutes from allocation until the worker shuts down after its idle period, hence --terminate in my script. On Desktop, the charge is per person using it, regardless of the number of minutes, within the fair use policy. I do not give prices or minutes from any account here. I describe the billing model from the documentation I am reading in July, and if Docker changes it, this paragraph will age, not the rest of the post.
The third reason is the trigger. In September 2022, I divided tests into three buckets, and container integration went to main and the nightly run, not every pull request. Cloud does not deserve a separate trigger “because cloud.” It changes where the containers run in the bucket to which they already belong. The pull request gate stays as it was, and the Cloud agent does not enter it.
The fourth reason is the hope that Turbo will create parallelism for me. Turbo mode gives a separate cloud environment to each test process, not each test. In .NET, that process is the VSTest test host, started separately for each assembly, so my single test project gets one worker even if xUnit runs classes in parallel within that process. Turbo is not part of the free plan, and --max-concurrency defaults to 4, meaning up to four workers at once, each counting minutes separately. The documentation itself recommends lowering this value if it does not change the run time.
The fifth reason is the most important because it concerns what Cloud does not change. A suite that flakes locally will flake in the cloud. In October 2021, I called a shared database killer number three, and that statement does not depend on which machine runs Postgres. Tests in one class share one container here, so they share data, and each must create its own rows or clean up only after itself. Cloud moves the database to another machine. It does not separate the data or release me from isolation.
The sixth reason concerns projects that are not mine. Running a client’s suite in Cloud means that its images, test data, and registry credentials pass through infrastructure that the client did not agree to use. That is the client’s decision, not mine, and it is not made silently. Because .NET has no per-project switch, Desktop is not in cloud mode on a machine where I run a client’s suite.
July pitfalls
None of the following is a reason to reject Cloud. Each is a reason to read the log before calling a red run a flake.
Network to the worker. Every test query to the database now travels through the agent tunnel instead of the loopback interface on my machine. The Cloud documentation says connection latency should be below 20 ms, and Desktop shows the latest measurement in the connection submenu. A test that executes a few queries will not notice. A suite that sends hundreds of small queries in every test will pay that latency for each one. There is also a new place where the connection can break, and a dropped tunnel halfway through a test looks like a database error in the log. On a red Cloud run, I read the Connected to Docker block before I read the SQL.
Cold pull. The image is pulled on the worker, not on the agent and not through my connection. The image cache on a self-hosted agent that saved minutes for years no longer helps. The documentation advises running the tests twice in succession on the first attempt to experience a warm environment. On a laptop, that is fair advice. On CI, each run starts a new agent, and with --terminate the worker ends with it, so I do not count on a warm cache between runs. The first pull is part of the cost of every run, exactly as it was on a hosted agent in 2021.
Private registry. The worker can see registries exposed to the internet, such as Docker Hub. It cannot see a company registry that is not exposed to the internet. The documentation addresses this with a registry proxy configured separately on every machine running the agent, for pulls only, with an allowlist of image names. It also describes two things that must be disclosed to the registry owner before enabling it. Registry credentials are visible to the library and Testcontainers Cloud: they pass through the service but are not stored there. Images from the private registry remain in the worker virtual machine until it is removed after roughly 30 minutes of inactivity. This requires a conversation, not a switch.
Network in the other direction. A container on the worker cannot see my laptop. A service running locally next to the tests, a service behind a VPN, and a host name resolved by company DNS do not exist from the worker’s side because the container host is the worker, not my machine. The first time, this looks like a DNS flake. It is not a flake: a name that the worker cannot resolve will never resolve, which is actually good news because the failure is repeatable. Testcontainers for .NET 3.9.0 has a mechanism for exposing host ports to containers, TestcontainersSettings.ExposeHostPortsAsync, but I have not tested it with Cloud and do not recommend it blindly.
Memory. Every cloud session gets 8 GB of RAM, and according to the documentation this can be neither increased nor limited. That is more than the entire hosted agent has, but it is the ceiling for one session, not an infinite cloud, and without Turbo mode all containers from one agent go to one environment. A suite that starts a database, a queue, and several services at once must fit.
Secret. The service account token is a secret like any other and has one place in the pipeline: the agent step. Pull requests from forks do not receive it and do not need it because the agent job runs on main. For GitHub Actions, the documentation provides a ready-made action with if: env.TC_CLOUD_TOKEN != '' and separately warns about jobs triggered by Dependabot, which cannot access regular repository secrets. That condition means that without the secret, the agent step is skipped and the tests silently run against the runner’s Docker. For pull requests from forks, that is reasonable behavior as long as someone reads the Host: line.
Desktop telemetry. Desktop tracks test sessions, including those on a local runtime, and collects them in a dashboard that combines desktop and CI data from everyone in the organization. Among other things, the dashboard answers which images the team uses most often. I have no problem with that in the blog examples repository. On a machine that runs a client’s suite, this is a question for the client before installation, of the same kind as the question of where its containers should run.
Finally, one subject returns in every July conversation but does not belong in this post. A language model is not the topic here. GPT-4o has been available since May 13, Claude 3.5 Sonnet since June 21, and neither changes where the Docker daemon runs or who pays for worker minutes.
Summary
In July 2024, Testcontainers Cloud is a tool for heavy dependencies and a constrained runner, not a new way to write tests. The blog’s default remains: code defines the container, isolation comes first, the trigger comes next, and I choose the runtime last and outside the code.
Three sentences that I want to be able to read a year from now: Testcontainers 3.9.0 chooses the runtime outside the test code; tc.host from the home directory takes precedence over DOCKER_HOST, and when it does not respond, the library falls back to the local socket without a warning. Preparing the synthetic suite for the cloud cost one line, replacing WithBindMount with WithResourceMapping, because the worker cannot see my disk, while the test and assertion remained untouched. On CI, Cloud removes the need for a socket and privileges from the agent, but costs worker minutes, a secret, and a network hop, so it goes only into the job on main that actually runs Testcontainers tests.
What does not follow from this? It does not mean that I am removing Docker from agents: the e2e job still starts the application through Compose, and the Compose file did not become obsolete. It does not mean that the cloud removes the need for isolation, because a shared database breaks tests on a worker just as it does on a laptop. It does not mean that Cloud replaces the open source libraries, because without them there would be nothing to offload. And it does not imply any GA date.
The cost accounting from December 2022 also remains unchanged. I use an in-process host for API layer logic, a container for what SQL knows better than my idea of SQL, a contract at the boundary with someone else’s system, and a browser for what a person sees. Cloud does not add a fifth item to that list. It moves the second one to another machine when mine stops being enough.
I define the condition under which Cloud becomes the default runtime for my integration job, rather than an option, as follows. Two runs of the same PostgresFixture class, local and cloud, produce the same result without any branch in the code. The job fails instead of silently falling back to the agent’s Docker when the agent does not connect. Worker minutes per run are visible and cheaper than the time lost on an undersized agent. And if the suite belongs to a client, the client knows where its containers run and has agreed to it. The examples repository code is already ready for the first two conditions today: the fixture has no runtime-dependent branch, and the agent step has no condition based on the token. I cannot verify the other two with a synthetic project, so until then I stay with the 2021 default wherever the agent can handle the images, and enable Cloud where it cannot.