All posts

Selenium Grid in Docker - Grid 3 and Grid 4 beta
Selenium Grid in Docker - Grid 3 and Grid 4 beta

Polski

Selenium Grid in Docker - Grid 3 and Grid 4 beta

How I run Selenium Grid 3 in Docker in March 2021, and what Grid 4 beta-1 images already offer.

QA

Why run a local Grid

My UI test suite grew to well over a hundred cases and no longer fit into a coffee break. I run them serially, one after another, in a single Chrome installed on my laptop. It takes more than forty minutes, and during that time the browser keeps jumping around my screen and I cannot work normally on that machine.

My first instinct is to parallelize the run in the runner. I set Parallelizable at the class level and get four Chrome instances at once - on the same desktop, with one user profile, and with one chromedriver in PATH. The result is predictable: the tests start flickering because the windows steal focus from one another, and one session can bring down another.

Grid solves exactly this problem. It is not a tool for a “huge browser farm,” but a way to stop a test from controlling a browser sitting next to it on the same desktop. The test sends commands over HTTP to the hub, the hub selects an available node, and the browser lives in a container, without a window and without taking my focus.

I wrote about how I start a single dependency in Docker for QA, and a month ago I moved that setup to a Compose file. I deliberately left the browser farm for a separate post at the time, and this is that post.

One thing is not covered here: running Grid on a shared server for CI. That is a different topic, with monitoring, restarts, and memory limits. Here, everything happens on my machine and disappears after the run.

Grid 3 or Grid 4 in March 2021

The short answer: Grid 3.

The latest stable server line is still 3.141.59. The version number dates back to 2018, which sometimes raises doubts, but the Docker images continue to be maintained, and that is what keeps them current. The release I use is 3.141.59-20210311 from March 11, just four days ago. The images from that release ship Chrome 89, Firefox 86, and GeckoDriver 0.29.0. The server is old, but the browsers are current.

Selenium 4 is in beta. The first beta, 4.0.0-beta-1, was announced on February 15 together with packages and images. A lot changes in it: the protocol is W3C only, and Grid itself has a new architecture with an event bus. The new Grid is described on the Selenium blog (beta announcement and description of the new Grid) and looks good, but I take the word “beta” literally.

My rule for now is simple. The daily regression run uses Grid 3 with a pinned tag. The Grid 4 beta runs alongside it, on separate ports, and helps me check how much work the migration will require. I do not mix these two worlds in one file or one test project.

The docker-compose file with a hub and node

The versions on which this setup runs for me:

$ docker version --format '{{.Server.Version}}'
20.10.5

$ docker-compose version --short
1.28.5

Engine 20.10.5 came with Docker Desktop 3.2.1 from March 5. I keep Compose at 1.28.5 from February 25. It is the same line I wrote about in February, so profiles and the longer form of depends_on work in the same way.

I put the docker-compose.grid.yml file next to the tests:

version: "3.8"

services:
  selenium-hub:
    image: selenium/hub:3.141.59-20210311
    container_name: selenium-hub
    ports:
      - "4444:4444"

  chrome:
    image: selenium/node-chrome:3.141.59-20210311
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      HUB_HOST: selenium-hub
      HUB_PORT: 4444
      NODE_MAX_INSTANCES: 2
      NODE_MAX_SESSION: 2
      SCREEN_WIDTH: 1920
      SCREEN_HEIGHT: 1080

Here is what matters.

HUB_HOST: selenium-hub is the service name, not localhost. The node is in its own container, and localhost means the node itself to it. Compose creates a network for the project in which services can see one another by name, so the node registers at http://selenium-hub:4444.

I publish only the hub port on the host. The node does not need to be visible from my machine because the hub talks to the node, and my test talks only to the hub.

shm_size: 2gb is not decorative. The default 64 MB in /dev/shm is too little for Chrome, and the browser crashes in the middle of a session. Anyone who prefers the older notation can mount the host memory instead:

services:
  chrome:
    volumes:
      - /dev/shm:/dev/shm

NODE_MAX_INSTANCES specifies how many Chrome instances can run on this node, while NODE_MAX_SESSION specifies how many sessions it can handle in parallel. I keep both numbers equal and do not set them too high because every session is a separate browser and a separate chunk of RAM.

I deliberately do not give the node a container_name. This lets me scale it and have more than one:

$ docker-compose -f docker-compose.grid.yml up -d --scale chrome=3

After a moment, I check what the hub thinks of itself:

$ curl -s http://localhost:4444/wd/hub/status
{"status":0,"value":{"ready":true,"message":"Hub has capacity","build":{"revision":"e82be7d358","time":"2018-11-14T08:25:53","version":"3.141.59"},"os":{...},"java":{...}}}

I trimmed the os and java blocks. What matters is "ready": true and the fact that the whole response has the JSON Wire shape with a top-level "status" field, because that is how Grid 3 answers. When no node has managed to register, the same address returns "ready": false and "message": "No spare hub capacity".

The console listing connected nodes is at http://localhost:4444/grid/console, and it is the first place I look when a test reports that no session is available. I clean up as usual:

$ docker-compose -f docker-compose.grid.yml down

A C# test through RemoteWebDriver

One thing changes in the test project: instead of a local ChromeDriver, I create a RemoteWebDriver with the hub address. I keep the client at a version matching the server:

<PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
<PackageReference Include="Selenium.Support" Version="3.141.0" />
<PackageReference Include="NUnit" Version="3.13.1" />

And the test itself:

using System;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Remote;

[TestFixture]
public class CartTests
{
    private IWebDriver _driver;

    [SetUp]
    public void SetUp()
    {
        var options = new ChromeOptions();
        options.AddArgument("--window-size=1920,1080");

        var gridUrl = Environment.GetEnvironmentVariable("GRID_URL")
                      ?? "http://localhost:4444/wd/hub";

        _driver = new RemoteWebDriver(
            new Uri(gridUrl),
            options.ToCapabilities(),
            TimeSpan.FromMinutes(2));
    }

    [Test]
    public void AddingProductChangesCartCount()
    {
        _driver.Navigate().GoToUrl("http://host.docker.internal:58080/shop");
        _driver.FindElement(By.CssSelector("[data-test=add-to-cart]")).Click();

        var cartCount = _driver.FindElement(By.CssSelector("[data-test=cart-count]"));
        Assert.That(cartCount.Text, Is.EqualTo("1"));
    }

    [TearDown]
    public void TearDown()
    {
        _driver?.Quit();
    }
}

The /wd/hub path is required here because Grid 3 speaks the JSON Wire protocol at that address. I put the address in an environment variable so I can later point the same code to another instance without recompiling it.

The third argument, TimeSpan.FromMinutes(2), is not excessive. The default command timeout in the client is short, while the first request after Grid starts waits for a session to be assigned and for the browser to start in the container. Without increasing this timeout, I got timeouts only on the first test in a run.

The address of the application under test is a separate trap. The browser no longer runs on my machine, so localhost in GoToUrl would point inside the node container. On Docker Desktop, I use host.docker.internal, while on Linux I simply connect the application to the same Compose network and refer to it by its service name. This is probably the most common reason for the question, “Why do all tests see a blank page after switching to Grid?”

I configure parallelism on the NUnit side, in the AssemblyInfo.cs file:

[assembly: Parallelizable(ParallelScope.Fixtures)]
[assembly: LevelOfParallelism(3)]

The number in LevelOfParallelism has to fit what is actually running on Grid. Three nodes with two instances each give six slots, so three sessions fit with room to spare. If I request more sessions than Grid has slots, the excess requests will wait in the hub queue and will most likely end in a timeout.

What you can already touch in Grid 4 beta

A very brief preview, so I know what is coming. The beta images have their own February 15 release, so I pin the tag just as I do with Grid 3:

version: "3.8"

services:
  selenium-hub:
    image: selenium/hub:4.0.0-beta-1-20210215
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4445:4444"

  chrome:
    image: selenium/node-chrome:4.0.0-beta-1-20210215
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      SE_EVENT_BUS_HOST: selenium-hub
      SE_EVENT_BUS_PUBLISH_PORT: 4442
      SE_EVENT_BUS_SUBSCRIBE_PORT: 4443

From the outside, the setup looks familiar: there is still a hub and still a node. The two ports next to 4444 inside the container are new. They belong to the event bus that the node and hub use to exchange registration and session state information. Instead of HUB_HOST, the node receives SE_EVENT_BUS_HOST. I publish the beta router on the host as 4445:4444 so it does not take 4444, which Grid 3 already uses.

The client is different too. For the beta, I use Selenium.WebDriver version 4.0.0-beta1 in a separate test project, and I do not try to share code with the production project. The beta endpoint is http://localhost:4445/, and the protocol is W3C only.

And that is where I stop experimenting for now. I checked that my suite can run after minor adjustments, saved a list of differences, and returned to Grid 3. I will revisit the topic when there is a stable release.

Typical failures

Four things cost me the most time.

Missing shared memory. A test crashes halfway through with a message about a closed session or a crashed tab, and docker-compose logs chrome shows that Chrome crashed. The answer is almost always a missing shm_size: 2gb. This is the first thing I check and the first thing I ask about when someone reports flickering tests on Grid.

Mixing the client and server. Selenium.WebDriver 3.141.0 pointed at Grid 4 beta, or a beta client pointed at Grid 3, produces errors that look like a network problem but are actually a protocol mismatch. I keep the client and server on the same line, without exceptions.

The latest tag. Selenium images are updated together with the browsers, so latest can move me from Chrome 89 to the next version in the middle of the week and break selectors or window behavior. It is the same rule as with the database: I pin the full tag with the date and update it deliberately.

localhost in the wrong place. Once in HUB_HOST on the node side, once in the application address on the test side. Each time, the same misunderstanding is to blame: localhost now means three different machines, depending on who is asking.

Grid does not fix tests. It only turns forty minutes of waiting into a dozen or so minutes and removes the browser from my desktop. If the suite was unstable when run serially, it will be unstable faster after parallelization - and whether I am testing the right things at all is still a question about the entire test process, not infrastructure.