Playwright component testing in 2023 - experimental ct-react 1.37

A feature that has been waiting since May 2022

The Playwright 1.22.0 release notes from 12 May 2022 had two items that interested me. The first was the toHaveScreenshot assertion, which I explored six months later when I compared visual regression tools. The second was component testing in preview, and I left that one for “someday.” Someday arrived now, fifteen months later, and that is the entire origin of this post.

The reason is mundane. 1.37.0 came out on 10 August, and I was updating the pin in a project I have been working on since last year. The release notes for this version contain three things: the blob reporter together with the merge-reports tool for merging reports from shards, support for Debian 12, and several fixes to UI Mode. None of them concerns components. But since I was already in package.json and the documentation, I opened the section I had been skipping since May 2022 and spent an evening with it.

Before I write anything about the mechanism itself, there is one point without which this post would be dishonest. After fifteen months, the package is still called @playwright/experimental-ct-react, and the word experimental is the first part of its name, not a footnote in the documentation. The documentation says directly that the feature is experimental and that the API may change. This is not a post about something finally reaching stable status, because nothing like that has happened. This is a post about an experiment, pinned to 1.37.0, on one synthetic component.

I am also not rewriting two of my older posts here. The first Playwright project describes an empty directory, the setup wizard, and the first green test on 1.22, and it stays that way. The April decision to move away from Cypress describes why I am in this ecosystem at all, and I am not reopening that discussion. There is only one question here: in August 2023, can I run a single component in a real browser without starting the whole application, and is this something I want in the repository?

I invented the component I use for this check specifically for the post. It is a login form with one email address field and a button in an application that does not exist. There is not a single line here from any project I work on, and no client will be named here.

What exactly I am mounting

The first thing worth understanding before running any command is that component testing means a separate package, a separate configuration file, and a separate run. It is not a switch in the existing playwright.config.ts. The setup wizard has a dedicated flag for it, and I run it at the current version because the wizard only needs to generate the skeleton, while I set the version numbers myself a moment later:

$ npm init playwright@latest -- --ct

The wizard asks for the framework and, after I select React, adds three things: the @playwright/experimental-ct-react package, the playwright-ct.config.ts file, and a playwright/ directory with two files. It writes whichever versions are the latest on the day it runs and adds carets, so then I do what I always do: clean up after the wizard and pin exactly 1.37.0 without a caret:

{
  "scripts": {
    "test": "playwright test",
    "test:ct": "playwright test -c playwright-ct.config.ts"
  },
  "devDependencies": {
    "@playwright/experimental-ct-react": "1.37.0",
    "@playwright/test": "1.37.0",
    "react": "18.2.0",
    "react-dom": "18.2.0"
  }
}

With a package containing the word experimental in its name, omitting the caret is not excessive caution but the minimum. 1.37.0 is the only version on which I checked anything in this post and the only one I can say works for me.

I keep Node on the 18 LTS line. Playwright 1.37 still formally supports 16, but that line reaches end of support on 11 September, less than a month from now, so pinning it in a new configuration would make no sense. Node 20 is already out, and I deliberately do not use it here: I do not want the only experimental part of the setup sitting next to another thing I have not checked in this project on CI.

The playwright/ directory is where the page on which the component will be rendered lives. It contains two files. An index.html with an empty container:

<html lang="en">
  <body>
    <div id="root"></div>
    <script type="module" src="./index.ts"></script>
  </body>
</html>

And an index.ts that, in my initial setup, contains only an import of global styles. This is the one place for things the component needs but does not import itself: a CSS reset, a font, or a theme provider. If the component works only inside a context, I either wrap it in that context in the test or move into the beforeMount hooks, at which point the setup stops being trivial. I record that as the first real cost of this route.

The component itself is as simple as I could make it because this post is about the mechanism, not the form:

import { useState } from 'react'

type LoginFormProps = {
  onSubmit: (email: string) => void
}

export function LoginForm({ onSubmit }: LoginFormProps) {
  const [email, setEmail] = useState('')
  const [error, setError] = useState<string | null>(null)

  return (
    <form
      onSubmit={event => {
        event.preventDefault()
        if (!email.includes('@')) {
          setError('Enter a valid email address')
          return
        }
        setError(null)
        onSubmit(email)
      }}
    >
      <label htmlFor="email">Email address</label>
      <input
        id="email"
        name="email"
        value={email}
        onChange={event => setEmail(event.target.value)}
      />
      {error && <p role="alert">{error}</p>}
      <button type="submit">Log in</button>
    </form>
  )
}

The config is shorter than the one I have for E2E because it has neither a baseURL nor a webServer. There is nothing to start because there is no application:

import { defineConfig, devices } from '@playwright/experimental-ct-react'

export default defineConfig({
  testDir: './src',
  testMatch: '**/*.ct.spec.tsx',
  use: {
    trace: 'on-first-retry',
    ctPort: 3100
  },
  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }]
})

Two things in this file deserve a sentence. I import defineConfig from the experimental-ct-react package, not from @playwright/test, and this is not cosmetic: it is a different runner with a different set of fixtures. ctPort is the port on which the dev server that builds the component will run, and I provide it explicitly so that it does not collide with anything I already have running locally.

There is one test, and it checks exactly one thing: an empty field does not let the form continue:

import { test, expect } from '@playwright/experimental-ct-react'
import { LoginForm } from './LoginForm'

test('empty email shows an error and does not call onSubmit', async ({ mount }) => {
  const submitted: string[] = []

  const component = await mount(
    <LoginForm onSubmit={email => submitted.push(email)} />
  )

  await component.getByRole('button', { name: 'Log in' }).click()

  await expect(component.getByRole('alert')).toHaveText(
    'Enter a valid email address'
  )
  expect(submitted).toEqual([])
})

I run it with npm run test:ct. The first run is noticeably longer because a Vite dev server starts underneath and builds the component. Subsequent runs take seconds on my machine, and that is the number I came here for.

That is the end of the new knowledge required. Everything below mount is regular @playwright/test: getByRole, click, toHaveText, the same timeouts, the same Trace Viewer, and the same retry behavior. mount returns a Locator attached to the root of the rendered component, so component.getByRole searches inside it rather than across the whole document. For someone who already has E2E tests in Playwright, the entry threshold is one new function and one new config.

One thing looks familiar but is not, and it is better to stumble over it now than three weeks from now. The test code runs in Node while the component lives in the browser, so props cross a process boundary. The onSubmit from my test will work, and the submitted array will indeed be filled when the form passes validation. But this is not a regular function call in the same place in memory: props must be transferable, so an object with methods or a closure over state held in Node is not something I can safely pass. This does not hurt with a form containing one field. With a component that receives an HTTP client in its props, it is the first wall I hit.

I will note one more sentence without turning it into the subject: sister packages following the same naming convention exist for Vue and Svelte, so the description above is not specific to React, but React is the only framework on which I checked it.

Where it sits in the pyramid

The most important question about a new tool is never “does it work,” but “what do I stop doing because of it?” The answer here is narrow, and I want to record it precisely.

A component test is faster than E2E because it does not do three things that my E2E tests must do: it does not log in, wait for the application to start, or need data in the database. The entire layer I described when I put Testcontainers and Playwright into one pipeline simply does not exist here. This is not optimizing a run but removing an entire stage from it.

At the same time, it is closer to the interface than a unit test in a simulated DOM. The component renders in a real Chromium browser, so a click on the button is a real click in a real layout, not an event dispatched programmatically in an environment that does not calculate CSS. An element covered by another element cannot be clicked because it genuinely cannot be clicked. For forms where half the bugs sit in the disabled state, focus, and tab order, this is a real difference.

What it does not replace, point by point: it does not replace a contract test because there is no API in my test, while the contract between consumer and provider is the only thing guarding against a field changing its name on the other side. It does not replace a server-side integration test because an in-memory host with a substituted dependency checks rules that do not exist in the component at all. And it does not replace one happy-path E2E test because only that test answers whether the form, endpoint, and post-login redirect meet in one place.

The practical division I intend to follow fits into two sentences. The states of one component, such as field validation, an error message, a disabled button, and a variant with long text, go into CT because checking them there takes seconds and requires nothing around them. Anything that crosses more than one screen or touches the backend stays in E2E, and I am not moving a single case from that set this month.

The price of the word experimental

Now for the part I want recorded in black and white so I do not have to explain my enthusiasm six months from now.

experimental in the package name means that the API may change in the next minor release without a deprecation period. This is not my prediction but what the documentation says about this feature. The consequence is singular and very concrete: the version is pinned exactly, its upgrade goes into a separate commit on a separate branch, and the only thing that commit does is change the number. If CT turns red after the bump, I want to see one change in the diff rather than search for it among twenty others.

The second price is architectural. Vite runs underneath, and this is its dev server, not my production build. In a project whose frontend also runs on Vite, the difference is small. In a project built another way, I have two bad options: configure ctViteConfig to imitate the real build or accept that the component under test is built through a different path than the component in production. The second option is honest, but it creates a class of bugs this test will never see, and I need to know that in advance.

The third price is in CI. Two configs mean two runs, two reports, and two places where something can fail. This is where the feature that arrived with 1.37.0 helps: the blob reporter and npx playwright merge-reports let me merge results from several runs into one HTML report. I picked up this version to merge E2E shards, but merging two different configs uses the same mechanism, and that is the only reason two runs do not currently sound like twice the mess to me.

My exit plan is simple, and it is the condition on which this enters the repository at all. If the maintenance cost of this configuration exceeds the benefit after any version bump, I remove the component test directory, remove the config, remove the package, and keep the E2E tests exactly where they already were. Nothing in this setup is a dependency of anything else, and I want it to stay that way.

What I am not doing after this evening is declaring it a team standard or a recommendation for the entire organization. I have one component, one test, and one pin. That is exactly what I have, and it is the full extent of what I am commenting on.

What I am not pulling into it

The list of things I am deliberately not pushing into component testing is more important here than the list of what worked because it determines whether this directory will still make sense in a year.

I am not pulling in the whole application. The router, layout, session provider, and post-login screen are not a component but an application with a worse shell. The moment mount receives a tree with five providers, I have a slow E2E test without a URL, which is the worst of both worlds.

I am not pulling in login. The entire authentication path, including browser state and the token, stays where it was: in the end-to-end tests.

I am not turning this into a screenshot machine. toHaveScreenshot also works on a component, and I know that, but visual comparison has its own history for me, described in the November review of visual regression tools. Adding a new source of reference images to that history in the same week when I run CT for the first time is a good way to break both things. Component testing and visual regression are two decisions, not one.

I am not putting Docker underneath it. There is no database, queue, or HTTP stub in this config, so there is nothing to containerize. That is actually the greatest advantage of this layer, and I do not intend to consume it by adding a real backend to the component.

I am not rewriting existing tests. I have not moved a single case from E2E to CT, and I deliberately do not start with a migration. New states of new components can be created here. Green tests that work stay where they are.

Summary

After one evening with @playwright/experimental-ct-react version 1.37.0, I have a conclusion that fits into one sentence: this is another rung on a ladder I already have, not a new framework and not something that has reached stable status.

I am breaking this down into three sentences that I want to be able to read a year from now. The mechanism is extremely simple for someone who knows @playwright/test because the new parts are one mount function and one additional configuration file, while everything below that uses the same locators and assertions. The benefit is real and narrow: I check states of a single component in a real browser in seconds, without logging in, seeding data, or waiting for the application. The price is also real: the package has experimental in its name, the dev server is Vite and does not have to match my production build, and CI gains a second run.

What does not follow from this? It does not mean component testing is ready for rollout across the entire organization because the documentation still warns about API changes. It does not mean I need E2E any less because I have not deleted a single test. It also does not mean this is the answer to a slow pipeline because my pipeline is slow due to data and the environment, not component rendering.

In a month, I will know two things I do not know today. First, how many new cases actually land in CT and how many I write in E2E out of habit because all my helpers are there. Second, what happens with the first version bump, because that is the only test that genuinely verifies the word experimental. Until then, the pin stays at 1.37.0, and I postpone the decision about whether this is a permanent rung until I have more than one form and one evening.