“AI will fix the brittle test”
It is 15 February. A month ago, in the post on agents, I set self-healing locators aside for a separate piece, because repairing a selector is a different category of tools from a loop with a goal. Since then, every conversation about a red E2E run has produced the same sentence: you no longer need to write stable locators, because a model or an “AI” product will swap the selector when the page changes. It sounds like a fix for flake that I have been treating with environment isolation since October 2021 and November 2021, not with magic in the runner.
My thesis for February 2024 is narrow, and I will defend it throughout this post. Self-healing is an old SaaS and OSS product, resold as AI, not a new button in Playwright. Healing a locator does not heal flake from the environment, from data, or from a race. It does not remove the review I left in July with hypotheses from the log.
Here is the state of the workshop today, so a year from now it will be clear what I mean. The runner is Playwright 1.41.1 from 19 January. In tests I still write getByRole and getByTestId, as I did in the first project and in the POM refactor. There is no official self-healing GA in Playwright 1.41. Auto-waiting and assertion retries are not locator repair by a model. Copilot Chat has been in stable VS Code since 29 December, and I do not use it for healing: chat sees the open file, not the DOM from the overnight run.
What will not be here. There will be no claim that Playwright shipped a healer in core. There will be no tutorial of Healenium, Testim, or Mabl as a February novelty. There will be no agent connected to CI. There will be one synthetic demo in my own repository of blog examples: a button id change versus a failing API timeout.
What is old
The wave everyone now discusses as a 2024 novelty is older than ChatGPT.
Testim advertised Smart Locators and “AI locators” at the Tricentis acquisition on 9 February 2022. That is two years before this post. Mabl collects several identifiers for an element and adapts the step when one of them disappears; the company has existed since 2017, and auto-heal is in the product blog, not in a February 2024 changelog. Healenium is a proxy and wrapper on Selenium: on NoSuchElement it swaps the selector for a variant that “healed” a previous run. The EPAM hub has had a 3.x line since October 2020 (v3.0.1 = 9 October 2020), and the GitHub repository since 7 October 2021. This is Selenium-shaped OSS, not a Playwright feature.
I am not pinning a Maven or NuGet patch for today, because I do not have a URL that would confirm it on 15 February. The dates I do have are enough: none of these three products “came out in 2024”. What came out is hype, in which an old mechanism gets the new word “AI” and lands in the same conversation as January’s agents.
The category difference is simple. An agent with a loop gets a goal and chooses its own tools. A healer gets a failing locator and swaps it for another that happens to hit the element this time. Both can produce a green result without knowing why it was red. Both deserve review like a diff, not trust like an assertion.
What healing treats, and what it does not
Healing treats one class of failure: the id, class, or path in the DOM changed, and the element is still on the page and still does the same thing. That is the same problem I have been solving since August 2021 with getByRole, getByTestId, and no static driver, not with swapping a selector on the fly.
It does not treat anything that is not a selector.
It does not treat a network timeout when the login API does not return in fifteen seconds. It does not treat a shared fixture when two workers write to the same user. It does not treat a race when the button is in the DOM but still disabled, because hydration has not finished before the click. It does not treat the font and viewport on the CI agent that my laptop does not have. These are the same buckets I split into a gate and a nightly in September 2022, and the same ones that in July the model was only allowed to name, and I had to reproduce.
A pixel from November 2022 does not help here either: a screenshot says the look changed, a healer says the selector changed. Neither says whether the product changed.
I will show this on a synthetic login test from the examples repository, not from a client project. Two variants of the same red, one healer, two different verdicts.
Variant A: on Friday someone changed the button id from login-submit to submit-login. The text stays “Log in”, the role stays button. A test written against #login-submit fails with NoSuchElement or a locator timeout.
TimeoutError: locator.click: Timeout 15000ms exceeded.
waiting for locator('#login-submit')A healer that collected several element attributes from the previous green run finds the button by text or by adjacency to the password field, swaps the selector, and the overnight run is green. This is the only case in which the mechanism does what it promises. The cost: the report has no trace that the product changed an identifier. If the id change was an accident, nobody will hear about it. If it was intentional and breaks the automation contract, the contract has just been rewritten in silence.
Variant B: the id is the same. The /auth API on the nightly environment does not respond, a consent modal or a spinner stays over the form, the button is in the DOM, but the click does not finish login. The test fails on a timeout of the assertion “the dashboard is visible”, not on NoSuchElement.
TimeoutError: expect(locator.getByRole('heading', { name: 'Dashboard' })).toBeVisible()
waiting for getByRole('heading', { name: 'Dashboard' })The healer has nothing to swap here, or, worse, it does: it finds another heading on the login page, treats it as a “healed” assertion target, and masks the API timeout. Variant B is environment flake. Treating it as locator healing is a category error, exactly the kind January’s loop made when it invented a data-testid that did not exist instead of checking a flag in configuration.
Healing “fixes” variant A and masks variant B. In nightly, without a review of the swap, I cannot tell them apart.
Playwright without official self-healing
Playwright 1.41.1 waits for the element, retries assertions, and records a trace. That is not self-healing. When a selector fails, I open the trace, read whether the element was in the DOM, whether it was invisible, whether it was covered, and only then change the test or the environment. Auto-waiting shortens the time I used to spend on sleep, and nothing else. It does not swap a locator behind my back.
I did not find an official Healer agent or self-healing GA in the v1.41.1 tag. I will not pin a feature that is not in the release notes. If someone sells “AI locators for Playwright” as a plugin or SaaS, that is a separate product, with a separate review and a separate place in the pyramid, not a feature of the runner I have in package.json.
When a selector fails, I have three questions, and none of them is “turn on heal”. Whether this test checks anything at all, or only waits for an element. Whether the change is in the product or in the environment. Whether I can reproduce the failure before I propose a fix. This is the same set I left with the agent loop in January and with log hypotheses in July. Healing answers a different question: “is there another selector with which this step will pass”. That question is useful during a POM refactor. It is harmful as an overnight automatic.
When someone installs a healer anyway
I will not succeed in banning this tool in someone else’s team. I can say under what condition it does not erase the signal.
Every selector swap must land in the report like a diff. Old locator, new locator, a snapshot of the element the new one hit, and the test that passed because of it. I review that the way I review ghost text since June 2022 and the way I review a model hypothesis since July: first I reproduce, then I agree or strike it out. A silent heal in nightly is worse than a red test, because a red test leaves a trace. A silent heal leaves green and a product regression that nobody will hear about until the “healed” selector hits the wrong button.
The contrast with July is mandatory here, because the two topics blur in conversation. In flake triage, the model gave three hypotheses from the log. I reproduced. No hypothesis changed the test without my commit. Self-healing works the other way: it changes the test or the locator at runtime, and the log, if it exists at all, describes success, not the cause. That is not triage. That is an automatic fix without an owner.
In the 2023 recap, I wrote how I tell that a tool has entered the team, not the hype. Two of those things are a named task and a person who owns the outcome. A healer has the first if we limit it to “suggest another locator during a POM refactor”. It does not have the second if it rewrites the suite overnight on its own.
For the record, one sentence about the date: on 15 February Google announced Gemini 1.5 Pro in limited preview. I do not have it as a daily model, and it is not the stack of this post.
Summary
Self-healing is this month’s topic because the word “AI” stuck to products I have known for years. In February 2024 I am not buying it as a substitute for isolation and review.
Three sentences I want to be able to read a year from now. Testim, Mabl, and Healenium are older than February’s hype; Playwright 1.41.1 has no official self-healing GA, and auto-waiting is not locator repair. A synthetic login test shows two reds: a button id change that a healer “fixes”, and an API timeout that healing masks or leaves untouched, in both cases without a verdict on whether the product changed. Review of the swap as a diff remains, exactly like review of a log hypothesis in July and review of an agent loop in January.
What does not follow. It does not follow that Smart Locators are worthless after a mass id change in a restyle. It does not follow that I will never put a “find the same button by role and name” heuristic in a POM helper. It also does not follow that this will not change if Microsoft ships a healer into Playwright core: it is not there today, so I have nothing to write about except the date.
The condition by which I will know the topic has matured is this: a selector swap is acceptable when it leaves an auditable diff, does not hide an environment timeout as a locator success, and does not take away from a human the question of whether the red test caught the product or flake. Until then the suite has no autonomous healer. It has isolation, a trace, and me.