Playwright can wait until a locator matches. That is not the same as waiting until the app did the thing you believe you are testing. If the condition already holds on a loading screen, on the default tab, or on the URL you were already on, the click can be a no-op and the suite still goes green.
This post is about that signal, not about how many scenarios you have.
Why this piece exists
Teams score e2e the way they score CI jobs: green or red. Green is supposed to mean “the user path works.” Red is supposed to mean “the change broke something or the environment is sick.”
In practice some of that green says nothing about the click. The assertion was true before the action. Playwright’s auto-wait does not help then: there is nothing to wait for, the condition already holds.
This is a cousin of two other costs I wrote about separately. The time spent understanding a change before you even pick scope is in the hidden tax on every pull request. A pipeline that cries wolf is in noisy CI. Here is a third pole: a test that stays quiet because it never asked a hard question.
I saw it on a demo SPA plus API stack (a RealWorld-style clone): after review the suite was broad, and a few assertions would still have passed if the tab never switched or the list was still on a spinner.
Three assertions that were already true
The pattern repeats in SPAs with tabs, Suspense, and forms that stay on the same URL after save.
| Symptom | What the test “thinks” | What was already true | Better hook |
|---|---|---|---|
toHaveURL on the settings page |
save performed a redirect | you were already there; a same-path redirect is not a navigation | wait for the profile PUT or for the button label to change |
| article card “is on the personal feed” | the right tab was clicked | the same card is visible on the global feed | assert feed=personal in the URL, then the card |
toHaveCount(0) after delete |
the title is gone from the list | during the spinner there is no card at all | wait until empty state or any preview appears, then assert count zero |
None of those tests are “bad because Playwright.” Playwright did what it was asked: wait until the locator is true. It was asked for a condition that can be true too early.
A short picture of the third row. After deleting an article the app returns home. The feed loads behind Suspense. The DOM does not have data-test="article-preview" yet. expect(preview).toHaveCount(0) passes immediately. A later visit to the old slug with “could not load article” may still save the scenario. The list assertion alone does not prove the delete.
A fourth variant: you waited on the wrong request
Another classic: waitForResponse whose predicate looks at a URL substring.
An API path .../follow and a bundler chunk whose name contains follow are the same hit for url.includes('follow'). The test “saw a response,” navigated away, and aborted the real POST. The backend log shows a canceled request or a 500; the UI may already have flipped the button optimistically.
Auto-wait is not lying here either. The matcher is. Wait for the HTTP method plus the API path (POST .../follow), not any URL that contains the word.
What this means in review
Before you add another scenario, ask of every assertion: could this pass if the click did nothing?
If yes, hooks that usually work:
- State that did not exist before the action: a query param (
feed=personal,favorited=,tag=), a button label that changes only after success. - The request that actually mutates: profile
PUT, followPOST, unfavoriteDELETE. Not a same-path redirect and not a webpack chunk. - A list that must appear first: the empty message or any preview, and only then “this title is absent.”
A shorter adjacent rule from the same lab, because it produces a different kind of false proof: if the test is not about login, do not prove the session through the login form. Seed the user via API, inject the token, and leave Playwright the clicks. The other way around too: do not follow or comment through the API and then “check the UI.” That e2e measures a read, not a path.
Models are good at locators and toBeVisible. They are worse at asking whether the condition already held at the start. That is not an argument against agents in QA. It is an argument for putting that one question on the test-review checklist, next to “is this even e2e, or an API hit.”
A pipeline that tests the wrong contract
There is another green job that lies for a different reason. An e2e suite pointed at frontend main will not see a contract fix that still sits on an open PR. You get a red settings spec because a blank password still goes out on PUT, or the inverse: you merge e2e before the product change and test main stays red until the other service lands.
That is not a locator flake. It is merge order. In a split stack (UI repo, API repo, e2e repo) land the contract first, then the test that stands on it. Otherwise a review that says “the test is wrong” burns the same attention budget I described in noisy CI.
Closing
A good e2e run should say: this user path reacted to the action. When the assertion is already true on the spinner, the suite adds confidence instead of proof - the mirror image of a pipeline that adds doubt instead of a verdict.
You do not need another framework for that. You need one question in review: would this pass if the click were a no-op. If yes, fix the hook before you add another happy path.