Flaky end-to-end tests: 10 causes and how to fix each one
Why end-to-end tests pass and fail without code changes, and concrete fixes for timing, locators, test data, dates, environments and animations.
A flaky test passes and fails against the same code. One flaky test is an annoyance. Ten of them and the team stops believing red builds, and that's when real regressions slip through.
The good news is that flakiness has a small number of root causes, and most have well-known fixes. Here are the ten we see most often in browser end-to-end tests, roughly in order of how common they are.
How to tell a flaky test from a real failure
Before fixing anything, confirm it really is flaky:
- Re-run the failing test alone, on the same build. If it passes without any change, it's a candidate.
- Run it repeatedly, for example 20 times. In Playwright:
npx playwright test checkout.spec.ts --repeat-each=20. - Compare the evidence. Look at the screenshot, trace and video of a failing run next to a passing one. The difference almost always points straight at one of the causes below.
Track a flakiness score per test (failures that later passed on retry, divided by total runs) so you fix the worst offenders first rather than whichever failed last.
1. Fixed waits instead of waiting for state
await page.click('text=Save');
await page.waitForTimeout(3000); // hope the save has finished
Fixed sleeps are wrong in both directions: too long on a fast day, too short on a slow one. Wait for the thing you actually need:
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Changes saved')).toBeVisible();
Playwright's web-first assertions (toBeVisible, toHaveText, toHaveURL) retry until they pass or time out, so they adapt to the speed of each run.
2. Brittle locators
Selectors such as #root > div:nth-child(2) > button break whenever the layout shifts, and some only match intermittently because element order changes as data loads. Locate elements by role, label and text instead. See Playwright locators that survive redesigns.
3. Locators that match more than one element
"Click the Delete button" is ambiguous on a page with ten rows. Depending on which row renders first, the test clicks a different one. Scope the locator to its container:
await page.getByRole('row', { name: /Invoice 1042/ })
.getByRole('button', { name: 'Delete' })
.click();
Playwright's strict mode helps here: an action on a locator that matches several elements throws instead of quietly picking the first.
4. Shared or leftover test data
Two tests edit the same account in parallel. Or yesterday's run created "Project A" and today's run fails because the name is already taken. Fixes:
- Give each test its own data, such as unique names with a timestamp or random suffix.
- Create what you need in the test (or through an API before it) instead of relying on existing records.
- Clean up afterwards, or use an environment that resets nightly.
5. Animations and transitions
A test clicks a button while a modal is still sliding in, and the click lands on the overlay. Wait for the end state (the dialog is visible and its button enabled) rather than the start. Playwright's actionability checks wait for elements to be visible, stable and enabled before clicking, which removes most of these. Disabling CSS animations in your test environment removes the rest.
6. Network timing and third-party scripts
Analytics, chat widgets and ad scripts load in their own time and can cover buttons or slow pages down. Options:
- Block third-party domains you don't need in tests.
- Wait for the specific response your step depends on:
const saved = page.waitForResponse(r => r.url().includes('/api/orders') && r.ok());
await page.getByRole('button', { name: 'Place order' }).click();
await saved;
7. Dates, times and time zones
Tests that pick "tomorrow" in a date picker fail on the last day of the month. Tests that check "Today" fail just after midnight UTC. Freeze the clock or compute dates explicitly:
await page.clock.setFixedTime(new Date('2026-11-14T10:00:00'));
Also run tests in a fixed time zone and locale (Playwright's timezoneId and locale options) so "11/04" means the same thing on every machine. We cover calendar widgets in depth in How to test date pickers and calendar popups.
8. Environment drift
Staging gets a deploy halfway through your test run. A feature flag is on for some users and off for others. Your test runner's browser version differs from a colleague's. Pin what you can: run suites against a known build, set feature flags explicitly for test accounts, and use the same browser version in CI and locally.
9. Order dependence
Test B only passes if test A ran first and left the user logged in. Run your suite in random order, or run single tests in isolation, to find these. Each test should set up its own state: log in, navigate and create data.
10. Resource limits
Tests that pass alone and fail in a big parallel run are often starved of CPU or memory. Reduce parallel workers, give the runner more resources, or split the suite. Timeouts that only trigger under load are the giveaway.
Should you just add retries?
Retries are a useful diagnostic: a test that fails and then passes on retry is flaky by definition, and recording that is how you build a flakiness score. But retries hide the problem rather than fix it, and they double the cost of real failures. Use one retry, track which tests needed it, and fix those tests.
A short checklist
- No fixed waits; every wait is for a visible state or a network response
- Locators use role, label, text or test ID, scoped to their container
- Each test creates its own data and doesn't depend on other tests
- Clock, time zone and locale are fixed
- Third-party scripts are blocked or accounted for
- Flakiness is tracked per test, and the worst offenders are fixed first
ReplayQA applies most of this checklist automatically: it waits for elements rather than time, locates them by role and label, records a trace and video of every run, and flags tests whose results flip without a code change. Try it on one of your flows.