Record-and-playback testing: why it failed and what changed

Ask an experienced QA engineer about record-and-playback testing and you'll probably get a wince. For twenty years the pitch was the same: click through your app once and get an automated test for free. And for twenty years the result was the same: tests that broke the week after they were recorded.

Recording a flow is still the fastest way to describe it. What has changed is what happens after the recording. Here's why the old tools failed, and what a modern approach does differently.

Why classic record-and-playback broke

Locators tied to page structure

Early recorders saved whatever selector was easiest to generate: an absolute XPath such as /html/body/div[2]/div/form/button[1], or a generated ID such as #ext-gen-1042. Both describe where an element is, not what it is. Any layout change, even one users never notice, broke the test.

Recording noise as steps

Recorders captured everything: every mouse move, every focus change, every accidental click. The resulting scripts were long, hard to read and full of steps that only mattered once.

Timing captured as sleeps

To make playback work, recorders inserted pauses copied from the recording session. A slow day meant failures, and a fast day meant wasted minutes. See flaky end-to-end tests for why fixed waits are the number one cause of flakiness.

No meaningful assertions

A recording knows what you clicked, but not what you were checking. Many recorded tests clicked through a whole checkout without ever confirming the order succeeded. They passed even when the feature was broken, which is worse than failing.

Opaque, uneditable output

When a recorded test broke, the fix was usually to record it again. Over time teams spent more effort re-recording than they'd saved.

What's different now

1. Accessibility-first locators

Modern browser automation, led by Playwright and Testing Library, locates elements by role, accessible name and label: "the button named Checkout", not "the third button in the second div". These describe what users see, so they survive redesigns and only break when the experience really changes. We go deeper in Playwright locators that survive redesigns.

2. Waiting for state instead of time

Current frameworks wait automatically until an element is visible, stable and enabled before acting, and their assertions retry until they pass. Recorded steps no longer need a sleep after every click.

3. Understanding intent, not just events

Vision-language models can watch a recording and describe what the person was doing, such as "choose 3 December in the check-out calendar", rather than replaying 14 raw clicks. They can also spot the outcome at the end of the video and turn it into an assertion, which fixes the biggest weakness of the old tools.

4. Proving the test before saving it

A draft test, whether written by a recorder, an AI or a person, should run once against the real site before it's trusted. If a step can't find its element, the fix happens immediately, while the recording is still fresh, not three weeks later in CI.

5. Keeping AI out of the replay

There's an important line between using AI to write a test and using AI to run one. AI agents that decide what to click at run time are impressive in demos, but they're slower, cost money on every run and, worst of all for a regression test, they may not do the same thing twice. A regression test is valuable because it's repeatable.

The approach that works: use AI once to understand the recording, validate the result in a real browser, then replay it deterministically. Bring AI back only to explain a failure.

When codeless testing makes sense, and when it doesn't

Good fits:

• Regression checks of stable, high-value user flows: sign-up, login, checkout, booking, core create, read, update and delete screens.
• Teams where the people who know the flows best aren't the people writing test code.
• Covering a backlog of manual regression test cases quickly.

Better done in code:

• Complex test data setup through APIs, or assertions on backend state.
• Heavily parameterised tests, such as one flow across 50 data combinations.
• Component and unit-level tests, which belong next to the code.

The two approaches work well side by side: engineers write the tricky tests in code, and everyone else turns recordings into regression coverage for the rest.

The short version

Record-and-playback failed because it recorded events and played them back blindly. Recording was never the problem. Modern tools record intent, locate elements the way people do, prove each test in a real browser, and replay it without guesswork.

That's the approach behind ReplayQA: upload a screen recording, get a validated test, and replay it on every release with no AI on each run.