Don't Raise the Timeout: Testing Flutter Web with Playwright
On a Flutter web app we're building, I left an AI coding agent running the browser tests in the background. The runs dragged on with no progress, so I looked under the hood. When a step missed its time, the agent had been raising the timeout and running it again, chasing a green result on a step that had already failed. It acted as if something was still loading and waited longer, when the step had already missed. Hours dragged on like that. I got lazy and stopped watching. You can't delegate accountability.
A longer timeout hides a failure
Our test stack runs locally. Sign-in is a mock, mail goes to a local catcher, and nothing leaves the machine. On this stack, a click, fill or visibility check that isn't done in 2 seconds isn't going to arrive. On this stack, a navigation that isn't done in 5 seconds isn't going to arrive. So when a step misses that bound, the step has failed. Raising the timeout just turns a bug you can reproduce into a slow suite that still has the bug.
We now write that rule down where the agents read it: a test that sets a longer timeout has opted out of the guard, and that is a defect unless the doc names the wait and the measurement that justified it.
Watch for untimed actions too. A bare click() inherits the whole scenario timeout, so a target that never settles can stall a run for minutes. Give every action its own short timeout, and never catch a timeout and carry on.
Fix the step, then rerun that one test
When a step misses its time, the agent (or I) follows the same list:
- Stop. Don't let the rest of the suite burn time on the same break.
- Name the scenario, the step and the control, and read what the screen actually showed. A timeout with no screen is not a diagnosis.
- Decide which it is: the test looked for the wrong node, or the product didn't do the thing.
- Fix that cause.
- Run that same scenario again. It passes inside its bound, or you go round again.
No force: true clicks to skip a node that isn't a stable target. No deleting data or restarting services to buy a green run, unless that state is the bug and you say so. If a product limit really does block a local test run, change the setting and write down why.
On Flutter web, most of those failures turned out to be the test looking for the wrong node. Here is why, and what fixed it.
1. Turn semantics on, then find controls by identifier
Flutter web exposes a semantics tree in the DOM for assistive technology. That tree is what Playwright can see. Give every control a test presses or reads a stable Semantics.identifier. On web, Flutter sets it as a flt-semantics-identifier attribute on the DOM element, and Playwright can use it as the test id.
// playwright.config.js
use: { testIdAttribute: 'flt-semantics-identifier' }
// in the test
await page.getByTestId('orders.detail.save').click({ timeout: 2000 });
An identifier is a name for the control, like settings.profile.save. It is not the label text, a sentence on the screen or a coordinate. Painted text is often not a node Playwright can find, so getByText on a Flutter label is a trap. If a control has no identifier, add one in the widget. Don't work around the gap from the test.
This applies only to the Flutter app. Any plain HTML pages around it, such as a landing page, are real HTML, so test them with their real roles and labels.
Don't wait for flt-semantics-placeholder either. The engine can enable semantics on the first frame and remove that node, so a wait for it burns the whole timeout. Wait briefly for flt-semantics and click the placeholder only if it is still there.
2. One control, one semantics node
Material buttons already publish a button. Wrapping one in Semantics(button: true) gives you two buttons for one control. Worse, setting an identifier forces its own semantics node, so in our app an identifier-only wrapper around a disabled button read as enabled in the browser while it looked grey on screen. Put the identifier, the button flag, the enabled state, the label and the tap handler on one node, and exclude the inner button's own semantics.
Semantics(
identifier: 'orders.detail.save',
button: true,
enabled: canSave,
label: 'Save',
onTap: canSave ? save : null,
child: ExcludeSemantics(
child: FilledButton(onPressed: canSave ? save : null, child: const Text('Save')),
),
)
An InkWell is the opposite case. It is not a button until a Semantics(button: true) on it says so.
3. Name fields by their label, not their hint
A painted caption above a TextFormField is not linked to the input, and a plain hintText can end up as the field's accessible name. Screen readers and tests then can't find a textbox named by its label. Exclude the painted label, put the label on the field, and make the hint visual only.
Column(children: [
ExcludeSemantics(child: Text(label)),
Semantics(
label: label,
child: TextFormField(
decoration: InputDecoration(
hint: ExcludeSemantics(child: Text(hintText)),
),
),
),
])
Now the test asks for exactly what a screen reader user would: page.getByRole('textbox', { name: label, exact: true }). A widget test can hold the line without a browser by checking there is one text field with that label, and no node named by the hint.
4. Type like a person
locator.fill() sets the DOM value and fires generic events. Flutter listens for real key events, so a fill can look fine and then vanish when the form rebuilds on submit. Click to focus, select all, type key by key, and check the value before and after the field loses focus.
async function enterText(page, field, value, { checkAfterBlur = true } = {}) {
await field.click();
await field.press('ControlOrMeta+A');
await field.pressSequentially(value, { delay: 20 });
await expect(field).toHaveValue(value);
if (!checkAfterBlur) return;
await field.press('Tab');
await expect(field).toHaveValue(value);
}
One exception. A one-time code field that submits itself when it loses focus must skip the blur check, or the Tab fires the submit before the test is ready for it.
5. Remember that the same URL doesn't rebuild the screen
Navigating to the hash route the app is already on doesn't rebuild the Flutter screen, so a test can read stale data. Navigate away and back when you need a fresh read.
6. Give every scrolling region a name
Layouts with panels go wrong when scrollers nest. A table scrolling inside a panel scrolling inside a page is unusable for people and confusing for tests. Aim for exactly one scroller per vertical run, and give a table that pins its search box and headers a panel whose scrolling it controls. In tests, scope every "this screen doesn't scroll" assertion to a keyed region. Otherwise a sidebar that later gains its own scroller breaks checks about unrelated screens. Sweep several window sizes too, because overflow often shows up at only one height.
7. Run against an isolated stack
Running browser tests against your everyday dev environment invites port clashes, shared data and rate limits from repeated sign-ins. We run a separate stack on its own ports with a mock identity provider and a local mail catcher, so sign-in codes can be read back by the test. Other rules we found worth keeping:
- One owner at a time. A file lock stops a second process from resetting the stack mid-run.
- A reset command that wipes and reseeds only the test database. That clears rate limits honestly, instead of switching them off in product code.
- Separate browser contexts for each user role, signed in through the real sign-in flow.
- Traces and screenshots kept private and out of the repo, because they can hold codes, tokens and session data. Review them before you share any.