Why Your Test Selectors Are Lying to You

Part 1 of "Testing the Way Users Actually Experience the Web" — a tutorial series on @afixt/usecase-runner.

Here is a Playwright test. It passes. It has passed every day for a year.

test('user can submit the form', async ({ page }) => {
  await page.goto('https://app.example.com/contact');
  await page.locator('#name').fill('Jane Doe');
  await page.locator('.btn-primary').click();
  await expect(page.locator('.toast-success')).toBeVisible();
});

And here is the markup it is testing:

<div class="form-row">
  <span class="label">Name</span>
  <input id="name" type="text" />
</div>
<div class="btn btn-primary" onclick="submitForm()">Send</div>
<div class="toast-success" style="display:none">Thanks!</div>

Nothing in that markup is usable by someone navigating with a keyboard or a screen reader. The input has no label — the <span> is visually next to it, but nothing associates the two, so a screen reader announces "edit text" and nothing else. The "button" is a <div>: it has no role, no accessible name, it isn't in the tab order, and pressing Enter on it does nothing because you can't reach it to press Enter. The success message appears visually and is announced to no one.

The test passes because it never asked any of those questions. #name finds the input by its id. .btn-primary finds the div by its class. click() dispatches a pointer event at its coordinates. Every one of those selectors reaches around the interface a real user would use and pokes the DOM directly.

That's the lie. A CSS selector is a statement about how the page is built. It tells you nothing about how the page is experienced.

@afixt/usecase-runner is built on one non-negotiable rule: every element is targeted through the accessibility tree, via Playwright's getByRole(), getByLabel() and getByText(). There is no XPath, and no CSS selector syntax in the authoring surface. If an element isn't exposed with the right role and accessible name, the test cannot find it, and it fails.

Two narrow escape hatches exist — id "..." and data-* "..." — for the case where a test has to reach the element a finding is about. They resolve through CSS, and that is the point: a step targeted that way proves the element is in the DOM and nothing about whether assistive technology can perceive it. Use them knowing what you have given up.

That failure is not a limitation of the tool. It is the finding.

Let's write the same test. Use cases live in YAML files with a .uc.yaml extension, and steps use a small domain-specific syntax:

# contact-form-success.uc.yaml
id: contact-form-success
title: 'Submit the contact form'
type: positive
description: 'A user can fill in their name and send the form.'

preconditions:
  - 'The contact page is reachable without authentication'

start_location: 'https://app.example.com/contact'
expected_result: 'The form is sent and a confirmation is shown'

data:
  name: 'Jane Doe'

steps:
  - locate: field "Name"
  - focus: field "Name"
  - enter: field "Name" value "{{ name }}"
  - locate: button "Send"
  - focus: button "Send"
  - activate: button "Send"
  - verify: live_region "Thanks!"

Run it directly, no code generation required:

npx usecase-runner run contact-form-success.uc.yaml

Against the markup above, the very first step fails:

Step 1: Locate field "Name" — FAIL
  Timed out waiting for getByLabel('Name') to be visible
  accessibility_notes:
    - No form control with accessible label 'Name' found
    - Possible causes: missing <label>, missing aria-label/aria-labelledby

The run keeps going (continue_on_failure is on by default, so one run gives you every finding, not just the first) and you get three more:

• locate: button "Send" — there is no element with role="button" named "Send". The <div> is invisible to the accessibility tree.
• focus: button "Send" — can't focus what can't be found.
• verify: live_region "Thanks!" — no role="status", role="alert", or aria-live region contains that text.

Four failures, and each one maps directly to a WCAG success criterion and a concrete code change. Nobody had to schedule an audit.

When a usecase-runner test fails, the reflex to "fix the selector" has nowhere to go — there is no selector to loosen. The only way forward is to fix the page:

<div class="form-row">
  <label for="name">Name</label>
  <input id="name" type="text" />
</div>
<button type="submit" class="btn btn-primary">Send</button>
<div role="status" class="toast-success" hidden>Thanks!</div>

Run it again. Seven passes. The test file didn't change at all, because the test was written against what a user experiences, and that is now correct.

This is the whole philosophy of the project in one paragraph: a test that targets the accessibility tree is simultaneously a functional test and an accessibility test, and it can't be one without the other. You don't add accessibility assertions to your flows. The flow is the assertion.

There are escape hatches — id "..." and data-* "..." targets exist for edge cases like a canvas surface or a third-party embed you don't control. They are deliberately ugly to write and stand out in review. What does not exist, and will not, is arbitrary CSS or XPath. If you find yourself reaching for an escape hatch on a button, a link, or a form field, the page has a bug.

npm install @afixt/usecase-runner @playwright/test
npx playwright install
npx usecase-runner init

init writes a sample sample-login.uc.yaml and a config file into the current directory. Point start_location at something you own, run it, and read the failures as a list of things a real user can't do.

Next in the series: A Test Your QA Team Can Actually Read — the full anatomy of a use case file, and why it is shaped like a manual test plan on purpose.