Accessibility Use Case Definition Language (UCDL) and Runner

This section is normative.

A conforming processor that emits reports MUST support the JSON form defined here. HTML and DOCX outputs are OPTIONAL but, where present, MUST preserve the same fields.

{
  "id": "<use case id>",
  "title": "<use case title>",
  "type": "positive | negative",
  "score": "pass | pass_with_conditions | inapplicable | fail | error",
  "tester": "<string>",
  "timestamp": "<ISO 8601>",
  "duration_ms": <integer>,
  "steps": [
    {
      "step_number": <integer>,
      "step_text": "<original step source>",
      "status": "pass | fail | skip",
      "outcome": "passed | failed | inapplicable | cannot_determine", // optional; see §11
      "duration_ms": <integer>,
      "error": "<message>",                  // optional, when status=fail
      "failure_reason": "profile_forbidden", // optional; see below
      "screenshot_path": "<path>",           // optional
      "accessibility_notes": [ "<string>" ], // optional
      "audit_result": <AuditResult>,         // optional, only for audit steps
      "contrast_result": <ContrastResult>,   // optional, only for contrast steps
      "lang_check_result": <LangCheckResult>,// optional, only for lang_check steps
      "read_image_result": <ReadImageResult>,// optional, only for read_image steps
      "sr_says_result": <SrSaysResult>       // optional, only for sr_says steps
    }
  ],
  "before_results": [ <StepResult> ],         // optional; present iff a `before` block ran
  "iterations": [                            // optional; present iff scope.for_each is set
    {
      "index": <integer>,
      "element_descriptor": "<string>",
      "status": "pass | fail",
      "step_results": [ <StepResult> ]
    }
  ],
  "iteration_summary": {                     // optional; present iff scope.for_each is set
    "total": <integer>,
    "passed": <integer>,
    "failed": <integer>,
    "inapplicable": <integer>,               // 1 when total=0, else 0
    "concurrency_used": <integer>            // effective parallelism (1 = sequential)
  },
  "environment": {                           // optional, RECOMMENDED; see below
    "processor": { "name": "@afixt/usecase-runner", "version": "<semver>" },
    "language_version": "<version of this specification>",
    "playwright_version": "<semver>",        // optional
    "browser": { "type": "chromium | firefox | webkit", "version": "<string>" },
    "os": { "platform": "<string>", "release": "<string>", "arch": "<string>" },
    "node_version": "<string>",
    "viewport": { "width": <integer>, "height": <integer> },              // optional
    "sr_driver": { "name": "virtual | voiceover | nvda", "version": "<semver>" }, // present iff a screen-reader session started
    "engine": { "name": "<package>", "version": "<semver>" },              // present iff an audit step ran
    "blocked_hosts": [ "<pattern>" ],                                       // present iff blocking was active (§12.1)
    "blocked_requests": <integer>                                           // present iff blocked_hosts is
  },
  "profile": "no-pointer"                    // optional; see below
}

before_results is OPTIONAL and MUST be emitted when — and only when — the use case declared a before block (§4.6) and that block ran. Its entries use the same step-result shape as steps.

It MUST NOT be merged into steps: §4.6 rule 3 requires setup results to be kept separate from the results of the behaviour under test.

A processor MUST emit it because it is the only diagnostic an error score carries. When a setup step fails the main steps never run, so steps is empty; without before_results a consumer can see that the case did not reach the behaviour it exists to judge, but not which setup step failed or why.

Because the before block is where a case authenticates, emitting its results can place credentials in a report. See §16.1.

failure_reason is OPTIONAL and MUST be emitted only when a step failed for a structural reason rather than because an assertion did not hold. A processor MUST NOT emit it on a passing step. The value MUST be one of:

The field exists so a consumer can distinguish "the profile forbade this step", "the runner lacked what the step needed" and "the engine could not load the snapshot" from "the interaction was attempted and did not hold" without string-matching error. All of them produce status: "fail", but they are different findings with different fixes — and the second and third are not findings about the page at all. A dashboard that counts failures SHOULD keep dependency_unavailable and snapshot_load_failed out of its accessibility totals: in neither case did anything about the page get examined.

A generated test has no failure_reason field. The engine and @afixt/test-utils are loaded when the generated file loads, so a missing one fails the file with the module loader's own error before any step runs. Where a generated test loads a dependency at run time — franc for lang_check, the virtual screen reader for sr_says — the error it throws MUST begin with the token [dependency_unavailable], so a CI log can be searched for the same reason the JSON report carries. Future structural reasons MUST be added to this enum rather than encoded in error.

profile is OPTIONAL and, when present, names the interaction profile the run was executed under. It MUST be recorded from the profile that was actually enforced, not from what was requested, so a report can never claim a profile the steps did not run under. Absence means the run was unrestricted.

environment is OPTIONAL in this version and RECOMMENDED; a later major will require it. It records what produced the result, as structured data, so a run can be reproduced and two runs that disagree can be diagnosed from their reports: the processor and its version, the specification version it implements, the automation engine version, the browser type and build, the operating system, the Node version, the viewport the run started at, the screen-reader driver a session was actually started with, and the audit engine when an audit step ran.

Every value MUST be read from the installed packages and the running process, never from a literal that can drift between releases. A value that cannot be read MUST be omitted rather than guessed. sr_driver MUST name the driver the session was started with, not the one requested — auto that fell back to virtual records virtual — for the same reason SrSaysResult.driver does (§9A.4). The HTML report SHOULD show the same record; the screen-reader line is where a virtual-SR run would otherwise be taken for a real one.

blocked_hosts and blocked_requests record that a run aborted requests under the configuration's blocked_hosts (§12.1): the patterns, and how many requests matched them. A processor that emits environment for such a run MUST include both, and MUST omit both when blocked_hosts is empty. blocked_requests is 0, not omitted, when blocking was active and matched nothing — that is the finding that the list blocked nothing, where an omitted count would read as an unknown one. A page with its tags blocked is not quite the page its users get, which is why the record says so. A generated test has no report of its own; the reference implementation records the patterns as a blocked_hosts test annotation.

tester remains as the one-line summary and for the manual-rubric mapping.

The tester field SHOULD include the processor name, the Playwright version (or "Playwright"), and the browser type. The reference implementation uses the form usecase-runner / Playwright / chromium.

The step_text MUST be the original, post-interpolation source string — the same text a human tester would have read. Implementations SHOULD NOT substitute a re-stringified form.

When emitted, the HTML report MUST be a self-contained, accessible document containing, at minimum:

• the use case id, title, type, score (visually distinguished), tester, timestamp, and total duration;
• a list of preconditions;
• a step results table with columns: step number, step text, status, duration, notes;
• when a before block ran, its results in a separate table from the step results table, for the same reason before_results is a separate key in the JSON report (§10.1);
• for audit steps with issues, an inline expandable disclosure containing the issue list (priority, check title, WCAG criteria, element snippet, remediation);
• for a step carrying a delegating-verb result (§9A), a summary of what that verb measured. A disclosure is not required — these payloads are counts rather than lists, and read well in the notes column.

For an sr_says step the summary MUST identify the driver. §10.1 already requires the driver in the JSON report, and the reasoning in §9A.4 applies at least as strongly here: the HTML report is the human-facing deliverable, so it is where a virtual-SR result would otherwise be taken for a real-SR one.

The HTML MUST itself be accessible: scoped table headers (<th scope="col">), text-and-icon status indicators, and prefers-color-scheme support are REQUIRED. Visual-only color coding is non-conforming.

Every string taken from the document or from the page under test — step text, accessible names, error messages, scanner element snippets and remediation text, spoken phrases — MUST be HTML-escaped when interpolated into the report. The report renders content the page under test controls, and an unescaped snippet is script injection into the tester's browser.

DOCX is RESERVED. The reference implementation does not currently emit DOCX. A future version of this specification may define a DOCX form matching AFixt's existing UseCaseTestData_-_EXAMPLE.docx template.

Note the difference described in §8.2:

• In codegen mode, step 1 is the implicit goto. User-authored steps begin at step 2.
• In direct execution mode, step 1 is the first user-authored step. The goto is performed before iteration and is not numbered in the report.

Implementations SHOULD normalize this in human-facing reports if mixing the two modes' outputs.