Preston Lamb

Introducing a11y-oracle: Testing That Goes Beyond Static Analysis

Published on Mar 05, 2026

tldr;

If you've worked on a project that takes accessibility seriously, you've probably run into a frustrating pattern. You install axe-core, wire it into your test suite, and within a few runs you're staring at a list of results labeled "Needs Review." Not passed. Not failed. Just... incomplete. And every single time your CI runs, there they are, cluttering your results, requiring someone to manually check whether that color contrast issue is real or a false positive.

I've been down this road more than once, and it always bugged me. These aren't theoretical edge cases — they're real WCAG requirements around color contrast, focus indicators, keyboard traps, and target sizes that axe-core simply can't verify by looking at the DOM alone. So I built a11y-oracle to fix it.

At its core, a11y-oracle is a set of npm packages that use the Chrome DevTools Protocol (CDP) to test accessibility in ways that static DOM analysis can't. It does three main things:

Speech assertions — Reads the browser's accessibility tree and produces standardized speech output (like what a screen reader would announce), so you can assert exactly what assistive technologies will communicate to your users.
Keyboard and focus testing — Sends real keyboard events through CDP, then validates focus indicators meet WCAG 2.4.12, checks tab order, and detects keyboard traps.
axe-core incomplete resolution — Takes those "Needs Review" results from axe-core and actually resolves them using screenshots, keyboard interaction, and CDP inspection.

It ships as 9 packages under the @a11y-oracle scope, but for most people you'll only need one or two. There are drop-in plugins for both Playwright and Cypress.

The Problem with Static Accessibility Testing

Let me be a little more specific about the problem. Tools like axe-core do an incredible job of catching accessibility issues through DOM inspection. Missing alt text, incorrect ARIA attributes, heading hierarchy problems — axe catches all of these reliably. But there's a whole category of WCAG requirements that need more than the DOM to verify.

Take color contrast, for example. axe-core can compute contrast ratios for simple cases where the text color and background color are straightforward CSS values. But what about text on a gradient background? Or text overlapping an image? axe-core marks these as "incomplete" because it literally can't determine the answer without seeing the rendered pixels.

The same goes for focus indicators. WCAG 2.4.7 (and the stricter 2.4.12 in WCAG 2.2) require that focus indicators are visible and have sufficient contrast. But you can't check that without actually focusing the element and comparing what changed on screen. axe-core can see that an element has an outline style, but it can't tell you whether that outline is actually visible against the background.

And then there are interactive checks — does a skip link become visible when focused? Does content that appears on hover stay visible when you move the mouse to it? Can you dismiss it with Escape? These all require actual interaction with the page.

The Approach: Use the Browser Like a User Would

The key insight behind a11y-oracle is that the Chrome DevTools Protocol gives us everything we need to test these things programmatically. CDP lets us:

Read the full accessibility tree (the same data screen readers consume)
Send native keyboard events (not synthetic JavaScript events — real hardware-level key presses)
Take screenshots and analyze pixels
Read computed CSS values from any element
Execute JavaScript in the page context

So instead of looking at the DOM and guessing, a11y-oracle interacts with the page the way a real user would, and then verifies the result.

Speech Assertions: Test What Your Users Hear

The part of a11y-oracle I'm most excited about is the speech engine. It reads Chrome's accessibility tree and produces a standardized string in the format:

[Computed Name], [Role], [State/Properties]

So for a button like , you'd get Products, button, collapsed. For a navigation landmark like

, you'd get Main, navigation landmark.

You might be wondering: why not just test with a real screen reader? The issue is that NVDA, JAWS, and VoiceOver all have their own verbosity rules that change between OS updates. If you tie your assertions to the exact phrasing of a specific screen reader, your tests will break every time the screen reader updates. a11y-oracle avoids that by producing a stable, predictable output based on the W3C specifications.

Here's what it looks like in a Playwright test:

import { test, expect } from '@a11y-oracle/playwright-plugin';

test('dropdown menu keyboard navigation', async ({ page, a11y }) => {
  await page.goto('/dropdown-nav.html');

  const speech = await a11y.press('Tab');
  expect(speech).toContain('Home');
  expect(speech).toContain('menu item');

  const products = await a11y.press('ArrowRight');
  expect(products).toBe('Products, menu item, collapsed');

  const clothing = await a11y.press('Enter');
  expect(clothing).toContain('Clothing');
});

And the Cypress equivalent:

describe('Dropdown Navigation', () => {
  beforeEach(() => {
    cy.visit('/dropdown-nav.html');
    cy.initA11yOracle();
  });

  afterEach(() => {
    cy.disposeA11yOracle();
  });

  it('Tab announces first menu item', () => {
    cy.a11yPress('Tab').should('contain', 'Home');
  });

  it('Enter opens submenu', () => {
    cy.a11yPress('Tab');
    cy.a11yPress('ArrowRight');
    cy.a11yPress('Enter').should('contain', 'Clothing');
  });
});

I find this approach helpful because you're testing the actual user experience. Instead of checking whether an element has the right ARIA attributes (which axe-core already handles well), you're verifying that those attributes produce the correct announcement. It's a higher-level assertion that catches a broader range of problems.

Unified State: Speech + Focus + Indicator in One Call

Beyond speech, a11y-oracle can return a unified A11yState snapshot that combines speech output with focus information and focus indicator analysis. This is useful when you want to verify multiple WCAG requirements in a single interaction:

test('Tab produces unified accessibility state', async ({ page, a11y }) => {
  await page.goto('/my-page.html');

  const state = await a11y.pressKey('Tab');

  // What would a screen reader announce?
  expect(state.speech).toContain('Submit');

  // What element is focused?
  expect(state.focusedElement?.tag).toBe('BUTTON');

  // Does the focus indicator meet WCAG 2.4.12 AA?
  expect(state.focusIndicator.isVisible).toBe(true);
  expect(state.focusIndicator.meetsWCAG_AA).toBe(true);
});

The pressKey method sends a real keyboard event via CDP, waits for focus to settle, and then collects all three dimensions in parallel. You're testing speech, focus management, and visual indicators all at once.

Resolving axe-core's "Needs Review" Items

This is probably where a11y-oracle will provide the most immediate value for teams that already have axe-core integrated. The @a11y-oracle/axe-bridge package takes axe-core's results and resolves the incomplete findings by actually performing the checks that axe-core couldn't.

Here's the basic usage:

import { resolveAllIncomplete } from '@a11y-oracle/axe-bridge';

const axeResults = await axe.run(document);
const resolved = await resolveAllIncomplete(cdpSession, axeResults);

// resolved.incomplete will have fewer (or zero) entries
// resolved.violations and resolved.passes will have the promoted findings

resolveAllIncomplete pipes the results through 10 specialized resolvers in sequence. Each one targets a specific axe-core rule:

Rule	WCAG	What the Resolver Does
`color-contrast`	1.4.3	Takes a screenshot and analyzes actual pixel colors
`focus-indicator`	2.4.7	Focuses the element, screenshots before/after, pixel-diffs
`target-size`	2.5.8	Measures bounding boxes and spacing between targets
`link-in-text-block`	1.4.1	Checks for underlines, borders, or font weight differences
`skip-link`	2.4.1	Focuses the skip link and verifies it becomes visible
`aria-hidden-focus`	4.1.2	Tabs through the page to see if hidden elements receive focus
`content-on-hover`	1.4.13	Hovers, checks persistence, tests Escape dismissal
`scrollable-region-focusable`	2.1.1	Checks scroll height and focusable children
`identical-links-same-purpose`	2.4.4	Normalizes and compares link destinations
`frame-tested`	N/A	Injects axe-core into cross-origin iframes via CDP

With Playwright, it looks like this:

import { test, expect } from '@playwright/test';
import { resolveAllIncomplete } from '@a11y-oracle/axe-bridge';

test('no unresolved accessibility issues', async ({ page }) => {
  await page.goto('/my-page.html');
  const cdp = await page.context().newCDPSession(page);

  const axeResults = await axe.run(document);
  const resolved = await resolveAllIncomplete(cdp, axeResults);

  expect(resolved.violations).toHaveLength(0);
  expect(resolved.incomplete).toHaveLength(0);

  await cdp.detach();
});

You can also configure individual resolvers or skip specific ones if they aren't relevant to your use case:

const resolved = await resolveAllIncomplete(cdpSession, axeResults, {
  wcagLevel: 'wcag22aa', // automatically derives contrast thresholds
  skipRules: ['frame-tested'],
});

Where This Helps

I think there are a few scenarios where a11y-oracle adds the most value:

Teams already using axe-core — If your CI pipeline has a growing list of "incomplete" results that nobody has time to manually review, the axe-bridge resolves most of them automatically. This gets you to a clean pass/fail signal instead of a perpetual "needs review" backlog.

Testing keyboard navigation — If you've ever tried to write E2E tests that verify keyboard accessibility, you know how tedious it is. a11y-oracle lets you send real keyboard events and assert on what a screen reader would announce, which is both more reliable and more meaningful than checking DOM attributes.

WCAG 2.2 compliance — WCAG 2.2 introduced stricter requirements around focus indicators (2.4.12) and target sizes (2.5.8). These are nearly impossible to test with DOM-only tools. a11y-oracle handles both through pixel analysis and bounding box measurement.

Catching regressions — Speech assertions are a great regression safety net. If someone changes an aria-label or removes an ARIA attribute, the speech output changes, and your test fails with a clear, readable message about what changed.

Limitations

I want to be upfront about a few things:

Chromium only — Because a11y-oracle relies on CDP, it only works with Chromium-based browsers (Chrome, Edge, Electron). It won't work with Firefox or WebKit. For most E2E testing scenarios this is fine, since most teams test against Chrome, but it's worth knowing.

Not a replacement for manual testing — Automated accessibility testing catches a lot, but it can't catch everything. Reading comprehension, meaningful alt text, logical content flow — these still need human judgment. a11y-oracle automates the mechanical checks so your team can focus their manual testing time on the things that actually require a human.

Not a screen reader — The speech output is based on the W3C accessibility tree specification, not any specific screen reader's behavior. If you need to verify what NVDA or VoiceOver specifically will say, you still need to test with those tools.

Getting Started

Installation depends on your test runner:

# Playwright
npm install @a11y-oracle/playwright-plugin

# Cypress
npm install @a11y-oracle/cypress-plugin

# axe-bridge standalone
npm install @a11y-oracle/axe-bridge

The Playwright plugin provides a test fixture that you import in place of Playwright's default. The Cypress plugin registers custom commands that you import in your support file. Both handle CDP session management for you.

The full documentation, including the complete speech output format, state mapping tables, and API reference for all 9 packages, is available in the GitHub repository.

Conclusion

Accessibility testing has come a long way with tools like axe-core, but there's always been a gap between what static analysis can check and what WCAG actually requires. a11y-oracle bridges that gap by using the Chrome DevTools Protocol to interact with pages the way real users do — pressing keys, checking focus indicators, analyzing rendered pixels, and reading the accessibility tree that assistive technologies consume.

If you're spending time manually reviewing axe-core's "Needs Review" list, or if you want to add speech and keyboard assertions to your E2E tests, give it a try. I'd love to hear how it works for your team.

Introducing a11y-oracle: Testing That Goes Beyond Static Analysis

tldr;

What is a11y-oracle?

The Problem with Static Accessibility Testing

The Approach: Use the Browser Like a User Would

Speech Assertions: Test What Your Users Hear