Home / Benchmark

Fifteen of our 28 design specs disagreed with the stylesheet

Every style here ships a DESIGN.md: the file you drop into a project so an agent knows the palette, the type stack and the radius values before it writes any interface code. It only works if the file is true. So we wrote a checker that reads the stylesheet and the spec and compares them token by token — names and values, not prose.

The first run disagreed with 15 of 28 specs. Not a matter of taste: 14 tokens were missing outright, and 8 carried a value that the stylesheet had already overridden somewhere further down. An agent following those files would have shipped the wrong header colour on eight styles, and nothing in the build would have complained.

Three things the check caught

1. The cascade was invisible to the generator

Themes are not always defined in one place. Six of them get a second body.theme-* block near the end of the stylesheet, where the header colour was darkened because the original pairing could not be read — glassmorphism had white text on a mint gradient, 1.16:1. The generator looked up only the first block, so it wrote the old value into the spec. Both values look reasonable in isolation. Only a comparison catches it.

2. Three quarters of the rules were the same sentence

A spec earns its place through the rules that only make sense for that style. Counting them: the 28 specs hold 176 Do / Don't items, and 36 of them are specific — the rest appear verbatim in more than half of the files. The 6 styles that were written with their own guidance account for every one of those 36; the other 22 carry a shared boilerplate set about not hard-coding hex values. Useful advice, but it is the same advice everywhere, so it cannot be the reason an agent gets one style right and another wrong.

3. Two tokens point at a variable that does not exist

--mono-font: var(--font-roboto-mono) appears in two themes, and --font-roboto-mono is declared nowhere in the stylesheet. The declaration is silently dropped, the code blocks fall back to the browser's default monospace — close enough to Roboto Mono that nobody noticed. This one is left open on the table below: fixing it changes how two live pages render, so it belongs in its own change rather than in a spec regeneration.

Every style, before and after

Coverage counts tokens whose name and value match what the browser applies. Rules is specific items over total Do / Don't items in that file.

How it is measured

The checker parses the stylesheet by regular expression — no CSS parser, no headless browser, no new dependencies — and merges every body.theme-{id} block in source order, so the last declaration wins, as it does in the browser. It then reads every row of the form | <token> | <value> | out of the spec and compares the two sets in both directions: tokens the stylesheet has and the spec lacks, tokens the spec claims that do not exist, and tokens where both name the same property but disagree on the value.

Two derived numbers matter more than the raw counts. Coverage is matched tokens over applicable tokens — 100% means an agent has everything it needs to reproduce the style. Specific rules counts Do / Don't items that appear in fewer than half of the specs; a rule repeated 28 times constrains nothing, because every style already has it.

Every number on this page comes from spec-check.json; the pre-fix snapshot is spec-check.baseline.json. Both are regenerated, never edited by hand. To reproduce:

node scripts/bench-spec-check.mjs          # summary table
node scripts/bench-spec-check.mjs --json   # the data this page renders
node scripts/bench-spec-check.mjs --strict # exit 1 if any spec is out of sync

What this is not

This does not rank AI tools. It measures one thing: whether a design spec matches the stylesheet it claims to describe. The next step — handing these specs to v0, Lovable, Cursor and Claude Code and scoring what comes back — is a separate benchmark, and it needs artefacts those tools produce. When there are enough of them to be worth publishing, the numbers will appear here under the same rule: every figure traceable to a raw record, and no score invented for a tool that cannot export code.

Browse the specs →