Tolerances (fixed now)

• Font family: exact match of first family name (case-insensitive); fallback stacks ignored. Brand fonts that aren't web-visible by name (e.g. Apple "SF Pro" serving as system) count as match if the skill names the same family or the documented system alias.
• Colors: pass if CIE76 delta-E <= 10 in Lab vs ground truth (primary, background, text each scored).
• Radius: pass if within +/-2px (or both "pill"/>=999 vs >=100px).
• Spacing base unit: pass if claimed base unit equals ground-truth mode, or the mode is an integer multiple (1-3x) of it.
• Token accuracy per site = passed tokens / tokens with ground truth. Skill beats baseline if its accuracy is higher by >= 0.10 averaged over sites; tie within 0.10 = no evidence of difference.

Pass/fail for the hypothesis

• "Accurate": mean skill token accuracy >= 0.70.
• "More useful than baseline": skill accuracy >= baseline + 0.10 AND mean rationale score greater than baseline's by >= 0.5 (1-5 blind judge, 2 judge repeats averaged).
• Usability: mean mock resemblance >= 3.0 of 5 (judge sees both screenshots, not told which method). Otherwise hypothesis is reported not supported / inconclusive.

Controls and bias handling

• Baseline: same model, same inputs (same site data given to both), prompt "extract this site's design tokens and explain the choices", no skill text.
• Both skill and baseline receive the identical page evidence (screenshot + DOM-derived page summary that is NOT the ground truth extraction; no computed-color table). If the skill itself drives the browser, that is recorded as its native mode and any deviation noted.
• Judge agents are blind: outputs labelled A/B randomly, site names retained but method hidden.
• Honest limits: n=5 sites, one model, LLM judges; no statistical significance claimed.

Amendment 1 (2026-10-03, still before any site load or model run)

After reading the skill: its Phase 1 input is a screenshot plus references/extract.js output (computed-style tables). So:

• Evidence given to the skill arm = viewport screenshot + extract.js JSON (verbatim file, run via Playwright library instead of the Playwright MCP; same 1440x900 viewport, 3s wait). Deviation recorded in insights.
• Baseline arm B1 ("same evidence"): identical screenshot + identical extract.js JSON, plain prompt, no skill text. This isolates the skill's prompt pipeline.
• Baseline arm B0 ("screenshot only"): the screenshot only, plain prompt. Extra control; shows what the extractor adds.
• Ground truth uses a different, independent script (CTA-based, in outputs/capture/capture.js) run in the SAME page load (the plan allows one load per site). Caveat: both read the same live DOM, so it is independent in method, not in source. Limitation will be reported.
• Skill's multi-page crawl and export targets are off (defaults: single page, skip export).
• Models run via claude -p (Claude Code CLI, sonnet) from a clean directory with no CLAUDE.md, settings, MCP or user skills, so global instructions cannot contaminate arms. Judges: same CLI, fresh sessions, 2 repeats each, blind A/B/C labels.
• Mock resemblance judged on the 1440x900 mock screenshot vs the real screenshot.
• Token set scored per site: bg, text, accent (CTA) colors; body font; heading font; CTA radius; spacing base unit.

Amendment 2 (2026-10-03, before any skill/baseline run)

Ground-truth script v1 (kept in outputs/capture/failed-first-attempt/gt-v1/) picked "Skip to content" links as the CTA on linear.app and vercel.com and a dark-tile paragraph as body text on apple.com. Fixed (v2: exclude /skip/ links, require on-screen, body text = first visible paragraph >40 chars) and recaptured. This is a 3rd load per site (1st failed on my own syntax error, 2nd had the GT bug); no model output had been seen. Extra scoring rules fixed now:

• Font match: compare lowercase with spaces/hyphens removed and the words variable/var/sans/display/text/pro-less suffixes ignored; "contains" in either direction counts (e.g. InterVariable ~ Inter, GeistSans ~ Geist, sohne-var ~ Söhne/Sohne, SFProDisplay ~ SF Pro).
• Colors with alpha are composited over the page background before delta-E.
• Ground truth is a proxy for the "true" token set (CTA color = accent). If a site's primary accent is judged by me not to be what the CTA shows, I will say so in RESULTS rather than change the rule.