Everything in this file, corpus/, answer-key.md, a-tasks.json, workspace-/, c1-, c2-workspace-start/, rubric.py was written before the first run. Treatment and baseline runs are separate stateless API calls (Gemini REST) that receive only the inputs listed for them and never see answer-key.md, a-tasks.json check fields, c1-must-have.json or rubric.py thresholds (the C2 auditor sees item descriptions only).
Model: worker gemini-3.1-flash-lite (cheap, weaker, so differences are visible); helper/judge gemini-3.5-flash. Reason for Gemini and not OpenRouter: the OpenRouter key has about 2k tokens of credit left (HTTP 402), recorded in outputs/logs/openrouter-402.txt. Single model family = limitation: results do not generalize across models.
A. AGENTS.md routing map
Design: workspace-base (V0, no routing file) vs workspace-agents-md (V1, AGENTS.md injected into the system prompt each run, mimicking "read before every prompt"). 8 tasks x 3 trials x 2 variants = 48 runs. Agent tools: list_dir, read_file, write_file, finish. Max 14 turns.
Metrics: success (programmatic check in a-tasks.json), files read (read_file calls), list_dir calls, tokens (prompt+output, summed over turns), wall seconds.
Staleness (A3): rename clients/ to accounts/ in both variants; V1 keeps its now-wrong map (V1-stale); also V1-fixed after the audit script flags drift. Tasks T1, T5, T7 x 3 trials x {V0-moved, V1-stale, V1-fixed}. Audit script: source/check_routing.py (exit 1 on dead route).
Pass criteria:
- H-A supported if V1 success rate >= V0 success rate + 10 points, OR files-read per task falls >= 25% at equal-or-better success. Token cost is reported either way (hypothesis says "at a token cost").
- Drift effect = V1-stale success minus V1-fresh success (and vs V0-moved). Audit catches drift if check_routing.py exits 1 on the moved tree and 0 on the fixed tree.
B. LLM wiki
Design: ingest corpus one doc at a time into wiki/ (index.md, log.md, entity pages, schema AGENTS-wiki.md) following the gist. Arms for the 12 questions, 3 trials each: (W) wiki agent reads index first then pages; (R) raw agent with list_dir/read_file on corpus; (L) all 15 raw docs stuffed in one prompt (extra control, since corpus is only ~1k words). Grading: regex rules in answer-key.md, deterministic. Lint: LLM reads the wiki (arm LW) or the raw corpus (arm LR) with the same lint prompt, 3 trials each, outputs JSON findings; matched against the key by the matching rule. Precision = hits/(findings), recall = items found/10 (also split contradictions/stale). Pass: H-B supported if wiki accuracy on cross+contradiction questions (8) exceeds raw-agent R by >= 1.0 question on average AND wiki tokens/question < R tokens/question. Lint passes if recall >= 0.8 and precision >= 0.6 for the wiki arm. Honest expectation: a 1k-word corpus is too small for the wiki to win on tokens; stuffing (L) may win everything. That is a result, not a failure of the experiment.
C. Skill-pack reimplementation (OUR OWN version, not the creator's files)
C1 Grill-me: interviewer (our prompt) vs simulated persona (c1-persona.md), max 10 questions, 3 trials; control = 1 open question. Coverage = # of 10 must-have regexes (c1-must-have.json) matched in about-me.md. Pass: treatment >= 9/10 mean and >= control + 4. Invention check by judge. C2 Audit: start workspace c2-workspace-start; rubric.py 20 items x 5 pts. LLM audit (3 runs) and deterministic scorer both on the start state, then level-up (LLM edits files with tools, no persona knowledge beyond workspace), then re-audit (3 runs). Control/noise = the 3 repeated baseline audits. Pass: mean LLM score rises >= 20 points and above baseline max, deterministic score rises, and every deterministic flipped item is attributable to a file the level-up created/changed (diff), and LLM-vs-script item agreement >= 80%. C3 Security read: table of what the described skills (onboard, audit, link, level-up, 3D brain) would be able to touch, derived from the ledger's video evidence, plus observed tool footprint of our reimplementation; check whether the homepage repo is reachable (HTTP status only, nothing downloaded or installed).