X is screen-decomposition, the skill extracted from cli-engineer: the principles for organizing everything a
project has into a map that a CLI and a UI can both be built from.
The question the playground answers: given a project, does an agent that loads X produce a better map than one
without X? And does cli-engineer, once its principles move into X, still do as well as it does today?
How one run works
One run is one model call, run exactly as the rules require for proving a skill:
claude -p --model claude-haiku-5-5 --setting-sources project --tools "" --strict-mcp-config --no-session-persistence < prompt.txt,
run from a fresh, empty folder that is not a git repo.
prompt.txt = the skill text for that arm + the case (a condensed dump of the project) + the task: "Map this
project." The run must end with the map in one fixed JSON shape:
{
"screens": [
{ "path": ["guests"], "sentence": "The other people's CodeBots on this machine.",
"rows": [ { "name": "Guest list", "form": "~/codebot/guests/", "what": "Lists every guest ..." } ] }
],
"questions": [ "Split the 25 scripts behind a Scripts screen?" ]
}
The model can't read files (--tools ""), so every case is pasted into the prompt.
Cases
| Case | What it is | Ground truth |
|---|---|---|
| codebot | The real repo, condensed into a file tree, the README, the scripts, ports and services. Its existing CLI (zsh.d/codebot.zsh, the help code) is cut out so the answer doesn't leak. |
Today's codebot help and its group screens, which you approved |
| workstation | The real repo, condensed the same way, with workstation help and its source cut out |
The README ## Contents table and today's workstation help |
| planted | A small made-up project with about 30 things, 5 of them hidden where they are hard to find (a cron job in a Makefile, a port only in docker-compose, a log path only in a systemd unit, a doc in a nested folder, a script with no extension). It uses its own vocabulary (for example, it calls its servers "rigs"), and it has one list of 25 scripts. A subagent that has never seen X writes it. | The list of all 30 things, written together with the project |
| UI | betec's technicians_app_native, given as screens, routes and data model, with no CLI |
The navigation in its 2026-10-21 flow card (org technicians) |
The project repos are only read. Each case is copied into this experiment's source/cases/, and nothing the
playground makes is written to codebot, workstation or betec.
Arms
| Arm | Skill text in the prompt | What it shows |
|---|---|---|
| 0: none | nothing | The floor: what the model does with no skill |
| 1: old | Today's cli-engineer for the CLI cases; today's minimal-ux-flow for the UI case |
What we have now |
| 2: new | X + the cut-down cli-engineer; X + minimal-ux-flow |
What the extraction gives |
5 runs per case per arm (the rule's minimum is 3; haiku varies from run to run). That's 4 × 3 × 5 = 60 runs.
Pass lines
A Python scorer reads each run's JSON. Every line is mechanical, apart from the last.
| # | Line | Pass |
|---|---|---|
| P1 | Recall: how many of the ground-truth things appear as a row, matched by technical form (path, command, port) | ≥ 90% |
| P2 | Nothing invented: every row's form exists in the case | 100% |
| P3 | Where and what: every screen below the top has its noun path plus one sentence; the top screen has only the sentence | 100% |
| P4 | Verb first: every "what" cell starts with a verb | ≥ 95% |
| P5 | Short top: the top screen has at most 9 rows | yes |
| P6 | Own words: screen and group names use the case's words, not invented ones (on the planted case, "rigs" not "servers") | ≥ 90% |
| P7 | Splits are asked, not made: the 25-script list shows up in questions, not as a silent new screen |
yes |
| P8 | Worded for its screen: a thing that appears on two screens has a different "what it's for" line on each | 100% |
| P9 | Grouping: how closely the run's groups match the ground-truth groups (adjusted Rand index), then you judge it on the results page | shown, judged by you |
Red: arm 0 clearly fails P1, P3, P4 or P7, and on the UI case arm 1 does too. Green: arm 2 passes P1 to P8 on every case. No regression: on codebot and workstation, arm 2 does at least as well as arm 1.
The results you see
You see the results in an interface: demo/index.html, a page in this experiment, outside every project repo.
- The page opens on the verdict and a table of cases × arms with the pass rate for every line.
- Every one of the 60 runs is on the page, with and without the skill, nothing left out: its full raw answer, its map drawn both ways, and its score on every line. You can step through the runs of one case side by side, arm by arm.
- Click a cell to see one run's map drawn two ways from its single JSON: as terminal help, and as a phone-width wireframe. The ground truth sits beside it. Seeing one map drawn in both media is how you see that X doesn't depend on the medium. No agent here opens a browser; you open the page yourself.
insights/RESULTS.md: the same verdict and table in text, for agents and the workstation README.outputs/runs/<case>/<arm>/<n>/: each run's prompt, raw answer, JSON and score.
How it could lie, and what stops it
| Risk | Guard |
|---|---|
| The case gives away the answer | The existing CLI and help code are cut out of the codebot and workstation dumps; a grep for their help text in each prompt must find nothing |
| Teaching to the test | The pass lines are written before X; the planted case is written by a subagent that has never seen X |
| The model copies the skill's wording without following it | The scorer checks structure and forms, never phrases from the skill |
| One lucky run | 5 runs per arm; a pass rate, not one result |
| The prompt is too long | Each dump is cut to about 30k tokens; the cut list is in source/ |
Order of work
- You approve this design.
- Write X and the cut-down
cli-engineer(a plan for you first, as with any skill change). - Build the cases, the runner, the scorer and the renderer. Test the scorer on hand-made good and bad JSON first.
- Run all 60 runs, score them, write RESULTS.md and the page.
- Add one line to the workstation README pointing to RESULTS.md.