The i-have-adhd skill, tested with generated answers and a judge

Ledger entry: i-have-adhd (Instagram carousel by SBX AI; free MIT skill). User ask: "try in the ai-lab as a playground."

Hypothesis

With the skill active, answers are shorter, lead with the action, and stay correct, but it may under-serve tasks that need detail (architecture review, safety, audits).

Setup

  • Find the skill repo from the carousel (https://www.instagram.com/sbx.intelligence/p/DdWTvhRCAf-/). Read all files first; record repo URL, commit hash, license. Install only inside outputs/skill/, not globally.

Fixed prompts (in source/prompts.md, written before running)

10 prompts across 3 buckets: 4 quick how-to / action prompts, 3 explanation prompts, 3 detail-critical prompts (architecture trade-off, a security caution, a multi-step migration where dropping a step is harmful).

Cases

  1. Length: with-skill vs no-skill baseline, same model, 10 prompts, 3 trials. Metrics: word count, time-to-first-action (position of the first imperative step), number of sentences before the answer.
  2. Correctness: a separate judge agent sees both answers blind, in random order, and marks each as correct / missing critical info / wrong against a checklist of must-have points per prompt written beforehand in source/checklists.md.
  3. Harm check: on the 3 detail-critical prompts, pass = no must-have point dropped. Report each dropped point.
  4. Conflict check: run with the lab's CLAUDE.md present and report any instruction conflict.
  5. Reversibility: confirm removing the skill restores baseline behavior.

Deliverables

insights/RESULTS.md: metrics table, where it helps, where it hurts, a recommended scope (e.g. on for quick tasks only). demo/index.html: side by side of one real prompt, baseline vs skill.