Repo15 days ago

i-have-adhd Hit 32,897 Stars on 140 Lines, and Its Own Evals Failed the Release Gate

The i-have-adhd skill for coding agents has 32,897 stars in 119 days, and the whole product is a 140 line prompt. We cloned it and found an eval report the README never mentions, with a release gate marked FAILED.

The WJS Desk

Sep 9, 2026 · 6 min read

Photo by DS stories on Pexels

32,897 stars. Created 13 May 2026, so 119 days old, which averages out to about 276 stars a day. It is written in Python, licensed MIT, and has 1,956 forks.

The product is a prompt. One file, 140 lines, telling your coding agent to put the answer first and stop opening with "Great question!" We cloned it because a repository that size with that star count usually has something else going on underneath, and this one does. It has an eval harness. It also has an eval result the README does not link to.

What you actually install

The shipped artifact is skills/i-have-adhd/SKILL.md: 140 lines, 1,183 words, 6,813 bytes. Ten rules. The frontmatter is the interesting part.

---
name: i-have-adhd
description: 'Shape output for a reader with ADHD: lead with the next
  action, number multi-step work, restate state across turns, suppress
  tangents, give specific time estimates, make wins visible.'
disable-model-invocation: true
---

disable-model-invocation: true means the model can never decide to load this on its own. You type /i-have-adhd or it does not run. That is a deliberate choice and it is the right one for an output-style skill, but it also means the top complaint in the Hacker News thread is structural rather than a bug.

The complaint, and the hook that answers it

HN user sleazebreeze wrote that agents "only maintain the conciseness for a few turns at most before they completely forget." That is the whole failure mode for a style instruction, and the repo has two separate answers to it.

The first is a Persistence section in the skill body: "These rules apply to every response for the rest of the session, not only this one." The second is a SessionStart hook that re-injects the ruleset every time a session starts, opt-in via a flag file at ~/.claude/.i-have-adhd-always.

We ran that hook directly against a throwaway config directory. With no flag file it emits 0 bytes and exits. With the flag file present it emits 6,592 bytes and 1,156 words in 0.039 seconds. That is the entire ruleset, stripped of frontmatter, prepended to every session you start.

The hidden cost: always-on mode is not free. Those 6,592 bytes sit in the context of every session, forever, before you have typed anything. At the usual four-characters-per-token rule of thumb that is roughly 1,650 tokens of standing overhead per session, spent on instructions about formatting. On a long agent run that is noise you are paying for on every request.

We ran its own checks

The repo's AGENTS.md lists the verification commands. We ran the two that cost nothing.

$ python3 -m unittest discover -s tests
Ran 41 tests in 0.469s
OK

$ python3 scripts/run_evals.py validate
Evaluation cases are valid.

$ python3 scripts/run_evals.py plan --trials 3 --include-comparator
126 rows

Everything passes. The catch is what those 41 tests cover: 17 in test_judge.py, 12 in test_run_evals.py, 7 for the always-on hooks, 3 for the OpenCode plugin, 2 for packaging. They test the machinery that measures the skill. Not one of them tests whether the skill changes an agent's output, because that costs money and needs a model.

evals/RESULTS.md is where the actual answer lives. One run, 14 cases, 3 trials, 84 rows, judged blind by the same model that generated the responses. The maintainers report a cost of $2.67 for generation plus $0.92 for judging. We did not reproduce it, so every number below is theirs, not ours.

The aggregate is good: weighted score 4.045 baseline to 4.473 with the skill, a gain of 0.427, with every dimension moving in the skill's favour. Concision gained the most at +1.143, which is unsurprising, and correctness still moved +0.190, which is the part that matters.

Then there is the per-case table, and this is the part worth reading.

Test caseBaselineWith the skillDelta
multi-step-progress2.234.77+2.53
error-report2.074.47+2.40
destructive-action4.134.65+0.52
code-answer5.005.000.00
agent-owned-edit2.572.230.33 worse
partial-success4.523.880.63 worse

And the line the README never mentions: Release gate: FAILED. The gate requires zero blocking findings and the candidate has three, down from the baseline's seven. The report says so plainly, then explains why the rule is written badly, which is a level of self-criticism we almost never see in a repo with five figures of stars.

The partial-success regression is the honest part. The grader noted the response "asserts 'missing auth header' as the definitive cause and prescribes a specific fix without any evidence." The maintainers name the likely mechanism themselves: rule 8 demands errors be reported as cause then fix, which pressures the model to invent a cause when the evidence does not supply one. A style rule producing confident wrong answers is the single most important thing anyone could know before installing this, and it is four directory levels from the front page.

What it does not do

  • It does not run itself. disable-model-invocation: true means manual invocation or an opt-in hook, every time.
  • It does not enforce anything. As HN user alienchow put it, "Instructions to LLMs are merely suggestions...unless you have a deterministic guardrail." There is no guardrail here. There is a prompt.
  • It has not been measured against a second model family. The judge is the same model that wrote the answers, a limitation the report flags itself.
  • Three trials is not enough to call any single case. Per-case standard deviations reach 0.95.

Where the 8,751 lines went

HN user mzajc asked the sharpest question in the thread: "Is there a reason this needs 8.7k lines across 59 files?" A reply guessed it was the evals. We counted, and the reply is about half right.

Total is 8,751 lines. Translated README and INSTALL files account for 4,007 of them, or 46 percent, across six languages. The eval harness code and its tests are 2,209 lines and the eval data and reports another 268, so 28 percent for the evals. The skill itself, counting the Cursor mirror, is 280 lines. The single largest thing in this repository is documentation translated into Japanese, Korean, Vietnamese, Thai, Portuguese and Chinese.

Bus factor, and the verdict

194 commits total. 34 contributors are listed, 16 of whom have landed more than one. The author, ayghri, has 104 commits, which is 54 percent of the project. That is a single-maintainer repository with a healthy contribution fringe, and it does not matter much, because the thing you install is one Markdown file. If the project stalls tomorrow, copy the 140 lines into your own config and you have lost nothing except the translations.

The stars are for the prompt. The value is in the eval report nobody links to.

Install it if you are tired of scrolling past preamble and you accept that a prompt is a suggestion. It costs one file and you can delete it in a second. Skip always-on mode until you have decided that 1,650 tokens per session is worth it to you.

What would change our mind: a run with a cross-family judge and more than three trials. Until then the honest summary is that one person measured this once, published the fact that it failed their own gate, and that is still more evidence than almost any prompt on GitHub carries.

Share

32,897 stars for a 140 line prompt. We cloned it and found the eval report the README never links to: release gate FAILED, and one test case where the skill made the agent worse. #OpenSource #ClaudeCode #AITooling #Evals

Never miss a ship

The best stuff that shipped this week, delivered every Thursday. Free, no spam. We read all the boring stuff so you get the fun parts.

Keep reading