diff --git a/README.md b/README.md index 34c97a3..238c1be 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,10 @@ More survivors in [examples/](examples/). The honest measurement is a real agent doing real work: a headless Claude Code session editing [tiangolo's full-stack-fastapi-template](https://github.com/fastapi/full-stack-fastapi-template) (a real FastAPI + React repo), scored on the `git diff` it leaves behind. Twelve feature tickets, the same agent with and without the skill, n=4, Haiku 4.5. +

+ Mean lines of code per task by arm on the agentic benchmark (Haiku 4.5): baseline 191, caveman 152, yagni-oneliner 128, ponytail 87 +

+ | vs no-skill baseline | LOC | tokens | cost | time | safe | |---|--:|--:|--:|--:|--:| | **ponytail** | **-54%** | **-22%** | **-20%** | **-27%** | **100%** | diff --git a/assets/benchmark-agentic.svg b/assets/benchmark-agentic.svg new file mode 100644 index 0000000..6903706 --- /dev/null +++ b/assets/benchmark-agentic.svg @@ -0,0 +1,23 @@ + + Mean lines of code per task by arm, agentic benchmark on Haiku 4.5 + Lines of code per task (12 real-repo features), Claude Code on Haiku 4.5 + Mean over 12 frontend + backend tickets, n=4. Lower is leaner. ponytail writes 54% less than a no-skill agent and stays 100% safe. + + baseline + + 191 + + caveman + + 152 (-20%) + + yagni-oneliner + + 128 (-33%) + + ponytail + + 87 (-54%) + + Range by task: -94% on over-build features (date/color picker, native input vs custom component) to ~0% on already-minimal backend code. LOC = git diff added lines. Writeup: benchmarks/results/2026-06-18-agentic.md +