From 1a9689af1d78e83dbcbf0bbb08b615cd41ed2e2c Mon Sep 17 00:00:00 2001 From: Emeriko Date: Thu, 18 Jun 2026 16:48:20 +0200 Subject: [PATCH] docs(readme): add agentic benchmark chart (LOC per arm) Bar chart of mean LOC per task by arm (Haiku 4.5) in the Numbers section, same system-gray style as the single-shot chart so it reads on both GitHub themes. The old single-shot chart stays in the collapsed "isolated generation" section. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 4 ++++ assets/benchmark-agentic.svg | 23 +++++++++++++++++++++++ 2 files changed, 27 insertions(+) create mode 100644 assets/benchmark-agentic.svg diff --git a/README.md b/README.md index 34c97a3..238c1be 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,10 @@ More survivors in [examples/](examples/). The honest measurement is a real agent doing real work: a headless Claude Code session editing [tiangolo's full-stack-fastapi-template](https://github.com/fastapi/full-stack-fastapi-template) (a real FastAPI + React repo), scored on the `git diff` it leaves behind. Twelve feature tickets, the same agent with and without the skill, n=4, Haiku 4.5. +

+ Mean lines of code per task by arm on the agentic benchmark (Haiku 4.5): baseline 191, caveman 152, yagni-oneliner 128, ponytail 87 +

+ | vs no-skill baseline | LOC | tokens | cost | time | safe | |---|--:|--:|--:|--:|--:| | **ponytail** | **-54%** | **-22%** | **-20%** | **-27%** | **100%** | diff --git a/assets/benchmark-agentic.svg b/assets/benchmark-agentic.svg new file mode 100644 index 0000000..6903706 --- /dev/null +++ b/assets/benchmark-agentic.svg @@ -0,0 +1,23 @@ + + Mean lines of code per task by arm, agentic benchmark on Haiku 4.5 + Lines of code per task (12 real-repo features), Claude Code on Haiku 4.5 + Mean over 12 frontend + backend tickets, n=4. Lower is leaner. ponytail writes 54% less than a no-skill agent and stays 100% safe. + + baseline + + 191 + + caveman + + 152 (-20%) + + yagni-oneliner + + 128 (-33%) + + ponytail + + 87 (-54%) + + Range by task: -94% on over-build features (date/color picker, native input vs custom component) to ~0% on already-minimal backend code. LOC = git diff added lines. Writeup: benchmarks/results/2026-06-18-agentic.md +