wangjunxiao/ponytail
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(readme): add agentic benchmark chart (LOC per arm)

Browse Source 
Bar chart of mean LOC per task by arm (Haiku 4.5) in the Numbers section, same
system-gray style as the single-shot chart so it reads on both GitHub themes.
The old single-shot chart stays in the collapsed "isolated generation" section.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Emeriko
2026-06-18 16:48:20 +02:00
co-authored by Claude Opus 4.8
parent 75fea8f743
commit 1a9689af1d
2 changed files with 27 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4
View File
@@ -46,6 +46,10 @@ More survivors in [examples/](examples/).
The honest measurement is a real agent doing real work: a headless Claude Code session editing [tiangolo's full-stack-fastapi-template](https://github.com/fastapi/full-stack-fastapi-template) (a real FastAPI + React repo), scored on the `git diff` it leaves behind. Twelve feature tickets, the same agent with and without the skill, n=4, Haiku 4.5.
<p align="center">
<img src="assets/benchmark-agentic.svg" width="860" alt="Mean lines of code per task by arm on the agentic benchmark (Haiku 4.5): baseline 191, caveman 152, yagni-oneliner 128, ponytail 87">
</p>
| vs no-skill baseline | LOC | tokens | cost | time | safe |
|---|--:|--:|--:|--:|--:|
| **ponytail** | **-54%** | **-22%** | **-20%** | **-27%** | **100%** |