wangjunxiao/ponytail
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(readme): make the benchmark chart multi-metric

Browse Source 
Grouped bars showing LOC, tokens, cost and time, each as a % of the no-skill
baseline (baseline = 100%, lower is leaner/cheaper/faster), instead of LOC
alone. Makes ponytail's win on every metric visible and shows caveman rising
above 100% on tokens/cost/time. Same system-gray palette so it reads on both
GitHub themes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Emeriko
2026-06-18 16:56:37 +02:00
co-authored by Claude Opus 4.8
parent 1a9689af1d
commit f0dad86a52
2 changed files with 50 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -47,7 +47,7 @@ More survivors in [examples/](examples/).
The honest measurement is a real agent doing real work: a headless Claude Code session editing [tiangolo's full-stack-fastapi-template](https://github.com/fastapi/full-stack-fastapi-template) (a real FastAPI + React repo), scored on the `git diff` it leaves behind. Twelve feature tickets, the same agent with and without the skill, n=4, Haiku 4.5.
<p align="center">
<img src="assets/benchmark-agentic.svg" width="860" alt="Mean lines of code per task by arm on the agentic benchmark (Haiku 4.5): baseline 191, caveman 152, yagni-oneliner 128, ponytail 87">
<img src="assets/benchmark-agentic.svg" width="860" alt="Each arm as a percent of the no-skill baseline across LOC, tokens, cost and time (Haiku 4.5). ponytail is lowest on every metric (LOC 46%, tokens 78%, cost 80%, time 73%); caveman rises above 100% on tokens, cost and time; yagni-oneliner LOC 67%.">
</p>
| vs no-skill baseline | LOC | tokens | cost | time | safe |