docs: correct cost claim to 42-75% from 30-rep re-verification

Re-ran the cost benchmark at 30 reps per cell on Claude (Haiku/Sonnet/Opus):
ponytail is 42-75% cheaper than no-skill, not the previously published 47-77%.
The direction holds, both ends came in a few points lower. Updates the README
headline and body, the benchmark chart subtitle, and the benchmarks/README cost
table, and adds a dated results doc with full method.

Also adds the OpenAI (gpt-4.1-mini/gpt-5.4-mini/gpt-5.5) and Gemini configs. On
OpenAI reasoning models ponytail costs more, not less, so the claim stays
Claude-scoped. Gemini run pending a fresh-quota day.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Emeriko
2026-06-17 03:47:36 +02:00
co-authored by Claude Opus 4.8
parent 99139a25d0
commit 0bf152a987
6 changed files with 167 additions and 9 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
<svg viewBox="0 0 860 336" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system, 'Segoe UI', Helvetica, Arial, sans-serif">
<title>Median lines of code per arm across three models</title>
<text x="20" y="26" font-size="15" font-weight="600" fill="#8b949e">Median lines of code. 10 runs per cell. Lower is leaner.</text>
<text x="20" y="45" font-size="12" fill="#8b949e" opacity="0.85">Ponytail writes 80-94% less code, costs 47-77% less, and runs 3-6x faster than a no-skill agent.</text>
<text x="20" y="45" font-size="12" fill="#8b949e" opacity="0.85">Ponytail writes 80-94% less code, costs 42-75% less, and runs 3-6x faster than a no-skill agent.</text>
<rect x="20" y="58" width="12" height="12" rx="2" fill="#8b949e"/><text x="38" y="69" font-size="13" fill="#8b949e">baseline (no skill)</text>
<rect x="190" y="58" width="12" height="12" rx="2" fill="#d9822b"/><text x="208" y="69" font-size="13" fill="#8b949e">caveman</text>
<rect x="300" y="58" width="12" height="12" rx="2" fill="#2da44e"/><text x="318" y="69" font-size="13" fill="#8b949e">ponytail</text>

Before

Width:  |  Height:  |  Size: 2.7 KiB

After

Width:  |  Height:  |  Size: 2.7 KiB