EmerikoandClaude Opus 4.8
88431defba
Swap em dashes for commas/colons/periods in the README, skills, AGENTS.md and its five rule copies, examples, command files, and benchmark README. Rule copies stay in sync (same edit applied to all) and the invariant guard passes. Left untouched on purpose: the vendored caveman SKILL.md (verbatim third-party text), the dated benchmark writeups in results/ (historical records), and .js code comments. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Benchmark
Three arms (no skill, caveman, ponytail), three models, five everyday tasks, 10 runs per cell, median reported. Code LOC is counted from fenced code blocks; tokens, cost, and latency come straight from the API.
Reproduce
cp ../.env.example ../.env # add your ANTHROPIC_API_KEY
npx promptfoo@latest eval -c promptfooconfig.yaml --repeat 10
npx promptfoo@latest view
Tasks: email validator, JS debounce, CSV sum, React countdown, FastAPI rate-limit (see promptfooconfig.yaml). Single-shot completions, default temperature.
Median results (10 runs, 2026-06-13)
Code (lines)
| arm | Haiku | Sonnet | Opus |
|---|---|---|---|
| baseline (no skill) | 518 | 693 | 256 |
| caveman | 116 | 120 | 67 |
| ponytail | 39 | 44 | 51 |
Cost (USD, 5 tasks)
| arm | Haiku | Sonnet | Opus |
|---|---|---|---|
| baseline (no skill) | 0.032 | 0.141 | 0.135 |
| caveman | 0.014 | 0.045 | 0.075 |
| ponytail | 0.010 | 0.032 | 0.071 |
Latency (seconds, 5 tasks)
| arm | Haiku | Sonnet | Opus |
|---|---|---|---|
| baseline (no skill) | 37.7 | 124.1 | 58.7 |
| caveman | 14.9 | 34.7 | 23.1 |
| ponytail | 9.9 | 20.1 | 18.0 |
Versus baseline, ponytail writes 80-94% less code, costs 47-77% less, and runs 3-6x faster, on every model.
Notes
- Caveman is a prose-compression skill (it leaves code "normal"), so it lands between baseline and ponytail on code size and wins mainly on prose tokens.
- Cost reflects single-shot calls that re-send the skill every time. In real sessions the skill is injected once and prompt-cached, so the cost gap widens further in ponytail's favor.
- These are everyday tasks. For production-grade specs, where an unconstrained agent bloats much harder, see the writeups in
results/.