EmerikoandClaude Opus 4.8
321a59c82f
Commit a promptfoo harness (config + arm prompts + LOC metric + vendored caveman SKILL) so anyone can re-run the comparison: no-skill vs caveman vs ponytail, across Haiku / Sonnet / Opus, 10 runs per cell, median reported. Replace the old unreproducible 6-task chart with assets/benchmark-3model.svg from this run, and reframe the README to the reproducible numbers: ponytail writes 80-94% less code, costs 47-77% less, and runs 3-6x faster than a no-skill agent on every model. benchmarks/README.md carries the median tables and the reproduce command. Drops nothing that is not measured. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
12 lines
151 B
Plaintext
12 lines
151 B
Plaintext
# Secrets — never commit API keys
|
|
.env
|
|
.env.*
|
|
!.env.example
|
|
|
|
# Dependencies
|
|
node_modules/
|
|
|
|
# promptfoo eval artifacts
|
|
.promptfoo/
|
|
benchmarks/output*
|