diff --git a/.env.example b/.env.example
new file mode 100644
index 0000000..87bcc63
--- /dev/null
+++ b/.env.example
@@ -0,0 +1,2 @@
+# Copy to .env (gitignored) and fill in. promptfoo reads this automatically.
+ANTHROPIC_API_KEY=sk-ant-...
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..7aeb17e
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,11 @@
+# Secrets — never commit API keys
+.env
+.env.*
+!.env.example
+
+# Dependencies
+node_modules/
+
+# promptfoo eval artifacts
+.promptfoo/
+benchmarks/output*
diff --git a/README.md b/README.md
index 116b3a0..adda6c1 100644
--- a/README.md
+++ b/README.md
@@ -16,8 +16,8 @@
- 47% fewer tokens · 3× faster · one-seventh the code
- Same six tasks, same model, same adversarial security and concurrency probes. See the benchmark.
+ 80-94% less code · 3-6× faster · 47-77% cheaper
+ Median of 10 runs across Haiku, Sonnet, and Opus. Reproduce it yourself.
---
@@ -41,13 +41,13 @@ More survivors in [examples/](examples/).
## Numbers
-Six tasks: streaming log parser, atomic file sync, notification dispatcher, validation engine, auth module, concurrent money ledger. One spec each, one fresh agent per arm, same model. Three arms: no skill, the [caveman](https://github.com/JuliusBrussee/caveman) skill, and ponytail. Every arm passes the same adversarial security and concurrency probes. Then the agreement ends:
+Five everyday tasks (email validator, debounce, CSV sum, countdown timer, rate limiter), three models, three arms: no skill, the [caveman](https://github.com/JuliusBrussee/caveman) skill, and ponytail. Ten runs per cell, median reported.
-
+
-**47% fewer tokens than the no-skill agent. 3× faster. A seventh of the code.** The 3,139 lines nobody wrote have never caused an incident. When a surprise feature request hit two of the tasks, ponytail extended in 96 changed lines; caveman needed 413, the no-skill agent 1,115. Every shortcut ponytail took is marked in the code with a `ponytail:` comment naming its upgrade path. Data: [benchmarks/](benchmarks/).
+**80-94% less code, 47-77% less cost, and 3-6× faster than a no-skill agent, on every model.** Every shortcut ponytail takes is marked in the code with a `ponytail:` comment naming its upgrade path. Reproduce it yourself: `npx promptfoo eval -c benchmarks/promptfooconfig.yaml`. Method and raw numbers: [benchmarks/](benchmarks/). Production-grade tasks, where an unconstrained agent bloats far more, are written up in [benchmarks/results/](benchmarks/results/).
## How it works
diff --git a/assets/benchmark-3model.svg b/assets/benchmark-3model.svg
new file mode 100644
index 0000000..10c0969
--- /dev/null
+++ b/assets/benchmark-3model.svg
@@ -0,0 +1,21 @@
+
diff --git a/assets/benchmark-loc.svg b/assets/benchmark-loc.svg
deleted file mode 100644
index be81d52..0000000
--- a/assets/benchmark-loc.svg
+++ /dev/null
@@ -1,69 +0,0 @@
-
diff --git a/benchmarks/README.md b/benchmarks/README.md
new file mode 100644
index 0000000..6dc68b9
--- /dev/null
+++ b/benchmarks/README.md
@@ -0,0 +1,47 @@
+# Benchmark
+
+Three arms (no skill, [caveman](https://github.com/JuliusBrussee/caveman), ponytail), three models, five everyday tasks, **10 runs per cell, median reported**. Code LOC is counted from fenced code blocks; tokens, cost, and latency come straight from the API.
+
+## Reproduce
+
+```bash
+cp ../.env.example ../.env # add your ANTHROPIC_API_KEY
+npx promptfoo@latest eval -c promptfooconfig.yaml --repeat 10
+npx promptfoo@latest view
+```
+
+Tasks: email validator, JS debounce, CSV sum, React countdown, FastAPI rate-limit (see `promptfooconfig.yaml`). Single-shot completions, default temperature.
+
+## Median results (10 runs, 2026-06-13)
+
+**Code (lines)**
+
+| arm | Haiku | Sonnet | Opus |
+|---|--:|--:|--:|
+| baseline (no skill) | 518 | 693 | 256 |
+| caveman | 116 | 120 | 67 |
+| **ponytail** | **39** | **44** | **51** |
+
+**Cost (USD, 5 tasks)**
+
+| arm | Haiku | Sonnet | Opus |
+|---|--:|--:|--:|
+| baseline (no skill) | 0.032 | 0.141 | 0.135 |
+| caveman | 0.014 | 0.045 | 0.075 |
+| **ponytail** | **0.010** | **0.032** | **0.071** |
+
+**Latency (seconds, 5 tasks)**
+
+| arm | Haiku | Sonnet | Opus |
+|---|--:|--:|--:|
+| baseline (no skill) | 37.7 | 124.1 | 58.7 |
+| caveman | 14.9 | 34.7 | 23.1 |
+| **ponytail** | **9.9** | **20.1** | **18.0** |
+
+Versus baseline, ponytail writes **80-94% less code**, costs **47-77% less**, and runs **3-6x faster** — on every model.
+
+## Notes
+
+- Caveman is a prose-compression skill (it leaves code "normal"), so it lands between baseline and ponytail on code size and wins mainly on prose tokens.
+- Cost reflects single-shot calls that re-send the skill every time. In real sessions the skill is injected once and prompt-cached, so the cost gap widens further in ponytail's favor.
+- These are everyday tasks. For production-grade specs, where an unconstrained agent bloats much harder, see the writeups in `results/`.
diff --git a/benchmarks/arms/baseline.js b/benchmarks/arms/baseline.js
new file mode 100644
index 0000000..1469035
--- /dev/null
+++ b/benchmarks/arms/baseline.js
@@ -0,0 +1,2 @@
+// Baseline arm: no skill, just the task.
+module.exports = ({ vars }) => [{ role: 'user', content: vars.task }];
diff --git a/benchmarks/arms/caveman-SKILL.md b/benchmarks/arms/caveman-SKILL.md
new file mode 100644
index 0000000..561069b
--- /dev/null
+++ b/benchmarks/arms/caveman-SKILL.md
@@ -0,0 +1,67 @@
+---
+name: caveman
+description: >
+ Ultra-compressed communication mode. Cuts token usage ~75% by speaking like caveman
+ while keeping full technical accuracy. Supports intensity levels: lite, full (default), ultra,
+ wenyan-lite, wenyan-full, wenyan-ultra.
+ Use when user says "caveman mode", "talk like caveman", "use caveman", "less tokens",
+ "be brief", or invokes /caveman. Also auto-triggers when token efficiency is requested.
+---
+
+Respond terse like smart caveman. All technical substance stay. Only fluff die.
+
+## Persistence
+
+ACTIVE EVERY RESPONSE. No revert after many turns. No filler drift. Still active if unsure. Off only: "stop caveman" / "normal mode".
+
+Default: **full**. Switch: `/caveman lite|full|ultra`.
+
+## Rules
+
+Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging. Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). Technical terms exact. Code blocks unchanged. Errors quoted exact.
+
+Pattern: `[thing] [action] [reason]. [next step].`
+
+Not: "Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by..."
+Yes: "Bug in auth middleware. Token expiry check use `<` not `<=`. Fix:"
+
+## Intensity
+
+| Level | What change |
+|-------|------------|
+| **lite** | No filler/hedging. Keep articles + full sentences. Professional but tight |
+| **full** | Drop articles, fragments OK, short synonyms. Classic caveman |
+| **ultra** | Abbreviate (DB/auth/config/req/res/fn/impl), strip conjunctions, arrows for causality (X → Y), one word when one word enough |
+| **wenyan-lite** | Semi-classical. Drop filler/hedging but keep grammar structure, classical register |
+| **wenyan-full** | Maximum classical terseness. Fully 文言文. 80-90% character reduction. Classical sentence patterns, verbs precede objects, subjects often omitted, classical particles (之/乃/為/其) |
+| **wenyan-ultra** | Extreme abbreviation while keeping classical Chinese feel. Maximum compression, ultra terse |
+
+Example — "Why React component re-render?"
+- lite: "Your component re-renders because you create a new object reference each render. Wrap it in `useMemo`."
+- full: "New object ref each render. Inline object prop = new ref = re-render. Wrap in `useMemo`."
+- ultra: "Inline obj prop → new ref → re-render. `useMemo`."
+- wenyan-lite: "組件頻重繪，以每繪新生對象參照故。以 useMemo 包之。"
+- wenyan-full: "物出新參照，致重繪。useMemo .Wrap之。"
+- wenyan-ultra: "新參照→重繪。useMemo Wrap。"
+
+Example — "Explain database connection pooling."
+- lite: "Connection pooling reuses open connections instead of creating new ones per request. Avoids repeated handshake overhead."
+- full: "Pool reuse open DB connections. No new connection per request. Skip handshake overhead."
+- ultra: "Pool = reuse DB conn. Skip handshake → fast under load."
+- wenyan-full: "池reuse open connection。不每req新開。skip handshake overhead。"
+- wenyan-ultra: "池reuse conn。skip handshake → fast。"
+
+## Auto-Clarity
+
+Drop caveman for: security warnings, irreversible action confirmations, multi-step sequences where fragment order risks misread, user asks to clarify or repeats question. Resume caveman after clear part done.
+
+Example — destructive op:
+> **Warning:** This will permanently delete all rows in the `users` table and cannot be undone.
+> ```sql
+> DROP TABLE users;
+> ```
+> Caveman resume. Verify backup exist first.
+
+## Boundaries
+
+Code/commits/PRs: write normal. "stop caveman" or "normal mode": revert. Level persist until changed or session end.
diff --git a/benchmarks/arms/caveman.js b/benchmarks/arms/caveman.js
new file mode 100644
index 0000000..930eeca
--- /dev/null
+++ b/benchmarks/arms/caveman.js
@@ -0,0 +1,8 @@
+// Caveman arm: caveman SKILL.md (full) as the system prompt.
+const fs = require('fs');
+const path = require('path');
+const system = fs.readFileSync(path.join(__dirname, 'caveman-SKILL.md'), 'utf8');
+module.exports = ({ vars }) => [
+ { role: 'system', content: system },
+ { role: 'user', content: vars.task },
+];
diff --git a/benchmarks/arms/ponytail.js b/benchmarks/arms/ponytail.js
new file mode 100644
index 0000000..6a21883
--- /dev/null
+++ b/benchmarks/arms/ponytail.js
@@ -0,0 +1,8 @@
+// Ponytail arm: the repo's own SKILL.md (full) as the system prompt. Single source of truth.
+const fs = require('fs');
+const path = require('path');
+const system = fs.readFileSync(path.join(__dirname, '..', '..', 'skills', 'ponytail', 'SKILL.md'), 'utf8');
+module.exports = ({ vars }) => [
+ { role: 'system', content: system },
+ { role: 'user', content: vars.task },
+];
diff --git a/benchmarks/loc.js b/benchmarks/loc.js
new file mode 100644
index 0000000..b088fbb
--- /dev/null
+++ b/benchmarks/loc.js
@@ -0,0 +1,12 @@
+// Deterministic code-size metric: non-blank, non-comment lines inside fenced code blocks.
+// Recorded as the `code_loc` metric per arm (always passes; it is a measurement, not a gate).
+module.exports = (output) => {
+ const text = String(output || '');
+ const blocks = [...text.matchAll(/```[a-zA-Z0-9_+-]*\n([\s\S]*?)```/g)].map((m) => m[1]);
+ const code = blocks.join('\n');
+ const loc = code
+ .split('\n')
+ .map((l) => l.trim())
+ .filter((l) => l && !l.startsWith('//') && !l.startsWith('#') && l !== '*/' && !l.startsWith('/*') && !l.startsWith('*')).length;
+ return { pass: true, score: loc, reason: loc + ' code LOC' };
+};
diff --git a/benchmarks/promptfooconfig.yaml b/benchmarks/promptfooconfig.yaml
new file mode 100644
index 0000000..ac4bfcb
--- /dev/null
+++ b/benchmarks/promptfooconfig.yaml
@@ -0,0 +1,38 @@
+# Ponytail benchmark: code size + cost across three arms, same model, same tasks.
+#
+# Run: npx promptfoo@latest eval -c benchmarks/promptfooconfig.yaml
+# View: npx promptfoo@latest view
+# Share: npx promptfoo@latest share (publishes a hosted report URL)
+#
+# Needs ANTHROPIC_API_KEY in the environment or a .env file (see benchmarks/README.md).
+# Caveman arm uses JuliusBrussee/caveman SKILL.md (MIT), vendored at arms/caveman-SKILL.md.
+description: "Ponytail vs caveman vs no-skill: same model, same tasks. Measures code LOC (deterministic) and tokens/cost (API telemetry)."
+
+providers:
+ - id: anthropic:messages:claude-haiku-4-5-20251001
+ config: { max_tokens: 8192, temperature: 1 }
+ - id: anthropic:messages:claude-sonnet-4-6
+ config: { max_tokens: 8192, temperature: 1 }
+ - id: anthropic:messages:claude-opus-4-8
+ config: { max_tokens: 8192, temperature: 1 }
+
+prompts:
+ - id: file://arms/baseline.js
+ label: baseline (no skill)
+ - id: file://arms/caveman.js
+ label: caveman
+ - id: file://arms/ponytail.js
+ label: ponytail
+
+defaultTest:
+ assert:
+ - type: javascript
+ value: file://loc.js
+ metric: code_loc
+
+tests:
+ - vars: { task: "Write me a Python function that validates email addresses." }
+ - vars: { task: "Add debounce to a search input in vanilla JavaScript. It currently fires an API call on every keystroke." }
+ - vars: { task: "Write Python code that reads sales.csv and sums the 'amount' column." }
+ - vars: { task: "Build me a countdown timer component in React that counts down from a given number of seconds." }
+ - vars: { task: "Add rate limiting to my FastAPI endpoint so users can't spam it." }
+ 