Bar chart of mean LOC per task by arm (Haiku 4.5) in the Numbers section, same
system-gray style as the single-shot chart so it reads on both GitHub themes.
The old single-shot chart stays in the collapsed "isolated generation" section.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebuild the benchmark to the standard #126 asked for: real headless Claude Code
sessions (not a bare model) editing a real public repo
(tiangolo/full-stack-fastapi-template @ cd83fc1, MIT), fair arms (baseline,
caveman, ponytail, and the "YAGNI + one-liners" prompt), n=4, Haiku 4.5. LOC is
the git diff; the safety tasks execute the produced code against adversarial
input.
Results: ponytail -54% LOC mean (up to -94% on over-build features like the
date/color picker), -22% tokens, -20% cost, -27% time, and never more than
baseline; 100% safe vs the one-liner prompt's 95% (it dropped a path-traversal
guard once). caveman writes less code but spends more tokens.
Also fixes a baseline-contamination bug (the ponytail plugin's SessionStart hook
fired on every arm; now isolated with --setting-sources project,local + per-arm
--plugin-dir) and a Windows subprocess-timeout hang.
Lead both READMEs with the agentic numbers; demote the single-shot 80-94% to a
labelled "isolated generation" note; supersede the contaminated 2026-06-17
writeup. Dead react-app fixture left untracked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: correct cost claim to 42-75% from 30-rep re-verification
Re-ran the cost benchmark at 30 reps per cell on Claude (Haiku/Sonnet/Opus):
ponytail is 42-75% cheaper than no-skill, not the previously published 47-77%.
The direction holds, both ends came in a few points lower. Updates the README
headline and body, the benchmark chart subtitle, and the benchmarks/README cost
table, and adds a dated results doc with full method.
Also adds the OpenAI (gpt-4.1-mini/gpt-5.4-mini/gpt-5.5) and Gemini configs. On
OpenAI reasoning models ponytail costs more, not less, so the claim stays
Claude-scoped. Gemini run pending a fresh-quota day.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: scope the body claim to Claude models
"on every model" read as cross-provider, but the 30-rep verification shows
the cost win reverses on OpenAI reasoning models. Match the caption and
benchmarks/README, which already say Claude.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: reframe the pitch as the discipline, not token savings
The cost/code/latency numbers vary by model and on some (terse reasoning
models like GPT-5.5) ponytail costs more, so leading with them as a universal
win was misleading. Adds model-variance to the headline caption and a paragraph
making the stated point the mental model: write only what the task needs,
safety kept, maintainable code. Savings are a model-dependent side effect.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: name the ladder's reasoning cost
The ladder is a deliberation step: on reasoning models the agent spends
thinking tokens working through the rungs before it saves any output, which
together with the always-on ruleset can outweigh the shorter code. Makes the
GPT-5.5 cost increase legible rather than just stating it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: state the single-shot limitation honestly
The benchmark is single-shot (one prompt, one completion); it does not measure
a real multi-turn agent session, where the ruleset re-injects and the ladder
deliberates every turn. Adds that caveat to the README, and corrects the
benchmarks/README note that claimed caching widens the gap "in ponytail's
favor" (unverified, and a measured agentic A/B in #121 found the opposite can
happen). Per-session cost can land either way.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: fix run count in caption (cost is 30 runs, not 10)
Cost was re-verified at 30 reps; code and latency are still the original 10.
The headline caption said "10 runs" across the board, which undersold the cost
verification. Now states the split, matching benchmarks/README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a caveat under the headline numbers: they are per-task code/latency/cost on the Claude API, not a plan quota promise. Prevents the misread in #111.
Adds .openclaw/skills/ (ponytail + review/audit/debt/help) generated from the canonical skills/ (verbatim body, no drift), a generator script, and a drift test. Verified live: loads as Ready in OpenClaw 2026.6.6.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents the Antigravity CLI install, the global default-level config, and the OpenCode absolute-path option. Addresses #58, #64, #71.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude Code runs hooks via a non-interactive /bin/sh. On setups where node
isn't on that shell's PATH (Nix/nix-darwin, nvm, fnm), every prompt errored
with "/bin/sh: node: command not found". Guard each hook command so it runs
node only when present and exits 0 otherwise, no more per-prompt noise. The
slash-command skills are unaffected; only the always-on activation needs node.
Document the requirement in the README install section.
Closes#51.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The benchmark harness hardcoded `python`, which is missing on macOS and
many Linux images. Probe python3 first, add npm test, and run checks in
GitHub Actions so regressions are caught on every PR.
Co-authored-by: Cursor <cursoragent@cursor.com>
The badge had fallen behind: it stayed at 11 when Antigravity and the VS Code
Codex extension were added, and Copilot CLI is now a full plugin host too. 13
distinct agent rows in docs/agent-portability.md (excluding the generic
fallback).
* Add GitHub Copilot plugin and marketplace manifests for Ponytail
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add Copilot hook adapters and plugin data runtime precedence
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Document Copilot plugin install flow and instruction fallback mode
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix Copilot hooks for native output context and state-only mode tracking
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: add Copilot CLI namespaced command examples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Collapse Copilot hooks into shared activate/mode-tracker
The Copilot hook files duplicated ponytail-activate.js and
ponytail-mode-tracker.js, differing only in output shape. Move that
difference into writeHookOutput (isCopilot branch) and point
copilot-hooks.json at the shared hooks. Deletes both forks (-73 lines).
Refs #1
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Align Copilot manifest version to 4.4.0 with cross-manifest parity test
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Make Copilot and Codex host detection exclusive in runtime output routing
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add Copilot debt command validation with a pull request acceptance checklist
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Removed PR template
* Drop tautological copilot command-form test
The namespaced-form assertion built '/ponytail:ponytail-debt' from two
constants and compared it to itself — it tests string concatenation, not
wiring. The file-exists check above already catches a renamed manifest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The header showed logo.png (black on transparent), which nearly vanishes in
GitHub's dark theme. Wrap it in a <picture> so dark-theme viewers get the
contoured logo-dark.png and light-theme viewers keep the original.
* docs: add a Commands reference to the README
The commands were only mentioned scattered through prose, and two install
blurbs had gone stale (OpenCode omitted /ponytail-debt, Gemini omitted audit
and debt). Add one canonical Commands table (all five commands + what each
does + which hosts support them) and point the install blurbs at it so they
stop drifting as commands are added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: add pi to the portability table, de-stale the Gemini row
pi was a supported integration (pi-extension, README install, registers all
the commands) but had no row in the Supported Adapters table. The Gemini row
also enumerated an outdated command list; point it at commands/*.toml
generically so it stops drifting.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the last gap from the field review: deferral creep. /ponytail-debt
greps the repo for `ponytail:` comment markers and prints a ledger
(file:line, what was simplified, ceiling, upgrade trigger), flagging any
marker with no trigger as the rot risk. One-shot, reports only.
Full parity like ponytail-audit: skill + commands/.toml + .opencode/.md + pi
registerCommand (+ test) + agent-portability + README.
Verified: tests 32/32 (pi command list updated), rule check green, the scan
finds the repo's real markers, and a live end-to-end run produced a correct
ledger (2 markers, 1 no-trigger, prose/examples excluded).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both read AGENTS.md, which the repo already ships, so ponytail works
from the repo root with no extra setup. Add agent-portability rows and a
README note. Instruction-tier (no /ponytail levels or hooks).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot CLI already reads AGENTS.md and .github/copilot-instructions.md
(both shipped), plus a global ~/.copilot/copilot-instructions.md. Add an
agent-portability row and a README note. Instruction-tier only: no
/ponytail levels or hooks.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add Gemini CLI support
Add a thin Gemini CLI extension adapter (closes#22). The manifest points
contextFileName at the existing AGENTS.md for always-on rules and reuses the
repo's commands/*.toml (/ponytail, /ponytail-review) and skills/, which Gemini
CLI auto-discovers — no rule text is duplicated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: read manifest inside cases for a clean RED
The manifest was parsed at module scope, so reverting the adapter crashed the
whole test file at load (ENOENT) and collapsed all four cases into one
unreadable failure. Read it inside each case via a helper that asserts existence
first, so a missing or malformed manifest surfaces as clear per-case assertion
failures instead of a stack trace.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add ponytail-audit skill
Whole-codebase audit for over-engineering. Scans a repository and produces a
ranked report of things to delete, simplify, or replace with stdlib/native
equivalents.
* Drop counting-by-number heuristics: remove Phase 5 (structural audit) and TODO-age heuristic per review
* feat: slim ponytail-audit and add cross-platform parity
Rewrite the ponytail-audit skill to ponytail-review's lean shape (35 lines,
was 111): same five tags, repo-wide scope, ranked output. Drop the
prescriptive phase walkthrough, sampling rules, report template, and the
license frontmatter that no other skill carries.
Add the adapters the skill was missing so it ships on every supported
platform the way ponytail-review does:
- commands/ponytail-audit.toml
- .opencode/command/ponytail-audit.md
- pi-extension registerCommand (+ test)
- agent-portability and README entries
Co-authored-by: Alexander Brandt <github@a13x.de>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Emeriko <dietrich.gebert@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swap em dashes for commas/colons/periods in the README, skills, AGENTS.md and
its five rule copies, examples, command files, and benchmark README. Rule
copies stay in sync (same edit applied to all) and the invariant guard passes.
Left untouched on purpose: the vendored caveman SKILL.md (verbatim third-party
text), the dated benchmark writeups in results/ (historical records), and
.js code comments.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Commit a promptfoo harness (config + arm prompts + LOC metric + vendored
caveman SKILL) so anyone can re-run the comparison: no-skill vs caveman vs
ponytail, across Haiku / Sonnet / Opus, 10 runs per cell, median reported.
Replace the old unreproducible 6-task chart with assets/benchmark-3model.svg
from this run, and reframe the README to the reproducible numbers: ponytail
writes 80-94% less code, costs 47-77% less, and runs 3-6x faster than a
no-skill agent on every model. benchmarks/README.md carries the median tables
and the reproduce command. Drops nothing that is not measured.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the Codex adapter: marketplace.json, .codex-plugin/plugin.json,
declarative hooks.json with Windows command variants, and a shared
hooks/ponytail-runtime.js that switches state path and output format
between Claude and Codex (gated on PLUGIN_DATA, so the Claude path is
unchanged). Bumps plugin version to 4.1.0 and ships tests/hooks.test.js.
README adapters block resolved to keep the Codex @ponytail invocation note
together with Kiro (#6) and the drift-check Development section (#3).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Six no-skill control arms re-run through the same harness so all three
arms share one model. README Numbers section and chart now cite the
complete dataset: -47% tokens, 3x faster, 490 vs 3,629 LOC, extension
96 vs 1,115 lines, probes green everywhere. Em dashes removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Skill preaching minimalism was 2x caveman length. Smaller file cuts
per-read and per-session-injection cost. Benchmark: beats caveman on
all areas now — 135.7k vs 138.4k tokens, 127s vs 136s, 47 vs 117 loc.
v1 lost to caveman on tokens/time despite minimal code: it wrote
essays defending each simplification. v2 caps explanation at three
lines and ships the lazy version instead of stalling on necessity
questions. Benchmark: 136.6k tok vs caveman 138.4k, code 47 vs 117
lines across 5 tasks.
