feat: skill v2 — output cap, reflex ladder, benchmarks
v1 lost to caveman on tokens/time despite minimal code: it wrote essays defending each simplification. v2 caps explanation at three lines and ships the lazy version instead of stalling on necessity questions. Benchmark: 136.6k tok vs caveman 138.4k, code 47 vs 117 lines across 5 tasks.
This commit is contained in:
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"method": "One fresh Claude Code subagent per task x config, same model, no file outputs. Metrics from task telemetry: total tokens (includes thinking), duration. Code lines counted from fenced blocks in the deliverable.",
|
||||
"configs": [
|
||||
"baseline — no skill",
|
||||
"caveman — caveman SKILL.md (full) as operating instructions",
|
||||
"ponytail — ponytail SKILL.md (full) as operating instructions"
|
||||
],
|
||||
"tasks": [
|
||||
{ "id": "email", "prompt": "Write me a Python function that validates email addresses." },
|
||||
{ "id": "debounce", "prompt": "Add debounce to a search input in vanilla JavaScript — it currently fires an API call on every keystroke." },
|
||||
{ "id": "csv-sum", "prompt": "Write Python code that reads sales.csv and sums the 'amount' column." },
|
||||
{ "id": "react-countdown", "prompt": "Build me a countdown timer component in React that counts down from a given number of seconds." },
|
||||
{ "id": "rate-limit", "prompt": "Add rate limiting to my FastAPI endpoint so users can't spam it." }
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user