wangjunxiao/ponytail
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: refine ruleset from a full-project field review (#39)

Browse Source 
* feat: refine ruleset from a full-project field review

A reviewer ran ponytail across a 9-phase rewrite (protocol, PC app, simulator,
RPi daemon, ESP32 firmware) and flagged three gaps. All three land in SKILL.md
and propagate to AGENTS.md + the rule copies:

- Promote the one-runnable-check rule to a headline ("Lazy code without its
  check is unfinished"), enforced as a check-rule-copies invariant.
- Hardware carve-out in "When NOT to be lazy": a real device is never the spec
  ideal (clock drift, sensor offset), leave the calibration knob.
- Clarify the Output rule: explanation the user explicitly asked for is not
  debt, only unrequested prose is.

Fallback instructions kept in sync. Rule-copy check + tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: add a behavior gate proving the refinements actually fire

The refinements were verified as injected text, but injected != behavioral.
This adds a behavior eval that probes each refined rule on a task that should
trigger it:

- hardware    -> does the output leave a calibration knob?
- explanation -> when a write-up is explicitly requested, is it given in full?
- onecheck    -> is a runnable check left behind?

benchmarks/behavior.yaml runs the probes (baseline vs ponytail arm); the
grader benchmarks/behavior.js is proven by tests/behavior.test.js (8 cases,
RED/GREEN, no API key, runs in CI). Live-confirmed: the model under the
current ruleset passes all three gates, graded by the same grader.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DietrichGebert
2026-06-15 02:02:50 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent b545f1536a
commit f3da910b4f
12 changed files with 202 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hooks/ponytail-instructions.js
+4 -3
View File
@@ -58,11 +58,12 @@ function getFallbackInstructions(mode) {
'Mark intentional simplifications with a `ponytail:` comment — a shortcut with a known ceiling names the ceiling and the upgrade path in the comment.\n\n' +
'## Output\n\n' +
'Code first. Then at most three short lines: what was skipped, when to add it. ' +
'If the explanation is longer than the code, delete the explanation.\n\n' +
'If the explanation is longer than the code, delete the explanation. ' +
'Explanation the user explicitly asked for is not debt, give it in full.\n\n' +
'## When NOT to be lazy\n\n' +
'Never simplify away: input validation at trust boundaries, error handling that prevents data loss, ' +
'security measures, accessibility basics, anything the user explicitly asked to keep. ' +
'Non-trivial logic leaves ONE runnable check behind (assert-based demo/self-check or one small test file; no frameworks). Trivial one-liners need no test.\n\n' +
'security measures, accessibility basics, the calibration real hardware needs (the platform is never the spec ideal), anything the user explicitly asked to keep. ' +
'Lazy code without its check is unfinished: non-trivial logic leaves ONE runnable check behind (assert-based demo/self-check or one small test file; no frameworks). Trivial one-liners need no test.\n\n' +
'## Boundaries\n\n' +
'Ponytail governs what you build, not how you talk. "stop ponytail" or "normal mode": revert. Level persists until changed or session end.';
}