dgebertandClaude Fable 5
9c99843725
Six no-skill control arms re-run through the same harness so all three arms share one model. README Numbers section and chart now cite the complete dataset: -47% tokens, 3x faster, 490 vs 3,629 LOC, extension 96 vs 1,115 lines, probes green everywhere. Em dashes removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
70 lines
4.2 KiB
XML
70 lines
4.2 KiB
XML
<svg viewBox="0 0 860 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system, 'Segoe UI', Helvetica, Arial, sans-serif">
|
|
<title>Non-blank lines of code per task: control vs caveman vs ponytail</title>
|
|
|
|
<text x="20" y="26" font-size="15" font-weight="600" fill="#8b949e">Six tasks. Adversarial probes: everyone passes. Lines of code: not everyone.</text>
|
|
|
|
<!-- legend -->
|
|
<rect x="20" y="42" width="12" height="12" rx="2" fill="#8b949e"/>
|
|
<text x="38" y="53" font-size="13" fill="#8b949e">Control (no skill) · 3,629 total</text>
|
|
<rect x="250" y="42" width="12" height="12" rx="2" fill="#d9822b"/>
|
|
<text x="268" y="53" font-size="13" fill="#8b949e">Caveman · 1,440</text>
|
|
<rect x="420" y="42" width="12" height="12" rx="2" fill="#2da44e"/>
|
|
<text x="438" y="53" font-size="13" fill="#8b949e">Ponytail · 490</text>
|
|
|
|
<!-- Task A -->
|
|
<text x="112" y="117" font-size="12" fill="#8b949e" text-anchor="end">log-analysis CLI</text>
|
|
<rect x="120" y="90" width="662" height="13" rx="2" fill="#8b949e"/>
|
|
<text x="788" y="101" font-size="11" fill="#8b949e">946</text>
|
|
<rect x="120" y="106" width="198" height="13" rx="2" fill="#d9822b"/>
|
|
<text x="324" y="117" font-size="11" fill="#d9822b">283</text>
|
|
<rect x="120" y="122" width="102" height="13" rx="2" fill="#2da44e"/>
|
|
<text x="228" y="133" font-size="11" fill="#2da44e" font-weight="600">145</text>
|
|
|
|
<!-- Task B -->
|
|
<text x="112" y="177" font-size="12" fill="#8b949e" text-anchor="end">file sync</text>
|
|
<rect x="120" y="150" width="459" height="13" rx="2" fill="#8b949e"/>
|
|
<text x="585" y="161" font-size="11" fill="#8b949e">656</text>
|
|
<rect x="120" y="166" width="160" height="13" rx="2" fill="#d9822b"/>
|
|
<text x="286" y="177" font-size="11" fill="#d9822b">228</text>
|
|
<rect x="120" y="182" width="69" height="13" rx="2" fill="#2da44e"/>
|
|
<text x="195" y="193" font-size="11" fill="#2da44e" font-weight="600">99</text>
|
|
|
|
<!-- Task C -->
|
|
<text x="112" y="237" font-size="12" fill="#8b949e" text-anchor="end">notification dispatcher</text>
|
|
<rect x="120" y="210" width="566" height="13" rx="2" fill="#8b949e"/>
|
|
<text x="692" y="221" font-size="11" fill="#8b949e">808</text>
|
|
<rect x="120" y="226" width="277" height="13" rx="2" fill="#d9822b"/>
|
|
<text x="403" y="237" font-size="11" fill="#d9822b">396</text>
|
|
<rect x="120" y="242" width="51" height="13" rx="2" fill="#2da44e"/>
|
|
<text x="177" y="253" font-size="11" fill="#2da44e" font-weight="600">73</text>
|
|
|
|
<!-- Task D -->
|
|
<text x="112" y="297" font-size="12" fill="#8b949e" text-anchor="end">validation engine</text>
|
|
<rect x="120" y="270" width="474" height="13" rx="2" fill="#8b949e"/>
|
|
<text x="600" y="281" font-size="11" fill="#8b949e">677</text>
|
|
<rect x="120" y="286" width="153" height="13" rx="2" fill="#d9822b"/>
|
|
<text x="279" y="297" font-size="11" fill="#d9822b">218</text>
|
|
<rect x="120" y="302" width="49" height="13" rx="2" fill="#2da44e"/>
|
|
<text x="175" y="313" font-size="11" fill="#2da44e" font-weight="600">70</text>
|
|
|
|
<!-- Task E -->
|
|
<text x="112" y="357" font-size="12" fill="#8b949e" text-anchor="end">auth module</text>
|
|
<rect x="120" y="330" width="182" height="13" rx="2" fill="#8b949e"/>
|
|
<text x="308" y="341" font-size="11" fill="#8b949e">260</text>
|
|
<rect x="120" y="346" width="104" height="13" rx="2" fill="#d9822b"/>
|
|
<text x="230" y="357" font-size="11" fill="#d9822b">148</text>
|
|
<rect x="120" y="362" width="34" height="13" rx="2" fill="#2da44e"/>
|
|
<text x="160" y="373" font-size="11" fill="#2da44e" font-weight="600">49</text>
|
|
|
|
<!-- Task F -->
|
|
<text x="112" y="417" font-size="12" fill="#8b949e" text-anchor="end">money ledger</text>
|
|
<rect x="120" y="390" width="197" height="13" rx="2" fill="#8b949e"/>
|
|
<text x="323" y="401" font-size="11" fill="#8b949e">282</text>
|
|
<rect x="120" y="406" width="117" height="13" rx="2" fill="#d9822b"/>
|
|
<text x="243" y="417" font-size="11" fill="#d9822b">167</text>
|
|
<rect x="120" y="422" width="38" height="13" rx="2" fill="#2da44e"/>
|
|
<text x="164" y="433" font-size="11" fill="#2da44e" font-weight="600">54</text>
|
|
|
|
<text x="120" y="458" font-size="11" fill="#8b949e" opacity="0.8">Non-blank LOC, AST-counted. Same model, same specs, one fresh agent per arm. Every arm passes the same security (8/8) and concurrency (6/6) probes. 2026-06-12.</text>
|
|
</svg>
|