fix(benchmarks): count unfenced code, ASCII-safe output, refresh llama3.2 results (#67)

Fixes the local benchmark LOC counter (counted only fenced code, scored bare output 0), makes summary output ASCII-safe (a Unicode arrow crashed the script on Windows cp1252), gitignores generated artifacts, and refreshes the llama3.2 writeup with n=5 data showing the LOC effect is within the noise floor. Follow-up to #63. Verified live.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DietrichGebert
2026-06-15 16:22:32 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 386f95734a
commit 2e6a93765a
4 changed files with 50 additions and 36 deletions
+3 -2
View File
@@ -1,9 +1,10 @@
// Deterministic code-size metric: non-blank, non-comment lines inside fenced code blocks.
// Deterministic code-size metric: non-blank, non-comment lines of code. Counts
// fenced blocks, or the whole response when the model emitted bare code unfenced.
// Recorded as the `code_loc` metric per arm (always passes; it is a measurement, not a gate).
module.exports = (output) => {
const text = String(output || '');
const blocks = [...text.matchAll(/```[a-zA-Z0-9_+-]*\n([\s\S]*?)```/g)].map((m) => m[1]);
const code = blocks.join('\n');
const code = blocks.length ? blocks.join('\n') : text;
const loc = code
.split('\n')
.map((l) => l.trim())