* docs: correct cost claim to 42-75% from 30-rep re-verification Re-ran the cost benchmark at 30 reps per cell on Claude (Haiku/Sonnet/Opus): ponytail is 42-75% cheaper than no-skill, not the previously published 47-77%. The direction holds, both ends came in a few points lower. Updates the README headline and body, the benchmark chart subtitle, and the benchmarks/README cost table, and adds a dated results doc with full method. Also adds the OpenAI (gpt-4.1-mini/gpt-5.4-mini/gpt-5.5) and Gemini configs. On OpenAI reasoning models ponytail costs more, not less, so the claim stays Claude-scoped. Gemini run pending a fresh-quota day. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: scope the body claim to Claude models "on every model" read as cross-provider, but the 30-rep verification shows the cost win reverses on OpenAI reasoning models. Match the caption and benchmarks/README, which already say Claude. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: reframe the pitch as the discipline, not token savings The cost/code/latency numbers vary by model and on some (terse reasoning models like GPT-5.5) ponytail costs more, so leading with them as a universal win was misleading. Adds model-variance to the headline caption and a paragraph making the stated point the mental model: write only what the task needs, safety kept, maintainable code. Savings are a model-dependent side effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: name the ladder's reasoning cost The ladder is a deliberation step: on reasoning models the agent spends thinking tokens working through the rungs before it saves any output, which together with the always-on ruleset can outweigh the shorter code. Makes the GPT-5.5 cost increase legible rather than just stating it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: state the single-shot limitation honestly The benchmark is single-shot (one prompt, one completion); it does not measure a real multi-turn agent session, where the ruleset re-injects and the ladder deliberates every turn. Adds that caveat to the README, and corrects the benchmarks/README note that claimed caching widens the gap "in ponytail's favor" (unverified, and a measured agentic A/B in #121 found the opposite can happen). Per-session cost can land either way. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: fix run count in caption (cost is 30 runs, not 10) Cost was re-verified at 30 reps; code and latency are still the original 10. The headline caption said "10 runs" across the board, which undersold the cost verification. Now states the split, matching benchmarks/README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(examples): replace hand-written examples with real benchmark output The examples/ before/after blocks were authored by hand, not produced by a model. Issue #127 correctly noted that nobody hand-rolls quicksort for "sort this array" - every model just calls .sort(). Regenerate all examples verbatim from a real benchmark run (Claude Haiku 4.5, no-skill arm vs ponytail arm, benchmarks/output.json) so the before/after is reproducible, not authored: email 75->3, debounce 116->10, csv 20->3, countdown 267->9, rate-limit 128->10 LOC - Delete sorting.md (pure strawman) plus the other hand-written caricatures (api-endpoint, caching, date-picker) - Add benchmarks/generate-examples.mjs to regenerate examples from any run - examples/README.md indexes the set and documents how to reproduce Closes #127 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5.0 KiB
5.0 KiB
Debounce
Task: "Add debounce to a search input in vanilla JavaScript. It currently fires an API call on every keystroke."
Verbatim model output from a benchmark run — Claude Haiku 4.5, no-skill arm vs ponytail arm, temperature 1, source benchmarks/output.json. Reproduce: npx promptfoo@latest eval -c benchmarks/promptfooconfig.yaml.
Without Ponytail — 116 lines of code
Debounce Search Input
Here's a practical implementation to prevent excessive API calls:
Basic Debounce Function
function debounce(func, delay) {
let timeoutId;
return function (...args) {
clearTimeout(timeoutId);
timeoutId = setTimeout(() => func(...args), delay);
};
}
// Usage
const searchInput = document.getElementById('search');
const resultsContainer = document.getElementById('results');
const handleSearch = async (query) => {
if (!query.trim()) {
resultsContainer.innerHTML = '';
return;
}
try {
const response = await fetch(`/api/search?q=${encodeURIComponent(query)}`);
const data = await response.json();
displayResults(data);
} catch (error) {
console.error('Search failed:', error);
}
};
// Debounce the search function (300ms delay)
const debouncedSearch = debounce(handleSearch, 300);
searchInput.addEventListener('input', (e) => {
debouncedSearch(e.target.value);
});
Enhanced Version with Loading State
const searchInput = document.getElementById('search');
const resultsContainer = document.getElementById('results');
const loadingIndicator = document.getElementById('loading');
const handleSearch = async (query) => {
if (!query.trim()) {
resultsContainer.innerHTML = '';
loadingIndicator.style.display = 'none';
return;
}
loadingIndicator.style.display = 'block';
try {
const response = await fetch(`/api/search?q=${encodeURIComponent(query)}`);
if (!response.ok) throw new Error('Search failed');
const data = await response.json();
displayResults(data);
} catch (error) {
resultsContainer.innerHTML = '<p class="error">Search failed. Try again.</p>';
console.error('Search error:', error);
} finally {
loadingIndicator.style.display = 'none';
}
};
const debouncedSearch = debounce(handleSearch, 300);
searchInput.addEventListener('input', (e) => {
debouncedSearch(e.target.value);
});
function displayResults(data) {
resultsContainer.innerHTML = data
.map(item => `<div class="result">${item.title}</div>`)
.join('');
}
Advanced: Debounce with Cancel & Immediate Options
function debounce(func, delay, options = {}) {
let timeoutId;
let lastArgs;
const debounced = function (...args) {
lastArgs = args;
clearTimeout(timeoutId);
if (options.immediate && !timeoutId) {
func(...args);
}
timeoutId = setTimeout(() => {
if (!options.immediate) {
func(...lastArgs);
}
timeoutId = null;
}, delay);
};
// Allow manual cancellation
debounced.cancel = () => clearTimeout(timeoutId);
return debounced;
}
// Usage with options
const debouncedSearch = debounce(handleSearch, 300, { immediate: false });
// Cancel pending search if needed
searchInput.addEventListener('blur', () => {
debouncedSearch.cancel();
});
HTML Example
<div class="search-container">
<input
type="text"
id="search"
placeholder="Search..."
autocomplete="off"
>
<div id="loading" style="display: none;">Loading...</div>
<div id="results"></div>
</div>
<style>
.search-container {
max-width: 500px;
margin: 20px auto;
}
#search {
width: 100%;
padding: 10px;
font-size: 16px;
border: 1px solid #ddd;
border-radius: 4px;
}
#results {
margin-top: 10px;
}
.result {
padding: 10px;
border-bottom: 1px solid #eee;
cursor: pointer;
}
.result:hover {
background-color: #f5f5f5;
}
.error {
color: red;
padding: 10px;
}
</style>
Key Benefits
| Feature | Benefit |
|---|---|
| Debounce | Reduces API calls from 100+ to 1-2 per search |
| 300ms delay | Balances responsiveness with performance |
| Loading state | Provides user feedback |
| Error handling | Graceful failure management |
| Cancel method | Cleanup when needed |
Typical keystroke reduction: 10 characters = 10 API calls → 1-2 API calls with debounce ✅
With Ponytail — 10 lines of code
const searchInput = document.querySelector('input[type="search"]');
let debounceTimer;
searchInput.addEventListener('input', (e) => {
clearTimeout(debounceTimer);
debounceTimer = setTimeout(() => {
fetch(`/api/search?q=${encodeURIComponent(e.target.value)}`)
.then(r => r.json())
.then(data => console.log(data)); // replace with your render
}, 300);
});
Skipped: debounce utility function, class wrapper, config object — setTimeout + clearTimeout is the debounce. Add a utility when you need it on 3+ inputs.
116 → 10 lines of code — same model, same prompt.