Error discovery turns reviewed outputs into eval rubrics

Hamel Husain and Shreya Shankar show an agent-assisted review loop: inspect real outputs, annotate failures, cluster them into criteria, then apply focused LLM judges. Why it matters: Auto-evals can surface obvious failures, but the video argues that product-specific judgment emerges from reviewing real examples and recording the reasons behind decisions.

Try this: Review 20 recent agent traces or editorial candidates, label each failure in place, and turn the repeated labels into separate pass/fail evals.