Langfuse derives eval metrics from traces

Lotte Verheyden's Langfuse Academy article proposes deriving agent eval metrics from failures observed in traces, alongside goal metrics and guardrails for known constraints. It keeps a candidate only when a change in the metric would change a decision. Why it matters: A library of generic checks can miss how a specific agent fails. The article's path is inspectable: review traces, name recurring failures, define a score, and connect that score to a deploy, prompt, or investigation decision.

Try this: Review 30 to 50 agent traces. Record a short failure note and pass/fail for each, cluster the notes into named categories, then create one boolean eval for a category only if its result can change a concrete decision.