DSPy prompt eval loop for SQL agents

Simon Willison used DSPy to evaluate Datasette Agent SQL prompts and found that an instruction meant to save context led the agent to guess column names and retry failed SQL. Why it matters: Agent prompt changes can introduce hidden regressions. Treating prompts as testable units helps teams optimize context use without degrading answer quality.

Try this: Before shortening a system prompt, capture a few realistic failure cases, define expected tool-use behavior, and compare prompt variants with an eval harness.