← Back to issue2 / 12 · Week of Sep 14, 2026

SlopCodeBench exposes the coding-agent maintenance gap

Dex Horthy’s three-problem SlopCodeBench run revealed requirements one checkpoint at a time and carried regressions forward: Opus 5 led with 4 of 17 strict passes, while no tested model finished a challenge with every inherited test green. Why it matters: An agent that closes isolated tickets can still leave a codebase harder to extend. Before delegating long-running backlog work, teams need evidence that each change preserves earlier behavior and limits the repair burden for the next change.

Try this: Run one representative agent task as three staged requirements against a disposable repo. After each step, rerun every earlier test and compare the diff, failed evals, and repair time before expanding unattended access.

Source
Dex Horthy — Why Software Factories Fail: Benchmarking the new frontier
View source →

Go deeper

  • SlopCodeBench paper ↗

    The benchmark design, 20-problem evaluation, and reported quality-degradation results.

  • SCBench problem set ↗

    Checkpoint specifications, reference solutions, and inherited regression tests used by the benchmark.

Get the field brief every week.

Important AI developments, useful explanations, and practical resources in one weekly read. Context to understand what matters, with links to the original sources and deeper reading.

Subscribe free →
Free weekly·No spam·Unsubscribe anytime