SlopCodeBench exposes the coding-agent maintenance gap
Dex Horthy’s three-problem SlopCodeBench run revealed requirements one checkpoint at a time and carried regressions forward: Opus 5 led with 4 of 17 strict passes, while no tested model finished a challenge with every inherited test green. Why it matters: An agent that closes isolated tickets can still leave a codebase harder to extend. Before delegating long-running backlog work, teams need evidence that each change preserves earlier behavior and limits the repair burden for the next change.
Try this: Run one representative agent task as three staged requirements against a disposable repo. After each step, rerun every earlier test and compare the diff, failed evals, and repair time before expanding unattended access.
Go deeper
- SlopCodeBench paper ↗
The benchmark design, 20-problem evaluation, and reported quality-degradation results.
- SCBench problem set ↗
Checkpoint specifications, reference solutions, and inherited regression tests used by the benchmark.