Your coding-agent harness can double the cost of a task

Databricks benchmarked coding agents on reviewed tasks from its multi-million-line codebase. With the same model and thinking effort, it found harness choice could change cost per task by more than 2x while quality stayed the same. Pi sent about 3x less context per turn and often finished in fewer runs. Why it matters: Model pricing is not the unit that teams actually buy. Context replay, tool defaults, and the number of turns determine the cost of a completed, verified task. A cheaper model can lose that comparison if its harness makes it work longer.

Try this: Run one representative task through two harnesses with the same model and effort. Record pass rate, total cost, turns, and context sent per turn in a trace, then compare cost per verified completion.