← Back to issue12 / 13 · Week of Aug 31, 2026
▶

Machinist measures coding-agent runs

Machinist is a software-factory control plane that queues coding tasks, dispatches workers to agent harnesses, and records run-level metrics such as time, tokens, commands, and outcomes. Why it matters: A shared agent workflow makes prompt, model, and harness changes comparable over time instead of relying on anecdotes. The practical gain is a repeatable path from run data to a specific workflow improvement, such as replacing a routine agent step with a deterministic script.

Try this: Run one recurring coding task through a fixed prompt and capture a trace with task time, token use, commands, and test outcome; compare that baseline with one changed model or harness.

Source
Owain Lewis — I Built A Self-Improving AI Software Factory
View source →

Get the field brief every week.

Important AI developments, useful explanations, and practical resources in one weekly read. Context to understand what matters, with links to the original sources and deeper reading.

Subscribe free →
Free weekly·No spam·Unsubscribe anytime