ScarfBench tests repository-level Java migration agents

IBM introduced ScarfBench as a public benchmark for AI-driven Java framework migration across Jakarta EE, Quarkus, and Spring, checking compilation, startup, runtime behavior, functional equivalence, and architecture preservation. Why it matters: Legacy-modernization agents need more than patch-level benchmarks. Migration quality depends on whether the converted application still runs, behaves correctly, and preserves design intent.

Try this: When evaluating a code-migration agent, borrow the ScarfBench checklist: compile the result, start it, run functional tests, and review whether the architecture changed in unintended ways.