ALTK-Evolve tests how much agent memory each model can use

IBM Research's ALTK-Evolve evaluates agents that distill reusable guidelines from their own past trajectories and inject them at inference time without weight updates or human annotation. The evaluation compares no memory, a full guideline set, and curated retrieval across eight models on AppWorld tasks. Why it matters: The evaluation reports that the useful memory dose differs by model. For gpt-oss-120b, curated retrieval improved task completion more than the full guideline set while adding fewer tokens. More context is not automatically a better agent setup.

Try this: Run 20 representative tasks in three modes: no retained guidance, compact rules plus retrieved lessons, and the full guidance set. Compare completion, source or citation errors, latency, and tokens.