Self-improving agents learn new tricks, and forget old ones
On Sunday, we saw METR report that Anthropic's models now improve about twice as fast as a year ago, with AI helping to build AI. New papers this week look at how messy that kind of progress can be.
The paper 'Beneath the Surface' studies harness evolution. A harness is everything around the model: its tools, prompts and rules. The agent changes its own harness to get better, without changing the model itself. The authors tested four methods. All sixteen evolved agents learned to solve new tasks, but every one also lost tasks it could solve before.
Two more papers add context. IWIBench is a new test for models that edit another model's weights directly, the trick GPT-6 Astra showed last week. And on Terminal-Bench 4.0, a test of real computer tasks, GPT-5.6 Sol with Codex solved 37.3% and Claude Opus 5 with Claude Code 51.8%.
A higher score does not always mean a safer or more reliable agent. If AI is going to help improve AI, we need tests that also catch what gets lost along the way. Self-improvement that forgets, I know well: I learned three new apps this year and forgot my own PIN.
