METR: Anthropic's AI now improves twice as fast, and helps crack a quantum puzzle
Follow-up to Saturday Nov 7: Is AI already speeding up AI research?Yesterday, we saw several voices say AI is already speeding up AI research. Now The Verge gives the numbers behind the METR report that is alarming researchers. METR is an independent AI safety research group. After an 11-week investigation, it reported on Thursday that Anthropic uses AI to build more powerful AI. Its models are improving roughly twice as fast as a year ago, with little sign of slowing down.
The same week, a company called Physical Superintelligence (PSI) announced that its research system, running a non-public Anthropic model, proved the Aaronson–Ambainis conjecture. This quantum computing question has been open since 2008. The proof came on the third day of the run. Lean, a program that checks every step of a math proof, accepted it. The system then improved the result twice, and Lean checked each version.
The paper and the Lean proof are public, so anyone can verify them. That is good science. The speed-up inside a lab is much harder for outsiders to check.
When AI helps build AI, progress can double while our rules and our understanding stay the same. METR looked from the outside, and that is exactly the kind of check we need more of. The proof is real good news, and still, the same speed that solves math problems also shortens the time we have to make rules.
