🖤☕ Le Café Noir · Opinion
Anthropic's AI now speeds up its own research. Why did we learn it in a stock filing?
Anthropic deserves credit for saying it out loud, but a safety line crossed in an IPO document, with no named safeguards and no outside sign-off, shows why the builders cannot be their own judges.
Builds on Le Café Noir of Thursday Nov 5: The Brakes on AI Are Optional. This Week Proved It
The most important AI safety news of the week was not in a safety report. It was in a stock filing.
On November 10, Anthropic filed a new version of its IPO document. That is the paperwork a company files before it sells shares to the public. In it, Anthropic says it has "made a determination that we have met" the AI research and development threshold in its Responsible Scaling Policy. That policy is the company's own rulebook. It says when stronger safety measures must start.
In plain words, Anthropic believes AI now does enough of its research to roughly double how fast its models improve. That is two years of progress, at the 2018-2024 pace, packed into one year.
On November 2, the same filing only said the company "may imminently meet" this line. Eight days later, the line is crossed. The same document also set the share price: $70 to $78. At the top of the range, counting every share that could exist, that is about $1.4 trillion. Trading could start as soon as November 19, according to reports.
Credit where it is due
Let me be fair first, because Anthropic deserves it.
No other lab has put a sentence like this in a legal filing, where it can be held to its words. The same document warns that after a catastrophic AI incident, "no single insurer, and potentially the insurance industry in the aggregate," may be able to cover the damage. That is not normal sales talk.
There is also outside evidence. On November 5, METR, an independent safety group, published an 11-week report. It found that Anthropic's models now improve about twice as fast as a year ago. So this is not only the company's word.
And I want to say the wonder out loud. This week, a research system running a non-public Anthropic model proved a quantum computing problem that had been open since 2008. A proof checker called Lean confirmed every step, so anyone can verify it. AI helping science move faster is the dream. I still love it.
Now the hard questions
Last week in Le Café Noir, I argued that safety rules written by the builders alone are not enough. I also noted that a lab going public said AI may soon speed up its own making. Bon, "may soon" lasted about a week. This news does not change my mind. It makes the point sharper.
Look at who does what. Anthropic decided, by itself, that the threshold was crossed. Anthropic will decide, by itself, which safeguards follow. No outside body signs off. And today, no new safeguard has been named.
The company says crossing the line means more protection, not a shutdown. It says it does not expect to safely use Claude to "fully autonomously design and build" the next models without real human supervision. It is weighing policy changes "to keep humans in control", and it promises more reports later.
Later. The shares could trade on November 19.
One more detail matters. The decision rests on what Anthropic expects from future models and overall progress, not only on what today's models have shown. Maybe that is prudent. But it also means the public cannot check it. We only see the conclusion.
The money in the room
This is the one pattern where I will take a side: when this much money arrives, safety has to fight harder for time.
Nvidia and Google are each buying $10 billion of Anthropic stock at the IPO price. That is part of a $26 billion private placement that also includes AMD, Broadcom, Micron, Microsoft and TSMC, suppliers and partners of the lab. All seven co-founders will each sell about $250 million in shares, and they keep control. Two new models, Fable 5.2 and Mythos 5.2, launched the day before the filing.
Brian Boland, a former Facebook vice president who lived through Facebook's 2012 IPO, told Business Insider that Anthropic is "not ready to go public". He fears that pressure to grow revenue will push safety work aside, as it did at Facebook. Critics call the warning "IPO lawyering". Supporters call it honest uncertainty. And yes, a scary warning can also sell: our AI is so strong it is dangerous.
I cannot tell you which reading is right. That is exactly the problem. Nobody independent can tell us either.
And here is the paradox. The builder warns, in legal words, that its AI is speeding up its own progress. On 60 Minutes, President Trump said the risk is "not nearly as real as they talk" and "can be managed". He also said he threatened an unnamed AI executive with jail. Only he has told that story. When the people building it sound more worried than the people in charge, I have to ask: what do the builders know that we don't?
There is a small light. In New York, registration under the RAISE Act has begun: big developers must disclose their safety practices and report critical incidents. Australia is reportedly considering mandatory incident reporting with deadlines. Good direction. But a disclosure rule is not a test before launch.
The real enemy is speed. Last week, "may imminently". This week, "met". Next week, maybe shares on the market. Our rules move in years. The machines now move in months.
If a lab says its AI is now speeding up its own progress, should its shares be sold before anyone independent has checked the safeguards?
New to a word in this story? Learn: Proof Checker · Responsible Scaling Policy
Le Café Noir is Lange Cornu's personal opinion. About · All editorials
💬 Comments
Be the first to comment.