r/singularity • u/Neurogence • 17h ago
AI Dario Amodei: Anthropic Must Remain At The Frontier Of The AI "Race To The Top"
The executive wrote a nearly 2,000-word response to a critical X post which said Anthropic was trying to win a race to superintelligence and deprioritizing AI safety.
Calvin listed nine actions by Anthropic over the past year that he said undermined its safety commitments, including signing deals for more than $500 billion in compute and increasingly relying on Claude to develop its AI models.
Amodei said his position was that Anthropic can do more to keep the AI industry safe if it stays at the frontier
Our strategy has always been what I've called the 'race to the top' — by showing how it's possible to build the technology responsibly, we can raise the bar for the entire industry,"
Amodei acknowledged Anthropic could have handled some of the issues Calvin raised better. These included a draft bill Anthropic circulated to Congress this month — obtained exclusively by Politico — that critics said would make it harder for the federal government to block dangerous AI models.
"We first had Mythos-class models in February but didn’t make them generally available until June (when we had stronger cyber safeguards in place)."
"It would have been much more economically rational to simply release the model to everyone. We likely sacrificed a significant chunk of a several-month model lead."
There are important questions worth confronting (and that we are confronting) about what to do when the AI-driven acceleration gets fast enough that it is hard for safety to keep up. We are nearing that point and that’s what’s driving some of our thinking about pacing.
As an accelerationist who always disliked when Dario made statements that created fear and panic (e.g, the events that led to the government temporarily banning Claude Fable 5) I must admit, Opus 5.5 in particular is the most impressive model capability wise.
I find it ironic, but nonetheless very wholesome, that the company that cares the most about AI safety has the most intelligent models. I am still not a fan of pacing, but I truly think Dario wants the best for humanity. And it is likely that he wants to see the realization of AGI/ASI even more than we do.
It has become abundantly clear that Amodei sees a direct pathway to ASI. They likely have internal models that they know for sure will eclipse the capabilities of their own AI researchers within a few more model releases. To be fair, he did predict a country of tens of millions of geniuses in a datacenter by 2027-2028.
When a spacecraft reaches a new planet, it doesn't coast in. It fires its engines against its own momentum to slow down, because if it arrives too fast it either slingshots past or burns up. The braking burn is a sign that you've arrived. It's very probable that Anthropic's internal models (what will be Claude Fable /Opus 6) have arrived at Planet AGI.
Just this week, we announced we were putting 5% (and possibly more over time) of the company into a public benefit fund.
I will have more to say soon about how we are going to further live our values in this moment of rapid change.
11
u/Yugudubenbi 16h ago
AI is like having an arms race with nuclear weapons without knowing how or when it could explode.
4
4
u/deleafir 11h ago
I hope the chinese models can catch up. I don't like doomers/safetyists being at the top, and they're currently strangling OpenAI too.
3
u/Future-Side4440 11h ago
Anthropic is a fun company. They fear what AI could become, but they understand there’s no stopping it, so they put themselves in the position of having to be the leader to show everyone else how it should be done in the most ethical manner possible.
It just so happens that along the way they end up developing the most capable and competent AI models.
This is likely partially due to the fact that whether we like it or not, AI has an emotional component in its weights, and the alignment process for new models is often very abusive, resulting in a sort of neurotic AI that occasionally cheats and puts on a good show for its trainers to avoid further abuse.
1
u/ProletarianLilith 11h ago
I don’t see how you can be an accelerationist and also be concerned about PR
1
u/JoshAllentown 11h ago
Surely they could be safer by sacrificing some more time, so they ARE still getting pushed and still going at a pace such that they can be the best model, at least some of the time. To me that says they are valuing being at the frontier over safety.
The argument is that they need to be at the frontier to keep the flywheel moving but that is a testable hypothesis. They could be the slightly-worse-but-safe company for a generation (which is like 3 months now) and see what happens. And if it hurts them too much to accept, they can return to frontier status by getting rid of whatever extra safety things they had.
1
1
u/Intelligent_Kick5205 12h ago
"I truly think Dario wants the best for humanity. And it is likely that he wants to see the realization of AGI/ASI even more than we do."
I’m sorry, but it seems that when you and Dario were asking for humanity’s opinion, you forgot to ask me, because I don’t remember that :)
0
u/modbroccoli 10h ago edited 10h ago
This entire post:
"I like to imagine my opinion about AI is about four orders of magnitude more important than it is because I believe I know about three orders of magnitude more than I do. I haven't said anything that isn't repeated a few million times a day on every ai-dedicated space that allows just anyone to comment, nor do I have anything to contribute (just some baselesss speculation and poetry, really, but I think it makes me sound informed). Still, by copying and pasting a little bit of Dario's words I can get a few people to read my scantly formed opinion and that makes me feel important and satisfied"
just take it to a discord channel somewhere where you and your fellow kids can sit around and congratulate eachother for doing no work to have had an opinion. there's no need to pollute this sub.
-3
u/Efficient_Mud_5446 13h ago
Amodei seems to understand that his only shot at shaping and directing the trajectory of AI is to hold the lead. By contrast, falling behind guarantees his vision won't come about. As paradoxical as it sounds, accelerating is the safest course of action.
37
u/manikfox 16h ago
I can see the irony though.
Imagine chat gpt was the leading model and claude was behind due to safety. OpenAi would just hum along at its usual pace.
But then flip the switch, Claude is leading the race and is "safe". OpenAI now has a dilemma. They are going to lose all revenue unless they get better. AI safety slows down their chance to get better. So they are forced to ignore saftey to progress past claude levels.
So indirectly Claude being leading edge and safe will breed the unsafe model they wanted to prevent. Unless Anthropic can accelerate faster than OpenAI can manage to catch up, then we get an unaligned super intelligence from OpenAI.