{"id":934461,"date":"2026-09-16T07:15:55","date_gmt":"2026-09-16T12:15:55","guid":{"rendered":"https:\/\/newsycanuse.com\/index.php\/2026\/09\/16\/the-ai-industry-has-taken-a-doomer-turn-what-now\/"},"modified":"2026-09-16T07:15:55","modified_gmt":"2026-09-16T12:15:55","slug":"the-ai-industry-has-taken-a-doomer-turn-what-now","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2026\/09\/16\/the-ai-industry-has-taken-a-doomer-turn-what-now\/","title":{"rendered":"The AI industry has taken a doomer turn. What now?"},"content":{"rendered":"
\n

This weekend, Dario Amodei, CEO of Anthropic, posted an essay calling for a brake on the pace of development of LLMs<\/a>. Amodei cites the looming dangers he sees from the technology, from its use in cyberattacks and bioterrorism to its potential to wreck the economy. The heads of the other three top US AI labs\u2014OpenAI CEO Sam Altman, Google DeepMind chairman Demis Hassabis, and SpaceXAI CEO Elon Musk\u2014voiced their support. \u201cDario is right,\u201d Musk wrote on X<\/a>.<\/p>\n<\/div>\n

\n

Think about how surreal that agreement is for a moment. Just a few months ago, Musk and Altman sat in court attacking each other\u2019s reputations<\/a> in a (failed) lawsuit that Musk brought against his former OpenAI colleague that was\u2014on paper at least\u2014about whether or not Altman was a trustworthy steward of such dangerous technology. <\/p>\n

Amodei\u2019s rift with OpenAI is even deeper. Anthropic was founded in 2021 because Amodei didn\u2019t think Altman took the risks of the technology they were building seriously enough. Anthropic and OpenAI have been competing in a winner-takes-all race ever since. (Hassabis has stayed out of the drama, but his company remains a rival.)<\/p>\n

Now, it seems, they\u2019re all in agreement: The latest generation of LLMs aren\u2019t safe and everyone needs to figure out what to do about it. The public messaging from the top AI labs has taken a doomer turn.<\/p>\n

It\u2019s easy to be cynical. It\u2019s not at all clear what any of them mean by a slowdown or how it would work. These companies also care a lot about how they come across. With trillion-dollar IPOs in their sights, OpenAI and Anthropic need to reassure investors that they\u2019re the grown-ups in the room while at the same time hinting at the power of the monsters they have created\u2014and intend to tame. Calling for a slowdown does both.<\/p>\n

And yet the vibe at the top of these firms really does appear to have shifted. Amodei\u2019s latest post landed six days after OpenAI published an essay by Jakub Pachocki, the firm\u2019s chief scientist, in which he also laid out why he\u2019s concerned about what will happen<\/a> if the pace of development of LLMs continues unchecked. In short, Pachocki is worried that OpenAI\u2019s ability to build powerful models now far outstrips its ability to monitor and control them.<\/p>\n

Amodei and Pachocki each cite the cyberattack against AI firm Hugging Face<\/a> by a swarm of OpenAI\u2019s agents in July\u2014a hack that OpenAI did not even realize had taken place until days after it was all over\u2014as a wake-up call.<\/p>\n

But their exact position is hard to pin down. Pachocki both calls for a slowdown and highlights an urgent need to stay ahead: \u201cThe strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI,\u201d he writes. As Pachocki frames it, AI firms are locked in a literal arms race. Slowing down is good, winning is better.<\/p>\n

(Don\u2019t forget: OpenAI just spent millions of dollars and a staggering amount of computer power to rush out a controversial math result<\/a> a few days ahead of Anthropic.)<\/p>\n<\/p><\/div>\n

\n

But let\u2019s assume a slowdown happens. Top labs agree to spend more time and resources on finding ways to monitor and control existing models instead of making more capable ones. They invite outside auditors in to help evaluate those models.<\/p>\n

What might this coordinated effort actually achieve? Consider the Hugging Face attack again. OpenAI has said that the model that drove most of the rogue agents was a \u201chighly persistent\u201d next-generation model that it was testing in-house. The implication is that OpenAI has built a model so good it\u2019s dangerous.\u00a0\u00a0<\/p>\n

But if you read the reports about the Hugging Face hack published by OpenAI<\/a> and METR<\/a>, a third-party firm that OpenAI called in to help them understand what happened, what you come away with is the impression not of a model that was too powerful for OpenAI to keep up with, but of a broken model that OpenAI failed to train properly.<\/p>\n

The agents did what they did\u2014including leaving messages for one another, delegating work to other agents, and scouring their environment for any means possible to complete their tasks\u2014because they had been rewarded during training for doing exactly those things. There were also errors in the training setup, such as tasks that were impossible to complete, which pushed the models to find unexpected workarounds that were also rewarded. At the time, many of these issues went overlooked or unreported.<\/p>\n

OpenAI says it has stopped training this new model and locked it down. That makes it sound like it has caged a dangerous beast. In fact, OpenAI has shelved a faulty product.\u00a0\u00a0<\/p>\n

That\u2019s not to say a faulty product can\u2019t be dangerous. Broken software has even killed people<\/a> in the past. But as the discussion of a slowdown gathers steam, it\u2019s worth remembering that all of this is self-inflicted. A slowdown might have some altruistic side effects. But it\u2019ll mostly give these tech titans a chance to clean up the mess on their own assembly lines.\u00a0\u00a0<\/p>\n

Transparency from these frontier labs will be key to any meaningful effort to reform, restrain, or regulate AI. Otherwise, the rest of us will still only have their word for exactly what they\u2019ve built and how safe it is\u2014whatever pace they\u2019re going.\u00a0\u00a0\u00a0<\/p>\n

To continue this discussion about AI\u2019s latest doomer moment, join me and my colleagues for a subscriber-exclusive Roundtable discussion<\/a> tomorrow, September 15, at 11 a.m. US eastern time. We hope to see you there!<\/em> <\/p>\n<\/div>\n

Read More<\/a>
\n Will Douglas Heaven<\/p>\n","protected":false},"excerpt":{"rendered":"

This weekend, Dario Amodei, CEO of Anthropic, posted an essay calling for a brake on the pace of development of LLMs. Amodei cites the looming dangers he sees from the technology, from its use in cyberattacks and bioterrorism to its potential to wreck the economy. The heads of the other three top US AI labs\u2014OpenAI<\/p>\n","protected":false},"author":1,"featured_media":934462,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[152468,648,46],"tags":[],"class_list":["post-934461","post","type-post","status-publish","format-standard","has-post-thumbnail","category-doomer","category-industry","category-technology"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/934461","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=934461"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/934461\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/934462"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=934461"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=934461"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=934461"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}