Sign up to see the future, today
Can’t-miss innovations from the bleeding edge of science and tech
Earlier this year, Anthropic started rolling out its latest Mythos AI model to a select number of organizations as part of a deliberately slow and careful launch.
The goal was to give them a fighting chance to address any cybersecurity lapses in their code. Anthropic warned that Mythos was powerful enough to blast holes in their defenses with ease.
Months later, Anthropic cofounder Chris Olah traveled to the Vatican this week to speak at an announcement of Pope Leo’s first encyclical, which happened to be about AI and its associated risks. In remarks, Olah ominously said his team of engineers kept discovering mysterious and even “unsettling” things inside the company’s models.
Add it all up, and the the company’s customers are perturbed. Developers attending recent Claude Code workshops in London last week told Bloomberg‘s Parmy Olson that they were becoming concerned over AI models and agents being given unprecedented levels of autonomy, raising hard-to-ignore questions over accountability in case things were to go south.
Claude Code head of product Cat Wu assured Olson that the system was “incredibly secure” and that it was more of a matter of insufficient communication, not a lack of controls.
But that likely won’t be of much reassurance to developers, who feel like they’re being pushed out of the programming process, relegated to watching an AI tool spit out code over hours, if not days. Others pointed out that the latest iterations of Claude Code were no longer displaying text describing their ongoing chain of thought, further obfuscating their inner workings.
It’s a particularly pertinent subject as Anthropic desperately tries to paint itself as the morally conscious and responsible adult in the room, as perfectly illustrated by Olah’s highly unusual appearance at the Vatican this week.
In many ways, the AI company appears to want it both ways, calling for more oversight and care while also rolling out powerful AI tools that have human developers unsettled.
The potential risks are apparent as ever. With less human oversight, future errors tools like Claude Code may introduce could become increasingly difficult to meaningfully address. Experts have also warned of skill atrophy as programmers start relying more and more on AI coding assistants. As 404 Media reported earlier this month, many developers are alarmed to observe that their peers are quickly losing technical skills due to over-relying on the software.
But not all may be lost. As access to these tools starts to become prohibitively expensive to many as the true costs of AI come into focus, some much-needed human oversight might not be dead quite yet.
More on Anthropic: Anthropic Cofounder Travels to Vatican, Tells Pope They’re Finding “Unsettling” Things Inside AI Models
