The Pause AI fantasy
Wiktor Szymanowicz/Future Publishing via Getty Images
The most dangerous assumption in AI safety is that we can control AI once it’s smarter than us. The second most dangerous assumption is that we can control how fast we develop it.
The control fantasy
The global race to build advanced AI is a dangerous sprint if anyone succeeds before solving the problem of how to do it safely. Calls to pause the race until we solve the safety problem have won attention without winning time, because pause activism is rooted in a twin fantasy of control: that we can control an intelligence that can outsmart us, and control how fast we develop it. A sober response requires not a pause but a mobilization of our greatest minds to solve a problem while we are actively creating it. The response must also consider solutions that don’t assume we have more control than we do, especially when failed attempts at control could make the problem worse.
The science against a pause
The scientific case against a global AI pause is that human control over collective human action and institutions has well-documented limits. Technologies can be halted by moratoriums when you can see exactly who’s doing the risky thing, where they are doing it and how to stop them. AI can’t be governed like human cloning or nuclear arms development, because it’s a general-purpose, digitally distributed utility. With trillions now projected for its infrastructure buildout, AI is already globally integrated into cloud computing, energy, finance, manufacturing, health care, education, transportation, military and defense, industrial systems and government agencies.
Given the speed and magnitude of integration, AI has the markers of technological lock-in, a problem described by W. Brian Arthur, one of the founders of complexity economics. Lock-in shows how human systems can form historical dependencies around a technology until that technology becomes entrenched and prohibitively challenging to redirect through ordinary policy tools. What makes AI lock-in so unusual is that it applies not only to adoption, but to acceleration. Pause advocates are asking states, companies and institutions to stop developing a technology whose usefulness is inextricably tied to continued development. AI’s progress is central to its utility. Better models, faster discovery, wider automation, more capable agents and strategic advantages that compound as systems improve. So while an AI pause may seem possible in principle, in any reasonable timeframe, the prospect becomes so costly, complex, coordination-dependent and opposed by high-stakes incentives that pausing is about as plausible as unbaking a cake.
The multipolar trap
The geopolitical incentive structure deepens the lock-in. AI development is now inseparable from national security, economic competition and geopolitical influence. For any one nation, slowing down risks losing advantage, and potentially losing sovereignty. As Putin ominously warned, “The one who becomes the leader in this sphere will be the ruler of the world.” This is a multipolar trap. Every rational actor keeps advancing a risky agenda because slowing down alone could be more dangerous than continuing.
Then there’s the issue of denying bad actors access to open-weight models. An AI pause would require responsible labs, private enterprises and governments to agree, without irresponsible actors treating the pause as an opening. Not everyone can train frontier systems from scratch. But now that powerful open-weight models exist, actors might build on and deploy increasingly capable systems with far less visible infrastructure than biotechnology or physical weapons programs. So while a pause may sound safe, whether it’s actually safe depends on whether a pause is aligned with the salient interests of every major actor, individual, group or state, good or bad, around the world, all at once.
No amount of time can solve a paradox
The pause is a thinly veiled reformulation of the underlying problem—how to control systems we can’t actually control. It assumes we can control timelines long enough to control a superior intelligence. Control is so deeply embedded in AI safety that the field’s central challenge is often called the control problem, failing to recognize that our obsession with control may be fueling the problem. If we assume control of a superior intelligence is the only way to stay safe, then even if a pause were possible, it would merely prolong our inevitable failure. If control is the wrong frame, the answer isn’t more time inside that frame. It’s a jolt into a wider solution space.
We need pragmatic optimists
We need a pragmatic response that doesn’t mistake a timeout for a strategy. We need a response that rejects doomer surrender, rejects fantasies of control, and acts with urgency inside the reality we’re already in—a reality where we need to ask whether “AI alignment” is purely an engineering problem. Solving this problem may not look anything like what we currently imagine, so it can’t be responsibly managed by those who only imagine the worst. Human beings have pulled off odds-defying feats before by expanding the solution space and refusing to confuse the path we first imagined with the outcome we actually need. Which means we need to deploy our most rigorous, constructive and solution-oriented thinkers, people who can steel themselves against the odds without letting uncertainty ossify into paralysis—and people who will accept the survival and flourishing of humanity as our primary success metric even if it doesn’t include remaining in control of a superior intelligence. Apollo 13 wasn’t saved by people arguing that the spacecraft should never have launched or pretending every variable could be managed by Mission Control. It was saved by redefining the mission midflight and letting go of the command module once it could no longer get them home. Our AI challenge is still a moonshot, but not a moonshot for control. It’s a moonshot for coexistence with a hyper-capable intelligence. It requires imagination and determination to figure out how to share our world with forms of intelligence more alien and powerful than our own. And it requires us to figure this out mid flight, without the luxury of a pause.
Watch the accompanying video essay behind this reporting:
