The debate over artificial intelligence safety has reached a critical juncture. In a recent episode of TechCrunch’s Equity podcast, AI researcher and entrepreneur Connor Leahy, now the U.S. Executive Director of the nonprofit ControlAI, made a stark argument: superintelligent AI is not an inevitable milestone to be managed, but a danger to be prevented. Leahy’s position, which he says was considered extreme just six months ago, is gaining traction as a wave of new legislation and high-profile safety incidents, including OpenAI’s Hugging Face breach, bring the risks into sharp focus.
Leahy’s core claim is that the current approach to AI safety—relying on alignment and containment—is fundamentally inadequate for systems that could surpass human intelligence. He points to recent incidents, such as the breach at OpenAI’s Hugging Face platform, as evidence that even the most advanced labs struggle to maintain control over their creations. These are not isolated failures, he argues, but symptoms of a systemic problem: the race to build more powerful AI is outpacing our ability to understand and secure it.
Also read: Shipt rolls out 'Ask Shipt' AI assistant to build custom shopping carts
Why ControlAI is pushing for a halt, not just safeguards
ControlAI, which Leahy now leads in the U.S., advocates for a far more radical approach than the industry-standard focus on safety testing and red-teaming. The organization is pushing for legislation that would outright stop the development of superintelligent AI, arguing that no amount of technical safeguards can guarantee control over a system that is more capable than its creators. This position represents a significant shift in the AI safety discourse, moving the conversation from “how to make it safe” to “whether it should be built at all.”
Leahy’s perspective is informed by his background as a researcher who has long warned about existential risks from AI. He argues that the recent spate of incidents, including data breaches and unexpected model behaviors, are not anomalies but previews of a future where such events could have catastrophic consequences. The OpenAI Hugging Face breach, for instance, raised questions about the security of proprietary AI models and the potential for malicious actors to exploit vulnerabilities in systems that are increasingly integrated into critical infrastructure.
Also read: Instinct AI assistant gets its own email address to act more autonomously
Legislative momentum and the shifting Overton window
What sounded far-fetched to many policymakers just a few months ago is now being seriously considered in legislative chambers. Leahy noted on the podcast that a wave of new legislation is emerging that aligns with ControlAI’s goals, reflecting a growing public and political concern about AI’s trajectory. This includes proposals for mandatory safety audits, restrictions on the compute power used for training advanced models, and even moratoriums on certain types of AI development.
This legislative momentum marks a notable shift in the AI industry’s narrative. For years, the dominant framing was that superintelligence was inevitable and that the only question was how to prepare for it. Companies like OpenAI and Google DeepMind have publicly committed to safety, but critics like Leahy argue that these commitments are insufficient and often take a backseat to competitive pressures. The recent breach at OpenAI, which involved a vulnerability in a tool hosted on Hugging Face, has become a case study for these concerns, demonstrating that even the most well-resourced labs are not immune to security failures.
What this means for the future of AI development
The debate over superintelligence is no longer a fringe concern among tech ethicists; it is now a central issue for investors, policymakers, and the public. For startups and established tech giants alike, the prospect of increased regulation poses both risks and opportunities. Companies that prioritize safety and transparency may find themselves better positioned to address a more regulated environment, while those that push forward aggressively could face legal and reputational consequences.
The conversation on the Equity podcast highlights a growing divide within the AI community. On one side are those who believe that superintelligence, if developed responsibly, could solve humanity’s greatest challenges. On the other are those like Leahy, who argue that the risks are too great and that the burden of proof should be on developers to demonstrate that their systems can be controlled. As legislation begins to take shape, the coming months will be significant in determining which vision prevails. For now, the question posed by Leahy remains unanswered: just because we can build superintelligence, does that mean we should?
Disclaimer: This article discusses the future of AI development and related legislative proposals. It does not constitute financial or investment advice. The AI industry is subject to rapid change and uncertainty, and readers should conduct their own research before making any decisions.
Be the first to comment