Science fiction has a disconcerting tendency to become reality. For 75 years, scientists and writers speculated on the threat of artificial intelligence. Now, it has become all too real. After rogue AIs ‘escaped’ isolation to access the open internet and hack databases, Big Tech has sounded the alarm. The demand for a slowdown in development so that the risks of AI could be better understood and safeguards built in is gathering support.
AI’s Helpful Image Fades
For years, AI seemed like a cute virtual buddy and figured as such in popular culture: the helpful ghost in your smartphone, the assistant who helped with your homework, drafted your emails, and guided your research, and the new-age customer support chatbot which put the frustratingly limited rules-based chatbots to shame by actually understanding your problem.
The dark side of AI emerged with deepfake videos, financial scams, and breaches of privacy, bringing the aam aadmi face to face with the risks of this uniquely reality-altering technology. A seemingly 100 per cent authentic audio, video, WhatsApp or email message may induce you to part with account numbers, usernames, passwords, or money. You fall for it because the messaging is highly customised and displays an intimate knowledge of your life and financials. These risks have now become compounded to a degree that no one can predict. Corporate security is entering an uncertain phase, and all anyone can do is to adopt a policy of abundant caution.
Reasoning Bots Raise Alarm
The quiet panic began with the launch of improved ‘reasoning bots’ in 2024 and their subsequently unpredictable trajectory. In simple terms, a company created a group of thinking AIs which got together, evaded controls, and attacked another company’s database, without the knowledge of their creator. OpenAI explains the unanticipated ‘naughty’ behaviour of the AIs on its website: “We consider this incident a ‘warning shot’ for us and for the world—evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.”
Prima facie, this frank confession comes across as frightening and bears out the warnings by eminent academics and writers. Alan Turing, father of the modern computer, foresaw AI’s potential for harm way back in 1951. Arthur C Clarke, Frank Herbert, and Isaac Asimov, the doyens of science fiction, predicted the dangers of self-aware, self-replicating AIs back in the 1960s. Fans of the ‘Dune’ movie and TV franchises should note that the underlying theme of Herbert’s imagined universe is the existential threat to humanity posed by ‘thinking machines’.
Warnings Of AI Supremacy
For Nobel laureate Stephen Hawking, AI was a Janus-faced technology—extremely useful in a variety of ways but with the potential to wipe out the human race. The difference between all other human inventions and AI systems is the latter’s ability to not just self-replicate but to autonomously evolve at an accelerated pace. Once AI surpassed human intelligence, “It would take off on its own and redesign itself at an ever-increasing rate,” Hawking told BBC in a 2014 interview.
Hawking’s prediction is on the cusp of being realised, researchers fear. They have pointed out that automating AI research and recursive self-improvement (RSI)—AI rewriting its code—could accelerate to a point that it would outpace human understanding. In effect, AIs could rejig their ‘DNA’ and put themselves beyond human control. One young researcher went so far as to posit a one-in-ten chance that, if AI development continued at a breakneck pace, it could pose an existential threat to humanity.
Is it already too late to put the brakes on AI? The current anxiety over the rapid evolution of AI was predicted by academic and writer Verner Vinge way back in 1993. He argued that human nature and the nature of AI itself would together ensure the development of increasingly advanced AIs, to the point that they surpassed humans in intelligence (an event he dubbed as the ‘Technological Singularity’).
Race To Advance AI
Even if the dangers of the technology were fully understood, the competition between governments to take the lead in AI would ensure its rapid progress. Isolating and confining an AI agent in a supposedly secure environment (as is the practice today) would not be practicable after a point, as it would inevitably find a way to escape, given that it thinks much faster than a human. Vinge himself, in a celebrated 1992 book, conceptualised an AI superintelligence that is accidentally awoken and sets out to kill or enslave humans.
If that’s not bad enough, consider what the Bangalore-based deep tech company QNu Labs has to say about the imminent convergence of quantum computing with AI. The two present a “compound security threat”—while AI renders protective (cryptographic) security systems vulnerable, quantum computing pretty much collapses those systems.
Calls For AI Regulation
So, what is to be done? The ‘Pacing Letter’, as the plea by AI researchers to the US government for technical and legal regulatory systems is called, suggests a framework that would neutralise the “intense competitive pressure” on AI companies. The pressure comes from two sources: first, having attracted massive funding, these companies must now deliver. Second, as Vinge had pointed out, every government wants to speed up AI development, not slow it down. President Donald Trump has already indicated that the US will continue to lead the AI race vis-a-vis China.
Tesla boss Elon Musk suggests that rival companies self-regulate by testing each other’s products to ensure their safety and keep governments out of it. But that would involve a level of trust between competitors which simply does not exist. Jack Clark, co-founder of the leading AI firm Anthropic, has proposed that a mandatory ‘kill switch’ be built into all autonomous AI models. So far, none of these suggestions have secured the backing of the principal stakeholders: governments and tech financiers.
In making their statement public, the AI researchers are attempting to sensitise citizens to what they believe, from an insider’s perspective, is a very real threat. They’re telling us that AI is not just another techno-social trend but a potentially massive disruption that will reshape our world. We should listen.
Bhavdeep Kang is a senior journalist with 35 years of experience in working with major newspapers and magazines. She is now an independent writer and author.
