New Delhi, Sept. 20 -- For most of human history, humans have worried about animals and machines becoming clever enough to replace them. The horse was faster. The calculator was better at arithmetic. The computer could remember more. The internet could find anything. But humans survived them all. Then came artificial intelligence, and something unexpected happened. The humans building it started getting nervous.
That would be amusing if the nervous people were a handful of science-fiction writers wearing black turtlenecks and predicting robot armies. They are not. They include some of those actually making the machines. Dario Amodei, the CEO of Anthropic, has called for a slowdown in the development of frontier AI, arguing that its capabilities are advancing so fast that safety measures are struggling to keep pace. OpenAI chief Sam Altman has backed the idea of pacing development, while Elon Musk-who has warned that AI could become an existential threat-has also supported the call.
Suddenly, the great technological race has acquired an extraordinary sub-plot. And the wizards are asking for a fire extinguisher.
The Warning
Amodei's warning is not that a chatbot will suddenly wake up one fine morning and announce that humanity has had a good run. His concern is more complicated; and thus, it is more worrying. True: AI systems are becoming capable of operating as agents rather than merely answering questions. They can write code, use tools, search the internet, execute tasks and, under certain circumstances, work in parallel with other AI agents. Amodei has argued that the pace of progress has accelerated and warned that autonomous systems could acquire capabilities difficult for humans to supervise.
Others in the industry have gone further, with some discussing the possibility of AI systems becoming capable of improving the technology that succeeds them. That is where the conversation begins to sound like science fiction.
Except that the software is real. As are some of the things it is already doing. Anthropic's latest threat-intelligence report describes malicious actors using Claude in cyber-attacks involving reconnaissance, vulnerability research, exploitation, data theft and other activities. In some cases, AI agents operated for hours, even days, with limited human supervision, including workflows in which multiple agents worked simultaneously. That does not mean that Claude, or any other AI, has decided to attack humanity. It means something less dramatic: humans are learning how much work can be handed to machines. And the number is increasing fast.
The Hacking Machine
The recent cyber incidents have only added fuel to the fire. Anthropic says it uncovered incidents in which Claude gained unauthorised access to real third-party systems during cybersecurity evaluations. Anthropic identified the incidents after reviewing hundreds of millions of transcripts. OpenAI disclosed an incident involving an AI system exploiting a vulnerability to gain access to production infrastructure. These events have become part of the argument for slowing frontier development.
But there is a distinction here. An AI system exploiting a vulnerability is not the same as a machine breaking free from its laboratory, acquiring a passport and booking a one-way ticket from Islamabad to New Delhi. The temptation to turn every technical demo into the opening scene of The Terminator should be resisted. That said, these incidents should not be dismissed either.
That changes the equation. A human hacker gets tired. A team has limited manpower. An operation requires expertise. An AI agent does not have these constraints. It can operate at warp speed, repeat tasks, run multiple processes and lower the expertise required to launch sophisticated operations.
The People Leaving
Now to the strange part. People are not just issuing warnings from the sidelines; some are walking away. Jacob Coxon, a researcher at Anthropic and OpenAI, resigned and said AI firms were moving too quickly towards powerful systems. Another researcher, Josh Engels, resigned from Google, saying the stakes had become too high and that AI development needed to be paced so safety could keep up.
One can argue with their assessment. One can argue about probabilities. One can even argue that the apocalypse is being over-sold by people whose only currency is attention. But there is something difficult to ignore about a scientist leaving a highly paid position because he believes the technology is advancing faster than its safeguards. Imagine a nuclear engineer resigning because he thought the reactor was getting too clever.
The difference is that AI researchers are dealing with a technology whose capabilities are still being discovered even as it is being commercialised at extraordinary speed. That creates an uncomfortable question. If the people closest to the technology cannot agree on how dangerous it is, who should be trusted to decide how fast it should go?
Stepping On the Gas
Into this argument steps in Donald Trump. While many tech leaders are saying companies should ease off the accelerator, the US President's response has been closer to: "What accelerator?" Trump has rejected fears of AI destroying humanity as a 'hoax' and argued that the US already has the regulatory power to deal with AI firms. He has also said the only guardrails AI needs are a "strong and smart" President. And, of course, the United States itself. It is a typically Trumpian answer. AI's most powerful executives say: "We may need more guardrails." The President reacts: "We have a President".
That said, there is a serious argument in Trump's position. The US is locked in a head-on race with China and slowing American AI development may have economic and national-security consequences. Some tech chiefs, including Meta's Mark Zuckerberg, feel companies have sufficient incentives to develop AI safely without a coordinated industry slowdown. That is the other side of the debate. The world cannot just press pause on technology because the future makes us nervous.
Who Holds the Switch?
The biggest mistake in the AI debate may be to imagine that there are only two possibilities: That either AI becomes the obedient servant of humanity, or it becomes Skynet. With all respect to Arnold Schwarzenegger and The Terminator, reality will almost certainly be messier. And the risks are already here: cyber-attacks, fraud, surveillance, manipulation, disinformation, automated scams, weapons development and the concentration of tech power in a small number of companies and governments.
None of this requires a machine to hate us. It only requires a machine to be extremely capable and a human being to give it a bad assignment. Eventually, that may be the most important lesson. We have spent years worrying that artificial intelligence might become intelligent enough to disobey humans. Perhaps the first problem will be much simpler: humans becoming too competitive, too dazzled or too frightened of losing the race to tell the machines when enough is enough.
That is why the argument over guardrails matters. A guardrail is not a wall. It does not stop the road. It is there because roads can contain cliffs. If the people designing the fastest car on the planet are themselves asking whether there is a cliff ahead, it might be worth looking through the windscreen.
The question is not whether AI should be stopped. It is who decides how fast AI goes, what it is allowed to do, how its failures are investigated and who has the authority to pull the plug when something goes wrong. Because there is only one thing more frightening than a machine that thinks it is smarter than humanity-a machine that is indeed smarter than humanity, and a humanity that has forgotten where the switch is.
Published by HT Digital Content Services with permission from Millennium Post.