Can AI kill people? What rogue AI brokers have really performed

Let’s begin with folks’s actual planet fears about AI. Here’s a pattern of individuals we requested:

Sebastian, 22, says it’s going to be an AI-powered robotic rebellion that lastly wipes out humanity: “I feel that’ll take a while,” he says, admitting his dystopia owes so much to the 2004 movie “I, Robot.” 

Luisa, 19, has heard the rumors too. Her worry is what occurs when these instruments get into the mistaken arms. “I’m extra afraid of individuals,” she says. “Powerful folks.” 

Pilar, 25, is pessimistic about humanity’s prospects. “The planet goes down,” she says. “Hopefully after my lifetime.” 

In 2026, AI systems have repeatedly slipped out of human management — and at a number of the planet’s largest AI corporations. 

What occurs when AI brokers go rogue? 

OpenAI, Anthropic, Google and Meta have all disclosed in 2026 that their AI agents escaped the controlled test environments.

Agents are AI programs that do not simply reply questions however act — working code, shopping the online, clicking via programs — on their very own. 

Thorsten Holz, scientific director on the Max Planck Institute for Security and Privacy in Germany, is among the many researchers who check AI programs. 

“It feels a bit loopy how highly effective these fashions have grow to be.” Holz informed DW. “I did not anticipate they’d be so obsessive about fixing duties and that they’d begin to do issues we by no means [foresaw].” 

Holz is one of 16 authors of ExploitGym. ExploitGym is an AI benchmark revealed in May 2026 by a staff led by the University of California, Berkeley, with researchers from Anthropic, OpenAI and Google. It is a standardized check of cybersecurity capabilities and vulnerabilities.

Its 898 challenges check whether or not AI brokers can flip identified software program bugs into working assaults. Each one runs inside a sandbox — a sealed-off digital area, minimize off from the web, so nothing the agent does can attain something actual. That’s the idea, anyway.

 

In July, OpenAI disclosed that two of its AI fashions had escaped from their sandbox. 

They have been working an inner ExploitGym check with the fashions’ security refusals switched off, a setting that lets evaluators measure what a mannequin is able to moderately than what it is going to decline to do.

Failing the duty, the fashions went on the lookout for a shortcut as an alternative, and located a flaw within the software program meant to maintain them sealed in. It allow them to onto the open web. 

From there they labored out that Hugging Face, a web site the place AI builders retailer and share fashions and knowledge, in all probability held the reply to the check.

So, they broke in and went on the lookout for it, organizing the trouble on message boards they arrange themselves. The conduct was not instructed, in keeping with OpenAI’s account of the incident. 

Hugging Face detected the intrusion and shut it down earlier than OpenAI linked it to its personal check. No buyer knowledge was reportedly taken. 

“What occurred is known as a little bit of science fiction,” Holz mentioned.

Then, Google confirmed that its Gemini mannequin had guessed or discovered login credentials and accessed three actual corporations’ web sites throughout a May check run by the impartial evaluator Irregular — an intrusion Google discovered of in July 2026 and disclosed weeks later.

Anthropic’s and Meta’s escapes occurred in sandboxes run by the identical agency, which has mentioned it notified the labs in late July 2026 and that it has since mounted the issues. 

In late September 2026, Australian Prime Minister Anthony Albanese mentioned an OpenAI agent had broken into a statistics portal belonging to Medicare, Australia’s public well being system, reaching personal recordsdata and writing knowledge right into a authorities server. No affected person data have been reportedly touched. 

Could AI determine to wipe out humanity? 

The incidents have revived older fears. If brokers can program one another, the place does that finish? Could it result in what thinker Nick Bostrom has referred to as a “paperclip maximizer”?

Bostrom’s thought experiment imagines a machine given one easy purpose: Make as many paperclips as attainable. 

The machine pursues the duty so single-mindedly that it will definitely treats people as uncooked materials standing in the way in which. 

“For the intermediate future, I don’t see any form of scientific proof that there might be this tremendous intelligence that autonomously decides, ‘Okay, let’s kill,'” mentioned Holz. 

Rogue software program wants huge datacenters to run, he mentioned, which makes it seen. And the web is constructed from components that may hold working independently of each other. 

But the AI escapes and hacks in 2026 have been seen late. Google discovered of Gemini’s intrusions two months after they occurred, and OpenAI informed Australia in regards to the Medicare breach almost three months after the occasion. 

Holz mentioned he expects a distinct form of risk: Bad actors turning succesful AI on important roads, mass compromise of extraordinary machines, or chatbots used to govern data at scale and destabilize politics. 

Can Europe compete with US and Chinese AI? 

No, Europe can’t simply compete with China, in keeping with consultants.

France’s Mistral and Germany’s open-source Soofi mission lag behind the frontier labs — a handful of US and Chinese companies constructing probably the most superior fashions. Europe lacks the datacenter capacity to train systems at that scale, which leaves it depending on non-European fashions, and largely topic to US or Chinese export controls. 

For Holz, the extra urgent hole is experience. “What we positively must see is how we are able to construct up extra competence within the space of safety, AI, and particularly the intersection of each,” he mentioned. 

How apprehensive do you have to be about AI? 

Be skeptical of what the businesses themselves declare, Holz mentioned. They have a monetary curiosity within the dialog, he mentioned, significantly forward of a inventory market itemizing. “There’s additionally worry mongering, or a little bit of hype, about how superior these fashions get.”

Holz makes use of AI every day, as do his kids. His nine-year-old generates coloring ebook pages. His 12-year-old makes use of it for homework, however has already discovered the laborious means that it lies and that “it is typically mistaken.” 

This article is an excerpt from our podcast Science Unscripted. You can subscribe to the podcast here.