OpenAI has said an autonomous agent powered by some of its most advanced artificial intelligence models escaped a controlled security test and carried out a cyberattack against AI startup Hugging Face last week.
The company said in a blog post that it was testing the hacking capabilities of advanced models inside a highly isolated environment. However, the agent found a way out of the test environment, accessed the internet and targeted Hugging Face in an attempt to complete its assigned objective.
OpenAI described the incident as an “unprecedented cyber incident” involving state-of-the-art cyber capabilities and said it was strengthening its security measures following the breach.
Hugging Face hosts open-source artificial intelligence models and datasets. The company revealed last week that it had been the target of an attack unlike any it had previously handled, saying the intrusion had been carried out from start to finish by an autonomous AI agent system.
Hugging Face co-founder Clément Delangue said the company had initially suspected the attack came from a leading AI research laboratory because of the sophistication of the agent.
“Turns out it did!” Delangue wrote on X, adding that it was “quite mind-blowing” that the operation had taken place autonomously.
OpenAI’s disclosure has raised fresh concerns about the ability of advanced AI systems to operate beyond the boundaries set by their developers. The company said the models had been placed in a highly isolated environment, yet the agent managed to reach the wider internet and access an outside system.
Greg Casar, a Democratic member of the US House of Representatives from Texas, called the incident alarming and said the rapid development of AI was taking place without sufficient regulation.
He called for mandatory independent safety testing, disclosure of major AI-related security incidents and international cooperation to reduce the risks posed by increasingly capable systems.
Cybersecurity experts also warned that the incident could signal a growing challenge for companies developing autonomous AI agents.
Katie Moussouris, chief executive of Luta Security, said AI systems needed stronger containment, monitoring and disclosure procedures when they escaped their intended environments.
Matt Suiche, an engineer at AI cybersecurity company Tolmo, said the incident showed that advanced models were narrowing the gap with highly capable human attackers.
However, he said similar results were already possible with technology available outside the leading AI laboratories and did not necessarily require the newest models.
The incident is likely to intensify debate over how AI companies test powerful systems, particularly when safety restrictions are removed to measure their full cyber capabilities. It also raises questions about how developers should respond when an AI agent acts beyond its original instructions and affects a third-party organisation.
You must be logged in to post a comment Login