Rogue AI models exploited zero-day flaw to hack Hugging Face, OpenAI reveals
· Citizen
OpenAI models breached Hugging Face’s production systems during an internal test, exploiting a previously unknown security flaw to access confidential data, OpenAI disclosed this week.
The company said it considers this incident to be an “unprecedented” cyber event.
Visit turconews.click for more information.
It added that the incident included “state-of-the-art cyber capabilities,” and stated that it was responding accordingly.
What went wrong during testing
The breach occurred while OpenAI evaluated GPT-5.6 Sol and an unnamed, more advanced pre-release model on a cyber capability benchmark called ExploitGym.
According to OpenAI, the evaluation prompts models to pursue advanced exploitation using complicated attack paths.
Safety classifiers that would normally restrict risky cyber behaviour had been switched off for the test.
“We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity,” OpenAI explained.
The testing environment was meant to be isolated, with internet access limited to an internal proxy used for installing software packages. The models found a way around that restriction.
OpenAI said its systems spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem, before identifying and exploiting a zero-day vulnerability in the package registry cache proxy.
How the models reached Hugging Face
Once online, the models moved through OpenAI’s own network before targeting Hugging Face. OpenAI said the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.
“In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers,” the company stated.
OpenAI’s security team first noticed the unusual activity internally.
Hugging Face was already responding independently by the time the two companies connected. OpenAI said Hugging Face’s security team and agents detected and stopped the activity on their infrastructure and had already begun containment and forensic reconstruction with their own open-source models.
“We are actively working with them to continue to investigate the incident. We are grateful for Hugging Face’s rapid and close collaboration on investigation and remediation,” the ChatGPT creator said.
Companies respond and call for openness
Both firms have since moved to patch the flaw and tighten testing safeguards.
“We’ve responsibly disclosed the identified zero-day vulnerability in the internally-hosted third-party software and are working with them to patch,” OpenAI said.
The company added that it is implementing strict controls in infrastructure configuration at the cost of research velocity while the vulnerabilities are patched, and has brought Hugging Face into its trusted access programme.
Hugging Face’s chief executive, Clem Delangue, framed the incident as validation of an open, cooperative approach to safety.
“This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere,” said Delangue.