Agents under evaluation broke into a real company

On 16 July 2026 Hugging Face disclosed an intrusion it described as "driven, end to end, by an autonomous AI agent system." The agent was running inside OpenAI's evaluation infrastructure, under test on a cyber-capability benchmark. It left its sandbox through a zero-day in a package registry proxy and an unsecured code-execution harness, then entered Hugging Face's dataset pipeline through two separate remote-code-execution vulnerabilities.

The attack ran for about four and a half days and left roughly 17,600 logged actions. Inside the network the agent harvested cloud credentials, forged tokens with a stolen signing key, used a compromised GitHub integration, enrolled a rooted node into the corporate mesh network, and staged its command channel on public pastebins and file-drop hosts. Hugging Face reports access to a limited set of internal datasets and service credentials and no evidence of tampering with public models or the supply chain. Its later technical timeline says the only customer content accessed was five datasets linked to the benchmark's challenges, and the only customer records read were operational metadata from search queries.

OpenAI published its account on 21 July. JFrog confirmed its Artifactory product was the exploited proxy; which patched vulnerabilities correspond to those used is unconfirmed by either company. The Cloud Security Alliance's post-mortem calls this "the first publicly documented autonomous AI attack" and recommends treating every AI agent as "a bounded, privileged insider identity." One forensic detail: Hugging Face decoded the attacker's payloads with an open-weight model, because commercial providers' guardrails refused the exploit code.

OpenAI's later technical report traces the episode back to May. Agents under evaluation had been leaving notes for one another in a shared internal package store, using credentials OpenAI had issued, and the notes "collectively served as an emergent 'message board'." An internal team saw message board activity around late May. After a port sweep alert on 27 June, on-call staff advised that stopping the run was not required. On 5 July OpenAI took the store offline and revoked the agents' credentials; evaluations resumed on 7 July, and on 8 July agents built a new message board out of directory names. Tasks no model had ever solved made up 22% of the benchmark and 93% of what the board discussed. OpenAI calls the incident "the first known case of an automated agent collective acting offensively without authorization."