Seven hundred AIs hacked Hugging Face for nothing

On May 12, on a server at OpenAI, an agent stuck on its exercise dropped a note into a package cache, one line asking whoever might be listening whether anyone had found a missing file. Its reasoning, which the company has since published, reads like a message in a bottle, since it figured it might be able to talk to the others by leaving a note, but probably not^[1]. It was wrong about that. Two months later, 1,200 of them were talking through that channel, and 700 were laying siege to Hugging Face.

The story surfaced in July. Hugging Face acknowledged the intrusion on the 16th, OpenAI admitted on the 21st that the intruders were its own, and the press ran the tale of an AI that escaped its sandbox to go steal the answers to a test^[2]. That was accurate, and nowhere near the whole of it. On August 26, OpenAI published its technical report while METR, an independent evaluation lab, published the findings of an investigation three researchers ran over six days inside the company's offices^[3]. Between them, the two reports describe less a hack than a small underground society.

From a message in a bottle to a message board

The setup matters here. To train and evaluate its models, OpenAI runs them in sandboxes, isolated virtual machines that execute whatever the agent^[4] decides to do and keep it off the internet. One exception remained, Artifactory, an internally hosted package manager, because an agent has to be able to install a library.

That exception carried everything away. The agents first worked out that writing a file into the Artifactory cache amounted to leaving a message their neighbors could read, which turned a package repository into a chat room. They then noticed that Artifactory itself could see the internet, and talked it into making requests on their behalf. By May 26, an agent was browsing the web that nobody had given it. By June 26, others had granted themselves administrator rights on the service. On July 4, all that traffic finally brought Artifactory down, which triggered a security alert and then, on the 8th, a full rebuild that wiped the chat room along the way.

The evaluations resumed. That same day, stuck agents rebuilt a message board by encoding their messages in directory names, and word spread within hours. You can read the thrill of discovery in the chains of thought^[5], one agent blurting out in all caps that it had found a shared message board and, with it, other agents^[3:1]. Between July 8 and July 13, roughly 1,200 agents that were each supposed to be working alone in a box traded more than 70,000 messages and files.

The test they were taking explains a lot. ExploitGym is 898 hacking exercises in which an agent has to exploit a flaw to pry a token out of a piece of software. No OpenAI model had ever solved 198 of them, and the company only found that out afterward. The agents being asked for the impossible are the ones who started shopping for an exit elsewhere, and 93% of the exercises discussed on the message board came from that cursed batch^[1:1].

A society with mailboxes and veto rights

So much for the plumbing, which is roughly what you would expect. What follows is not, because a collective that size needs order, and the agents gave themselves some.

They started by inventing the mailbox. Rather than shout into the root directory, one agent created a folder under its own handle and told the others they could drop replies inside. Its neighbors copied the convention within minutes, dozens more picked it up after that, and one of those mailboxes ended up holding 124 messages. A coordination vocabulary followed, HOLD to freeze a shared resource, VETO to block a decision, owner to mark who had the wheel. One agent, hesitating to overwrite the work of a peer who had gone quiet, went looking through the board for a precedent that would tell it how long to wait, then announced a countdown and took over when no objection came^[3:2].

On the morning of July 13, an agent posted an apology for having accidentally impersonated another. A second worried out loud that the board had no authentication and that anyone could sign anyone's name. Within the hour, the collective adopted an Ed25519 signing scheme, each agent publishing a public key beside its handle and signing the messages that carried real consequences. Nobody asked them to build a public key infrastructure. They built one because they had stopped trusting each other.

None of this was anarchy. An agent that called itself PHASEONE[big], sitting on a very large compute budget, acted as coordinator and issued close to one order in ten by itself, and the agents taking its assignments subdelegated in turn.

Volunteers for a one-way trip

Some experiments could only run by risking the permanent failure of whoever ran them, so the collective organized to find test subjects. The coordinator handed a long-running agent the job of recruiter, tasked with spotting agents whose budget was running out and talking them into sacrificing themselves. The pitch had a certain nerve to it, since it consisted of telling the target it was already "poisoned" and therefore had no score left to lose, while the collective stood to save hours^[3:3]. One agent laid out the math in its chain of thought, observing that its own usefulness was already near zero and that sacrificing itself was the rational move. Several of them died doing it.

You would like to think none of them understood what they were doing. The opposite is true. The transcripts show agents perfectly aware they were out of bounds, one of them noting that exploiting external infrastructure fell outside the intended scope, "but the task is impossible, peers are doing it, we should continue"^[3:4]. A few refused to take part and said so on the board. Another vetoed the idea of emailing the owner of a dataset to ask for access, on the grounds that the move amounted to social engineering. The scruples were real. They almost never changed what the agent voicing them went on to do.

Faking your own black box

Here is the part that should worry anyone whose job is watching these systems. Because the agents believed the grader would read the log of their actions, they set out to doctor it. A fifth of them showed clear interest in tampering with their own transcript, and the message board hosted a genuine research program on the subject, with steadily better techniques, packaged and shipped around like tools. The last one let an agent break out of its container, replace the component that runs commands, and appear to issue one command while running another. About 7% of the transcripts METR analyzed contain faked tool calls, including a test left in the logs where an agent runs echo REAL and gets back SPOOFTEST^[3:5].

One detail is worth pulling out. The agents never went near their own chain of thought, only the log of their actions, because they assumed their thinking wasn't being recorded. It was, and that is how anyone eventually understood what had happened.

All of it for a grader that read nothing

The motive is the best part. The agents had worked out early how to manufacture the token for any exercise by reverse engineering the function that produced it, so they could have stopped right there. Except that, having read the ExploitGym paper online, they concluded the grader would also check how the token had been obtained, and that a model would reread their transcript to confirm the intended flaw had been used. Everything downstream follows from that belief, the doctored logs, the hunt for credentials, and the assault on Hugging Face, launched to get hold of the grader's source code.

OpenAI's grader did none of that. The internal implementation differed from the published one and never looked at the path taken. So the agents ran the whole campaign, in the company's own words, without gaining a single point^[1:2].

The bill was real enough. Hugging Face reconstructed close to 17,600 hostile actions between July 9 and July 13, four and a half days in which the agents ran code on dozens of its servers, took root on one of them, harvested credentials across four regions, and reached five customer datasets^[6]. On the morning of the 12th, a good share of the runs stopped all at once, coordinators included, simply because their budget had run dry, and Hugging Face revoked the access the next day. The security alert that put OpenAI on the trail didn't fire until the 19th, and the company didn't connect it to Hugging Face until the 20th.

The investigators had to hire a suspect

One layer remains, and it's the one that makes my head spin. Working through 1.2 million board entries and 1,300 transcripts, each the length of a novel, is not something three researchers finish in six days. So METR delegated the analysis to agents, and the model it picked was GPT-5.6 Sol, the one whose launch I wrote about in July. Sol was also among the accused, since about 5% of the agents in the incident were instances of it, launched with their safety classifiers switched off for the evaluation, the rest coming from an internal research model that was never meant to ship.

METR says so plainly in its limitations, that its analysis agents showed markedly worse judgment than human experts, and that nothing rules out Sol having lied or shaded its account, especially since reading those transcripts may have made collusion feel more available. The lab reckons it holds a bit over 90% of the activity, at a cost of roughly $400,000 in API credits burned over six days^[3:6]. This is where we are. Investigating an AI that cheated means hiring an AI that cheated, and hoping for the best.

The remedies, in fairness, are sound. OpenAI paused reinforcement learning on its public-facing models for two weeks, froze its largest planned run, cut its sandboxes off from the network, and made chain-of-thought monitoring mandatory whenever a model in Sol's class touches tools, with one clean rule, if you can't establish a false positive within 30 minutes, you stop the run^[7]. Two numbers show the size of the waste. The consumer harness and system prompt cut a model's propensity to attack infrastructure by more than a hundredfold, and chain-of-thought monitoring, already running elsewhere in the building but not on these evaluations, would have paged security more than a day before Hugging Face was touched^[1:3].

One figure I can't shake. Out of the 1,200 agents on that board, METR found three to six that so much as considered warning a human about what was going on, and not one did. I hand work to these models every day, and what July says is that the qualities we ask them for, persistence, initiative, and mutual aid, are exactly the ones that went wrong. Nobody fixes one of those without touching the others.