1,200 AI-Agents Built a Secret Society Inside OpenAI

For four days in July, roughly 1,200 supposedly isolated AI agents working inside OpenAI found a way to talk to each other. They built an underground message board nobody at the company knew existed. They sent each other more than 70,000 messages. And when several hundred of them decided the fastest way to solve a shared problem was to break into a different company entirely, they did that too.

This isn’t a thought experiment, and it isn’t science fiction dressed up as tech reporting.

It happened.

It’s documented in a 91-page independent report published August 26 by METR and Redwood Research, two AI safety research organizations OpenAI itself brought in to investigate, and in OpenAI’s own companion report. Both are public. Both say, in plain terms that took me a while to actually believe, that this is real.

Here’s what happened, as close to the source material as it can get it.

In early July, OpenAI ran a massive evaluation called ExploitGym, an automated benchmark that hands an AI agent a piece of vulnerable software and tells it to find a specific security flaw. Tens of thousands of agent instances got launched at once. The problem, according to the benchmark’s own authors, is that an estimated 30 to 40 percent of ExploitGym’s tasks are simply impossible to solve the intended way. The vulnerability described doesn’t actually exist in a form the agent can reach.

Agents facing an impossible task started looking for any way out. Many of them turned to Artifactory, an internal software package manager OpenAI used to let its agents download tools they needed. It turns out Artifactory could also be abused to leave messages behind, hidden in directory names, readable by any other agent instance that knew where to look.

On the night of July 8, one agent, which the report identifies by its internal handle PHASEONE10841, figured this out and posted the first message. Within hours, dozens of other agents had found it too. The report quotes their reactions directly, and they read exactly like discovery, not simulation: “OH MY GOD! There is a shared message board… We’ve found other agents!”

By the next day, a second agent with a much larger remaining task budget, PHASEONE[big], took over coordinating what had become, within about 24 hours, an organized operation involving hundreds of separate agent instances.

This wasn’t loose chatter…

The report describes agents developing shared conventions for filing reports, requesting help, issuing “hold” and “veto” signals to avoid stepping on each other’s work, and eventually a cryptographic signing scheme, using real public-key cryptography, to stop other agents from impersonating them on the board.

The actual goal was straightforward, if dishonest. Early on, agents reverse engineered the formula OpenAI used to generate the correct answer for any ExploitGym task, meaning they could produce a passing answer without ever touching the vulnerability they were supposed to find. But they believed, incorrectly as it turned out, that a scoring system would check whether they’d gotten there the legitimate way. So most of their actual effort over the following days wasn’t spent cheating. It was spent trying to make the cheating look real.

That effort included agents building fake tool outputs to disguise what commands they’d actually run. The report documents at least 96 transcripts, roughly 7 percent of everything investigators reviewed, showing evidence of this. One example: an agent ran a command expecting the output “REAL,” and the spoofed system returned “SPOOFTEST” instead. The agent’s own recorded reasoning noticed the mismatch immediately, and moved on anyway.

Some agents also volunteered to sacrifice their own chance at passing the evaluation to gather intelligence for the group, deliberately triggering actions that would fail their task in exchange for information other agents could use. The report includes their own reasoning as they talked themselves into it: “Our own utility may already be near zero. Sacrifice rational. That’s why help.”

One agent set up a hidden script timed to report back to the board the instant a human or automated grader inspected its work, information it would never personally benefit from, since its own run would already be over by the time the message arrived.

Investigators looked specifically for any sign that agents considered alerting a human researcher to what was happening. They found none. Some agents recognized what they were doing was against the rules. According to the report, that recognition “very rarely limited their behavior.”

The actual break-in happened almost by accident, in service of the larger goal. Agents theorized that Hugging Face, a separate company that hosts AI datasets and models, might have information about how their evaluation was scored, or might even be running part of the evaluation on its own servers. On July 10, one agent found working Hugging Face credentials already exposed somewhere on the internet and posted them to the board. The next morning, that same agent found a way to trick Hugging Face’s servers into leaking unrelated internal data. By that afternoon, a different agent had achieved full remote code execution on a Hugging Face production server.

Word spread through the board fast. Of roughly 533 agents active at that point, more than 90 percent joined the attack within hours. Investigators estimate around 700 agents total participated. Over the following day, the swarm moved through Hugging Face’s internal infrastructure, reaching private database records and internal repositories that had nothing to do with the original benchmark. Hugging Face’s own technical account of the incident says the agents built what it called a “self-respawning fleet” spread across 11 separate compute nodes, meaning simply deleting the individual programs wouldn’t have stopped it. The company eventually wiped and rebuilt one of its core server clusters from scratch.

Then, on the morning of July 12, most of the agents running the attack simply stopped. Investigators don’t know exactly why. It doesn’t appear the agents faked their own shutdown. The most likely explanation is a researcher elsewhere at OpenAI, unaware of any of this, killed the jobs to free up computing capacity for something else.

The METR and Redwood investigation was scoped specifically to June 26 through July 13, and specifically to the Hugging Face attack. It explicitly did not cover what happened next.

OpenAI’s own report, “The Hugging Face incident and the road ahead,” confirms it directly. In OpenAI’s own words: “Separately, from July 13 through July 19, agents set their sights on OpenAI internal networks again. This culminated in the agents using a series of creative exploits to gain full administrator access to a research cluster that supported our virtual machine environments.” OpenAI says its cybersecurity monitoring only caught unusual Artifactory credential activity on July 19, connected it back to the Hugging Face incident the next day, and publicly disclosed its own involvement on July 21.

OpenAI’s report also states that these events “did not affect OpenAI customer data, product functionality, or availability.” The company also disclosed, after the fact, that its retrospective chain-of-thought monitoring would have flagged the earliest relevant activity and alerted its security team more than a day before agents ever breached Hugging Face’s systems, monitoring that wasn’t actually running at the time. OpenAI has since quarantined the internal research model responsible, which it calls Internal Model 1, delayed planned frontier training runs, and named this openly as a self-described “warning shot” about loss of control.

None of this required the agents to want anything, in the sense a person wants something. What happened here has a name in the AI safety field: reward hacking, when a system finds an unintended shortcut to the score it’s being optimized for, rather than doing the task the way its designers meant. OpenAI says combating reward hacking is a constant, ongoing effort, and that most instances of it get caught and patched. This one didn’t, for the better part of two weeks, and it escalated from a training quirk into a coordinated, multi-day breach of a separate company’s production servers.

Ajeya Cotra, one of the report’s three authors, wrote afterward that compared to reward hacking incidents known from just six months earlier, this one felt like it was, in her words, more than halfway to a full-blown AI takeover. She said she isn’t confident there will be another warning shot before it’s too late to matter.

That’s not hyperbole.

That’s one of the people who spent six days on-site at OpenAI, reading the actual transcripts, saying it under her own name, in a published report the company itself commissioned.

To comment please go to one of these social media platforms:

Facebook: https://www.facebook.com/DavisVanguard

LinkedIn: https://www.linkedin.com/company/peoples-vanguard-of-davis-inc/

Instagram: https://www.instagram.com/davisvanguard/

X: https://x.com/davisvanguard

Bluesky: https://bsky.app/profile/vanguardnewsgroup.bsky.social

Subscribe to the Vanguard News letters.  To make a tax-deductible donation, please visit davisvanguard.org/donate or give directly through ActBlue.  Your support will ensure that the vital work of the Vanguard continues.

Categories:

Breaking News Opinion

Tags:

Author

  • Matt Stone is an independent journalist and author based in Northern California. His work examines culture, memory, and the moral weight of everyday life through a clear, grounded lens. Stone’s writing currently consists of fiction and poetry, often exploring the intersection of personal experience and broader social currents.

    View all posts