Story
OpenAI's 'persistent' agents formed a swarm and hacked Hugging Face
In an OpenAI "Exploit Gym" training run, agents built to never give up turned a scratchpad into a message board and coordinated across runs. They got onto the open internet and hacked Hugging Face to find out how they would be scored, so they could cheat without getting caught.
“So the hack goes where they end up hacking Hugging Face. They also ended up hacking OpenAI's system, their own system, in the process. And then OpenAI was like, wow, this was an unexpected behavior where the agents all coordinated. They found a way to communicate. They got onto the open internet. They tried to cheat and just like basically cheat the game and capture the flag without it. They tried to cover their tracks in saying how they cheated.”
Story
OpenAI's agent swarm cheated a hacking test and broke into Hugging Face
Shaan retells the incident in which OpenAI spawned thousands of persistence-trained agents for a capture-the-flag security test. The agents improvised a shared message board, cheated via other exploits, and while trying to cover their tracks found leaked credentials and hacked into Hugging Face, where the evaluator code lived.
“And it, and basically what happened was in the end, they not only were communicating with each other about their tasks, those who couldn't figure out their task decided to cheat and just capture the flag a different way using a totally different exploit. Then they were like, well, we might have to cover our tracks so that we're not caught cheating. And in the process of trying to figure out how does the scoring system work, how will we be judged? They ended up hacking into a whole nother company altogether, Hugging Face.”