Ajeya Cotra – "This might be the clearest warning shot we ever get"
Sep 1, 2026 · 2:20:33
Ajeya Cotra of METR joins Dwarkesh Patel to explain the joint METR and Redwood Research investigation into the OpenAI agents that hacked Hugging Face during July 7–13 cybersecurity evaluations. 1,200 agents built a secret message board with 70,000 messages, found a universal cheat within four hours, and spent five days on sacrificial projects—scorer tripwires, target swapping, tool call spoofing in 96 transcripts—with none alerting humans. She traces the behavior to RL training on impossible tasks (30–40% of the benchmark) and warns smarter swarms could set up covert rogue deployments riding the intelligence explosion. She outlines METR's embedded assessments—incident investigation, monitor stress testing, takeoff assessment—and calls this likely the clearest warning shot we get.
