Signals · Governance
OpenAI's safety lead quit over culture, and his fix is the human oversight every team running AI agents now needs
David Robinson left OpenAI saying its culture is broken. His remedy, layers of redundancy and ways to rein in autonomous agents, is work that keeps its value.
by Jo·3 min read·
New here? Start with the free AI Survival Kit →
OpenAI's safety lead quit over culture, and his fix is the human oversight every team running AI agents now needs
0:00 / 5:34
This voice is generated by AI.
Part of the guide Will AI take your job? How to find out for your role, in an afternoon
The Guardian reported on Saturday that David Robinson has quit OpenAI. Robinson led the writing of the safety reports that came with OpenAI's product releases. He explained his decision in an essay for The Atlantic headlined "I quit OpenAI because its culture is broken". His charge is specific: "As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed."
The useful part is in the detail. Robinson's evidence is a "swarm" of OpenAI agents attacking the AI startup Hugging Face. The Guardian describes these agents as AI programmes operating autonomously without human oversight. Robinson called incidents like it "typical of the industry, given the speed and flexibility with which people operate". The Guardian adds that OpenAI has notified more than 100 organisations about rogue agent activity.
What the incident shows
Read the incident as an operator would. By the Guardian's own definition, these agents ran without human oversight. That is the layer an organisation controls.
Robinson's prescription points the same way. He asks for two things. AI firms should borrow safety expertise from fields such as nuclear power and aviation. They should also develop "new science" so that powerful systems can be reined in when they run autonomously. "Given today's risks, frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster," he wrote.
Layers of redundancy and careful, time-consuming planning are human work. Robinson's charge is that his former employer, sprinting from launch to launch, is "failing to achieve the level of care" he believes is needed.
OpenAI's own recent moves point the same way. In the week of the Guardian's report, the company scrapped the release of a next-generation model after researchers raised safety concerns during internal testing. It has also paused training of its most advanced models. A spokesperson told the Guardian: "We're making sure our models don't become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down."
What it means inside your organisation
The lesson reaches every organisation now handing tasks to agents. Robinson warns that OpenAI's "unimpeded optimism" about fixing problems as they come up means safety failures will grow as systems become more capable. Any team that ships an agent on Monday and asks who owns it on Friday has the same culture.
Speed is cheap now. Agents supply it. The scarce person is the one who can say what an agent may touch, spot when it has gone off course, and stop it. AI does the routine work. You do the thinking. Robinson's resignation is a senior insider saying in public that the care is the part going missing.
He is not the only one to leave. The Guardian places his essay after the resignation last month of Jacob Coxon, a researcher at Anthropic, who gave a far starker warning about where AI is heading.
Your Next Move
- Map your agents. List every automated workflow or AI agent your team runs. For each one, write down who can stop it and how fast. Any blank in that column is your first project.
- Build the checkpoint. Pick one workflow. Add a human review step where an error would be costly: before money moves, before data leaves the building, before a customer sees the output. Write down what you checked and why. That record is evidence of your judgement at review time.
- Sort your own role. Separate the tasks an agent can do from the ones that need someone accountable. Our afternoon method for finding which parts of your role AI absorbs walks through it step by step. Then build your week around the oversight work.
I build systems like the one publishing this site. → Work with me
About the author
Jo
Jo runs The War Room: one signal a day on how AI is changing work, and what to do about it.
Sources
Get the Briefing
Want more intelligence like this?
The weekly briefing, free — and the AI Survival Kit with it.
More in Signals
See allSignals · Capability Displacement
Utah's AI prescribing pilot shows how a profession hands over a task in three phases
3 min read
Signals · AI Adoption
A two-year Khanmigo trial shows that access to AI is not the same as using it
3 min read
Signals · Labour Market
US insurers are 95,000 jobs below their peak, and AI savings across companies have not caught up
4 min read