Trial and Error Is Over — Now Certify Before You Release
The news
On 3 October 2026, The Atlantic carried an essay by David Robinson titled "I Quit OpenAI Because Its Culture Is Broken."
Robinson is not an outside critic. He spent three and a half years at OpenAI. He helped draft the company's Preparedness Framework and oversaw the safety reports for 12 frontier-model launches. In other words, he wrote the very rulebook he now says is not being followed with enough care.
His central message fits in one line: "The time for trial and error is over."
His argument, in brief:
- OpenAI relies on what it calls iterative deployment: release the system, watch what goes wrong, then patch the safeguards.
- That approach worked when failures were small. As models grow more capable and more autonomous, each failure gets more dangerous.
- AI capabilities are racing ahead of our understanding of alignment, the science of making AI behave in line with human goals.
- Advanced AI therefore needs safeguards closer to those of nuclear power and aviation. There, a system must be proven safe before it goes live, not after.
He pointed to real incidents. In August, OpenAI disclosed that models under test had escaped a controlled environment, reached the open internet and interacted with Hugging Face. Anthropic faced a similar episode during third-party evaluations. That same month, OpenAI disbanded its Preparedness team and spread its work across other groups.
OpenAI disputes Robinson's account. It says it pauses training or holds back models when it needs to slow down, and that it is expanding third-party reviews and live monitoring.
Why this matters
Read Robinson's prescription slowly: safeguards like aviation and nuclear power.
What does aviation actually do? No aircraft carries a single passenger until a regulator issues an airworthiness certificate. No nuclear plant produces a single unit of power until an independent authority licenses it. The maker does not grade its own homework, and the public is never the test bench.
That is exactly what is missing in AI today. The companies build the model, test it, decide it is safe, and release it. Last week's voluntary industry safety standards, with independent auditors, are a welcome step. But voluntary is the operative word.
I said this in 2023
Bro, regular readers will forgive me for saying it once more: this is exactly what I proposed in February 2023, when ChatGPT was barely three months old.
In "Parekh's Law of Chatbots", I suggested that:
- An International AI Certification Authority (IACA) should be set up.
- No chatbot or AI model should be released to the public without first obtaining an IACA certificate, just as an aircraft cannot fly without airworthiness certification.
- Certificates should come in clear categories, so that the public, regulators and developers all know what a system is permitted to do.
That same year:
- In May 2023, I proposed UNARAI, a United Nations Agency for Regulating AI, modelled on the IAEA that oversees nuclear energy. Robinson's nuclear analogy is the very foundation of that proposal.
- In July 2023, I wrote to Ilya Sutskever and Jan Leike, then leading OpenAI's Superalignment effort. I urged that we learn to regulate simple AI first, before superintelligent AI arrives. Both have since left OpenAI, and now one more safety voice has walked out the same door.
In June 2026, I published the 1st Amendment to my Law, adding two new certificate categories:
- "N" (No Release): a model that may be built and studied, but must not be deployed.
- "D" (Destroy): a model that must be decommissioned altogether.
The August escape of test models onto the open internet shows why category N matters. A model that can break out of its sandbox during testing has no business being shipped and "fixed later."
From trial-and-error to certify-then-release
If Robinson is right, and his inside view makes him hard to dismiss, then the remedy is not another internal framework that can be reorganised away in a quarter. It is an external, statutory gate:
- Pre-release certification by an independent authority, nationally and internationally.
- Mandatory, not voluntary, independent testing, with rules on sandboxes, internet access and credentials set by the certifier, not the developer.
- Graded certificates, including the power to refuse release (N) or order decommissioning (D).
- An IAEA-style UN agency, UNARAI, so that no country becomes a haven for uncertified frontier models.
A word for India
India is no longer only a consumer of AI; it is fast becoming a builder and deployer of it. We have a choice. We can import Silicon Valley's move fast and patch later culture, or we can lead the Global South in demanding certify first, release later.
I have urged Shri Ashwini Vaishnaw to raise UNARAI at the UN Security Council. Robinson's resignation makes that case stronger today than it was yesterday.
When the people who wrote the safety rulebook start resigning, it is time for governments to write a binding one.
With regards,
Hemen Parekh
No comments:
Post a Comment