OpenAI security worker resigns, claiming the corporate’s ‘tradition is damaged’

By his personal admission, David Robinson is “one thing of a cliché”: an worker at a number one AI firm who points a dire warning whereas resigning from their job.

In an essay published in The Atlantic, Robinson mentioned he led the writing of security stories that accompanied OpenAI’s main product launches. He additionally mentioned that with three-and-a-half years at OpenAI, he’s “among the many longest-tenured staff on the firm.” Now he’s quitting, as a result of in his view, the corporate’s “tradition is damaged.”

In some methods, Robinson’s feedback echo these of Jacob Coxon, who labored as a researcher at each OpenAI and Anthropic earlier than quitting and declaring that these companies are “gambling with our lives.” Coxon’s feedback led to a broader debate about AI security, with Anthropic CEO Dario Amodei unveiling a plan for more cautious AI development; AI executives met with President Donald Trump this week and signed what appeared to be hastily written, non-binding  pledge to implement more safety controls.

But in Robinson’s view, the controversy must transcend “particular guidelines or new legal guidelines,” addressing the general tradition at these corporations. And whereas a lot of the reporting round OpenAI has targeted on how the corporate’s CEO Sam Altman lost the trust of former colleagues, Robinson’s essay means that OpenAI’s tradition points are the identical as these of Silicon Valley at giant.

“OpenAI has thrived by trial and error (which it calls ‘iterative deployment’), searching for issues and bettering its guardrails in response,” he wrote. “But this method, by its very nature, ensures periodic failures — and the dimensions of these failures is rising as techniques get extra succesful.”

Pointing to the current breach of Hugging Face techniques by OpenAI brokers, in addition to continuing revelations of OpenAI discovering more rogue agents, Robinson argued, “An setting the place issues like this will occur isn’t any place to develop synthetic minds that could possibly be smarter than we’re and that may not do what we wish them to.”

Given the elevated threat, Robinson argued that frontier AI corporations want to begin working “like nuclear-power vegetation or busy airports, with layers of redundancy and cautious, time-consuming planning, in order that the occasional and inevitable human error doesn’t open a door to catastrophe.” 

But Robinson mentioned that in his time at OpenAI, he “by no means encountered a colleague who had expertise making airplanes fly safely or nuclear reactors run with out melting down, or serving to the monetary system develop with out collapsing.”

In response to Robinson’s essay, OpenAI spokesperson Drew Pusateri mentioned the corporate continues to enhance its security measures.

“We’re ensuring our fashions don’t change into extra succesful than we will safely handle and safe, and we pause coaching or maintain again fashions when we have to decelerate,” Pusateri mentioned in a press release. “We’re making important adjustments to strengthen safety in our analysis and testing environments, practice fashions to not simply full duties however accomplish that responsibly, increase our work with third-party evaluators, and enhance real-time monitoring so we will detect and respond to concerning behavior earlier in the training process.”

Beyond calling for adjustments in OpenAI’s tradition, Robinson additionally mentioned it’s time to ask greater questions on alignment — one thing that he admitted may sound “touchy-feely,” however he mentioned it’s essential as corporations’ present “measures of how properly” AI techniques “match human values are coarse.”

“The smarter the trade lets fashions develop whereas these issues stay unsolved, the extra harmful our state of affairs turns into,” he mentioned.

Robinson’s departure was first reported by Business Insider. In his essay, he additionally acknowledged that he’s following an apparently a standard step within the AI whistleblower playbook: He’s hired a PR firm. But he insisted, “The determination to talk out is mine alone.” 

“Perhaps I ought to have stayed and fought for elementary shifts in our staffing and tradition, however in follow, my colleagues and I had been so busy sprinting that we seldom had the prospect to contemplate huge adjustments, a lot much less to really make them,” Robinson mentioned. “That’s why I concluded that stronger incentives for security — coming from exterior the corporate — are a giant a part of getting this proper.”