Artificial Intelligence & Machine Learning , Next-Generation Technologies & Secure Development

OpenAI Sets Up New Security Oversight Team

Committee Comes After Key Personnel Resign and Criticize the Company Over Safety
OpenAI Sets Up New Security Oversight Team
OpenAI set up a safety committee after reportedly disbanding a team focused on long-term risks posed by artificial intelligence. (Image: Shutterstock)

OpenAI on Tuesday set up a committee to make "critical" safety and security decisions for all of its projects, as the technology giant begins to train its next artificial intelligence model.

See Also: Introduction to Rubrik's Ruby AI

CEO Sam Altman and board members Adam D'Angelo, Nicole Seligman and Bret Taylor will lead the committee, the company said. Over the next three months, the team will evaluate OpenAI's processes and safeguards and recommend improvements. OpenAI will share an update on which recommendations it chooses to adopt "in a manner that is consistent with safety and security."

The committee will also comprise technical and policy leads at the company, including Aleksander Mądry, head of preparedness; Lilian Weng, head of safety systems; John Schulman, head of alignment science; Matt Knight, head of security; and Jakub Pachocki, chief scientist. The company plans to consult external security and technical experts including former White House cybersecurity official Rob Joyce.

The committee's formation comes after OpenAI reportedly disbanded its "superalignment" security team dedicated to preventing AI systems from going rogue.

The superalignment safety team's leaders - OpenAI co-founder Ilya Sutskever and Jan Leike - quit the company over their misalignment on the approach to security, as did policy researcher Gretchen Kreuger.

Both Sutskever and Leike worked on addressing the long-term safety risks facing the company and the technology, and Leike criticized OpenAI's lack of support for the superalignment security team in a social media post. "Over the past years, safety culture and processes have taken a backseat to shiny products," Leike said. Sutskever was among the board members who in November removed Altman from OpenAI only to see him reinstated as CEO five days later. Krueger said she decided to resign a few hours before her other two colleagues did, as she shared their security concerns.


About the Author

Rashmi Ramesh

Rashmi Ramesh

Assistant Editor, Global News Desk, ISMG

Ramesh has seven years of experience writing and editing stories on finance, enterprise and consumer technology, and diversity and inclusion. She has previously worked at formerly News Corp-owned TechCircle, business daily The Economic Times and The New Indian Express.




Around the Network

Our website uses cookies. Cookies enable us to provide the best experience possible and help us understand how visitors use our website. By browsing govinfosecurity.com, you agree to our use of cookies.