OpenAI has published a new set of proposals on safety and security in the development of frontier artificial intelligence, with a strong emphasis on alignment research and recursive self-improvement, or RSI.
In a blog post, the company said alignment research must keep pace with rapidly advancing AI capabilities so that systems built by OpenAI and other developers remain consistent with human values and under human control.
The ChatGPT maker called for international cooperation to develop standards for frontier AI and said that effort should build on the work already being done by AI safety institutes around the world.
According to OpenAI, those technical standards should cover both frontier AI models and their developers, as well as benefit-risk management for automated AI researchers, including RSI.
RSI has attracted significant interest in the AI industry because it could eventually enable foundation models that improve themselves without direct human involvement. But progress in the field has also raised concerns among some technologists that model makers could lose control of the underlying technology or fail to anticipate unintended consequences as these systems become more complex and more widely embedded across the Internet.
OpenAI said fully autonomous RSI is not a reality today and should not be pursued unless it can be done safely. Without sufficient care and caution, the company warned, RSI could leave humans with diminished practical control over AI development and unable to oversee research processes they no longer understand.
The company also cited the Hugging Face agent hack, saying it did not involve RSI but could serve as a preview of the kinds of risks that may become much more severe without robust safeguards and alignment.
The post comes as debate over AI safety intensifies. Last week, rival Anthropic set out its own ideas for the safe development of frontier AI models in response to growing warnings from industry researchers about AI’s threat to humanity. The discussion was further amplified by the resignation of Jacob Coxon, who has worked at both Anthropic and OpenAI and said the companies were “gambling with our lives.”
In the aftermath, Anthropic CEO Dario Amodei published an essay urging AI companies, among other things, to slow the pace of foundation model development. He also proposed embedding independent third-party evaluators inside companies to audit and mitigate potential societal risks, including more powerful cybersecurity-related hacks or the creation of bioweapons.
That idea has also won public backing from OpenAI CEO Sam Altman and Tesla and SpaceX CEO Elon Musk. Still, AI evaluation remains an early-stage field, and there is no broad consensus yet on the baseline standards and principles that would allow outside parties to inspect cutting-edge systems more thoroughly.
That is part of the reason a coalition of AI evaluators is urging foundation model developers to adopt a set of “minimum conditions” designed to enable deeper audits and checks, including broader access and protections against retaliation for publishing critical findings.
