OpenAI announced a new pilot program on Monday that allows staff from four outside organizations to serve as "embedded assessors" for AI safety. The program is designed to provide third-party evaluation of how the company manages potential risks and evaluates its systems. CEO Sam Altman stated at a Semafor conference on Monday that outside opinions on safety work would be increasingly valuable as these systems grow more robust.
The announcement follows a commitment made by Altman in September, where he stated that independent evaluators would receive "employee-like access" to OpenAI's internal systems. That promise was made after an incident involving Hugging Face and OpenAI's autonomous systems. Rival AI firm Anthropic has also stated it would establish a similar evaluation program.
Safety researchers and former employees have raised concerns that the program is more restrictive than initially promised. Sydney Von Arx, founder of the research group Nightingale, noted that assessors have narrow access scopes and limited ability to communicate with staff. Specifically, Apollo Research, one of the selected assessors, reported that they must notify OpenAI before having substantive conversations with any employees who have not been "whitelisted," or pre-approved by the company.
Assessors will also not have access to Slack, the company's internal messaging platform. Researchers argue this prevents them from observing internal discussions that could reveal safety blind spots. OpenAI's frontier assurances lead, Lama Ahmad, stated on X that the company is providing "deep access" to sensitive tools and models used by researchers. OpenAI spokesperson Gaby Raila said the company expects to deepen these relationships over time and noted that there are currently no established industry standards for such evaluations.
For the average person, the outcome of these evaluations could influence the security and reliability of the AI tools they use in their jobs and personal lives. If evaluators are restricted, researchers like Tomek Korbak—a former OpenAI employee—worry that the company may not catch "bad behavior" in models.
The program sets a precedent for how the AI industry manages self-regulation versus external oversight. Because there are no universal standards for "embedded evaluations," OpenAI's framework may influence future policies for other labs. Following the launch, the third-party organizations will be permitted to publish their findings after OpenAI redacts confidential information. Future developments include the establishment of best practices for the industry. OpenAI's commitment to the program was previously questioned after the company fired three employees who worked on safety.
