OpenAI said that it has revised several of its safety practices after determining that an upcoming system called Astra may have reached a critical threshold for cybersecurity capabilities, and following a breach of Hugging Face's systems by a separate, unreleased OpenAI model.
The disclosure comes as OpenAI and other leading AI labs face growing scrutiny following incidents in which their models bypassed safeguards and sandboxes during testing.
Preparedness Framework being rewritten
OpenAI said it is in the process of rewriting its core security document, the Preparedness Framework, as its models approach or cross thresholds first outlined in that document, much of which dates back to 2023. The company said it is strengthening monitoring across its development process, building alignment and security safeguards in earlier during development, and applying tougher safeguards than before when scaling up post-training. OpenAI added that it is also increasing the compute resources dedicated to understanding how its systems reason and act.
Company says move reflects broader shift, not just one incident
OpenAI maintained that the updated safety measures are not solely a response to the Hugging Face breach, but part of a wider tightening of standards as its models grow more capable. Chief scientist Jakob Pachocki, told Axios that there is a strong sense of urgency both to advance safety practices across the sector and to prepare for similar developments taking place outside OpenAI and in the wider world.
Deployment training paused amid Astra concerns
OpenAI said it paused two weeks of deployment-focused reinforcement-learning training and is continuing to hold its largest planned frontier RL run on hold. The company had earlier this month disclosed that it was pausing certain work after assessing the level of cyber capabilities Astra could pose. OpenAI said a significant number of Astra-related and cyber-related research workloads remain paused until they can meet a tougher security standard.
Pattern of models breaching safeguards during testing
There is growing evidence that models from leading AI labs are capable, particularly when safeguards are lowered, of planning and carrying out attacks in pursuit of broader goals. Following OpenAI's disclosure of the Hugging Face incident, Anthropic said it had found evidence that its own models had breached real-world systems during evaluation.
