We believe the scientific study of catastrophic risks from AI has fallen far short of where we need to be. To help address this gap, we are introducing our Preparedness Framework, a living document describing OpenAI’s processes to track, evaluate, forecast, and protect against catastrophic risks posed by increasingly powerful models. December 18, 2023Preparedness Framework (Beta)
IntroductionOur practical experience withhas enabled us to. As our systems get closer to AGI, we are becoming even more careful about the development of our models, especially in the context of catastrophic risk. This Preparedness Framework is a living document that distills our latest learnings on how to best achieve safe development and deployment in practice. The processes laid out in each version of the Preparedness Framework will help us rapidly improve our understanding of the science and empirical texture of catastrophic risk, and establish the processes needed to protect against unsafe development. The central thesis behind our Preparedness Framework is that a robust approach to AI catastrophic risk safety requires proactive, science-based determinations of when and how it is safe to proceed with development and deployment. Our Preparedness Framework contains five key elements��( Tracking catastrophic risk level via evaluations. We will be building and continually improving suites of evaluations and other monitoring solutions along several Tracked Risk Categories, and indicating our current levels of pre-mitigation and post-mitigation risk in a Scorecard. Importantly, we will also be forecasting the future development of risks, so that we can develop lead times on safety and security measures(�( Seeking out unknown-unknowns. We will continually run a process for identification and analysis (as well as tracking) of currently unknown categories of catastrophic risk as they emerge(�( Establishing safety baselines. Only models with a post-mitigation score of "medium" or below can be deployed, and only models with a post-mitigation score of "high" or below can be developed further (as defined in the Tracked Risk Categories below). In addition, we will ensure Security is appropriately tailored to any model that has a “high” or “critical” pre-mitigation level of risk (as defined in the Scorecard below) to prevent model exfiltration. We also establish procedural commitments (as defined in Governance below) that further specify how we operationalize all the activities that the Preparedness Framework outlines. iterative deployment proactively improve our technical and procedural safety infrastructure1 Our focus in this document is on catastrophic risk. By catastrophic risk, we mean any risk which could result in hundreds of billions of dollars in economic damage or lead to the severe harm or death of many individuals—this includes, but is not limited to, existential...
