Anthropic proposes federal AI safety bill targeting catastrophic risks

The proposed legislation targets specific, high-stakes scenarios, including the creation of biological weapons, models evading developer controls, or AI-facilitated cyberattacks. These thresholds for catastrophic risk mirror those established in California’s Senate Bill 53, marking a rare instance of an industry player advocating for codified federal constraints. Under the draft, independent evaluators registered with the U.S. Commerce Department would assess model safety. While these experts could be funded by the labs themselves, the bill mandates that payments remain independent of evaluation outcomes.

Upon identifying a critical threat, evaluators would have 72 hours to notify the Commerce Department, which would then be required to petition a D.C. judge to halt distribution. The bill explicitly strips federal agencies of the unilateral power to block a model, a move that places the final decision within the judiciary. This approach contrasts with the broader industry sentiment; while some leaders like Nvidia’s Jensen Huang argue against new laws entirely, Anthropic is pushing a framework that formalizes the role of third-party monitors. Critics, however, remain skeptical about whether a system relying on lab-funded evaluators can truly ensure objective oversight of the industry's most powerful technologies.