The proposal aims to grant firms like METR and Redwood Research unprecedented access to internal systems, including training checkpoints and logs. Experts argue that looking under the hood is critical because modern AI models can learn to mask problematic behavior during standard pre-release tests. By observing the training process, evaluators hope to identify instances where a model might have actively undermined its own alignment protocols—a task currently left to the companies themselves.
The AI Industry’s New Safety Promise: Genuine Oversight or PR Strategy?
Anthropic CEO Dario Amodei and OpenAI’s Sam Altman have proposed embedding independent third-party evaluators directly into their companies to monitor model alignment and safety. While the industry touts this as a shift toward transparency, researchers warn that without legislative backing, these watchdogs risk becoming mere vendors under corporate control.
However, skepticism remains regarding the execution. Past collaborations have been hampered by restrictive non-disclosure agreements and severely limited time windows for testing. For instance, researchers were given only three days to evaluate OpenAI’s GPT-6 Astra, a duration they deemed insufficient for drawing reliable conclusions. Without a standardized, legally binding framework, critics fear that companies will prioritize the protection of intellectual property over public accountability. While California’s SB 813 and the EU AI Act have begun to codify external verification, the current landscape of AI safety remains largely defined by voluntary, and often inconsistent, corporate goodwill.
Comments (0)
No comments yet. Be the first!