The handover discussion in Silicon Valley is tied to the prospect of recursive self-improvement: AI systems improving themselves and producing more capable successors, potentially reaching a level of superintelligence that people could no longer control. Altman said he and Brockman differ on how best to prevent that outcome and what difficulties to expect, but share the goal of keeping people in control and avoiding an accidental or deliberate transfer of authority.
Brockman said increasingly capable models raise the stakes for alignment, safety and security, and may require new monitoring methods. Researchers now monitor models’ chain of thought—the reasoning they write out while completing tasks—but he warned that more advanced systems can do more without displaying that reasoning. As a result, chain-of-thought monitoring may become less effective, he said, adding that the challenge is technical.





Comments (0)
No comments yet. Be the first!