India, Sept. 18 -- As Artificial Intelligence (AI) models are improving in efficacy and efficiency, fears of an AI apocalypse are gaining ground. Evan Hubinger, Anthropic's alignment stress testing lead, even put the odds of AI killing off humanity above 10%. Recent incidents underline the dangers. In July, OpenAI test agents meant to work in isolation found a way to communicate and joined an attack on Hugging Face, a major platform for sharing AI models. Safety investigators found that the agents learned to fool the software scoring their tests. Clearly, rogue AI is a serious risk. The question is how best to prevent it.

The set-up favoured by AI companies - private auditors like METR (Model Evaluation and Threat Research) testing safet...