India, Sept. 10 -- Anthropic researcher Jacob Coxon resigned on Wednesday and wrote on X that AI companies were "racing straight to self-improving superintelligence and gambling with our lives". An expert in pretraining - the first and largest stage of building an AI model, in which it learns from vast quantities of text before being trained to do anything in particular - Coxon worked first at OpenAI and then at Anthropic. Anthropic's alignment science lead, Evan Hubinger, replied to Coxon's post and agreed with him. "Jacob is correct here-we really do earnestly believe AI could kill all humans!" Hubinger wrote. "I personally think it is >10% within the next decade."

This is, of course, not the first time such alarm has been sounded. Geo...