Artificial intelligence: is there really more than a 10% chance it could kill us?
As an Anthropic researcher puts the odds of AI-driven human extinction above 10%, this piece asks what that figure really means, whether such warnings also serve the AI giants' commercial interests, and why Europe is regulating without waiting for an answer.
By Martin Bertrand
1,145 views
Photo: by Martin Bertrand
The figure is striking. On September 8, Evan Hubinger, head of alignment science at Anthropic, the company behind Claude, publicly estimated that there was a “greater than 10% chance in the next decade” that artificial intelligence could cause human extinction. His statement came just hours after Anthropic researcher Jacob Coxon resigned, warning about a race towards increasingly powerful systems that could escape human control. The episode reignited an old debate: those building the most advanced AI systems are also among those issuing the strongest warnings about their dangers.
Those building AI are sounding the alarm, and some are calling for a slowdown
Dario Amodei, CEO of Anthropic, subsequently called for slowing down, without stopping, the development of AI capabilities. Sam Altman, CEO of OpenAI, and Elon Musk, founder of xAI, supported the idea. These warnings did not begin in September 2026. In 2023, Altman and Amodei were already among the leading figures in AI publicly warning about scenarios that could go as far as human extinction.
The question, however, is what the famous 10% figure actually means. It is not an empirical measurement of risk from which a direct probability can be established. Hubinger is making a judgement about a future scenario based on several assumptions that are difficult to verify today. But he is not alone in suggesting figures of this magnitude. A survey of 2,778 researchers who had published at major AI conferences asked them to assess various scenarios linked to advances in artificial intelligence. For one question concerning a loss of control that could lead to human extinction or the severe and permanent disempowerment of humanity, the median response was 10%.
Participants were given several different formulations of questions dealing directly with extinction or a permanent loss of control. Depending on the wording, between 41.2% and 51.4% of the researchers concerned assigned a probability above 10% to one of these scenarios. This is different from the 38% of respondents who, in another and broader question, gave at least a 10% chance to “extremely bad” outcomes such as human extinction. These figures therefore measure experts’ probabilistic judgements, not an objective estimate of the risk. The authors themselves note that experts do not necessarily have any particular ability to predict developments of this kind over the long term.
Could this fear also serve the interests of AI giants?
Arthur Mensch, CEO of Mistral AI, offers a far more critical reading of these warnings. In an interview with Le Monde published on September 24, he accused some American AI labs of turning their own technical incidents into a “marketing argument” and attempting “regulatory capture”. In his view, presenting AI as an extremely dangerous technology could help justify rules that would effectively reserve the market for a handful of companies able to bear the costs and constraints imposed by regulation. “AI is software.
It can be controlled,” he said.
This criticism does not prove that the concerns expressed by OpenAI or Anthropic are artificial. Anthropic has placed AI safety at the centre of its public positioning since its
creation and devotes part of its research to the risks of losing control over advanced systems. But Mistral’s position is not economically neutral either. The French company advocates a more open approach to AI and is trying to gain ground against its American competitors. It therefore also has a direct interest in how future rules are written.
The debate is therefore not only about different views of risk. It also reflects competing business strategies. Major American labs can genuinely fear certain dangers while also having an interest in shaping the regulation that will govern them. Their competitors can, in turn, challenge these scenarios while defending a more open market.
Europe has already started regulating
The European Union has not waited to find out whether the risk of extinction is 5%, 10% or 25%. The AI Act, which entered into force in 2024, is gradually introducing different obligations. Rules applying to providers of general-purpose AI models have been in force since August 2, 2025. They include requirements on technical documentation and copyright compliance, with additional obligations concerning risk assessment and mitigation for models considered to pose systemic risks. Since August 2, 2026, the European Commission has had enforcement powers over these obligations for providers of general-purpose AI models, including the ability to impose fines.
Henna Virkkunen, Executive Vice-President of the European Commission responsible for tech sovereignty, security and democracy, has said that the most advanced models create risks “on an entirely new scale”. She presents the AI Act as a risk-based framework intended to provide legal certainty for companies while protecting the public interest.
The European regulation therefore does not settle the question of the 10%. It starts from a more concrete premise: some risks already exist, while others, potentially far more serious, are considered plausible enough to justify preparation. The real question becomes less about fixing a percentage that cannot be verified today, and more about deciding which risks should be regulated, how far that regulation should go, and how much influence the companies concerned should have in shaping the rules.