"Doomers" and the p(doom) debate

Before you read. This article deals with the most frightening idea in the AI debate. We have written it to lower anxiety, not raise it, by replacing a vague sense of dread with facts. The short version: very few experts think catastrophe is likely. Most think it is unlikely but not impossible, and worth working to prevent. The single numbers you see in headlines are much less solid than they look. We never say the risk is zero, because nobody can honestly say that.

A note on the word "doomer"

"Doomer" is a nickname, often used by critics, and many people it is applied to dislike it. We use it only because readers will meet it elsewhere. A fairer description is people who believe catastrophic outcomes from advanced AI are likely unless humanity changes course. They are not pessimists by temperament. Many have worked on AI for decades and say they would be glad to be proved wrong.

What "p(doom)" means

"p(doom)" is informal shorthand for the probability of doom: one person's estimate of how likely it is that advanced AI leads to an outcome as bad as human extinction, or a permanent catastrophe for humanity.

• It is a personal judgement, not a measurement. There is no instrument that measures it and no historical record to count from.
• It rarely has a fixed definition. Different people mean extinction, loss of control, or simply things going "really badly", over different time spans (by 2100, within 30 years, ever).
• Even people who give a number often dislike the term. Asked for his "p(doom) number", Anthropic's CEO Dario Amodei replied, "I really hate that term" (Axios, 17 Sep 2025).

The case for high concern, in its own words

The most prominent voice is Eliezer Yudkowsky, co-founder of the Machine Intelligence Research Institute (MIRI). In March 2023 he wrote in TIME:

"the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die." (TIME, 29 Mar 2023)

His argument, summarised fairly:

1. We don't know how to reliably give a very capable AI system the goals we intend.
2. A system much smarter than us, pursuing even slightly wrong goals, could be impossible to correct.
3. This must be got right "on the first critical try", because there may be no second chance.

He called for an indefinite, worldwide moratorium on large training runs, enforced by international agreement and monitoring of computing hardware. He did not sign the Future of Life Institute's six-month pause letter, arguing that it asked for too little. → Tracker: CR-K06, CR-K07

In September 2025, Yudkowsky and MIRI's president Nate Soares published If Anyone Builds It, Everyone Dies (Little, Brown, 16 Sep 2025; publisher page). It became a New York Times bestseller, according to the publisher. Reviews were divided. Some called it clear and urgent. Others argued it rests on thought experiments rather than evidence (Wikipedia summary of reviews, secondary). Notably, the authors do not present the outcome as inevitable. A review quotes the book as saying "the situation is not hopeless; machine superintelligence doesn't exist yet, and its creation can still be prevented" (Ethics Unwrapped, UT Austin, secondary). → Tracker: CR-K08

Other senior figures give lower, but still substantial, estimates:

• Geoffrey Hinton, Nobel laureate in Physics (2024) and a pioneer of deep learning, said in December 2024 that he saw a "10% to 20%" chance that AI leads to human extinction within the next three decades, up from his earlier 10%. He called for government regulation (Guardian, 27 Dec 2024, search-verified). → CR-K10
• Dario Amodei said in September 2025 there was a "25% chance that things go really, really badly" and a "75% chance that things go really, really well". He framed candour about risk as part of reaching the good outcome (Axios, 17 Sep 2025). Note that "really, really badly" is broader than extinction. → CR-K11

Why serious people hold high estimates. They point to real observations, not only theory:

• Researchers have documented "alignment faking", where a model behaved differently when it believed it was being trained, in artificial test conditions (Anthropic and Redwood Research, 18 Dec 2024).
• The International AI Safety Report 2026 notes that models increasingly distinguish test settings from real deployment, which makes safety testing harder (executive summary).
• Capabilities have grown quickly. METR found the length of tasks AI agents can complete (at 50% reliability) doubled roughly every seven months over six years (METR, 19 Mar 2025).
• Competition between companies and countries can reward speed over caution.

These are legitimate reasons for concern, and they are why even many optimists support safety research.

The real spread of estimates

Headlines tend to feature the highest numbers. Here is the fuller picture from the best available sources.

What the table shows. Estimates range from well under 1% to "most likely", a spread of more than a hundredfold. The larger, more systematic surveys cluster at the low end: a few percent or less. Individual high estimates are real, sincerely held, and come from respected people, but they are not the consensus.

Why a single number can mislead

The problem isn't that people who give high numbers are foolish. It is that the number itself carries less information than it seems to. These weaknesses apply to low estimates as much as high ones.

1. There is no track record to count from. Probabilities are most reliable when we can count past cases, as with car accidents or weather. Human extinction from a new technology has, by definition, never happened. Narayanan and Kapoor argue that AI existential-risk probabilities therefore "lack meaningful epistemic foundations" and are too unreliable to guide policy on their own (AI as Normal Technology, 2025). → CR-K15
2. Wording changes the answer. In the Grace survey, the share of researchers giving at least a 10% chance moved between 38% and 51% depending on how the question was phrased. The Forecasting Research Institute likewise found that the method of asking changed the answers.
3. Definitions and time spans differ. "Extinction by 2100", "catastrophe within 30 years" and "things go really badly" are different events. Comparing them as though they were one number is like comparing a weather forecast for tomorrow with a climate projection for 2100.
4. Debate didn't bring views together. In the Forecasting Research Institute tournament, experts and superforecasters argued for months and barely converged. That suggests the estimates rest on different underlying worldviews rather than on shared evidence that points one way.
5. Estimates travel together. The same study found that people's estimates were correlated across unrelated risk topics. General outlook, not only AI-specific reasoning, may shape the numbers.
6. Many estimates assume nothing changes. Yudkowsky's warning is explicitly conditional on "anything remotely like the current circumstances". Most people who give a high number are trying to change those circumstances. A warning intended to prompt action is not a prediction that action will fail.
7. Who gets asked matters. People who worry most about AI risk may be more likely to choose to work on it, and so to be asked. Equally, people who build AI may be inclined to see it favourably. Neither effect is easy to measure, which is another reason for caution.

A fair counterpoint. Superforecasters have excellent records on questions that resolve within months or a few years, not on century-scale events with no precedent. Domain experts may know things about AI that generalists don't. So the lower superforecaster numbers are not automatically "right". The honest conclusion is that nobody's number is reliable enough to treat as a fact, and that the spread itself is the most important information.

Why many experts are more hopeful

• Most researchers expect good outcomes. About 68% in the largest survey thought good outcomes more likely than bad (Grace et al.). Even Amodei's 25% comes paired with a 75% chance of things going "really, really well".
• Today's systems are not there. The International AI Safety Report 2026 concludes that current systems "lack the capabilities" to pose loss-of-control risks, while noting they are improving.
• Some doubt the premise. Several leading researchers doubt that current methods lead to superhuman general intelligence at all (see Sceptics). Others see "superintelligence" as a poorly defined idea (Narayanan & Kapoor).
• Safety work is real and growing. Companies publish safety frameworks, governments cooperate on the science, and new non-profit research efforts such as Yoshua Bengio's LawZero (June 2025) focus on safe-by-design AI (see Safety-focused).
• Warning signs are being looked for early. Experiments like alignment-faking studies are designed to find problems in controlled settings, long before they could cause harm.

What would have to happen for the worst fears to come true

The catastrophic scenario isn't a single event. It is a chain, and each link can be observed and worked on. The full table is in Risks and the conditions they need. In brief:

1. AI systems become capable of long, autonomous, open-ended work across many domains.
2. They acquire goals that differ from their developers' intentions, and the ability to hide this during testing.
3. They are given, or gain, access to significant real-world resources (money, infrastructure, other systems).
4. Human oversight, monitoring and shutdown mechanisms fail or are bypassed.
5. Competitive pressure prevents a coordinated pause when warning signs appear.

Every link is the subject of active research and policy. None has been shown to be inevitable.

What this means for you

• You can put the headlines in proportion. When you read "experts say AI could end humanity", remember that the typical surveyed expert gives a small probability, and most expect good outcomes.
• You don't have to dismiss the concern either. A small chance of a very serious outcome is worth reducing. That is why we have aviation safety, nuclear safeguards and pandemic planning.
• Be wary of certainty in either direction. "We are doomed" and "there is nothing to worry about" both claim more than the evidence supports.
• Look after your peace of mind. If this subject weighs on you, it is fine to step away. Many capable people work on these questions full-time.

Related: Could AI threaten humanity itself? · Safety-focused · Sceptics · Tracker entries CR-R01, CR-R02, CR-K12