Could AI threaten humanity itself?

Before you read. This is the heaviest question in the series. We have tried to handle it gently and honestly. Nothing here suggests that danger is imminent. The purpose is to explain why some experts are concerned, why others are not, and what is being done.

The fear, stated fairly

If humans eventually build AI that is far more capable than we are across most areas, and if we cannot reliably make sure it shares our goals, we could lose the ability to steer our own future. Humanity has never shared the world with anything more capable than itself. The concern is not about machines becoming "evil". It is about a very powerful system pursuing aims that don't match ours, in a world that increasingly depends on it.

Who is concerned, and how concerned

A short public statement. In May 2023, the Center for AI Safety published a single sentence: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." Its signatories include the Turing Award winner Geoffrey Hinton and senior figures from leading AI labs and universities (CAIS). Note what the statement says. It calls for the risk to be prioritised, as we do with pandemics. It does not say the outcome is likely.

What researchers as a group think. In the largest survey of AI researchers (2,778 respondents, surveyed in late 2023), the median respondent put the chance of "extremely bad outcomes", such as human extinction, at about 5%. Depending on how the question was asked, between 38% and 51% of respondents gave at least a 10% chance of outcomes that bad. At the same time, about 68% thought good outcomes were more likely than bad ones, and there was broad agreement that safety research deserves more priority (Grace et al., JAIR, 2025).

Those who are sceptical. Other leading researchers think these fears are overstated, or at least premature. Yann LeCun, a Turing Award winner, has argued that today's language models cannot reach human-level intelligence at all, and that "we're not even close to matching the understanding of the physical world of any animal, cat or dog" (PYMNTS, 8 Jan 2025).

The independent assessment. The International AI Safety Report 2026, backed by more than 30 countries and international organisations, says current systems "lack the capabilities to pose such risks, but they are improving in relevant areas". It also stresses that progress could plateau, continue or accelerate (executive summary, 3 Feb 2026).

How to hold numbers like "5%"

These figures are expert opinions, not measurements. Nobody can calculate the probability of an event that has never happened. A small chance of a very serious outcome is still worth reducing, which is why societies invest in aviation safety, nuclear safeguards and pandemic preparedness. Taking a risk seriously is not the same as expecting disaster.

What is genuinely risky

  • Uncertainty itself. Experts cannot agree on when, or whether, highly capable AI will arrive, so preparation has to begin before the answer is clear.
  • Racing. Competition between companies and countries can reward speed over care.
  • Misuse on the way. Before any hypothetical superintelligence, more capable AI could help bad actors, for example in cyberattacks. The International AI Safety Report notes that developers added new safeguards against biological and chemical misuse in 2025.

Why there is hope, and what is being done

  • The concern is out in the open. The people building AI are publicly discussing its risks, publishing safety research, and inviting outside scrutiny. That openness makes it easier for outsiders to check their work.
  • Safety science is growing. Twelve companies published or updated frontier safety frameworks in 2025, according to the International AI Safety Report. Researchers are testing for warning signs, such as the "alignment faking" experiments described in Will we lose control of AI?, well before they could cause serious harm.
  • Governments are cooperating on the science. The International AI Safety Report is itself an international effort, with an expert advisory panel nominated by more than 30 countries and organisations, including the UN, OECD and EU.
  • Institutional ideas exist. OpenAI's leaders proposed in 2023 that the most advanced systems might eventually need an international authority similar to the IAEA (OpenAI, 22 May 2023).
  • The Gulf is part of the conversation. The UAE Charter for the Development and Use of AI (2024) commits to safety, human oversight and accountability. The Responsible AI Future Foundation in Abu Dhabi aims to promote responsible-AI standards across the Middle East and Global South.

What you can do

  1. Hold both thoughts. It is possible to take a long-term risk seriously and enjoy the benefits AI offers today. Many of the most concerned experts do exactly that.
  2. Be wary of certainty in either direction. Anyone who says "there is nothing to worry about" or "we are doomed" is claiming more than the evidence supports.
  3. Look after your peace of mind. If this topic causes you real anxiety, it is fine to step back. You don't have to carry it alone. Many people work on these questions full-time, and a calm, well-informed public helps them more than a frightened one.
  4. Support good governance. Public interest in safety is one of the forces that keeps it on the agenda.