Control and long-term risk
Loss of control, catastrophic and existential risk, and the debate around them.
- If AI becomes far more capable than us, how would we live alongside it? A calm guide to a speculative question
Some experts think AI could one day exceed human ability in most areas. Others think today's methods won't get there. This page explains the terms, sets out the range of expert views without taking sides on timelines, and describes how humans and very capable AI might interact, with speculation clearly labelled.
- CR-R01
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
- CR-R02
Median researcher: 5% chance of "extremely bad outcomes (e.g. human extinction)". 38–51% give at least 10%. About 68% think good outcomes more likely than bad.
- CR-R03
More than half of researchers say misinformation, authoritarian control and worsening inequality warrant substantial or extreme concern.
- CR-R04
On loss of control: "Current systems lack the capabilities to pose such risks, but they are improving in relevant areas." Models increasingly distinguish test from deployment.
- CR-R05
12 companies published or updated frontier AI safety frameworks in 2025.
- CR-R06
Experimental evidence of "alignment faking" in Claude 3 Opus under artificial conditions (12% prompted; 78% alignment-faking reasoning after RL). Authors: does not show malicious goals.
- CR-R07
Eventually an IAEA-like international authority may be needed for systems above a capability threshold.
- CR-R08
"AI seems likely to enable much better propaganda and surveillance."
- CR-R09
Content Credentials are "not a cure-all for misinformation". They show provenance, not truth.
- CR-R10
Neuralink: 21 trial participants; "zero serious device-related adverse events" so far.