Claims about AI’s impact from weak to strong
Introduction
There are lots of claims about AI's impact going around by prominent thinkers (e.g. the AI 2027 scenario or that there'll be 'A country of geniuses in a datacenter'). Here, I present an ontology of claims about AI's impacts and risks roughly sorted by expected value / going from weak to strong claims (from 'AI probably won't ever have much of an impact' to ‘AI will most definitely lead to extinction’).
22 claims about AI's impact sorted from weak to strong
Here are the claims about AI impacts sorted by impact:
- We'll probably never build transformative AI (not even in a hundred years), because it is too difficult. Things will likely go on like they always have.
- AI is hyped and TAI is a long way off: Current AI is just a stochastic parrot or next-token prediction, and the models are not as smart as industry hype wants to make us believe (cf Gary Marcus work). The economic, military and societal impact are limited. AI might have some applications, but also major externalities that risk slowing down innovation and progress: Coders might think they get better when using AI, but they're actually getting worse. AI slop actually slows down innovation and growth. AI firms are overvalued, they don't have a revenue model, and so, AI is a bubble waiting to pop. Then there might be another AI winter.
- Gains from AI will be large, but not transformative: Most top economists think generative AI has a measurable impact on national innovation.[1] The recent Nobel prize winners' macroeconomic model finds AI automation will cause at most ~1.7% GDP growth over 10 years (though these models assume that no further innovation in AI will happen).[2] This is very high, but not transformative.
- The stock market values AI more highly: Based on stock market caps, AI might create trillions of consumer surplus per year.[3] At 2% of World GDP, that's even larger than what economists expect, but not transformative AI.
- AI progress has stalled[4] or is going to slow dramatically, because the scaling paradigm has hit steeply diminishing returns to further investment.[5]
- Even if AI progress stops today, we would have a decade of major changes across entire professions & industries (medicine, law, education, coding…) as we figure out how to actually use it.[6]
- AI is or will be just as revolutionary as smart phones or the internet: Many top economists think that AI will substantially boost our per capita income over the next 30 years—perhaps more than the internet.[7],[8] cf AI as Normal Technology, Most AI value will come from broad automation, not from R&D | Epoch AI
- AI will be even more revolutionary than the internet, which will lead to widespread economic disruption (e.g. job displacement, concentration of power). Countries that do not develop AI could be left behind economically (cf. Europe 2031).
- AI and science: 'AI has already started to contribute to the hard sciences in meaningful ways [...] By 2027 or 2028, it will be clear to all that AI models are making important contributions to science.'[9]
- Narrow AI capabilities might pose significant threats or global catastrophic risks (e.g. AI-uplift causing engineered pandemics)[10]
- AI disruption is baked in: AI progress will continue and there will be major societal disruption (e.g. misinformation, cyberattacks AI girlfriends, etc.)
- Situational awareness[11]: AI will speed up via 100x more: money, compute, efficient algorithms & tinkering. That's enough for AGI automating all cognitive jobs with vast scale & speed. Such abilities will be key for int'l military competition, and so, the deep state will take over AI R&D & key data centers, locking down AI secrets to race vs China. AGI will be like the Manhattan project.
- At some point, we will create AI systems that have long-term goals (and can make and execute complex plans), excellent situational awareness, and highly advanced capabilities relative to today's systems and human abilities.[12]
- AI 2027:[13] AGI by 2027 and geopolitical tensions are likely; and secrecy and competition will erode safety and public oversight.
- Transformative AI (TAI): Real chance that in the next ~5 years, at least one of a select handful of 'frontier' AI companies will build that outperform all humans in a number of relevant intellectual domains (e.g. coding, science, military operations, running companies, etc.). The future will be largely unrecognizable.
- Even near‑human‑level LLMs have a non-trivial chance of a covert takeover.[14]
- A country of geniuses in a datacenter.[15] AI will have highly advanced capabilities relative to today's systems and human abilities. Even if humanity ultimately survives TAI, we should still worry about everything transforming. TAI will emerge in late 2026 or early 2027. Powerful AI systems will have the following properties: Intellectual capabilities matching or exceeding that of Nobel Prize winners across most scientific disciplines.
- AI significantly surpasses human intelligence in a broad and not in a jagged way: All previous weaker points implicitly assume that AI is ~bounded by human intelligence and that those geniuses in the datacenter aren't much smarter than Einstein. AI's IQ scores have improved rapidly recently (70 IQ → 120), but don't feel much smarter because useful originality rises steeply only at very high IQ. And so, if they gain another 10 IQ points, AIs will still produce slop. But if they increase by another 30, they may cross a threshold and start providing useful original insights.[16] An even smarter AI might have a much bigger impact, the combinatorics of the consequences are hard to grasp. Just consider the Bitcoin creator making billions by pseudonymously publishing a paper.
- AI will cause catastrophic misuse (e.g. bioweapons, cyberweapons)[17]
- Vulnerable World Hypothesis:[18] it is near inevitable that AI will cause global catastrophes
- AGI is an existential risk: If anyone builds AGI, everyone will die.[19]
- High p(doom): It's likely that someone builds AGI at some point, and thus it’s likely that then everyone will die.
- Short timelines and high p(doom): It's near inevitable that someone will build AGI and soon, and thus, everyone will likely die soon.
- Slow doom: “AI catastrophe looks like being with your family and friends as usual, and over an extended period of time watching one another succumb to things like starvation due to having no money, illness due to having no money, or literally overheating or being crushed in the midst of a breathtakingly fast technological buildout that doesn’t need any of the specific conditions we depend on to live, doesn’t care about us, and does need to do vasts amount of computation quickly. Meanwhile, there won’t be a moment when you know this has happened—there will be increasing confusion and misinformation and decreasing resources put into making it legible to you. It will probably look more like the atrocities we are used to than the perfect, clean instantaneous end to the human story.”[20]
- Fast doom: “In some prominent AI catastrophe stories, a single AI system suddenly emerges fantastically more powerful than anyone else and builds technology to quickly kill everyone, perhaps before they notice.”[21]
Conclusion
Some of these claims about AI are underspecified or underoperationalized, but often they have three different properties:
- Impact: Going from 'AI won't affect humanity much' to 'AI will cause extinction'.
- Probability: what are the odds of such an impact? Is it near impossible i.e 0%? 1%, 10%, 50%? Is it near inevitable (i.e. 100%)?
- Timelines: when will AI have this impact? Today? In 1 / 10 / 100 years? Never?
The claim's implied expected damage is a function of these three properties. For instance, only recently one paper has defined AGI formally as as matching the cognitive versatility and proficiency of a well-educated adult and dissected intelligence into ten core cognitive domains (e.g. reasoning, memory, and perception) where current AIs show highly “jagged” cognitive profiles (e.g. they have poor long-term memory).[22]
Some concluding thoughts:
- Wisdom of the crowd: Many reasonable people have advanced arguments for claims across the whole range of AI impacts. Based on the wisdom of the crowd effect, the truth might be more likely to be in the middle than in the extreme positions.
- It seems overdetermined that some weaker claims are wrong: AI seems to be quite useful (has mundane utility) to many people already. Even if AI progress stalls, working on AI policy/safety could be overwhelmingly important / impactful.
- I feel like the strongest claims are sometimes a Motte and Bailey i.e. bait-and-switch between a 'motte' (an easy-to-defend, weaker claim like 'AI already has a large impact') and a 'bailey' (a hard-to-defend and more controversial claim. Sometimes one gets a 'AI will kill everyone soon, have you even used the latest model? Don't you see the mundane utility?'' vibe from proponents of the stronger claims). To be fair, the more AI progress we see, the more this really is suggestive evidence that TAI/AGI is possible, imminent, and risky. But some proponents of the stronger claims sometimes act as if their claims are 'oh so overdetermined'.
https://docs.google.com/document/d/1mrNOkFY90K8zBXh5750pyrYkvY8Y4HLyou_uGSro7gs/edit?tab=t.0
Notes
[2] Acemoglu - The simple macroeconomics of AI
[3] The estimates use NVIDIA's stock price, which implies the GPU market will go quickly to $180B, and then increase at the average amount after that. Todd estimates costs for AI software firms and assumes they will have a 30% margin, which suggests they'll have revenues of $800B per year. If AI firms capture 25% of the value they create, that leaves a consumer surplus of $2.4T. The market expects AI software to create trillions of dollars of value by 2027
[9] Is AI Stalling Out? Cutting Through Capabilities Confusion, w/ Erik Torenberg, from the a16z Podcast
[13] Summary — AI 2027 Summary: 2025: AI accelerates—hype, investment, unreliable but useful agents, ongoing AGI skepticism. 2026: China, short on compute, concentrates millions of GPUs in a Centralized Development Zone (~10% global). 2027: U.S. automates AI R&D, then reveals adversarial misalignment; China steals model weights; U.S. involvement grows. Then it’s either a race with rapid deployment, state capture, robot buildup, AI unleashes bioweapon, wipes out humans, expands into space. Or slowdown.
[16] Dan Hendrycks on X: "AI models are dramatically improving at IQ tests (70 IQ → 120), yet they don't feel vastly smarter than two years ago. At their current level of intelligence, rehashing existing human writings will work better than leaning on their own intelligence to produce novel analysis. https://t.co/iXmMxW0wqo" / X