Is AI scaling hitting diminishing returns? (04 Nov 2025)
Until 2018, frontier models' training compute grew 7x/year due to a 'compute overhang'; you could run small-scale ML experiments cheaply and deep learning scaled well on small GPU clusters.[1] Scaling data and compute, but not algorithms, drove most AI progress. While training methods and fine-tuning have evolved, the fundamental architecture of the GPT hasn't changed much.[2] The main 'algorithmic breakthrough' was using exponentially more training data.[3]
In small models (e.g. by Google, FB, Alibaba, Mixtral etc.), compute and capabilities (e.g. reasoning) correlated very strongly (~95%)[4]:
Source[5]
Since 2018, models have needed larger dedicated clusters and better hardware and frontier model growth slowed to 4x more compute per year.[6]
In 2022, OpenAI trained GPT-4 for ~$50M.[7],[8] When it came out two years ago in 2023, everyone was surprised how much smarter it was than GPT-3. This was reflected in big performance jumps on benchmarks for language understanding, math, science, and coding. Of course, all benchmarks are wrong models of intelligence and capabilities,[9] but some are still useful proxies for the intuitions that progress has been decelerating. While AI has gotten a bit better since, and we might find yet more ways to monetize it, we haven't seen as big a jump in capabilities as the one from GPT-3 to 4.[10] Scale used to be all we needed. The models just wanted to learn.
In 2023, Google trained Gemini 1 for ~$130M and 2.4x more compute than GPT-4.[11],[12] While it did slightly better on some benchmarks, Google has a lot of data and the most compute of any firm and can easily spend $1B to train an AI.[13] If scale was all you needed for a GPT-3 to 4 jump, why didn't Google just release a 10x bigger model? Hassabis cited both practical limits (e.g. how much compute can you actually fit in one data center), but also said that scaling more than 10x between models is suboptimal: you want to adjust 'the recipe' (e.g. intermediate data points to help course-correct the hyperparameter optimization).[14] And so, some say that 2023 was slowest year of AI progress for some time to come[15] and forecasts also suggest only a ~15% chance of next gen models plateauing.[16]
In 2024, performance no longer improved linearly, but sublinearly:[17]
- Google released Gemini 2.5, surely trained on at least ~10x the compute as GPT-4 (Anthropic's CEO says we'll have ~$1B models in 2024). DeepMind's CEO was reportedly disappointed by Gemini's performance gains.[18]
- Sutskever, co-founder OpenAI, also said results from scaling up pre-training have plateaued[19]- a trend that can be seen across frontier AI firms according to industry insiders[20],[21] (e.g. OpenAI started training GPT-5 only in May '24,[22] Musk's xAI is training a model on a cluster of 100k H100s chips, Meta is training Llama 4 on even more[23]).
In 2025, GPT-5 turned out to use less training compute than GPT-4.5 (but GPT-6 probably won't).[24] Many have been disappointed by the release (though see[25]).
And so, the scaling paradigm has hit diminishing returns, because generally, returns to scale past the point of increasing returns are often logarithmic.[26] As a rule of thumb, past the point of increasing returns to scale, which we've now reached, the next dollar spent on say compute, data or AI scientists at the, say, $1B mark get you 10x less than at the $100M funding mark.
By 2028, there might be 160x more compute per training than the 100K H100s of 2024, but it’ll require 5 GW of power and cost ~$140B[27]). While by 2030 we might train AI that is 10,000x bigger than GPT-4—the same increase from GPT-2 to 4[28]— this would cost on the order of hundreds of billions.[29]
Due to diminishing returns the money spent leading up to it is 1000x less effective than what we spent around the $100M mark (at which scaling has already stopped delivering). A fortiori, if we already see diminishing returns, then this will only get worse as the low hanging algorithmic improvements fruits are plucked.
There is a case to be made that AI hype is continuously moving the goalpost:
- 'Pretraining scaling is all you need' until it stopped delivering.
- Inference scaling was hailed as the new paradigm but is more expensive and on challenging questions which are outside a given base model's capabilities or under higher inference requirement, pretraining is likely more effective for improving performance.[30]
- Agents were supposed to be the big thing of 2025 and create a revenue feedback loop in, but didn't deliver.
- Now it’s agent swarms[31] and
- Automating AI R&D (RSI) has been hailed as the most recent approach.
Economic data suggests that TAI is not near:
- AI firms are not acting as if this can go on: AI is undoubtedly booming: firms now invest $175B per year in AI[32] and in 2023 funding for generative AI increased by 8x.[33] In 2025, AI investment might be $200B globally.[34] But if firms believed scaling would get us to TAI by 2030 or expected progress to continue at the same rate, they would train larger models. Instead, firms focus on lowering inference costs and adding new features,[35] (e.g. voice assistants,[36] models that can run locally,[37] and search.[38]), but don’t do much better on benchmarks.
- In industry, anecdotally, the excitement has calmed down. One AI expert who looked at the 900 most popular open source AI tools says: 'in early 2023, all AI conversations I had with companies centered around GenAI, but the recent conversations are more grounded.'[39]
- Stock markets also do not expect transformative AI soon: Based on stock market caps, AI might create $2.4T a year in consumer surplus.[40] At 2% of World GDP, that's big, but not TAI.
- Interest rates, too, do not imply TAI. Markets don't expect aligned or unaligned AGI in the next 50 years, because if they did, interest rates would be higher than they are now.[41]
- Macroeconomic models suggest gains from AI will be large, but not transformative: Most top economists think generative AI has a measurable impact on national innovation,[42] and many agree that AI will substantially boost our per capita income over the next 30 years—perhaps more than the internet.[43],[44] The recent Nobel prize winners' macroeconomic model uses estimates of how many tasks in a variety of jobs will be automated by AI and then estimates the productivity gains from this automation. They find that the upper bound on GDP growth due to AI is ~1.7% over 10 years.[45] This is even much less growth than what the stock market expects, though these models assume that no further innovation in AI will happen.
And so, in sum, AI might not improve as fast as before without feedback loops (e.g. AI improving itself) and bring about transformative AI in the near-term (i.e. a new, qualitatively different future as per Karnofsky's definition[46]).
Notes
[2] A 61x increase in effective compute see Measuring the Algorithmic Efficiency of Neural Networks
[12] Epoch’s Gemini Ultra cost.ipynb
[25] Is AI Stalling Out? Cutting Through Capabilities Confusion, w/ Erik Torenberg, from the a16z Podcast
[27] https://www.lesswrong.com/posts/fdCaCDfstHxyPmB9h/vladimir_nesov-s-shortform?commentId=vpCuEPiJkwoDxhrqN
[40] The estimates use NVIDIA's stock price, which implies the GPU market will go quickly to $180B, and then increase at the average amount after that. Todd estimates costs for AI software firms and assumes they will have a 30% margin, which suggests they'll have revenues of $800B per year. If AI firms capture 25% of the value they create, that leaves a consumer surplus of $2.4T. The market expects AI software to create trillions of dollars of value by 2027
[41] The argument is as follows:
- If we align AI, we'll earn more. There's diminishing utility to more income, so the more confident we are in AI, the more we'll front load consumption (e.g consume/borrow today) knowing that we don't need to save much to have a high income later. If we save less, interest rates go up.
- In an unaligned-AGI scenario, we'll die. The more confident we are in this scenario, the less we save, and instead they'll increase consumption/borrowing, as there's no use for savings if we all die. If we lend less, interest rates go up.
[45] Acemoglu - The simple macroeconomics of AI