Moravec's law: AI will come faster than you think, even if you take into account Moravec's law
To know when AI will surpass human intelligence, it might be good to know how sophisticated the parts of our brains are that give rise to our intelligence.[1] Moravec's paradox states that, surprisingly, reasoning needs much less compute than sensorimotor skills, even though e.g. driving comes easier to us than math. Put simply: 'For the brain, easy things are hard, and hard things are easy.'
To unpack that: sensing (e.g. seeing, hearing, and feeling) and moving come really easy to us (e.g. we find it easy to tie our shoes or solve captchas where we have to recognize street signs). Capabilities that have evolved earlier should be harder to reproduce with AI, because evolution applied much more optimization pressure on these older capabilities (i.e. vision has evolved earlier than cognition).[2] For most of our evolutionary history, brains have have been selected based on the fitness of sensorimotor processes i.e. movement [3] and sensing (mostly vision).[4]
If a lot of the brain is used for sensorimotor skills, and only very little for higher level cognitive abilities, then we don't need a lot compute for AI. Crucially, the parts of the brain needed for cognition are actually not big and, as such, do not use much compute. We don't have much bigger brains than many other animals, and so the brain size/compute needed for higher cognition separating us from e.g. chimps and whales can't be that large. Consider that:
- Animals with small brains are often surprisingly smart and animals with large brains are sometimes less smart than those with bigger brains
- Human-level cognition does not need much compute
In contrast, higher-level cognitive processes like math, science, chess, translating languages, or writing feel much harder, even though they do not actually use a lot of compute. This is because for most of our evolutionary history, we were not selected based on this. In fact, computers with little compute are better than humans in chess or math, but worse at solving captchas and driving cars. Similarly, despite much hype, robots are still much less dexterous than animals. Most of the brain's computations are for sensorimotor processes in rich real-world environments that are not needed for AIs in abstracted environments.
I think we even fall prey to Moravec's bias, our systematic inability to overcome Moravec's paradox. Moravec's bias makes us believe that sensorimotor tasks (e.g. robotics, computer vision) are easier to automate than cognitive tasks (e.g. mathematical cognition, language). But, in fact, it's the opposite: cognitive tasks seem actually much easier to automate. This biases us to underestimate how hard it is to automate tasks like driving and how easy it is to automate verbal and mathematical intelligence. Moravec's bias also makes us believe that AI will develop more slowly than it actually is, because we systematically overestimate how much compute is needed for higher-level human cognition and transformative AI. For that reason, we might underinvest in AI safety.
This hypothesis explains away certain surprising data and makes concrete predictions.
If Moravec’s bias is true, then we should see the following:
- We continuously underestimate AI progress in computationally easy domains (e.g. language, machine translation, chess, math, etc.) and overestimate how much compute we need for it.
- We continuously overestimate AI progress in computationally hard domains (e.g. vision, robotics) thinking that they should be relatively easy to solve and underestimate how much compute we need for it.
This bias seems pervasive even amongst experts, who do not seem to update, and keep being surprised by lack of progress in robotics ('self-driving cars are always 5 years away'), which they overinvest in, while also being surprised by AI progress in cognitive areas, which they underinvest in (e.g. GPT-4).
Crucially, if AI will solve cognitive tasks before solving sensorimotor tasks, and we're close to solving sensorimotor tasks, we might be quite close to TAI.
Finally, just like optimism bias causes Hofstadter's law ('Everything takes longer than you think, even if you take into account Hofstaedter's law'), I argue how, corollaritively, Moravec's bias causes Moravec's law:
'Automating sensorimotor tasks (cf. robotics, driverless cars) always take longer than you expect, but automating cognitive tasks (cf. LLMs, math) never takes as long as you expect—even if you take into account Moravec's Law.'
Notes
[1] “A better understanding of the computational requirements for human-level performance (considering both competence and throughput) would enable a better understanding of AI prospects.” Could 1 PFLOP/s systems exceed the basic functional capacity of the brain?
[3] Wolpert - What's a Brain For: A Moving Story