Did Deep Learning Recommender Systems cause the overreaction to Covid?
Surprisingly often, I run into people, who are pretty smart and otherwise not into conspiracy theories, but maybe not that technical, who genuinely believe that their smartphones are constantly recording everything they say to serve them better ads on their feeds. I'm amused and bemused by that, because they don't seem to be doing much about this and seem somehow alright with continuing to live in their digital panopticon, nor do they run experiments to prove this to others or whatever.
On the face of it, this is of course ridiculous. Not sure if I should spell it out, but like, why would your Apple or Google phone collaborate with Facebook to record everything you say. Also that would be way too much data and processing. Maybe some people will say 'yeah but something like that really happened with Amazon Echo' or something… but for a really good conspiracy to be true, by definition, you need evidence that an event is unlikely AND that it really happened. But if you get evidence that the event is very unlikely, that's always also evidence that it hasn't actually happened. And if you get evidence that the event really happened, that's always also evidence that it isn't unlikely. That's why extraordinary claims require extraordinary evidence.
But I digress, the point is a different one: What's going on here?
I think maybe normal people are picking up on the fact that modern recommender systems might actually be much more powerful and sophisticated than many people give them credit for. Especially if you're like me and you use a good adblocker and are into digital minimalism and maybe have an X.com sub to get rid of the ads and so on, you haven't actually noticed how good modern recommender systems have become and it might even be somewhat more subtle.
To me the power of the ML that powers the feed seems somewhat overstated... like... sure they can target your demographic pretty well to sell you products and it's somewhat eerie that Instagram serves me videos at the intersection my nationality x living abroad in the country I live in, and, okay they can guess that I like videos that make fun of people who are really into LCD soundsystem and Paul Thomas Anderson, but remember StumbleUpon?
IIRC, StumpleUpon, was a website that must have emerged around 2010, and it recommended websites from all around the web and you could either hit like or dislike and it was an early adopter of ML. The first time I used it, I rated like 20 or at most 100 websites or so, and it recommended the website of the research institute I was working at the time, which had only like 100 employees. And back then I was real surprised and was like 'Woah ML'. I found it really counterintuitive that they could pinpoint my preferences with so few datapoints (I thought I was more unique).
But since then, my intuition has always been that recommender systems seem to have stagnated quite a lot? Like, I rarely buy products I get advertised on feeds and while I still block addictive social media braincrack on my computer, I find that other websites like chess.com and Hackernews are pretty much just as addictive, even though they don't use ML and the algorithms they use can literally be written on the back of an envelope.
But there have been two updates for me recently:
1. I came across a paper, showing that, even pre-ChatGPT, in early 2022, Facebook had trained an 12T parameter Deep Learning Recommender System model on exabytes of data and 1 zetaFLOPs of compute using thousands of A100s.[1] And even back in 2017, they had already built a datacenter with 20k V100 to compute to train and optimize people's feeds for engagement and click-through rate prediction as is used for maybe something like GPT 3.5. With all that compute, shouldn't we have noticed more of an eerie difference in terms of the addictiveness or engagement or something? Subjectively, the only thing I've noticed is that, as far as I can recall, I started buying lots of cheap, silly little life-improvement-y things on Amazon (which presumably used somewhat similar technology) around 2015. Digital advertising global annual revenue is like $.5T a year.
2. I listened to the excellent Sentinel Podcast[2] recently and the guest was saying:
'There's a darker version of that. Around 2018 or 2019, there was a panic when a few academics published papers about recommender systems that shape you to want to use more recommender systems. It's similar to addiction, where an appetite becomes self-reinforcing.
You can imagine the part of you that wants to go outside never gets rewarded because if it did, you'd go outside and they'd lose your user minutes. Instead, the parts of you that want to stay home are amplified. This could happen at a deep, memetic level. These powerful deep learning systems have tons of data and understand human psychology in ways we don't.
They might spread patterns of thought—for example, that it's not safe to go outside. They don't have to tell you crime is high; they can just show you enough stories of people being murdered or assaulted. That's a very fit meme, not just because it's triggering, but because it reinforces staying at home over the long term.
No one has evidence this is happening, but it's plausible and the incentives are there. In 2018, most algorithms were short-term, optimizing for clicks or watch time. Now, more algorithms use long-horizon reinforcement learning, which we've gotten better at. In principle, these algorithms could learn that showing someone a violent news story makes them more likely to have higher watch time a month later because they're scared.'
I wonder if these recommender systems made us believe that the covid was worse than it actually was. I'm not an expert but my theory of covid is that, ex-ante, we thought that covid might cause ~100x as many DALYs as the flu (which is already really bad), and that justified tanking the world economy and take a $10T hit based on the precautionary principle. But ex-post, covid was 'only' ~10x as many DALYs as the flu, and so now most people think we overreacted.[3]
I worry about what's yet to come, when due to massive demand for AI compute, the marginal cost of training ever more powerful recommender systems will drop a lot and maybe create weird effects (cf Sort By Controversial | Slate Star Codex).