Let's Fund: What was the impact of your donation?
Our grantees helped increase clean energy R&D funding by billions, and the most important scientific journal has adopted a better publication format
Hi, it's Hauke, the founder of Let's Fund. We research pressing problems, like climate change or the replication crisis in science, and then crowdfund for particularly effective policy solutions.
Ages ago, you signed up for my newsletter. I promised it'd be low-traffic, and boy did I deliver (in that I never delivered an email). But now I've evaluated the $1M+ in grants you donated, and they had a big impact. Below I present the Better Science / Registered Report campaign evaluation but stay tuned for the climate policy campaign impact evaluation (spoiler: clean energy R&D increased by billions of dollars).
Let's Fund: Better Science
We crowdfunded ~$80k for Prof. Chambers to promote Registered Reports, a new publication format, where research is peer-reviewed before the results are known. This fundamentally changes the way research is done across all scientific fields. For instance, one recent Registered Report studied COVID patients undergoing ventilation.[1] (but there are examples in other areas including climate science,[2] development economics,[3] biosecurity,[4] farm animal welfare,[5] etc.).
Registered Reports have higher quality than normal publications,[6] because they
- make science more theory-driven, open, and transparent
- find methodological weaknesses prior to publication and also potential biosafety failures of dangerous dual-use tech (e.g. gain of function research)[7]
- get more papers published that fail to confirm the original hypothesis
- increase the credibility of non-randomized natural experiments using observational data
If Registered Reports become widely adopted, it might lead to a paradigm shift and better science. 300+ journals have already adapted Registered Reports. And just last week Nature, the most prestigious academic journal, adopted it:
https://twitter.com/chrisdc77/status/1628431986464526336
This is big and Registered Reports might soon become the gold standard.
Why? Imagine you're a scientist with a good idea for an experiment with a high value of information (think: cancer cure). If that has a low chance of working out (say 1%), you previously had little incentive to run it.
Now, if your idea is really good, and based on strong theory, Registered Reports de-risks running the experiment. You can first submit the idea to Nature, and the reviewers might say: 'This idea is nuts, but we agree there's a small chance it might work and if you run the experiment, we'll publish this independent of results.' Now you can go ahead and spend a lot of effort on running the experiment, and if it doesn't work, you still get a publication in Nature (which you wouldn't with null results).
This will lead to more high risk, high reward research (share this post or the tweet with academics! They might thank you for the Nature publication).
Many people were integral to this progress, but I think Chambers, the co-inventor and prime proponent of Registered Reports deserves special credit:
Chris Chambers giving a talk on Registered Reports
In turn he credited you:
https://twitter.com/chrisdc77/status/1305230530292649984
You may feel smug.
If you want to make a bigger donation (>$1k), click here. There are proposals to improve Registered Reports even further, like trinity review which integrates Registered Reports with ethics and funding reviews, and peer community review which reduces review burdens (see The past, present and future of Registered Reports for more).
Feel free to ask questions in the comments and share this blog.
Notes
[1] Core warming of coronavirus disease 2019 (COVID-19) patients undergoing mechanical ventilation—A protocol for a randomized controlled pilot study
[3] Registered Reports: Piloting a Pre-Results Review Process at the Journal of Development Economics
[5] Measurement properties of pain scoring instruments in farm animals: A systematic review protocol using the COSMIN checklist | PLOS ONE
[6] Initial evidence of research quality of registered reports compared with the standard publishing model | Nature Human Behaviour
[7] Registered Reports are a de facto scientific self-oversight mechanism, as peers review the study design before it is performed and thus in time to recommend and implement additional safety steps. Unlike institutional review boards, Registered Reports selects peer-reviewers in related disciplines without institutional conflicts of interest. This could supplement IRBs, and one could envision that an IRB would require a Registered Reports. COVID-19 and the gain of function debates