MIT publicly retracts famous article about artificial intelligence
Beware of academic and commercial snake oil
Last year, an article from the Massachusetts Institute of Technology (MIT) made bold claims about the role of artificial intelligence (AI) in materials science:
AI-assisted researchers discover 44% more materials.
This led to a 39% increase in patent filings and a 17% rise in downstream product innovation.
AI automates 57% of “idea-generation” tasks.
These are massive improvements in productivity - especially in a field that creates so much technology of practical importance.
The only problem? The research was likely based on invalid data.
On November 4, 2024, Aidan Toner-Rodgers published the preprint "Artificial Intelligence, Scientific Discovery, and Product Innovation" via the National Bureau of Economic Research (NBER). It significantly influenced the field of AI and online discussions about its impact on science.
Tyler Cowen wrote about it on the blog Marginal Revolution.
Freakonomics recorded a podcast episode that cited this research.
Planet Money wrote a newsletter article that mentioned this article.
Daron Acemoglu is a winner of the Nobel Prize in Economics and was one of the economists cited within the article. In a statement, he acknowledged that this article
is already known and discussed extensively in the literature on AI and science…
However, MIT published a public retraction of this article on May 16, 2025, requesting its withdrawal from arXiv and its submission to the Quarterly Journal of Economics. An internal review found
no confidence in the provenance, reliability, or validity of the data and the veracity of the research.
MIT has not disclosed the full details behind its retraction. However:
NBER first published this article on November 4, 2024
MIT published its retraction statement on May 16, 2025
Thus, MIT took 6 months to establish its lack of confidence in the data. Based on this timeline, it is reasonable to presume that Toner-Rogers did not disclose his data initially. Otherwise, scholars would have tried to reproduce his results fairly quickly to check his bold claims.
I urge all scientists, scholars, researchers, and AI practitioners to take 2 lessons from this saga:
Beware of “snake oil” in AI - whether it is in academia or the private sector. Currently, there is a lot of research and business about AI that is more hype than substance. This is sad but predictable, because fraudsters are exploiting the naivete of uninformed people who want to benefit from this exciting technology.
Data-driven research works best when the researchers post their code and data publicly for other scholars to reproduce the results. Due to the private or confidential nature of some data (such as information about medical patients), I recognize that this is not always realistic. I encourage you to visit the website of The Open Science Foundation; it provides good guidelines about how to do this.
MIT’s statement does not name the author of this paper, but does refer to him as a “former second-year PhD student in the Department of Economics”. Based on this statement and the severity of this offense, I presume that MIT has expelled Aidan Toner-Rogers.
For scientific research to be valuable, scientists must conduct their research honestly and open it for peer review (ideally for anyone to scrutinize its veracity). This is the only way for the rest of society to trust the integrity of the scientific process.