Science Is Drowning in AI Slop
Image Credit: Pexels
Scientific journals, long the trusted conduits for verified knowledge, are now inundated with low-quality or fraudulent content generated or assisted by large language models, often termed "AI slop." The issue came into sharp personal focus for University of Oslo psychology professor Dan Quintana earlier this month, when he spotted a fabricated citation to a nonexistent paper bearing his own name while peer-reviewing a seemingly routine manuscript for a respected journal. This discovery shattered his assumption that such "phantom citations" were confined to lesser outlets, revealing the problem's penetration into higher-tier publications.
The surge traces to AI tools that boost productivity—particularly aiding non-native English speakers—but also enable paper mills to mass-produce deceptive manuscripts at industrial scale. These operations recycle templates, fabricate plausible data, images (including microscopic or histological visuals), and even entire studies in high-stakes fields like cancer research and AI itself, where modest claims rarely prompt replication. Major AI conferences such as NeurIPS and ICLR have seen submission volumes double or more in recent years, with analyses detecting numerous hallucinated citations that evaded peer review; in some cases, reviews themselves appear largely AI-generated, including covert prompts hidden in manuscripts to elicit glowing assessments.
Preprint servers like arXiv, bioRxiv, and medRxiv face similar pressures, with submission rates climbing steadily since ChatGPT's release and researchers evidently using LLMs producing about a third more papers. Moderation catches some obvious junk, yet improving models increasingly slip through, raising alarms about an existential crisis, if genuine signal drowns in manufactured noise. Experts describe an escalating arms race between fraudsters and detection tools, warning that unchecked proliferation risks permanent epistemological pollution: a scientific literature where AI writes, reviews, and cites itself in a self-reinforcing loop, eroding trust in the entire body of research.
