Authors who submit a paper containing unchecked output from a large language model (LLM) will risk a year’s suspension from arXiv. The preprint server, long the main channel for circulating papers in physics, mathematics, computer science and other quantitative fields before peer review, has clarified its content policy in response to a rising tide of AI-generated submissions.
The threshold for suspension is “incontrovertible evidence of hallucinatory AI generation”, in the words of the arXiv scientific director Steinn Sigurðsson. Examples include hallucinated references, citations to non-existent papers, and meta-comments left by the model, such as an instruction to fill in the real numbers from an experiment. arXiv uses a detection algorithm to identify suspect papers, with readers also able to submit “Code of Conduct” complaints if papers have been released without being caught. “After the suspension period, the author can request reinstatement,” Sigurðsson says, “but in a number of cases, depending on the reason for the suspension, the author will be asked to submit work that has passed peer review with a reputable journal or conference. Typically, after three such submissions, there is no further restraint.”
The pressure behind the move is one of volume. Rejection rates have climbed, and the number of submissions held back for review, flagged by a quality-assurance tool or a moderator and often rejected in the end, has climbed faster still. “The increase in such cases puts a major strain on both the staff and the volunteer moderators,” says Sigurðsson.
arXiv typically holds all co-authors jointly responsible for a paper’s content. But that applies only when everyone listed has consented to it, a condition a fabricated paper may not satisfy. In such cases, arXiv relies on rules it already has. “Adding an author to a paper without their knowledge is a major academic misconduct,” Sigurðsson says, “and would generally lead to referral to the relevant academic institutions.”
For an offending submission from a large team of thousands, arXiv would not suspend the entire author list. It would defer instead to the collaboration, trusting it to identify the member who posted without clearance and to impose the primary sanction. That person would still be flagged for close inspection – and may be suspended by arXiv itself. “We have had several instances,” says Sigurðsson, “both where a member of a collaboration has submitted a manuscript to arXiv without clearing it through the internal process, or even where completely unaffiliated submitting authors have added a collaboration without notifying it at all.”
The one-strike rule on AI hallucinations is a matter of enforcement, rather than a new rule. arXiv has long held broad policies on content and scholarly standards, and recent internal discussions focused on how to apply them consistently to AI-generated content.
As LLMs develop, however, quality-control mechanisms are liable to date quickly. The difficulty, Sigurðsson says, is “not the lack of possible tools and counters, but the work involved in testing the tools and implementing them in a production system. This is due especially to their short useful lifetime, as AI tools evolve and users react to arXiv measures.”
