Online tools will help publishers detect fake research papers.Credit: Getty
Publishers are testing prototypes of automatic systems to flag submitted manuscripts bearing the hallmarks of paper mills — businesses that produce fake research papers.
The tools, which will eventually form part of an online integrity hub, are the result of a year-long collaboration between 24 publishers and scholarly analytics providers. Together, the companies are trying to stamp out the growing scourge of fabricated scientific articles.
Paper-mill submissions to journals have increased massively over the past few years, says Jana Christopher, an image-integrity analyst at FEBS Press in Heidelberg, Germany. “If we did nothing about this, then the literature will just become really unreliable,” she says. “That’s something we can’t afford.”
Fighting fake science
In recent years, journals have retracted hundreds of papers over fears that the work contained fabricated data and images. As a result, publishers have been trying to beef up their defences to stop such papers getting through their submission systems. Several private companies have sprung up offering software that can spot potentially manipulated images or flag characteristics that suggest paper-mill activity.
‘Papermill alarm’ software flags potentially fake papers
Since 2020, the International Association of Scientific, Technical and Medical Publishers (STM) — a global trade association based in The Hague, the Netherlands — has been working with publishing companies to develop shared standards for software that can detect potential problems with images during peer review. It announced in December 2021 that it was developing the STM Integrity Hub, which will provide tools that allow editors from any publisher to check submitted articles for research-integrity issues.
Companies involved in the initiative include BMJ, Elsevier, Frontiers, IOP Publishing, the JAMA Network, Sage Publishing, Taylor & Francis, Wiley and Springer Nature (Nature’s news team is independent of its publisher). A small group is now testing prototypes for two of the three proposed tools. STM declined to name the publishers involved in the trials and says that it is too early to have any meaningful data on the tools’ effectiveness.
‘Tip of the iceberg’
Nature understands that Elsevier, Taylor & Francis and Frontiers are among those testing the prototypes.
Sabina Alam, director of publishing ethics and integrity at Taylor & Francis, says that suspicions about paper mills comprise around half of all ethics cases the publisher is dealing with. “The problem is significant not just because of volume, but also because there are different types of paper mill, and they are all highly adaptive. So, investigating the issues within a shifting-sands setting poses a lot of challenges,” she says.
“I suspect that what we have identified thus far is only the tip of the iceberg,” says David Knutson, a spokesperson for open-access publisher PLOS in San Francisco, California. “A single paper-mill case may affect tens, hundreds or even thousands of articles across multiple publishers.”
The integrity hub’s first tool works by scanning papers for more than 70 signals that could indicate that the manuscript has been generated by a paper mill. Those involved remain tight-lipped about what those signals are, so as not to tip off fraudsters. But previous public work has suggested red flags such as formulaic article titles and layouts, bar charts with identical profiles claiming to represent data from different experiments, suspicious-looking author e-mail addresses or strange turns of phrase that could indicate the use of automated translation software.
Duplicate submissions
The second tool is designed to alert editors when someone has submitted a paper to several journals at once. Paper mills use this tactic to try to get papers accepted more quickly (it is considered inappropriate to submit a full manuscript to multiple journals at once).
Christopher says that ferreting out these duplicate submissions will be a “really big and important step” in tackling paper mills. Previously, journals had no way of knowing whether a paper that they might be considering for publication was also under review elsewhere.
Publishers unite to tackle doctored images in research papers
Sharing data securely between publishers is legally problematic, because of data processing and anti-competition laws. The manuscripts that researchers submit to journals are confidential, and cannot be shared easily between journals and publishers. But the hub is putting in place a series of technical measures so that only minimal snippets of information are collected from publishers. “The hub works in such a way that information can be correlated and compared to each other,” says Hylke Koers, who heads STM Solutions — a subsidiary of STM that is developing the hub. Information will also be encrypted for security.
Such collaboration between publishers is crucial, says Elena Vica, head of research integrity at open-access publisher Frontiers in Lausanne, Switzerland. “If we don’t work together, the problem may only be shifted from one journal or one publisher to another,” she says.
The final technical element of the hub will be an analysis of available software that can spot manipulated images in manuscripts.
Assistance for editors
Joris van Rossum, director of research integrity at STM, says that the organization hopes to have versions of the paper-mill detection and duplicate-submission alerts available for wider use by early next year.
The applications will support rather than replace editors and peer reviewers, says Nicola Nugent, publishing manager for quality and ethics at the Royal Society of Chemistry in London, which has been involved in developing the hub. Alerts will still need to be acted on by people, but some degree of automation is important, because “publishers are often working at significant scale, assessing high volumes of submissions”, she says.
As well as the online tools, STM is working with the Committee on Publication Ethics, an advisory body based in Eastleigh, UK, to provide publishers with guidance and policies on how to deal with integrity issues. It is producing a series of workshops for publishers to share information and learn from each other.
So far, the hub has been funded by an initial, undisclosed, investment by STM. Next year, the company will be seeking voluntary funding from hub members and looking at how to make the project financially sustainable. “It’s still too early to speculate, but we expect to recover at least part of the operational costs by a fee for integrating the hub into the publishers’ editorial systems,” says van Rossum.
