# Machine learning based screening of potential paper mill publications in cancer research: methodological and cross sectional study

Baptiste Scancar, Jennifer Byrne, David Causeur, Adrian Barnett

► To cite this version:

Baptiste Scancar, Jennifer Byrne, David Causeur, Adrian Barnett. Machine learning based screening of potential paper mill publications in cancer research: methodological and cross sectional study. BMJ - British Medical Journal, 2026, 392, pp.e087581. 10.1136/bmj-2025-087581. hal-05488821

HAL Id: hal-05488821

https://institut-agro-rennes-angers.hal.science/hal-05488821v1

Submitted on 25 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

OPEN ACCESS

<figure><img src="image_3.png" /></figure>

Check for updates

# Machine learning based screening of potential paper mill publications in cancer research: methodological and cross sectional study

Baptiste Scancar,<fnref n="1" /> Jennifer A Byrne,<fnref n="2" />,<fnref n="3" /> David Causeur,<fnref n="1" /> Adrian G Barnett<fnref n="4" />

## ABSTRACT

## OBJECTIVES

To train and validate a machine learning model to distinguish paper mill publications from genuine cancer research articles, and to screen the cancer research literature to assess the prevalence of papers that have textual similarities to paper mill papers.

## DESIGN

Methodological and cross sectional study applying a BERT (bidirectional encoder representations from transformers) based, text classification model to article titles and abstracts.

## SETTING

Retracted paper mill publications listed in the Retraction Watch database were used for model training. The cancer research corpus was screened by the model using the PubMed database restricted to original cancer research articles published between 1999 and 2024.

## POPULATION

The model was trained on 2202 retracted paper mill papers and validated on independent data collected by image integrity experts. 2.6 million cancer research papers were screened.

## MAIN OUTCOME MEASURES

Classification performance of the model. Prevalence of papers flagged as similar to retracted paper mill publications with 95% confidence intervals and their distribution over time, by country, publisher, cancer type, research area, and within high impact journals (top 10%).

## RESULTS

The model achieved an accuracy of 0.91. When applied to the cancer research literature, it flagged 261 245 of 2 647 471 papers (9.87%, 95% confidence interval 9.83 to 9.90) and revealed a large increase in flagged papers from 1999 to 2024, both across the entire corpus and in the top 10% of journals by impact factor. More than 170 000 papers affiliated with Chinese institutions were flagged, accounting for 36% of Chinese cancer research articles. Most publishers had published substantial numbers of flagged papers. Flagged papers were overrepresented in fundamental research and in gastric, bone, and liver cancer.

## CONCLUSIONS

Paper mills are a large and growing problem in the cancer literature and are not restricted to low impact journals. Collective awareness and action will be crucial to address the problem of paper mill publications.

## Introduction

Research paper mills are “contract-cheating organisations which provide undeclared services to support research manuscripts and publications.”<fnref n="1" /> Research paper mills fabricate and submit manuscripts for their customers. The first research paper mill activity was reported in the 2010s,<fnref n="2" /> and has since increased in volume and sophistication. More than 400 000 papers suspected to have originated from paper mills have been published in the past 20 years,<fnref n="3" /> with the paper mills earning tens of millions of dollars annually.<fnref n="4" /> This issue gained visibility when Wiley—after acquiring Hindawi—retracted nearly 11 000 suspected paper mill papers and shut down 19 journals over two years.⁵

Research paper mills maximise their earnings by quickly producing industrial quantities of low quality research papers.⁶ ⁷ To produce manuscripts at scale, fabrication has likely relied on templates with premade sentences where domain specific terms vary.⁸ Suspect papers can include incorrect reagents,⁹ fabricated data and experiments,<fnref n="2" /> ¹⁰ and photoshopped or reused figures.¹¹ Paper mill papers are often generic, poorly written, lack coherence between sections,⁸ ¹² and may offer superficial research justifications.⁷ Paper mills sell manuscripts to researchers eager to increase their number of publications,¹⁰ creating author groups who have never worked together or made any intellectual input.¹³ Paper mills often cite their own productions,¹⁴ possibly as part of their service to clients who pay for citations.¹⁵ Paper mills may even bribe editors and manipulate peer review to facilitate publication, as shown by online discussions between researchers and likely paper mill contacts.<fnref n="4" />

## WHAT IS ALREADY KNOWN ON THIS TOPIC

Paper mills are fraudulent organisations that produce manuscripts for sale using textual and structural templates, with an estimated prevalence of around 3% in biomedical research

Evidence suggests that cancer research may be a major target of paper mills, but the field lacks large scale estimates to assess the severity of the problem

Machine learning approaches have shown potential for detecting paper mill manuscripts from textual features, but no scalable approach has been publicly developed in cancer research

## WHAT THIS STUDY ADDS

A scalable machine learning classifier can distinguish paper mill from genuine research using only titles and abstracts, suggesting that these sections contain template-like features

Large scale screening of cancer research showed that papers with textual similarities to paper mill manuscripts have increased over time and are widespread across the literature, including in high impact journals

Coordinated action from publishers and policy makers is urgently needed to address paper mill activity

<footnote n="1">¹IRMAR UMR 6625 CNRS, L'Institut Agro, Rennes, France</footnote>

<footnote n="2">²NSW Health Statewide Biobank, NSW Health Pathology, Camperdown, NSW, Australia</footnote>

<footnote n="3">³School of Medical Sciences, Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia</footnote>

<footnote n="4">⁴School of Public Health and Social Work, Queensland University of Technology, Kelvin Grove, QLD 4059, Australia</footnote>

<footnote>Correspondence to: A G Barnett a.barnett@qut.edu.au; (ORCID 0000-0001-6339-0374)</footnote>

<footnote>Additional material is published online only. To view please visit the journal online.</footnote>

<footnote>Cite this as: BMJ 2026;392:e087581 http://dx.doi.org/10.1136/bmj-2025-087581</footnote>

<footnote>Accepted: 30 November 2025</footnote>
Paper mill manuscripts can be simultaneously submitted to several journals until acceptance, wasting the time of editors and reviewers.² ¹¹ ¹⁶ The reported proportion of paper mill submissions to journals ranges from 2% to 46%.² Paper mills likely target journals where fabricated manuscripts have already been accepted,² increasing their chances of success. They may also focus on high impact journals because the prices they charge—according to Abalkina's investigations of a paper mill¹⁶—can be directly linked to the journal's impact factor.¹⁷

The overall prevalence of paper mill papers in biology and medicine is estimated to be 3%,³ but cancer research and particularly molecular oncology could be more affected.¹ This can be explained by high publication pressure,⁷ a specialised field with simple-to-fake data and techniques,¹ and limited peer review capacity⁶—making fake papers easier to produce and harder to detect.

The rise of artificial intelligence (AI) could exacerbate the paper mill problem through automated image and text generation, making detection more challenging.¹⁸ Some publishers are using screening tools to detect manuscripts from paper mills.¹⁹ ²⁰ Integrity sleuths have also developed detection methods, such as identifying awkward rewording of scientific terms, known as “tortured phrases,”²¹ or nucleotide sequence reagent verification.⁹ Paper mill manuscripts may have missing or unusual acknowledgments, funding, or ethics statements.¹¹

Previous publications have documented recurrent formatting templates among suspected paper mill papers, including identical layouts and nearly identical textual formulations in figure legends.⁸ ⁹ ¹³ ²² ²³ Bless and colleagues²⁴ reported “distinct phrase patterns and higher word repetition” and “lower lexical diversity” in full texts and abstracts of retracted papers. Also, Bless and colleagues²⁴ and other studies²⁵ ²⁶ have shown that machine learning methods can predict retractions and paper mill products from text using data from Retraction Watch—a non-profit group that records paper retractions²⁷—in a cross disciplinary context rather than within a specific scientific domain.

The performance of such an approach has never been tested in cancer research, and we believe it has the potential to systematically screen papers to assess the prevalence of papers sharing textual similarities with paper mill publications. We hypothesise that paper mills use text templates, which likely extend to titles and abstracts. We also believe that these templates are domain and publication type specific and could constitute strong signals to machine learning models. Therefore, we aim to use cancer research titles and abstracts from retracted paper mill papers as input to a BERT based (bidirectional encoder representations from transformers)²⁸ machine learning pipeline for a text classification task. BERT learns from examples to recognise patterns in text, enabling it to identify when new papers share similarities with retracted paper mill papers. Using only titles and abstracts enables

easy access to training data and offers scalability for producing broad estimates.

The first objective of this study was to train and evaluate our model's ability to reliably classify titles and abstracts from retracted papers attributed to suspected paper mill activity and genuine cancer research papers. The second objective was to use our model to screen millions of cancer research papers to assess the prevalence of flagged papers over time across countries, publishers, and cancer research subdomains, and to examine how they have evolved in high impact factor journals (defined as decile 1 or D1—the top 10% of journals in this study). Throughout we use flagged papers to refer to articles whose titles and abstracts are textually similar to retracted papers tagged as “paper mill” in Retraction Watch. Flagging is a statistical screen, not an attribution of misconduct.

This research aims not only to assess the potential of machine learning for detecting cancer research papers that share textual features with paper mill manuscripts but also to raise awareness of potentially fabricated papers among stakeholders in cancer research. We believe that such research could support editorial triage, help protect genuine researchers from citing or using fake research, inform funding and institutional policies on research integrity, and provide insights into the scale and dynamics of paper mill activity within cancer research.

## Methods

### Cancer research corpus

#### PubMed database preprocessing

To create a comprehensive cancer research dataset, the entire biomedical research corpus from the PubMed database (https://pubmed.ncbi.nlm.nih.gov/) was downloaded in March 2025. The following data were extracted from each of more than 38 million papers: PubMed Identifier (PMID), title, abstract, original language, journal name, journal ISSN (International Standard Serial Number), publication date, first author's affiliation, publication type, and MeSH terms (medical subject headings). The data were preprocessed following the method used by González-Márquez and colleagues.²⁹ We excluded abstracts that were non-English, empty, truncated, and unpunctuated to avoid these features influencing the language model. The text was transformed into standardised tokens (units of text such as words or punctuation marks) for analysis. Abstracts of less than 250 tokens and more than 4000 tokens were removed because these were generally non-standard abstracts and were also rare (<1%). After this initial filtering, the research dataset contained 24.8 million papers published between 1975 and 2025.

Next, the dataset was filtered to retain only papers published within the target timeframe (1999-2024): papers before 1999 or after 2024 were excluded, and duplicates were removed, reducing the dataset to 20.2 million papers. We only included papers classified as “journal article” and excluded other publication types, such as literature reviews and clinical trials.
### Box 1: Cancer related keywords

astrocytoma, carcinoembryonic antigen, carcinoid, carcinogen, carcinogenesis, carcinoma, cancer, checkpoint inhibitor, chemotherapy, chordoma, ependymoma, glioblastoma, glioma, leukaemia, leukemia, lymphoma, macroglobulinaemia, macroglobulinemia, medulloblastoma, melanoma, mesothelioma, metastasis, metastatic, myelodysplastic syndrome, myeloma, myeloproliferative neoplasm, neuroblastoma, nsclc, oncogene, oncogenesis, oncology, pheochromocytoma, radiation therapy, radiotherapy, retinoblastoma, sarcoma, seminoma, tumor, and tumour.

These types may also be targeted by paper mills, but would require separate models because we expect paper mills to use manuscript type specific templates. All retraction notices, corrections, and expressions of concern were also removed. After this second filtering step, 17.4 million papers remained.

### Cancer research filtering

The cancer research corpus was derived from the remaining papers using a two level keyword filtering strategy. Box 1 shows the keywords searched for in titles and abstracts of the 17.4 million papers. These keywords were adapted from MeSH terms and National Cancer Institute³⁰ terminology. The keyword matching was designed to be specific to cancer while also retaining the broadest coverage of cancer research papers. We acknowledge that some non-cancer related papers may be included and that not all cancer specific terms have been used.

Substring matching ensured that terms such as “osteosarcoma” were captured under broader categories like “sarcoma.” Papers matching multiple keywords were included only once to avoid duplicates. UK and US spellings of each cancer related term were used. All levels of the search were combined to produce a final cancer research dataset of 2 647 471 papers published across 11 632 journals. Supplementary file 1 provides a flowchart detailing the filtering strategy from the initial PubMed database to the final cancer research corpus.

Data extracted for visualisation purposes are the first author's country of affiliation, the publisher, the type of cancer investigated, the main cancer research areas, and the SCImago journal impact factor.³¹ The type of cancer investigated and the main cancer research areas were inferred from the titles, abstracts, and MeSH terms of the papers using AI based labelling. Supplementary file 2 describes all extraction methods. A detailed breakdown of these variables describing the cancer research corpus is provided in supplementary file 3.

### Paper mill datasets

We developed our model using two sources of paper mill papers: papers tagged as originating from paper mills in

the Retraction Watch database²⁷; online lists compiled by image integrity experts—also called integrity sleuths—where evidence of image manipulation was found. A compilation of paper mill papers is available online in the “Spreadsheet of spreadsheet” thanks to anonymous PubPeer contributors.³² PubPeer³³ is a website (https://pubpeer.com/) that allows users to leave comments after publication about potential research integrity issues or other concerns. The website has been used in research on integrity³⁴ and has played an important role in high profile retractions.³⁵

The Retraction Watch dataset was used for model development (training, optimisation, and internal validation), whereas the experts' dataset was reserved for external validation of model performance. From the 64 457 total retractions recorded in the Retraction Watch database as of June 2025, we removed papers that were not in the PubMed database, and those that were not retractions (eg, expressions of concern, corrections, or reinstatements). The “Paper Mill” tag in the retraction reason field was used to identify 5657 retracted publications. Only those whose PMIDs matched entries in our cancer research corpus were retained, reducing the number to 2 270 retracted papers. We excluded papers for which the original text had been replaced by the retraction notice, reducing the number of retracted paper mill papers to 2 202.

To validate the model's ability using new data, we used 3094 suspected paper mill papers from the integrity experts' dataset for external model validation, excluding those that overlapped with the Retraction Watch set. These papers were also extracted from the cancer research corpus.

Supplementary file 4 presents visualisations of data used at the training stage from the Retraction Watch and the integrity experts' datasets. These data show the distribution of publishers, countries of the first authors' institutions, cancer types, and research areas among paper mill papers, as well as the single word, two word, and three word combinations most prevalent in the titles of paper mill papers.

### Model selection and training

#### Controls selection

Our training dataset was chosen to be balanced, with 50% paper mill papers and 50% presumed genuine papers. Controls were selected from the cancer research corpus with the aim of including as few paper mill papers as possible to minimise bias and enhance training performance. Given the difficulty of assessing the genuine status of cancer research papers in large samples, control papers were selected from high impact factor journals (decile 1) and countries under-represented in the Retraction Watch database (using the country of the first author's institution). Throughout this paper, controls should be regarded as proxies for quality research, not verified genuine papers.

To avoid a potential bias in the English diction used by Chinese authors, we included 101 papers (5%) in the control set authored by researchers from

Protected by copyright, including for uses related to, text and data mining, AI training, and similar technologies.
Chinese institutions that were published in four high impact journals: *Cell*, *Cancer Cell*, *Molecular Cell*, and *The EMBO Journal*. We also included 600 of the most cited Taiwanese cancer research papers (28%) listed in OpenAlex³⁶ because Taiwan is a predominantly Mandarin speaking country, but is less represented in the Retraction Watch database (with only one recorded retraction owing to paper mill involvement in the cancer research corpus). Another 33% of the control papers were randomly selected from Swedish, Finnish, and Norwegian institutions in cancer research because these countries have no recorded instances of paper mill retractions in the Retraction Watch database. The remaining 33% consisted of a random selection of papers published in *Cell*, *Cancer Cell*, *Molecular Cell*, and *The EMBO Journal* from countries other than China, Taiwan, Norway, Sweden, and Finland.

For the external validation dataset, papers from the integrity experts' dataset were combined with a similar number of control papers randomly sampled from authors from Swedish, Finnish, and Norwegian institutions, and other papers published in the four high impact factor journals mentioned above. None of the controls or paper mill papers used in the training set overlap with those in the external validation set.

All papers in the control sets were manually verified as free of research integrity concerns on PubPeer (June 2025). A paper was removed if any problem was reported. We assumed that all papers in the paper mill datasets are indeed paper mill products, and that the controls are legitimate scientific papers. However, we acknowledge that the ground truth is unknown and that, despite our efforts, some papers may have been mislabelled.

### Model selection and training

We chose to use only titles and abstracts to train the model because these data were often available (full texts are frequently behind paywalls). Each paper's title and abstract were combined. We framed the detection of paper mill papers as a binary text classification problem—authentic or fraudulent—to provide supervision signals to the model.

The 4404 labelled papers from the Retraction Watch dataset and its controls (2202 paper mill papers and 2202 controls) were split into 70% for training (n=3082), 17.5% for optimisation (n=771), and 12.5% for internal validation (n=551). The model was trained and tuned using the training and optimisation sets. Splitting was performed at the paper level and stratified by class label to preserve the 1:1 ratio of paper mill and control papers across all subsets.

Model performance (accuracy, sensitivity, and specificity) was first evaluated on the internal validation set—which was never seen during training or optimisation—and then on an external validation set composed of papers from research integrity experts (3094 paper mill papers and 3100 controls). The latter dataset was entirely independent from Retraction Watch and used to assess the model's generalisation to new data. Table 1 provides a summary of all datasets and their respective roles. Each metric was derived from the confusion matrix using a probability threshold to prioritise specificity and minimise false positives. Reported values correspond to a single final model obtained after hyperparameter optimisation over 150 trials. Confidence intervals were not computed because performance was estimated on fixed validation sets. Supplementary file 2 gives further details.

We selected BERT to analyse the text for its strong performance and relatively low computational costs because of its moderate number of parameters.²⁸ The model was initialised from the public BERT base uncased checkpoint and fine tuned on our dataset with a new classification head. The model was initially compared with RoBERTa, BioBERT, PubMedBERT, Longformer, and Clinical Longformer³⁷⁻⁴¹ to assess the potential of biomedical specific pretraining and extended input capacity. Supplementary file 5 gives details of comparative model testing.

All journal specific formatting was removed from the abstracts. Because BERT truncates inputs beyond its 512 token limit, each title and abstract were split into individual sentences to prevent information loss. Each sentence was labelled (paper mill or not) and fed into BERT during the training phase. During inference, predictions were made at the sentence level and final classification probabilities were obtained by averaging the positive class probabilities across the title and abstract. Supplementary file 2 describes the optimisation method. A logistic regression model was also tested on BERT's output probabilities to incorporate sentence ordering; however, it did not outperform the simple averaging approach and so was not used (supplementary file 2).

### Additional validation

To further evaluate the model, we checked whether 873 problematic cancer research papers reported in three previous studies where paper mill involvement was suspected⁴²⁻⁴⁴ were flagged by the model (table 1). These papers were not necessarily from paper mills because similar errors can sometimes occur in genuine research. This analysis was not intended to measure

<table><caption>Table 1 | Summary of the five datasets and their use in the experiment design</caption><thead><tr><td>Source</td><td>Purpose</td><td>Sample size</td></tr></thead><tbody><tr><td>Retraction Watch</td><td>Model training</td><td>1541 cases and 1541 controls</td></tr><tr><td>Retraction Watch</td><td>Model optimisation</td><td>385 cases and 386 controls</td></tr><tr><td>Retraction Watch</td><td>Internal validation</td><td>276 cases and 275 controls</td></tr><tr><td>Integrity experts</td><td>External validation</td><td>3094 cases and 3100 controls</td></tr><tr><td>Cell lines and nucleotide sequences</td><td>Unlabelled testing</td><td>873 papers with likely high paper mill representation</td></tr></tbody></table>
model performance directly, but rather to assess the model's ability to flag papers that had already been reported as suspicious. Although we did not expect all these papers to be flagged, we did anticipate that most would be.

None of these papers were included in the training or validation sets. All papers overlapping with paper mill papers or controls from the training, optimisation, or validation sets were removed during data extraction (n=29). No content re-evaluation was performed, and integrity related information was not disclosed to the model. These papers, involving misidentified or non-verifiable nucleotide sequences (primers and other oligonucleotides used for amplification, detection, or gene knockdown), or both, or cell lines, were retrieved from three sources: 193 cancer related papers identified by Oste and colleagues,⁴² 113 cancer related papers in high impact factor journals—*Molecular Cancer* and *Oncogene*—reported by Pathmendra and colleagues,⁴³ and 567 cancer related papers listed by Park and colleagues.⁴⁴

### Screening the cancer literature

Each of the 2.6 million papers of the cancer research corpus published between 1999 and 2024 was screened with our fine tuned version of BERT trained to identify papers with textual features associated with retracted paper mill papers. We refer to papers that were classified as resembling retracted paper mill papers as “flagged papers.” None of the papers seen by the model during training were excluded from the cancer research corpus (n=3853). These papers were considered part of the corpus and represented a small percentage of flagged papers and so not large enough to influence the overall conclusions.

The 95% confidence interval for the proportion of flagged papers was estimated through bootstrapping (1000 resamples with replacement). All figures are presented as percentages of the total number of papers to illustrate the proportion of non-flagged papers compared with flagged papers. Data were visualised using the ggplot2 R library.⁴⁵ Supplementary file 6 gives a diagram of the study design.

### Patient and public involvement

There was no involvement in this research of patients or the public because publication policy is indirectly related to patient health and the key stakeholders are publishers and other researchers.

### Results

#### Model performance and errors

##### Training data

Retraction Watch covered papers from 2007 to 2024, with a peak in 2022, while the integrity experts' data

included papers from 2010 to 2024, with a peak between 2019 and 2020 (supplementary figure S4). In the Retraction Watch dataset, the most frequent publisher is John Wiley and Sons (via Hindawi), followed by Spandidos Publications, and Informa (supplementary table S4.2). In the experts' dataset, Springer Nature has the most suspected paper mill papers, followed by John Wiley and Sons, and Spandidos Publications (supplementary table S4.4). Papers from Chinese institutions are highly represented in both datasets (supplementary tables S4.2 and S4.4). Common topics in paper titles for both datasets include microRNAs, long non-coding RNAs, and lung cancer related terms (supplementary tables S4.1 and S4.3). Many papers cover topics related to gene and preclinical research. The most frequent cancer types are lung, liver, colorectal, gastric, and brain cancer (supplementary tables S4.2 and S4.4). The most prevalent research fields are cancer biology and cancer therapy, followed by cancer diagnosis (supplementary tables S4.2 and S4.4).

#### Performance

Table 2 summarises model performance metrics for both datasets. The classification model achieved an accuracy of 0.91 in detecting paper mill papers in the Retraction Watch dataset (502 of 551 papers were correctly classified as paper mill associated or genuine), with a sensitivity of 0.87 (239 of 276 papers correctly identified) and a specificity of 0.96 (263 of 275 genuine papers correctly identified). The titles and abstracts of three retracted paper mill papers, all correctly predicted with high probability, are presented as examples in supplementary file 7. In validating on unseen data for the integrity experts' dataset, the model had a similar classification accuracy of 0.93 (5771 of 6194) with a sensitivity of 0.87 (2698 of 3094) and a specificity of 0.99 (3073 of 3100).

In further validation sets, the model flagged 67% (130 of 193) of problematic papers found by Oste and colleagues,⁴² 75% (425 of 567) of problematic papers found by Park and colleagues,⁴⁴ and 66% (75 of 113) of problematic papers found by Pathmendra and colleagues.⁴³ About 72% of the combined problematic papers were flagged by the model.

#### Misclassifications

We combined the misclassifications from the internal and external validation sets because their characteristics were similar (supplementary file 4). Pooling these data increased the sample size and therefore improved the robustness of analyses. False positives—control papers incorrectly predicted to resemble paper mill papers—were rare, with only 39 out of 3375 across the Retraction Watch and integrity experts' datasets. This small number of false positives did not allow for the identification of generalisable patterns. In contrast, 433 out of 3370 paper mill papers were false negatives—paper mill papers incorrectly predicted as authentic.

<table><caption>Table 2 | Model performance over internal and external validation sets</caption><thead><tr><td>Set</td><td>Accuracy</td><td>Sensitivity</td><td>Specificity</td></tr></thead><tbody><tr><td>Internal validation</td><td>91% (502/551)</td><td>87% (239/276)</td><td>96% (263/275)</td></tr><tr><td>External validation</td><td>93% (5771/6194)</td><td>87% (2698/3094)</td><td>99% (3073/3100)</td></tr></tbody></table>

Protected by copyright, including for uses related to text, data mining, All training and similar technologies.
Protected by copyright, including for uses related to text and data mining. AI training, and similar technologies.

<table><caption>Table 3 | Characteristics of false negatives (n=433) by publication year, publisher, first author's country, and cancer type</caption><thead><tr><th>Variables and most overrepresented categories</th><th>False negative (%)</th><th>Overall (%)</th><th>Difference (%)</th><th>χ<sup>2</sup> P value</th></tr></thead><tbody><tr><td><strong>Year</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>2014</td><td>12</td><td>3</td><td>9</td><td>&lt;0.001</td></tr><tr><td>2015</td><td>13</td><td>5</td><td>8</td><td></td></tr><tr><td><strong>Publisher</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Rapamycin Press LLC</td><td>7</td><td>3</td><td>4</td><td>&lt;0.001</td></tr><tr><td>John Wiley and Sons</td><td>14</td><td>12</td><td>2</td><td></td></tr><tr><td><strong>Country</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>China</td><td>90</td><td>47</td><td>43</td><td>&lt;0.001</td></tr><tr><td><strong>Cancer type</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Gastric cancer</td><td>8</td><td>4</td><td>4</td><td>&lt;0.001</td></tr><tr><td>Liver cancer</td><td>10</td><td>6</td><td>4</td><td></td></tr><tr><td>Colorectal cancer</td><td>12</td><td>8</td><td>4</td><td></td></tr><tr><td>Lung cancer</td><td>12</td><td>8</td><td>4</td><td></td></tr></tbody></table>

For each variable, the 10 most frequent categories were retained. A single Pearson's χ² test of independence was carried out for each variable. Categories listed that were significantly overrepresented for each variable are defined by standardised residuals >2 (this is not tested and only descriptive).

Percentages are shown within the false negatives and overall pooled validation dataset (n=6745). The difference is the false negative minus overall. Data are percentages only; absolute counts not shown.

Table 3 summarises the main characteristics of false negative predictions by the model and an extended analysis is given in supplementary file 8. False negatives are disproportionately associated with first authors affiliated in China as they account for 90% of false negatives (compared with 47% in the overall validation dataset). Papers published in 2014 (+9%) and 2015 (+8%) were more often misclassified as false negatives by the model. Gastric cancer (+4%), liver (+4%), colorectal (+4%), and lung cancer (+4%), as well as publishers such as Rapamycin Press LLC (+4%) and John Wiley and Sons (+2%) are slightly overrepresented among the false negatives. Epidemiology papers are slightly overrepresented among false negatives, while text analyses do not show overrepresented topics other than general cancer words in the false negatives (supplementary file 8).

### Flagged papers in cancer literature

After applying our model to the cancer research corpus from 1999 to 2024, 261245 papers were flagged as including textual characteristics of retracted paper mill papers. This number represents 9.87% (95% confidence interval 9.83 to 9.90) of all original cancer research papers.

### *Trends in flagged papers*

The number of flagged papers showed a clear and rapid increase between 1999 and 2022 (fig 1), peaking in 2022 before showing a slight decline in 2023 and 2024. The number of flagged papers each year followed an exponential trend from 1999 to 2022 (R² for exponential fit=0.92). Although the percentages of flagged papers remained around 1% in the early 2000s, they progressively rose to exceed 15% (26457/171656) of the annual cancer research output by the early 2020s.

### *Countries of flagged papers*

The percentages of flagged papers for each country show that papers from China were most frequently flagged, representing 36% of cancer papers with

177907 of 497672 flagged papers (fig 2), followed by Iran with 20% of flagged papers, accounting for 6801 of 33935 papers. Papers from four other countries were also frequently flagged: Saudi Arabia (16%, 1607/10308), Egypt (15%, 2229/14618), Pakistan (13%, 883/6578), and Malaysia (13%, 870/6615). The United States was the second country in terms of number of flagged papers, with 10511 flagged papers representing 2% of US cancer research papers.

### *Publishers and journals of flagged papers*

The publisher Verduci Editore had the highest percentage of flagged papers with about 67% (2834/4199) in its cancer research journal—*European Review for Medical and Pharmacological Sciences* (fig 3). The second publisher in terms of percentage was International Scientific Literature, with about 45% (1261/2776) of papers flagged in one journal—*Medical Science Monitor (MSM)*. The next four publishers were E-Century Publishing Corporation (44%, 4620/10510), Spandidos Publications (38%, 19043/49796), Ivyspring International Publisher (30%, 3065/10069), and IOS Press (30%, 3079/10302).

The largest publishers—such as Springer Nature, Elsevier, and John Wiley and Sons—have a relatively low percentage of flagged papers (around 10%), but account for the highest absolute numbers of flagged papers across more than 500 journals, with flagged paper numbers of 40293 (Springer Nature), 39755 (Elsevier), and 28330 (John Wiley and Sons).

### *Cancer types of flagged papers*

Among all cancer types, gastric cancer papers show the highest percentage of flagged papers, with 22% (18398/82690) of these papers flagged (fig 4). Bone cancers, such as osteosarcoma, follow with 21% (8458/39433) of papers flagged, followed by liver cancer at 20% (26730/136719). Most cancer types fall within a range of 10-15% of flagged papers. Breast, skin, prostate, and blood cancers show the lowest percentages of flagged papers. In terms of absolute
Number of flagged papers per year in cancer research corpus*

Journal articles published from 1999 to 2024

*Flagged papers are textually similar to retracted paper mill papers

thebmj

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

Article DOI: 10.1136/bmj-2025-087581 • Download data

Fig 1 | Number of papers each year in cancer research corpus flagged because their titles and abstracts were similar to those of retracted paper mill papers. 95% confidence intervals estimated using bootstrap resampling. An interactive version of this graphic and downloadable data are available at https://public.flourish.studio/ visualisation/27240897/

numbers, lung (28435) and liver (26730) cancer account for the highest numbers of flagged papers.

Cancer research area of flagged papers

Flagged papers are largely concentrated in cancer biology and fundamental research, as well as in treatment development or evaluation and in diagnosis and prognosis, with percentages exceeding 10% (fig 5). By contrast, areas such as survivorship, supportive care, and end of life; epidemiology and population studies; and health systems, policy, and implementation had lower percentages of flagged papers (<2%).

Flagged cancer research papers in high impact factor journals

The percentage of flagged papers in the top 10% of journals from the cancer research corpus by impact factor (decile 1) shows a clear increase over time (fig 6). While the percentage remained low in the early 2000s, a sustained increase occurred in the following years, reaching over 10% in 2022 (3383/28991). The minimum impact factor required to be among the top 10% of journals for each year (cutoff impact factor) also increased from 3 in 1999 to 7 in 2021.

Discussion

Principal findings

We have fine tuned a BERT machine learning model that achieves good accuracy in flagging suspected

paper mill papers, using combined titles and abstracts from Retraction Watch and image integrity experts' datasets. The model also flagged many papers previously identified as containing incorrect nucleotide sequences or cell lines,⁴²⁻⁴⁴ despite not having access to this information, showing its ability to detect problematic papers without circular validation.

The model's performance on Retraction Watch and integrity experts' data supports the hypothesis that paper mills can rely on textual templates to create the titles and abstracts of cancer research papers. Although some of the text signals might be because of geographical writing styles, specific terminology, or area specific research topics, the analysis of misclassified papers showed little systematic bias with a low number of false positives. Concerns about our model only identifying the writing style of Chinese authors are somewhat ameliorated by the high number of false negatives for authors from China, whereas we would be concerned about undue geographical discrimination if any country was overrepresented in the false positives.

The model flagged almost 10% of cancer papers, a higher proportion than the previous 3% estimate of paper mill paper prevalence in biomedical research,³ although those estimates were derived from different datasets, time periods, and detection criteria. The positive predictive value depends on the true prevalence of paper mill papers, which remains unknown but is likely higher in cancer research. Assuming a 10% true prevalence and the observed model performance (sensitivity and specificity), the positive predictive value would be approximately 0.75 meaning that around 30% of flagged papers would represent false positives. This reinforces the need to not solely rely on the model prediction to conclude that a manuscript is from a paper mill.

The exponential rise of flagged papers in figure flattens after 2022. Three potential hypotheses can explain this phenomenon: the result of the publishers and research community fighting back against paper mills; a shift to new templates by paper mills after the rise of AI; and the known delays in PubMed listing all papers for the most recent years. The relatively low number of flagged papers before 2010 may reflect the distribution of the training data, which primarily includes retracted paper mill papers published between 2013 and 2023, rather than indicating a near complete absence of such features during these years. This finding could also reflect the use of proxy labels, rather than verified ground truth. Some unlabelled older papers may be paper mill products, inflating apparent time trends and contributing to temporal drift.

The higher proportion of flagged papers in gastric and liver cancer research may partly be explained by the high prevalence of these cancers in China.⁴⁶ However, their marked overrepresentation among misidentified cell lines—25% and 15% of all such lines, respectively⁴⁷—is striking. Given that some misidentified cell lines, such as BGC-823 and BEL-

Protected by copyright, including for uses related to text and data mining, AI training, and similar technologies.
Protected by copyright, including for uses related to text and data mining, AI training, and similar technologies.

Percentage of flagged papers per country in cancer research corpus*

<figure><img src="image_6.png" /></figure>

Across 25 countries with highest numbers of flagged papers based on first author's
affiliation country

*Flagged papers are textually similar to retracted paper mill papers

<figure><img src="image_7.png" /></figure>

Article DOI: 10.1136/bmj-2025-087581 • Download data

Fig 2 | Percentage of papers in cancer research corpus flagged because their titles and abstracts were similar to those of retracted paper mill papers across 25 countries with highest numbers of flagged papers based on first author's affiliation country. An interactive version of this graphic and downloadable data are available at https://public.flourish.studio/visualisation/27226146/

7402, appear almost exclusively in publications from Chinese institutions,⁴⁷ this pattern may also reflect vulnerabilities exploited by paper mills when popular research topics are targeted. Furthermore, this pattern could result from inertia because early templates were reused and adapted repeatedly in these domains.

The rise in the percentage of flagged papers in high
impact journals suggests that paper mill papers are not
just a problem for low impact journals. The concurrent
increase in impact factors and the spread of flagged

papers suggest that both phenomena may stem from
the pressures of the publish-or-perish culture.⁴⁸
The increase in flagged papers in high impact factor
journals highlights an important limitation of using
impact factors as proxies for research quality.⁴³

Strengths and limitations of the study

The consistent model performance across different
validation sets confirms its ability to reliably identify
textual features characteristic of paper mill papers
Protected by copyright, including for uses related to text and data mining, AI training, and similar technologies.

<figure><img src="image_8.png" /></figure>

Article DOI: 10.1136/bmj-2025-087581 • Download data

<caption>Fig 3 | Percentage of papers in cancer research corpus flagged because their titles and abstracts were similar to those of retracted paper mill papers for 25 publishers with highest numbers of flagged papers. An interactive version of this graphic and downloadable data are available at https://public.flourish.studio/visualisation/27226940/</caption>

and strengthens the hypothesis that these papers share common title and abstract templates. Several indirect indicators support the validity of the model's predictions: the exponential trend over time in flagged papers coincides with the known development of paper mill papers.² China being the leading country in terms of flagged papers is consistent with findings of paper mill paper origins.⁹ ⁴⁹ ⁵⁰ Publishers with a high percentage of flagged papers in this study have been found to publish problematic papers, and some have

been flagged as “in conflict with academic rigour” by the Chinese government.⁵⁰⁻⁵² Flagged papers were more common in fundamental research, which coincides with evidence from the literature.⁵⁰ Finally, suspected paper mill papers have already been identified in high impact factor cancer journals.⁴³

Our training set of paper mill papers has limitations. The tag “paper mill” in the Retraction Watch database reflects only Retraction Watch staff’s interpretation of the publisher’s retraction notice. No uniformity exists
Protected by copyright, including for uses related to text and data mining, AI training, and similar technologies.

## Percentage of flagged papers in cancer research corpus according to cancer type*

thebmj

Studied cancer types determined by AI labelling. Papers not discussing a specific cancer type are not shown

*Flagged papers are textually similar to retracted paper mill papers

<figure><img src="image_9.png" /></figure>

Article DOI: 10.1136/bmj-2025-087581 • Download data

Fig 4 | Percentage of cancer research papers according to cancer type flagged because their titles and abstracts were similar to those of retracted paper mill papers. An interactive version of this graphic and downloadable data are available at https://public.flourish.studio/visualisation/27227226/

in the way publishers investigate fraudulent papers and there is no standard way of interpreting the retraction notices; therefore, the “paper mill” qualification likely reflects varied levels of evidence. The papers listed online by research integrity experts include evidence of image manipulation, which can occur within settings beyond paper mills. Additionally, experts’ methods and transparency may vary. Research on paper mills remains limited,¹ which could mean that the currently identified paper mill papers represent only a fraction of their actual prevalence in the scientific literature. Consequently, the model will likely not detect textual features associated with paper mill papers as a whole, but rather those represented in the training set.

The overrepresentation of authors from Chinese institutions among retracted papers suspected of originating from paper mills introduces a potential bias.

Despite efforts to balance by language in the control set, a residual risk remains that the model may learn to associate linguistic patterns of Chinese scientific writing with paper mill content, rather than identifying features specific to fraudulent manuscripts. However, the model misclassification analysis shows few false positives, and an overrepresentation of Chinese papers among false negatives—which does not indicate systematic over-flagging. Furthermore, the paper mills’ country of origin can differ from the authors’ countries and Abalkina showed that a Russian paper mill sold publications to “more than 800 scholars affiliated with more than 300 universities from at least 39 countries.”¹⁶ The misclassifications may instead reflect blind spots in the training data where certain textual features in the wider population of paper mill papers were underrepresented or absent in our training data.
Protected by copyright, including for uses related to text and data mining. All training and similar technologies.

## Percentage of flagged papers within each research area of cancer research corpus*

Research areas determined by AI labelling. Each paper could be assigned to multiple research areas

*Flagged papers are textually similar to retracted paper mill papers

thebmj

<figure><img src="image_10.png" /></figure>

Article DOI: 10.1136/bmj-2025-087581 • Download data

Fig 5 | Percentage of cancer research papers within each research area flagged because their titles and abstracts were similar to those of retracted paper mill papers. Because categories were assigned using a multi-labelling tool, each paper may appear in several categories and sum of all categories does not equal total number of flagged papers. Percentages represent proportion of papers flagged as similar to retracted paper mill papers within each research area, calculated as (flagged papers in area/total papers in area)×100. An interactive version of this graphic and downloadable data are available at https://public.flourish.studio/visualisation/27227359/

Additional sources of bias may stem from the composition of the control set. Controls were not randomly sampled from the broader cancer research literature to avoid including undetected paper mill papers. As a result, only a limited number of verified high quality Chinese publications were included in the controls to minimise potential overlap with paper mill content. The assumption—supported by retraction data—that articles published in selected high impact journals or authored by Taiwanese, Swedish, Norwegian, and Finnish research teams can serve as proxies for high quality controls is open to criticism. Although this strategy may enhance contrast between genuine and fabricated texts, it may also limit the model's ability to detect more nuanced cases of fabrication. We also acknowledge that only limited human verification was conducted to assess the validity of the ground truth labels, and that potential misclassifications may have affected the model's quality. Future research could manually verify a larger and more diverse set of Chinese publications to improve label accuracy.

The non-explainability of deep learning models prevents us from directly identifying the features captured by BERT. Flagged papers can include actual paper mill features; other features of misconduct;

original work copied by paper mills; original work drawing inspiration from paper mill papers; and mistakes by the model. This research does not aim to directly identify paper mill papers or to accuse anyone of fraud. It focuses on studying overall patterns and trends rather than individual cases. The classifier is a probabilistic model, not a definitive arbiter of misconduct. As such, all flagged papers represent statistical predictions based on textual features and should be interpreted as signals requiring human judgment and further verification, not as confirmed cases of fraud.

## Future directions and implications

Our model could be continuously improved by updating the training set with the latest confirmed paper mill papers. Because only titles and abstracts were used to train the model, incorporating full text data or selected sections of the full text has the potential to further enhance its performance. Future work could explore alternative, less computationally intensive training strategies, which might prove as effective as our fine tuned BERT classifier, considering that we evaluated only a limited set of configurations. Experimenting with other aggregation strategies and post hoc calibration methods may also help to
Percentage of flagged papers in top 10% of journals of cancer research
corpus*

Top 10% of journals computed using SClmago journal impact factor for each year (cut-off impact factor)

*Flagged papers are textually similar to retracted paper mill papers

thebmj

<figure><img src="image_11.png" /></figure>

Article DOI: 10.1136/bmj-2025-087581 • Download data

<caption>Fig 6 | Percentage of cancer research papers flagged because their titles and abstracts were similar to those of retracted paper mill papers in top 10% of journals by impact factor according to publication year. An interactive version of this graphic and downloadable data are available at https://public.flourish.studio/visualisation/27227402</caption>

improve the robustness and interpretability of model
predictions.

We expect the paper mills to react and innovate as
detection methods like ours threaten their income.
The release of OpenAI's ChatGPT-3.5 in 2022 and the
rise of generative AI might further blur the boundaries
between genuine and fabricated texts, rendering future
automated detection of fraudulent features more
challenging. While efforts to combat paper mills may
evolve into an arms race,¹ this problem has reached an
unacceptable scale. Inaction risks allowing paper mills
to spread further, potentially compromising entire
journals and publishers—as already seen in the case of
Hindawi.⁵

Our model is currently integrated into the online
submission systems of three journals from a major
publisher and is being used to screen cancer related
manuscripts. To prevent unfair paper mill attributions,
final decisions are always made by humans, using
their expertise and a multitool detection approach to
support decision making. The predictions are intended
to be informative and to precede a second phase of
scrutiny. Therefore, submissions are never rejected on
the basis of the tool alone but only following serious

integrity findings. Authors are not informed if their
paper is flagged to prevent paper mills from adapting
their templates. This collaboration did not influence,
the writing of this manuscript and no data were
collected throughout this pilot testing phase.

Conclusion

This study shows that using machine learning to
identify papers resembling retracted paper mill papers
from titles and abstracts is feasible and effective. Our
findings reveal concerning trends in cancer research
publishing. The rising percentage of flagged papers
indicates that paper mills have grown in ambition and
now target higher impact factor journals, highlighting
the need for vigilance among journals, reviewers, and
researchers. Although our model has clear limitations,
it provides useful insights and highlights the need for
collective awareness to curb the spread of paper mill
publications.

The authors thank Nathalie Bock for her valuable review of
this manuscript; members of the high performance computing
environment teams of Aqua (Queensland University of Technology),
Bunya (Queensland Cyber Infrastructure Foundation, on behalf
of the University of Queensland) and Tesla (Institut de recherche
mathématique de Rennes, France) for granting access to their

Protected by copyright, including for uses related to text and data mining, AI training, and similar technologies.
Protected by copyright, including for uses related to text and data mining, AI training, and similar technologies.

resources and for their technical support; and the teams at PubMed, Retraction Watch, SClmago, and the contributors from PubPeer, especially anonymous user Hoya Camphorifolia, for sharing their data and insights.

**Contributors:** AGB and JAB conceptualised the study. BS, AGB, DC, and JAB developed the methodology. BS developed and executed the code. BS, AGB, DC, and JAB performed the analysis. BS and AGB wrote the manuscript. BS, AGB, DC, and JAB reviewed and edited the manuscript. JAB and AGB secured the funding. AGB, JAB, and DC supervised the work. All work reported in the paper was performed solely by the authors. AGB is the guarantor of the study. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

**Funding:** This study was funded by the National Health and Medical Research Council (ideas grant 2029249; Problematic Articles and Literature Reviews in Molecular Cancer Research). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

**Competing interests:** All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare support from the National Health and Medical Research Council for the submitted work; no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

**Ethical approval:** Not applicable.

**Data sharing:** All data used in this study are publicly available: PubMed annual XML datasets (https://ftp.ncbi.nlm.nih.gov/pubmed/ baseline/), Retraction Watch database,²⁷ and the integrity experts' dataset.³² The fine tuned models and code developed in this study are not publicly disclosed to prevent potential misuse by people seeking to evade fraud detection. The list of flagged PMIDs is available from the corresponding author (AGB) upon reasonable request.

**Transparency:** The corresponding author (AGB) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

**Dissemination to participants and related patient and public communities:** This study did not involve human participants. Findings will be shared with the scientific community through publication, media interviews, press coverage, social media, and presentations focused on research integrity.

**Provenance and peer review:** Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

1. Byrne JA, Park Y, Richardson RAK, Pathmendra P, Sun M, Stoeger T. Protection of the human gene research literature from contract cheating organizations known as research paper mills. *Nucleic Acids Res* 2022;50:12058-70. doi:10.1093/nar/gkac1139

2. Committee on Publication Ethics. STM. Paper mills research report and recommendations. Technical report. Vol 6. 2022. https://members publicationethics.org/sites/default/files/paper-mills-cope-stm-research-report.pdf

3. Van Noorden R. How big is science's fake-paper problem? *Nature* 2023;623:466-7. doi:10.1038/d41586-023-03464-x

4. Joelving F, Retraction Watch. Paper trail. *Science* 2024;383:252-5. doi:10.1126/science.ado3039

5. Van Noorden R. More than 10,000 research papers were retracted in 2023—a new record. *Nature* 2023;624:479-81. doi:10.1038/d41586-023-03974-8

6. Byrne JA, Grima N, Capes-Davis A, Labbé C. The possibility of systematic research fraud targeting under-studied human genes: causes, consequences, and potential solutions. *Biomark Insights* 2019;14:1177271919829162. doi:10.1177/1177271919829162

7. Byrne JA, Christopher J. Digital magic, or the dark arts of the 21st century-how can journals and peer reviewers detect manuscripts and publications from paper mills? *FEBS Lett* 2020;594:583-9. doi:10.1002/1873-3468.13747

8. Christopher J. The raw truth about paper mills. *FEBS Lett* 2021;595:1751-7. doi:10.1002/1873-3468.14143

9. Byrne JA, Labbé C. Striking similarities between publications from China describing single gene knockdown experiments in human cancer cell lines. *Scientometrics* 2017;110:1471-93doi:10.1007/s11192-016-2209-6

10. Hvistendahl M. China's publication bazaar. *Science* 2013;342:1035-9. doi:10.1126/science.342.6162.1035

11. Parker L, Boughton S, Lawrence R, Bero L. Experts identified warning signs of fraudulent research: a qualitative study to inform a screening tool. *J Clin Epidemiol* 2022;151:1-17. doi:10.1016/j.jclinepi.2022.07.006

12. Else H, Van Noorden R. The fight against fake-paper factories that churn out sham science. *Nature* 2021;591:516-9. doi:10.1038/d41586-021-00733-5

13. Porter SJ, McIntosh LD. Identifying fabricated networks within authorship-for-sale enterprises. *Sci Rep* 2024;14:29569. doi:10.1038/s41598-024-71230-8

14. Candal-Pedreira C, Guerra-Tort C, Ruano-Ravina A, et al. Retracted papers originating from paper mills: a cross-sectional analysis of references and citations. *J Clin Epidemiol* 2024;172:111397. doi:10.1016/j.jclinepi.2024.111397

15. Ibrahim H, Liu F, Zaki Y, Rahwan T. Citation manipulation through citation mills and pre-print servers. *Sci Rep* 2025;15:5480. doi:10.1038/s41598-025-88709-7

16. Abalkina A. Publication and collaboration anomalies in academic papers originating from a paper mill: Evidence from a Russia-based paper mill. *Learn Publ* 2023;36:689-702doi:10.1002/leap.1574.

17. A Sting Inside a Papermill. For Better Science. 2025. https://forbetterscience.com/2025/05/19/a-sting-inside-a-papermill/

18. Gu J, Wang X, Li C, et al. Al-enabled image fraud in scientific publications. *Patterns (N Y)* 2022;3:100511. doi:10.1016/j.patter.2022.100511

19. STM Integrity Hub - STM Association (accessed 30 June 2025). https://stm-assoc.org/what-we-do/strategic-areas/research-integrity/integrity-hub/

20. Frontiers Media. Editorial policies and publication ethics (accessed 27 October 2025). https://www.frontiersin.org/guidelines/policies-and-publication-ethics

21. Cabanac G, Labbé C, Magazinov A. Tortured phrases: a dubious writing style emerging in science. Evidence of critical issues affecting established journals. *arXiv* 2021;2107.06751

22. Galli C, Sala R, Colangelo MT, Guizzardi S. Tamquam alter idem: formal similarities in a subset of reports on anti-inflammatory compounds in the years 2008-2019. *Scientometrics* 2022;127:3879doi:10.1007/s11192-022-04434-2

23. Galli C, Sala R, Colangelo MT, Guizzardi S. Between innovation and standardization, is there still a room for scientific reports? The rise of a formatting tradition in periodontal research. *Publications* 2019;7:67doi:10.3390/publications7040067

24. Bless C, Waldis A, Parfenova A, Andueza Rodriguez M, Marfurt A. *Analyzing the Evolution of Scientific Misconduct Based on the Language Of Retracted Papers*. Proc Fifth Work Sch Doc Process, 2025: 57-71. doi:10.18653/v1/2025.sdp.1-6

25. Fletcher AHA, Stevenson M. Predicting retracted research: a dataset and machine learning approaches. *Res Integr Peer Rev* 2025;10:9. doi:10.1186/s41073-025-00168-w

26. Chen L, Fu S, Freedman G, et al. Pub-Guard-LLM: detecting fraudulent biomedical articles with reliable explanations. *arXiv* 2025;2502.15429

27. Retraction Watch Database (accessed 16 April 2025). http://retractiondatabase.org/RetractionSearch.aspx?

28. Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. *Proc Conf North Am Chapter Assoc Comput Linguist* 2019;1:4171-86

29. González-Márquez R, Schmidt L, Schmidt BM, Berens P, Kobak D. The landscape of biomedical research. *Patterns (N Y)* 2024;5:100968. doi:10.1016/j.patter.2024.100968

30. National Cancer Institute (accessed 20 May 2025). https://www.cancer.gov/

31. SClmago. SJR-SClmago Journal & Country Rank (accessed 2 June 2025). https://www.scimagojr.com/

32. PubPeer user—Hoya Camphorifolia. Spreadsheet of spreadsheets (accessed 12 May 2025). https://docs.google.com/spreadsheets/d/1zKxaquq42hwHyGzslF38pFyC8xtU8IzmmOFMGYITDI/edit?gid=1473413779#gid=1473413779

33. PubPeer—The online Journal club (accessed 12 May 2025). https://pubpeer.com/

34. Zhu H, Jia Y, Leung SW. Citations of microRNA biomarker articles that were retracted: a systematic review. *JAMA Netw Open* 2024;7:e243173. doi:10.1001/jamanetworkopen.2024.3173

35. Schrag M, Patrick K, Bik E. Academic research integrity investigations must be independent, fair, and timely. *J Law Med Ethics* 2025;53:1-4. doi:10.1017/jme.2025.38
Protected by copyright, including for uses related to text and data mining, AI training, and similar technologies.

36 Priem J, Piwowar H, Orr R. OpenAlex: a fully-open index of scholarly works, authors, venues, institutions, and concepts. *arXiv* 2022;2205.01833

37 Liu Y, Ott M, Goyal N, et al. RoBERTa: a robustly optimized BERT pretraining approach. *arXiv* 2019;1907.11692

38 Lee J, Yoon W, Kim S, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. *Bioinformatics* 2020;36:1234-40. doi:10.1093/bioinformatics/btz682

39 Gu Y, Tinn R, Cheng H, et al. Domain-specific language model pretraining for biomedical natural language processing. *ACM Trans Comput Healthc* 2022;3:1-23doi:10.1145/3458754

40 Beltagy I, Peters ME, Cohan A. Longformer: the long-document transformer. *arXiv* 2020;2004.05150

41 Li Y, Wehbe RM, Ahmad FS, Wang H, Luo Y. Clinical-Longformer and Clinical-BigBird: transformers for long clinical sequences. *arXiv* 2022;2201.11838

42 Oste DJ, Pathmendra P, Richardson RAK, et al. Misspellings or "miscellings—non-verifiable and unknown cell lines in cancer research publications. *Int J Cancer* 2024;155:1278-89. doi:10.1002/ijc.34995

43 Pathmendra P, Park Y, Enguita FJ, Byrne JA. Verification of nucleotide sequence reagent identities in original publications in high impact factor cancer research journals. *Naunyn Schmiedebergs Arch Pharmacol* 2024;397:5049-66. doi:10.1007/s00210-023-02846-2

44 Park Y, West RA, Pathmendra P, et al. Identification of human gene research articles with wrongly identified nucleotide sequences. *Life Sci Alliance* 2022;5:e202101203. doi:10.26508/lsa.202101203

45 Wickham H. ggplot2: elegant graphics for data analysis. *J R Stat Soc Ser A Stat Soc* 2016;174:245-6

46 Xie W, Yang T, Zuo J, et al. Chinese and global burdens of gastrointestinal cancers from 1990 to 2019. *Front Public Health* 2022;10:941284. doi:10.3389/fpubh.2022.941284

47 Souren NY, Fusenig NE, Heck S, et al. Cell line authentication: a necessity for reproducible biomedical research. *EMBO J* 2022;41:e111307. doi:10.15252/embj.2022111307

48 Vasconez-Gonzalez J, Izquierdo-Condoy JS, Naranjo-Lara P, Garcia-Bereguiain MÁ, Ortiz-Prado E. Integrity at stake: confronting "publish or perish" in the developing world and emerging economies. *Front Med (Lausanne)* 2024;11:1405424. doi:10.3389/fmed.2024.1405424

49 Chambers H. Unmasking the fraud: how paper mills are undermining scientific publishing. *Dev Med Child Neurol* 2024;66:1262-3. doi:10.1111/dmcn.15997

50 Candal-Pedreira C, Ross JS, Ruano-Ravina A, Egilman DS, Fernández E, Pérez-Ríos M. Retracted papers originating from paper mills: cross sectional study. *BMJ* 2022;379:e071517. doi:10.1136/bmj-2022-071517

51 Bik EM, Casadevall A, Fang FC. The prevalence of inappropriate image duplication in biomedical research publications. *mBio* 2016;7:e00809-16. doi:10.1128/mBio.00809-16

52 Zhang L, Wei Y, Sivertsen G, Huang Y. The motivations and criteria behind China's list of questionable journals. *Learn Publ* 2022;35:467-80doi:10.1002/leap.1456

Web appendix: Supplementary files