# Automatic extraction of tortured phrases from the scientific literature: screening for hallmarks of fraudulent publications

Alexandre Clausse

<figure><img src="image_1.png" /></figure>

## To cite this version:

Alexandre Clausse. Automatic extraction of tortured phrases from the scientific literature: screening for hallmarks of fraudulent publications. ISSI'25: 20th International Conference on Scientometrics and Informetrics, Jun 2025, Yerevan, Armenia. hal-05148333

HAL Id: hal-05148333

https://hal.science/hal-05148333v1

Submitted on 7 Jul 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Automatic extraction of tortured phrases from the scientific literature:
screening for hallmarks of fraudulent publications

**NAME:** Alexandre Clausse.

**INSTITUTION:** Toulouse III - Paul Sabatier university, Toulouse, France.

**ADVISOR(S):** Guillaume Cabanac, Cyril Labbé, and Pascal Cuxac.

**STATUS IN PROGRAM:** 2^nd^ year PhD student.

## DESCRIPTION OF RESEARCH:

In the modern research world, the "publish or perish" work environment leads a small amount of unscrupulous people to adopt strategies in order to improve their metrics, for career prospects (Biagioli & Lippman, 2020). These strategies range from questionable research practices to proven fraud, including falsification, fabrication, and plagiarism. Some fraudsters do not hesitate to resort to paper-mill services (Abalkina et al., 2025), including the buying of articles featuring patchworks of plagiarized contents. These can be hidden using paraphrasing tools such as spinners (e.g., SpinBot) which randomly replace words with synonyms given a copied-pasted content.

These tools, used to rephrase scientific content, produce lexical artifacts known as tortured phrases (Cabanac, Labbé & Magazinov, 2021). These are established scientific terms replaced by synonyms, making them nonsensical given their fields of study. As an example, "amino corrosive" is a tortured phrase for "amino acid", which is meaningless in the medical field (Cabanac, Labbé & Magazinov, 2022). Fraudulent articles featuring tortured phrases are part of scientific misconducts, increasingly polluting the scientific literature (Van Noorden, 2023). As an example, the Hindawi publisher has been victim of a large-scale manipulation of their editorial process¹ and published many articles containing tortured phrases.

These misconducts, in violation with the Merton's principles of 'disinterestedness' and 'organized skepticism' (Merton, 1942), spread widely. To tackle this issue, a post-publication peer review (PPPR) approach has been set up in 2021. It features the Problematic Paper Screener (PPS) and PubPeer (Cabanac, Labbé & Magazinov, 2022; Barbour & Stell, 2020). It consists in feeding the PPS database with tortured phrases identified from flagged suspect contents (i.e., fingerprints) in order to find them in other articles, and (re)assess them by posting comments on PubPeer. Despite its snowball effect, this solution is time consuming and relies on domain-experts, and is mostly limited to STEM studies.

Tortured phrases present several forms and are not standardized in the PPS database, making their automatic detection difficult to be implemented. For example, fingerprints are stored as "neural organization", "deep learning AND profound learning", "128 pieces~5" (meaning that "128" and "pieces" may be

1. https://www.wiley.com/en-us/network/publishing/research-publishing/open-access/hindawi-publication-manipulation-whitepaper
separated with at most 5 genuine words – 128 bits). Moreover, some of them present polysemy issues (e.g., “profound learning” which is the tortured version of “deep learning” in computer science, but which is genuine in pediatric psychology).

As a PhD student in computer science with a major in text mining, I develop my skills in natural language processing to address this issue, by developing scalable solutions to automatically detect tortured phrases. I strive to promote open science and responsible conduct of research by training university staff on scientific integrity. I also teach databases to undergraduate students. Through my academic journey, and as a member of the ERC NanoBubbles project<fnref n="2" /> on “how, when and why science fails to correct itself”, I worked on several related projects.

First, the NETSCITY platform<fnref n="3" /> allows people to visualize scientific collaborations worldwide, both at the country and city scales, given a bibliographic dataset extracted from several sources (such as Scopus, Web of Science, and Dimensions), including the authors affiliations. I worked on this project to explore the geography of retracted articles through a spatial analysis of data extracted from the Dimensions bibliometric search engine (Cabanac et al., 2023). I also intend to use this tool to visualize the geographic distribution of articles featuring tortured phrases.

Then, I extended the notion of “tortured phrase” to abbreviations, as spun scientific concepts mismatching their abbreviations (Clausse et al., 2023): these are called “tortured abbreviations”. As an example, the “lingering brain organization (ResNet)” is a tortured version of “residual neural network (ResNet)”, in computer science. I developed a software using filtering rules based on syntactic characteristics, to extract mismatching abbreviations. I used a dataset of 150 open-access fraudulent articles flagged on the PPS, which have been manually annotated, and found 185 new fingerprints to be added to the PPS database. This work has been covered in *Science* (O’Grady, 2024).

I lately continued this project during a research stay at GESIS – Leibniz Institute for Social Sciences (Köln, Germany), to assess the ability to generalize of such a solution (Clausse et al., 2025). I focused on the presence of tortured phrases in humanities and social sciences (HSS). To do so, I used 42 HSS fingerprints from the PPS, and 121 generated tortured abbreviations using SpinBot on both the European Language Social Sciences Thesaurus (ELSST) and the Thesaurus for the Social Sciences (THESOZ). I found 26 problematic articles related to education, phycology, and economics studies, and reassessed 4 of them on PubPeer since they contain several distinct tortured phrases. I also found new filtering rules to improve the tortured abbreviations detector.

Finally, I am currently working on a new tortured phrases detection software using regular expressions. The main motivations are to develop a software that is independent from any search engine, in addition to capture the different forms of tortured phrases from the PPS list of fingerprints. I computed the overlapping

<footnote n="2">2 https://nanobubbles.hypotheses.org/</footnote>

<footnote n="3">3 https://irit.fr/netscity/</footnote>
between the PPS results and the one from this approach at the article level, using
the Hindawi XML corpus, and found more than 3k problematic articles between
2020 and 2022 in 139 journals, including 1,948 mutual results with the PPS. I
analyzed a subset of 20 documents, which highlighted several improvements.
Firstly, it proves necessary to revisit the list of fingerprints as some of them lack
crucial information and may be false positives. Secondly, I need to test another
configuration by including references, as some of them contain tortured phrases.
Finally, I need to explore Dimensions' indexing issues such as indexed references
and acknowledgments, and ignored characters, both returning false positives. As of
today, only 1,777 of these flagged articles have been retracted so far.

Following the PPPR approach is still the most reliable, since false positives can be quickly assessed by domain experts, while the current developed solutions have to be improved. These solutions are proposed to publishers so that they can use them as filtering checks in their editorial workflow. Future work will focus on using language models and large language models. I am currently working on two projects. The first one uses several language models used as classifiers to extract noun chunks containing tortured phrases from genuine ones. The second one implements large language models in order to prompt a tortured phrases extraction task.

## Acknowledgments

The NanoBubbles project has received Synergy grant funding from the European Research Council (ERC), as part of the European Union's Horizon 2020 program, grant agreement number 951393.

This article has benefited from an invited research stay at GESIS - Leibniz Institute for the Social Sciences, Köln, Germany.

## References

Abalkina, A., Aquarius, R., Bik, E., Bimler, D., Bishop, D., Byrne, J., Cabanac, G., Day, A., Labbé, C. & Wise, N. (2025). 'Stamp out paper mills' – science sleuths on how to fight fake research. *Nature*, 637. (pp. 1047-1050). DOI: https://doi.org/10.1038/d41586-025-00212-1

Barbour, B. & Stell, B.-M. (2020). PubPeer: scientific assessment without metrics. In Biagioli, M. & Lippman, A. (Eds.), *Gaming the Metrics: Misconduct and Manipulation in Academic Research* (pp. 149-155). The MIT Press. DOI: https://doi.org/10.7551/mitpress/11087.003.0015

Biagioli, M. & Lippman, A. (Eds.). (2020). *Gaming the Metrics: Misconduct and Manipulation in Academic Research*. The MIT Press. DOI: https://doi.org/10.7551/mitpress/11087.001.0001

Cabanac, G., Labbé, C. & Magazinov, A. (2021). Tortured phrases: a dubious writing style emerging in science. Evidence of Critical issues affecting established journals. arXiv preprint. DOI: https://doi.org/10.48550/arXiv.2107.06751
Cabanac, G., Labbé, C. & Magazinov, A. (2022). The ‘Problematic Paper Screener’ automatically select suspect publications for post-publication (re)assessment. Presented at WCRI 2022: 7th World Conference on Research Integrity. arXiv preprint. DOI: https://doi.org/10.48550/arXiv.2210.04895

Cabanac, G., Labbé, C. & Magazinov, A. (2022). “Bosom peril” is not “breast cancer”: How weird computer-generated phrases help researchers find scientific publishing fraud. *Bulletin of the Atomic Scientists*. URL: https://thebulletin.org/2022/01/bosom-peril-is-not-breast-cancer-how-weird-computer-generated-phrases-help-researchers-find-scientific-publishing-fraud/

Cabanac, G., Clausse, A., Jégou, L. & Maisonobe, M. (2023). The geography of retracted papers: showcasing a Crossref-Dimensions-NETSCITY pipeline for the spatial analysis of bibliographic data. *27th International Conference on Science, Technology and Innovation Indicators (STI'23)*. URL: https://hal.science/hal-04731543

Clausse, A., Cabanac, G., Cuxac, P. & Labbé, C. (2023). Mining tortured abbreviations in the scientific literature. *Presented at the 8th World Conference on Research Integrity (WCRI'24)*. URL: https://hal.science/hal-04311600

Clausse, A., Badalova, F., Cabanac, G. & Mayr, P. (2025). Unveiling tortured phrases in Humanities and Social Sciences. ArXiv preprint. DOI: https://doi.org/10.48550/arXiv.2502.04944

Merton, R.-K. (1942). Science and technology in a democratic order. *Journal of Legal and Political Sociology, 1*(1) (pp. 115-126). DOI: https://doi.org/2027/mdp.39015008014428

Van Noorden, R. (2023). How big is science's fake-paper problem? *Nature*, 623 (pp. 466-467). DOI: https://doi.org/10.1038/d41586-023-03464-x