Investigating the detection of Tortured Phrases in Scientific Literature (2022)

Abstract: With the help of online tools, unscrupulous authors can today generate a pseudo-scientific article and attempt to publish it. Some of these tools work by replacing or paraphrasing existing texts to produce new content, but they have a tendency to generate nonsensical expressions. A recent study introduced the concept of “tortured phrase”, an unexpected odd phrase that appears instead of the fixed expression. E.g. counterfeit consciousness instead of artificial intelligence. The present study aims at investigating how tortured phrases, that are not yet listed, can be detected automatically. We conducted several experiments, including non-neural binary classification, neural binary classification and cosine similarity comparison of the phrase tokens, yielding noticeable results.

## introduction scientific texts generated by computer programs can be meaningless, and fake generated papers are served and sold by various publishers with the estimation of 4.29 documents every one million reports @xcite . but generated texts are also meaningful: with the inputs of a thousand articles, new books are now produced (e.g. beta writer, 2019). despite the ability of text-generators to produce counterfeit publications, meaningless generated papers can be easily spotted by both machines and humans @xcite . texts produced by neural language models are more difficult to spot @xcite . these neural language models can produce paraphrased texts that are closer to human-written texts @xcite , and therefore machineparaphrased texts are harder to differentiate from the human-written texts. online tools such as spinbot, and spinner chief are used to paraphrase texts. however the capacity of a paraphrasing software to assist a writer can be harmful to the scientific literature. @xcite screened recent publications (e.g. in the journal microprocessors and microsystems) and discovered over 500 meaning less phrases in those scientific papers. they called it "tortured phrases", unexpected odd phrases replacing the lexicalised expression, such as counterfeit consciousness instead of artificial intelligence (i.e., the expected phrase). the database of tortured phrases, and articles that contain them, have since been expanded to over 9000 publications in different domains such as computer sciences, biology or medicine. in this paper, we investigate strategies to automatically detect new (i.e. unlisted) tortured phrases. focusing solely on tortured phrases detection, and not paraphrased text in general, we will use recent machine learning techniques and state-of-the-art language models. our methods were trained on a corpus composed of 141 known tortured phrases, taking their sentences as contexts, and aims at detecting never-seen-before tortured phrases. all code and corpus used are available online. ## related works up to now, no dataset has been built for the automatic detection of tortured phrases. in @xcite , authors and contributors collected a set of tortured phrases and their expected phrases that we will use as dataset. @xcite used spinbot and spinnerchief to paraphrase original data from several sources such as an arxiv test sets, graduation theses, and wikipedia articles. their study aims at detecting whether a paragraph is machine-paraphrased or not. the authors tested classic machine learning approaches and neural language models based on the transformer architecture @xcite , such as bert @xcite , roberta @xcite , albert @xcite , longformer @xcite , and others. they showed that such approaches can complement text-matching software, such as plagscan and turnitin, which often fail to notice machine-paraphrased plagiarism. because paraphrasing tools like spinbot and spinnerchief can generate tortured phrases, the dataset created by @xcite surely contains such phrases. but the task we aim at, i.e. detecting new tortured phrases, is more specific than detecting paraphrased text. thus, we investigated three supervised machine learning classifiers: random forest, perceptron, and transfomer-based model. random forest classifier @xcite is an ensemble learning method that builds decision trees and classifies each data according to the most selected class. perceptron @xcite is a linear classifier used to classify vectors of numbers. term frequency-inverse document frequency (tf-idf), glove @xcite , and bert @xcite were used for word vector representation. these models were chosen for their state-of-the-art performances and to compare how a model with fixed vectors (i.e. glove) compares with a model using dynamic ones (i.e. bert). ## building datasets of tortured phrases this experiment uses two data sources: the tortured phrases @xcite database and the contexts containing tortured phrases from @xcite . we automatically extracted the context of known tortured phrases from the corpus of @xcite to build a training set. tortured phrases identified by @xcite consists of 558 tortured phrases (e.g. we extracted the paragraphs containing tortured phrases @xcite to build a training and evaluation corpus. this resulted in 1, 104 paragraphs with tortured phrases and 1, 668 paragraphs without tortured phrases (randomly extracted from the non-paraphrased original data). ## experiment and result we investigated binary classifiers to check the difference between paragraphs or five-grams containing tortured phrases in several settings. the paragraphs and five-grams containing tortured phrases are considered positives, with label '1', while negative paragraphs are labeled as '0'. accuracy, precision, recall, and f-measure are used to evaluate the classification performances. classifiers: random forest and perceptron in this experiment, five-grams data are used in the classification. the five-grams are converted to a numerical representation using sklearn tf-idf count vectorizer and split randomly 80% for training and 20% for testing. we used the scikit learn library for the random forest and perceptron with the default value of all parameters. the result in after observing the accuracy, precision, recall, and f1-score, we see that the models perform well based on tf-idf vector representation. however, it is believable that the models learned to classify five-grams based solely on specific words: since the training and test data were split randomly, a tortured phrase can be present in both sub-sets. ## conclusions in this research, we aimed at detecting new tortured phrases. we studied different classification approaches and examined the characteristics of tortured phrases using cosine similarity. the result of perceptron and random forest classifier are high, but we intuitively suspect they are not reliable due to the word representation using tf-idf vectorization. the transformer-based classifier model with paragraph data provided the best result among transformer models. however, we suspect that the model learned to classify paragraphs based only on a few tokens, and classifying paragraphs is not sufficient to detect the exact tortured phrases. thus, the transformer-based classifier model five-gram data yields the best result with balanced classes (i.e. results above .70 for all metrics). we also studied the use of cosine similarity between the phrase tokens to identify new tortured phrase. this showed that language model with fixed vectors (e.g. glove) could be used to classify part of the phrases. future research should include more human evaluation of tortured phrases and a bigger dataset tortured phrases with their context. to improve the classification, future work could investigate support vector machine (svm) and naïve bayes model (nb): svm performs better with a small dataset and binary class, while nb can provide probabilities of a prediction.finally, computing the cosine similarity of the tortured phrase and expected phrase pairs within the whole context to see tortured phrases' performance.