The Influence of the Perplexity Score in the Detection of Machine-generated Texts (2024)

Abstract: The high performance of large language models (LLM) generating natural language represents a real threat, since they can be leveraged to generate any kind of deceptive content. Since there are still disparities among the language generated by machines and the human language, we claim that perplexity may be used as classification signal to discern between machine and human text. We propose a classification model based on XLM-RoBERTa, and we evaluate it on the M4 dataset. The results show that the perplexity score is useful for the identification of machine generated text, but it is constrained by the differences among the LLMs used in the training and test sets.

## introduction large language models (llms) present a large number of capabilities, ranging from text summarization and information extraction to text paraphrasing @xcite . one of those abilities is text generation, which is approaching to the human written performance @xcite . however, they also present some pitfalls that can lead to privacy and security leaks. for instance, the tendency to hallucinate of llms may lead to privacy violations by exposing sensitive data @xcite . likewise, the generative capacity of llms is an extremely positive skill for many applications, but it may be used to generate deceptive and malicious content, which can be used as a source of security leaks @xcite @xcite . hence, we need the automatic identification of machine generated text to warm about it to the readers. we can consider the language generated by each person that follows a particular probability distribution. although, the small nuances among the use of language of each person, the spoken and written language by humans follow a common probability distribution. similarly, the language generated by llms follows a specific probability distribution, with some disparities between llms, but with a large difference with respect to the human language. perplexity is a measure of uncertainty in the value of a sample from a discrete probability distribution @xcite . accordingly, a low value of perplexity means a reduced uncertainty score that the sample is drawn from a probability distribution, otherwise it is likely that the sample does not belong to the distribution. hence, perplexity can be used to discern whether a span of text follows the probability distribution of the language usually generated by a llm or by a human. in this work, we claim that perplexity can be used as a classification signal for identifying span of text generated by machines, with the aim of warming readers and protecting them from deceptive content. we thus propose a classification system built upon the xlm-roberta language model @xcite , whose input are the word embeddings vectors of each input token and the perplexity score of the input text. we evaluate the classification model on the m4 dataset @xcite used in the task 8 of semeval @xcite . moreover, we analyze whether there is any influence in the nature of the llm used to calculate the perplexity score and the one used to generate the evaluation texts. the results show that the perplexity is a useful signal to identify machine-generated texts, but it is limited to a small difference among the probability distribution of the llm used to calculate its score and the one used to generate the text to classify. this work is organized as follows: next section highlights the most salient related works. section 3 justifies the use of perplexity as classification signal. our proposal is described in section 4, and the experimental framework in section 5. then, we analyze the results in section 6, and we remark the main conclusions in section 7. ## related work llms are able to generate text very similar to what a human can do. accordingly, differentiating a machine-written text from a human one is very challenging @xcite . the automatic detection of these kinds of text is crucial to security scenarios like phishing, fake news, identity fraud, and others. powerful models are open to use by anyone with the capability to connect to the internet, such as those ones in hugging face 1 . this facility for the user to be able to generate any type of text with hardly any resources demonstrates the importance of obtaining a system that can differentiate when a text is artificially generated. the need of recognizing machine or artificial intelligence (ai) generated text comes from the first uses of grover @xcite for the generation of propaganda. since that moment several models and methodologies have been published to detect this automatic generated text, because humans struggle at it @xcite . we mainly find two approaches to face up the challenge of detecting ai generated text. on the one hand, the proposals based on used of linguistic features, as for instance tf-idf @xcite or the use of fluency features as the flesch score @xcite . on the other hand, the works ground in the use of language models. for instance, in @xcite , the authors fine-tuned a roberta model to detect gpt-2 generated texts. likewise, in @xcite is shown that features derived from bert outperform linguistic and other features stemmed from other neural models. the literature of machine generated text detectors is wide @xcite , but as far as we know, perplexity has not been used yet as feature to guide the identification of machine generated text. in this paper, we claim to use perplexity as a classification signal, and it shows to give a strong performance as we show in the subsequent sections. ## perplexity as feature perplexity is a metric from information theory that indicates how well a probability distribution or model predicts a given sample. its usefulness resides in facilitating the comparison of various probability models @xcite . a low value 1 https://huggingface.co/models of perplexity means that a sample may be derived from the probability distribution, since there is a low value of uncertainly, otherwise the perplexity value is large. perplexity is usually lower in texts generated by ai and their texts rather express feelings and use unusual words. @xcite show a difference in performance between perplexity-based and machine learning-based classification, the latter being better than perplexity-based classification. consequently, the use of both parameters, text, and perplexity, in training a classifier may be interesting to study in this task, demonstrating that the use of perplexity in texts generated by the llm itself results in highly accurate results. the perplexity of human-generated text tends to be higher than that of machine-generated text according to @xcite , because the perplexity is calculated according to a specific llm, which generates language that follows a different probability distribution than the human language. hence, we calculate the perplexity score of the dataset text that we will use for training and evaluation of the system (see section 5.1). to calculate the perplexity score we used the language model perplexity (lm-ppl) python library. 2 the lm-ppl computes an ordinary perplexity for recurrent lms such as gpt3 @xcite . we calculate the perplexity score of each instance using the gpt2 language model @xcite . the perplexity pp of a discrete probability distribution p is a widely used concept in information theory, where h(p) is the entropy of the distribution, and x ranges over the events. ## machine-detection system we have developed a fine-tuning classification model based on xlm-roberta for differentiating text authorship. this machine-detection (md) system uses the text and the perplexity associated with to fuse the two feature sets, we use the multimodal toolkit library, which offers several fusion methods. in this case, we have selected a specific approach that involves multi-layer perceptron (mlp) partitioning for categorical and numerical features. subsequently, the output of the transformer is concatenated with processed numerical and categorical features before reaching the final classifier. once it reaches the classification head, the system is trained. to optimize this training, we performed a hyperparameter optimization (see section 5.2). we depict the system in figure 1. ## experimental framework we have developed a training system including data from all llms as a baseline for our experimental framework. one training has been conducted using perplexity and the other without it. we have also assessed the performance of our proposal when the training and test texts have been generated using the same llm, and we compare them when that difference is not done. this proves that the use of perplexity improves the performance of the system when the model is trained and evaluated using the machine-generated text by the same llm. these two baselines allow us to compare them with our proposed system, demonstrating that the use of perplexity improves the hit rate in identifying the authorship of the text when training and predicting the generated text with the same linguistic model. baseline one -fine-tuning the system without perplexity value is a fine-tuning using the xlm-roberta-large, trained with a balanced dataset where the machine-generated text used is comprised of all the texts of the llms. baseline two -fine-tuning and perplexity this baseline is similar to the previous system but with the addition of perplexity. the same dataset is used in this system. this system is the same as we propose, the only difference is the training data used. ## results and discussion as shown in the disparity between baseline one and two lies in the macro-f1 score demonstrating the improvement of the system when perplexity is added to the training. while baseline one exhibits superior precision in generating machine text, baseline two demonstrates a broader efficacy. notably, baseline two excels in discerning between human and machine-generated text owing to its balanced consideration of the macro-f1 score for both categories. the results of bloomz have obtained a macro-f1 score of more than 0.90, almost perfect. in contrast, the dolly shows lower results than baseline one and two. the analysis reveals no significant correlation between the average perplexity of a model. the bloomz has an average perplexity similar to that of a text written by a human being, but its results are much higher. using the methodologies defined in this study, evidence emerges for the effectiveness of using per-plexity in conjunction with textual features to classify authority. on the other hand, in cases where there is certainty about the uniformity of the llm model across machine-generated text, the effectiveness of such classification depends on the models used and the methodologies employed to calculate the perplexity score. our hypothesis holds in most cases. with the systems that have been trained with the chatgpt, cohere, and bloomz models we obtain a macro-f1 superior to baseline two, being remarkable improvement where the same llm models are used to train and evaluate the experiments. even, in systems such as the one used by davinci where the macro-f1 is lower than baseline two, we can see an improvement in accuracy. ## conclusions the results obtained have shown that the performance obtained has been improved for most of the llm models that have been worked with. this shows that as long as the same llm generates the machine-generated data the proposed system using perplexity and text can with a high probability of success differentiate between whether a text is machine-generated or human-generated. it is also worth noting the difference in the results between the baselines exposed. this also proves that the additional information on the perplexity of each text is useful information for the authority recognition of the generated text, even if it has been trained by different llms. following the positive results obtained in md system, our next objective will be to classify texts independently of their origin. for this purpose, we will apply the same methodology with considerable modifications. such modifications may include the integration of a new model to calculate text perplexity or the use of several models to generate a vector of perplexities.