S im LLM : Detecting Sentences Generated by Large Language Models Using Similarity between the Generation and its Re-generation (2024)

Abstract: Large language models have emerged as a significant phenomenon due to their ability to produce natural text across various applications. However, the proliferation of generated text raises concerns regarding its potential misuse in fraudulent activities such as academic dishonesty, spam dissemination, and misinformation propagation. Prior studies have detected the generation of non-analogous text, which manifests numerous differences between original and generated text. We have observed that the similarity between the original text and its generation is notably higher than that between the generated text and its subsequent regeneration. To address this, we propose a novel approach named SimLLM, aimed at estimating the similarity between an input sentence and its generated counterpart to detect analogous machine-generated sentences that closely mimic human-written ones. Our empirical analysis demonstrates SimLLM’s superior performance compared to existing methods.

## introduction the rise of generative ai, especially large language models, has had a substantial impact across various applications. however, it also presents challenges, such as academic dishonesty and the spread of disinformation, stemming from the misuse of generated text. therefore, our goal is to create a strategy to detect and mitigate the negative effects associated with the improper use of generated text. detection of text generated by large language models uses three main techniques. firstly, supervised learning methods @xcite @xcite train classifiers on datasets of original and generated text, though this requires large volumes of training data. the zero-shot approach (bhattacharjee and @xcite eliminates the need for training but is sensitive to out-of-distribution text. recent research explores watermarking methodologies @xcite to force models to produce predefined words, aiding detection, but this requires modifying the models, which is impractical for proprietary models like chatgpt. previous studies mainly address non-analogous text with substantial differences between original and generated content. in contrast, we focus on analogous generated text, where changes to the original text are minimal. motivation an ai model aims to extensively optimize original data to generate new data. this process often results in a significant disparity between the original and the generated data. when the model optimizes the generated data to create re-generated data, the already optimized nature of the generated data limits further optimization. as a result, the gap between the generated and re-generated data diminishes. to illustrate, we randomly selected a human sentence (h) from the extreme summarization (xsum) dataset @xcite (figure 1 ). then, a large language model, specifically chatgpt (gpt 3.5-turbo), was tasked with generating a machine sentence (m chatgp t ) conveying an opposite meaning to the original text. chatgpt and llama 2 70b were utilized to proofread both the human-written text (h chatgp t and h llam a ) and the machine-generated text (m chatgp t -chatgp t and m chatgp t -llam a ), with the respective subscripts indicating the sequence of using the large language models. analysis showed that proofreading of the human text by chatgpt introduced numerous disparities between h and h chatgp t , whereas fewer differences were observed between m chatgp t and m chatgp t -chatgp t . in this example, while there were ten word differences between h and h chatgp t highlighted in underline, ## related work the methods previously used to detect text generated by large language models can be classified into three approaches. the first strategy involves training models on large datasets to identify generated text characteristics, such as openai's fine-tuning of the roberta model @xcite . some researchers have analyzed probability distributions in large language models' hidden layers @xcite , while others have used a paraphraser in a gan to train the detector component @xcite . the intrinsic dimension of the embedding space from long texts has been estimated to understand the workings of these models better @xcite . other approaches include building a proxy model to estimate generated text's perplexity @xcite , using positive-unlabeled learning @xcite to improve performance on short text, and highlighting human text's coherence to spot machinegenerated text discrepancies @xcite . some researchers have also incorporated top similarity texts from the training set into prompts and used in-context learning to boost detector and attacker capabilities @xcite . however, this approach is sensitive to out-of-distribution texts. watermarking is another method where a language model is guided to generate text that meets specific criteria, acting as a watermark to identify generated content. for example, @xcite instructed the model to use only a certain set of "green" words, avoiding the "red" ones. however, this method's downside is that it requires modifying the original models, which is impractical for real-world use, especially considering the proprietary nature of many large language models. the third strategy involves zero-shot detection, where research identifies generated text without training. @xcite employed this method by prompting chatgpt to detect generated texts from various large language models. @xcite noted that large language models often predict the next word in a text sequence with high probability, which can be assessed through ranking, logarithms, and entropy. other researchers have improved performance by combining ranking and logarithms @xcite , or by introducing a method where original words are randomly perturbed and the change in log probability is analyzed @xcite . close to our work, @xcite and @xcite evaluated the similarity between input text and revised text. however, these approaches face challenges in identifying out-of-distribution text. ## conclusion this paper presents a novel method, named simllm, designed to identify sentences generated by large language models. specifically, we augment the original input sentence by integrating re-generated alternatives from candidate large language models. subsequently, this augmented data is input into a classifier to ascertain the origin of the text, whether it is human-generated or from a large language model. experimental results from diverse large language models demonstrate the superior performance of simllm compared to existing methods across various scenarios.