Spotting AI ’s Touch: Identifying LLM -Paraphrased Spans in Text (2024)

Abstract: AI-generated text detection has attracted increasing attention as powerful language models approach human-level generation. Limited work is devoted to detecting (partially) AI-paraphrased texts. However, AI paraphrasing is commonly employed in various application scenarios for text refinement and diversity. To this end, we propose a novel detection framework, paraphrased text span detection (PTD), aiming to identify paraphrased text spans within a text. Different from text-level detection, PTD takes in the full text and assigns each of the sentences with a score indicating the paraphrasing degree. We construct a dedicated dataset, PASTED, for paraphrased text span detection. Both in-distribution and out-of-distribution results demonstrate the effectiveness of PTD models in identifying AI-paraphrased text spans. Statistical and model analysis explains the crucial role of the surrounding context of the paraphrased text spans. Extensive experiments show that PTD models can generalize to versatile paraphrasing prompts as well as multiple paraphrased text spans.

## introduction recent advances in large language models (llms) @xcite @xcite have raised concerns about potential misuse, including student plagiarism @xcite and the spread of fake news @xcite . a line of work @xcite @xcite focuses on ai-generated text detection, which assigns a label of "human-written" or "machine-generated" to a text. in addition to pristine ai-generated texts, ai paraphrasing is frequently utilized to polish writings or enhance textual diversity. ## dataset construction in general, pasted consists of in-distribution training, validation and test sets, along with a generalization testset. we first randomly collect 10% of the original texts from the ai-generation detection dataset @xcite to collect original texts which encompasses various writing tasks and large language models. the original texts can be either authored by humans or ai, both of which have practical applications. paraphrasing human-authored compositions can infringe upon composition copyright, while effective paraphrasing can help machine-generated news evade detection. we consider two paraphrasing styles: contextagnostic paraphrasing and context-aware paraphrasing. context-agnostic paraphrasing modifies texts without considering the surrounding context and is more commonly used. context-aware paraphrasing considers the context, bringing larger challenges to detection, as the paraphrases are more coherent and consistent with the context. in-distribution data. to simulate real-world scenarios, we employ a sampling process that randomly paraphrases a text-span of several consecutive sentences in the original text. the selected text span consists of 1 to 10 sentences. for contextagnostic paraphrasing, we use a powerful commercial llm (chatgpt (openai, 2023b)) to construct paraphrases given the independent candidate text span without considering the context. for contextaware paraphrasing, we consider dipper @xcite , an 11b model which supports paraphrasing conditioned on the surrounding context. we present several data cases in appendix c. we conduct paraphrasing on both human and machine texts and collect 83,089 instances (28,473 original texts and 54,616 paraphrased texts) after text pre-processing and filtering. we split the data into train/validation/test sets, with an 80%/10%/10% partition. detailed data statistics can be referred to in appendix b. ## method a ptd model decomposes the given text into sentences and predicts a label or score for each sentence based on the entire text. the overall pipeline of ptd is depicted in figure 2 . as shown, the text is split into sentences, with a predefined symbol (e.g., "</@xmath0. model prediction: … currently, the uk lacks official ai regulation. however, the government suggests integrating concerns regarding ai activity into the responsibilities of existing regulatory bodies. i decided to try to put things right. ## understanding paraphrased compositions we statistically compare original texts and paraphrased texts to assess their distinguishability. initially, we examine whether paraphrases display distinct word distributions. we randomly divide the dataset into two halves and calculate the kullback-leibler (kl) divergence of the top 100 word frequency distribution between original texts and paraphrased texts. we average results across five seeds to reduce randomness. the kl divergence between the original texts and partially paraphrased texts (0.0018) is significantly lower than that between the paraphrased text spans and the corresponding original text spans (0.056). in other words, when considering the full context, the distinct writing pattern of paraphrasing can be overshadowed, emphasizing the necessity for fine-grained detection. this finding is also evident in figure 3 , which shows the perplexity distribution. the perplexity distribution of paraphrased text closely aligns with that of original text. note that the original text can either be sourced from human or machine, which forms isolated perplexity distributions @xcite . nevertheless, the perplexity of paraphrased text spans exhibits a distribution centered around a value of 12, indicating a strong writing pattern regardless of the data source. therefore, sentence-level detection captures paraphrasing patterns more precisely compared to text-level detection. we further compare the original and the paraphrased text spans under different data sources and paraphrasing methods. results show that both human-written and machine-generated sources yield similar word distribution divergences (0.083 v.s. 0.052). however, context-agnostic paraphrasing results in a significantly different word distribution than context-aware paraphrasing (0.22 v.s. 0.041), indicating that context-agnostic approaches are more lexically diverse but may largely deviate from the original style. word clouds for both paraphrasing methods are shown in appendix e, demonstrating the novel word distribution introduced by context-agnostic paraphrasing. ## analysis in this section, we set the decision boundary to maintain an fpr of 1% on the validation set to accommodate various testsets across all analytic experiments and report detection accuracy. effect of surrounding context. as discussed in section 7, paraphrased texts exhibit strong writing patterns different from the surrounding texts. we conduct an ablation study on the effect of the surrounding context, with results shown in the models in the table are trained merely on the paraphrased sentences, without considering the surrounding context. as shown, both classification and regression (lexical) models suffer substantial performance degradation when missing the context information, with over 100% performance drop on detection accuracy. to gain further insight, we analyze perplexity variations across token positions around the boundary of the paraphrased text span. the results are presented in figure 6 , where we observe perplexity impulses at both boundaries of the paraphrased text span. context-agnostic paraphrases typically exhibit higher token perplexity from the beginning of the paraphrase. while context-aware paraphrases display lower token perplexity before reaching the end of the paraphrase, they encounter a substantial increase in perplexity afterwards. this indicates that paraphrased sentences cannot perfectly integrate into the original text, even if context is considered during paraphrasing.