Spotting AI’s Touch: Identifying LLM-Paraphrased Spans in Text (2024)

Abstract: AI-generated text detection has attracted increasing attention as powerful language models approach human-level generation. Limited work is devoted to detecting (partially) AIparaphrased texts. However, AI paraphrasing is commonly employed in various application scenarios for text refinement and diversity. To this end, we propose a novel detection framework, paraphrased text span detection (PTD), aiming to identify paraphrased text spans within a text. Different from text-level detection, PTD takes in the full text and assigns each of the sentences with a score indicating the paraphrasing degree. We construct a dedicated dataset, PASTED, for paraphrased text span detection. Both in-distribution and out-of-distribution results demonstrate the effectiveness of PTD models in identifying AI-paraphrased text spans. Statistical and model analysis explains the crucial role of the surrounding context of the paraphrased text spans. Extensive experiments show that PTD models can generalize to versatile paraphrasing prompts and multiple paraphrased text spans.

heading: 1 Introduction
text: Recent advances in large language models (LLMs) (Touvron et al., 2023; Brown et al., 2020; OpenAI, 2023b) have raised concerns about potential misuse, including student plagiarism (Wahle et al., 2022) and the spread of fake news (Mitchell et al., 2023). A line of work (OpenAI, 2023a; Li et al., 2023b; Mitchell et al., 2023; Yang et al., 2023b) focuses on AI-generated text detection, which assigns a label of “human-written” or “machine-generated” to a text. In addition to pristine AI-generated texts, AI paraphrasing is frequently utilized to polish writings or enhance textual diversity.
* Equal contribution. Work was done during Yafu Li’s internship at Tencent AI Lab.
† Corresponding authors.
However, there is limited research on the finegrained detection of texts partially paraphrased or polished by AI. AI paraphrasing can suffer from issues like hallucination (Ji et al., 2023; Zhang et al., 2023) and bias (Navigli et al., 2023), which pose significant challenges to achieving trustworthy AI. For instance, strict oversight of AI paraphrasing is necessary in education to ensure the content remains factual and harmless. Moreover, fine-grained paraphrasing detection can provide statistical evidence for nuanced evaluations. As illustrated in Figure 1, the paraphrased text-span detector assigns a higher probability to the sentence "Currently, the UK lacks official AI regulation" being paraphrased by AI compared to the other sentences within the longer text. Compared with text-level labelling, finer-grained predictions can support more ethical decisions regarding the applications of AI generation, such as in cases of plagiarism.
To this end, we propose a new task called Paraphrased Text span Detection (PTD). PTD predicts a score for each sentence within a long text, identifying AI paraphrased spans comprising consecutive sentences. The sentence-level score also
ar X
iv :2
40 5.
12 68
9v 2
[ cs
.C L
] 2
9 M
ay 2
02 4
7088
reflects the degree to which the paraphrased text deviates from the original text, i.e., paraphrasing degree, as shown in Figure 1. Since there is no existing data for training such a detection system, we assemble a dataset called PASTED (paraphrased text span detection dataset), including original texts and corresponding paraphrased texts. We obtain these paraphrases by paraphrasing parts of the original texts while keeping other sections intact. The original texts encompass not only human-written compositions but also machine-generated texts. Our goal is for a model trained on our dataset to detect AI-paraphrased texts from new domains and unseen models. To achieve this, we construct an additional generalization testset to evaluate out-ofdistribution (OOD) performance, where texts are paraphrased by a novel paraphraser with different prompts.
The PTD approach is based on the observation that AI-paraphrased text exhibits distinct writing patterns compared to both the original text and its surrounding context. This observation is supported by statistical findings and model analysis. Formally, a PTD model encodes the full text and assigns a sequence of labels (classification model) or scores (regression model) to all sentences in the text. For classification, the labels indicate whether a sentence is paraphrased or not. For regression, the scores quantify the degree of paraphrasing by calculating the difference between each paraphrased sentence and its aligned original text span. To construct reference scores, a dedicated algorithm is devised for aligning paraphrasing pairs.
In-distribution performance shows that all methods effectively distinguish paraphrased text spans, with AUROC exceeding 0.95. While the classification model achieves higher detection accuracy, regression models provide more accurate predictions of the degree of paraphrasing, aligning with OOD performance. The aggregate regression model, which considers all types of divergences, achieves the best overall performance. Identifying contextaware paraphrases poses greater difficulty. On the other hand, models obtain better performance on texts with more paraphrased sentences. Results from the generalization testset demonstrate that all methods can generalize to OOD texts and various paraphrasing prompts despite a performance decrease compared with ID.
Analysis reveals that while paraphrased text displays distinct writing patterns, the surrounding context in the original text significantly plays
a crucial role in model detection. Empirical results demonstrate that all models can generalize to texts with multiple paraphrased spans, despite being trained on texts with only one paraphrased span. Moreover, our PTD models can robustly resist minor-perturbed texts that should not be regarded as paraphrases, resulting in few misclassifications. Lastly, we demonstrate the effectiveness of our PTD model in defending against paraphrasing attacks and protecting traditional AI-generation detection systems. The data and code are available at https://github.com/Linzwcs/PASTED.

heading: 2 Related Work
text: AI-generated text (AI-generation) detection has received increasing attention. Li et al. (2023b); Chakraborty et al. (2023) systematically discuss the differentiability of AI texts. Various features are explored for detetcion, including n-gram features (Badaskar et al., 2008), entropy (Lavergne et al., 2008; Gehrmann et al., 2019), perplexity (Beresneva, 2016) and model-wise features (Li et al., 2023a). A more direct method involves training neural classifers (Bakhtin et al., 2019; Fagni et al., 2021; Uchendu et al., 2020; OpenAI, 2023a; Li et al., 2023b). Adversarial learning is utilized for robustly detecting AI generations (Hu et al., 2023; Koike et al., 2023b). Another approach proposes training-free methods for detecting AI generation. Methods by Mitchell et al. (2023); Bao et al. (2023) utilize negative curvature regions in the log probability of a model to identify machinegenerated text. Additionally, Yang et al. (2023b) compare n-gram features between human-written and AI-generated continuations of text. Different from text-level detection, we propose a more finegrained approach that identifies paraphrased sentences within a larger body of text. Sentence-level AI-generation detection proposed by Wang et al. (2023) constructs data by breaking long generations into sentences and trains sequence labeling models accordingly. In contrast, we propose detecting paraphrased spans, which can consist of one or multiple sentences within longer texts, focusing on the distinct paraphrasing patterns compared with the original text and the surrounding context. Sadasivan et al. (2023) propose that AI-generation detection can be vulnerable against paraphrasing attacks. Tripto et al. (2023) discuss the authorship of AI-paraphrased texts. Krishna et al. (2023) utilize retrieval to assist detectors to defend against
paraphrasing attacks, while Yang et al. (2023a) construct a dedicated dataset with ChatGPT polished texts to increase detection robustness. In contrast, we present a novel framework that identifies AIparaphrased text spans within a given text and reflects the degree of paraphrasing for each sentence.

heading: 3 Problem Definition

heading: 3.1 AI-generated Text Detection
text: Given a text sequence x, the AI-generated text detection model predicts a label y of “human-written” or “machine-generated” based on a probability distribution: p(y|x).

heading: 3.2 Paraphrased Text Span Detection
text: A piece of text x can be segmented into a series of sentences: {s1, s2, · · · , sn}. A Paraphrased Text Span Detection (PTD) model is optimized to predict a sequence of binary labels {c1, c2, · · · , cn} or continuous scores {r1, r2, · · · , rn} for these sentences, identifying whether each sentence si has been paraphrased by an AI model given the full text x. For each sentence si within the text x, the PTD model is tasked with either: (1) Producing a probability distribution p(ci|x, i) over binary labels, where ci indicates whether the sentence si is paraphrased; (2) Outputing a continuous score e(x, i) that represents the extent of paraphrasing for the sentence si. In addition to detection, the model is expected to provide insights into the degree of deviation of the paraphrased text from the original text, quantifying changes in meaning, structure, or other linguistic features that signify paraphrasing.

heading: 4 Dataset Construction
text: In general, PASTED consists of in-distribution training, validation and test sets, along with a generalization testset. We first randomly collect 10% of the original texts from the AI-generation detection dataset (Li et al., 2023b) to collect original texts which encompasses various writing tasks and large language models. The original texts can be either authored by humans or AI, both of which have practical applications. Paraphrasing human-authored compositions can infringe upon composition copyright, while effective paraphrasing can help machine-generated news evade detection. We consider two paraphrasing styles: contextagnostic paraphrasing and context-aware paraphrasing. Context-agnostic paraphrasing modifies texts without considering the surrounding context
and is more commonly used. Context-aware paraphrasing considers the context, bringing larger challenges to detection, as the paraphrases are more coherent and consistent with the context.
In-distribution Data. To simulate real-world scenarios, we employ a sampling process that randomly paraphrases a text-span of several consecutive sentences in the original text. The selected text span consists of 1 to 10 sentences. For contextagnostic paraphrasing, we use a powerful commercial LLM (ChatGPT (OpenAI, 2023b)) to construct paraphrases given the independent candidate text span without considering the context. For contextaware paraphrasing, we consider Dipper (Krishna et al., 2023), an 11B model which supports paraphrasing conditioned on the surrounding context. We present several data cases in Appendix C. We conduct paraphrasing on both human and machine texts and collect 83,089 instances (28,473 original texts and 54,616 paraphrased texts) after text pre-processing and filtering. We split the data into train/validation/test sets, with an 80%/10%/10% partition. Detailed data statistics can be referred to in Appendix B.
Generalization Testset. In addition to the indistribution testset, we construct an additional generalization testset where texts are paraphrased by a novel LLM with different paraphrasing prompts. We employ the same sampling process to generate paraphrases on the out-of-distribution testset from Li et al. (2023b). Recent research (Koike et al., 2023a; Kumarage et al., 2023; Lu et al., 2023) demonstrates that prompt engineering can be employed to evade detection effectively. To this end, we utilize an unseen paraphraser, GPT-4 (OpenAI, 2023b), and explore various prompt variants with increasingly complex instructions for generating elaborate paraphrases. The prompts used for data construction are presented in Appendix D. Ultimately, our OOD evaluation comprises a total of 9,372 instances (1,562 original texts and 7,810 paraphrased texts).

heading: 5 Method
text: A PTD model decomposes the given text into sentences and predicts a label or score for each sentence based on the entire text. The overall pipeline of PTD is depicted in Figure 2. As shown, the text is split into sentences, with a predefined symbol (e.g., "
") appended to the end of each sentence.
The paraphrased text span detector then processes the full text and assigns a score to each sentence boundary token, indicating the probability of the sentence being paraphrased.

heading: 5.1 Sentence-level Classification
text: We treat each paraphrased sentence in the paraphrased text span as “paraphrased” and others as “original”, utilizing cross-entropy (CE) to optimize the model.
LCE(θ) = − n∑
i=1
[ci log(pθ(ci|x, i))
+ (1− ci) log(1− pθ(ci|x, i))] (1)
where θ denotes the model parameters.
Limitations of Classification. A major issue of classification arises from its assumption that all paraphrased sentences are equally labeled as "paraphrased", which can overlook the varying degrees of difference involved in paraphrasing. For instance, when given the sentence “I lost my keys yesterday”, the model receives the same label for both paraphrases “Yesterday I lost my keys” and “Yesterday my keys went missing”. However, the latter exhibits disparities with the original sentence in terms of word choices and syntax structure. This lack of calibration in predicted probabilities can reduce reliability and interpretability. Therefore, classification models are less resistant to minor text perturbations which should not be regarded as paraphrases (Section 9).

heading: 5.2 Sentence-level Regression
text: To this end, we propose span-level regression, which leverages various difference quantification metrics to inform to what extent each text span has been modified during paraphrasing. Instead of assigning labels to sentences, our model is trained to predict a difference score ri. This score is calculated using a difference scoring function f(si, ti), such as BLEU score (Papineni et al., 2002). Here, ti represents the text span in the original text that aligns with the current paraphrased sentence si. For training on the original text in our dataset, ti simply refers to si itself. By computing difference scores for each sentence, we optimize our model using mean squared error loss:
LMSE(θ) = 1
n
n∑
i=1
(f(si, ti)− eθ(x, i))2 (2)
Aligning Paraphrased Text Spans. Accurately aligning paraphrased sentences with the original text spans is necessary for reliable indication of the paraphrasing degree as regression labels. A major challenge arises when paraphrasing a text span containing consecutive sentences, which can result in a different number of sentences (41%/45%/14% ratio for fewer, equal, and more sentences in paraphrased text). Furthermore, some paraphrased texts involve reordering at the sentence level. We provide a case illustration in Appendix A. To this end, we propose an alignment algorithm in light of sentence similarity (Reimers and Gurevych, 2019) to align paraphrased sentences with their original counterparts. Our approach involves greedily traversing each paraphrased sentence and identifying a span
of consecutive original sentences that share a high semantic similarity. If no suitable span is found, we resort to finding the most semantically similar original sentence (Appendix A).
Difference Quantification Function. We consider a range of different functions to quantify the paraphrasing degree. Given the aligned text span pairs, the most straightforward metric is the lexical divergence between them, which can be measured using common similarity-based metrics, e.g., BLEU (Papineni et al., 2002). To capture grammatical divergence, we also consider the text similarity score of the part-of-speech sequences. We subtract the BLEU score from 1 to denote the divergence score. For a more comprehensive measure of syntactic divergence, we can calculate the tree edit distance between syntax trees (Zhang and Shasha, 1989) at the third level (Bandel et al., 2022). To normalize the tree edit distance, we divide it by the maximum number of nodes in both trees. This normalization results in a difference score ranging from 0 to 1, where 0 indicates identical trees. Finally, we can aggregate all these divergence metrics, which measure the paraphrasing degree from different granularities and views. Specifically, we train the regression model to fit a aggregate divergence function encompassing a set of metrics {f1, f2, . . . , fd}:
L̂MSE(θ) = 1
n
n∑
i=1
d∑
j=1
(fj(si, ti)− eθ(x, i)j)2
(3) The scores of each dimension are averaged as the final aggregate score.

heading: 6 Experiment Setup
text: Settings. We tokenize text into sentences using the NLTK sentence tokenizer (Bird et al., 2009). To perform context-agnostic paraphrasing, we utilize the GPT-3.5-Trubo API. For context-aware paraphrasing with Dipper, we use the default setting with lexical diversity set at 80 and order diversity set at 60. For the generalization testset, we use GPT4 API “gpt-4-1106-preview”. To measure sentence similarity in aligning paraphrased sentences, we employ a sentence-transformers model* (Reimers and Gurevych, 2019). For constructing regression labels, we obtain part-ofspeech tags and constituency parses using the
*https://huggingface.co/sentence-transformers/allMiniLM-L6-v2
Stanza parser (Qi et al., 2020). We quantify lexical and POS divergence using a 4-gram sentence-level BLEU score and calculate tree edit distance with the ZSS algorithm (Zhang and Shasha, 1989). Following Li et al. (2023b), we train classifiers and regression models based on Longformer (Beltagy et al., 2020) by adding a linear layer. All models are trained for 2 epochs on 1 V100 GPU with a batch size of 12. We used the Adam optimizer (Kingma and Ba, 2015) with a learning rate of 0.005 and set the dropout rate at 0.1. We use GPT-2 large (Radford et al., 2019) to compute text perplexity for experiments in Secition 7 and Section 9.
Evaluation Metrics. To assess the performance of paraphrased sentence detection, we utilize two metrics: AUROC (Area Under the Receiver Operating Characteristic curve) and detection accuracy. AUROC measures a classifier’s capability to differentiate between positive and negative classes, with a value of 1.0 indicating perfect classification and 0.5 representing random guessing. Following Krishna et al. (2023), we fix the false positive rate (FPR) of 1% and adjust the decision boundary accordingly to report accuracy, ensuring that human-authored text is rarely classified as machinegenerated. We denote this metric as Accuracy (FPR 1%). To assess the estimation accuracy of paraphrasing degree, we calculate the Pearson Correlation between model-predicted scores and reference scores. For reference scores, we adopt the lexical diversity and syntactic diversity proposed by Bandel et al. (2022), quantifying lexical and syntactic diversity in generated paraphrases. A high correlation indicates that the model accurately predicts differences between paraphrases and the corresponding original sentences, i.e., the paraphrasing degree. We denote these two correlation scores as Lexical Corr. and Syntactic Corr., respectively. For classification models, we utilize the prediction confidence of the classification model as a proxy for predicting paraphrasing degree.

heading: 7 Understanding Paraphrased Compositions
text: We statistically compare original texts and paraphrased texts to assess their distinguishability. Initially, we examine whether paraphrases display distinct word distributions. We randomly divide the dataset into two halves and calculate the KullbackLeibler (KL) divergence of the top 100 word frequency distribution between original texts and para-
phrased texts. We average results across five seeds to reduce randomness. The KL divergence between the original texts and partially paraphrased texts (0.0018) is significantly lower than that between the paraphrased text spans and the corresponding original text spans (0.056). In other words, when considering the full context, the distinct writing pattern of paraphrasing can be overshadowed, emphasizing the necessity for fine-grained detection.
This finding is also evident in Figure 3, which shows the perplexity distribution. The perplexity distribution of paraphrased text closely aligns with that of original text. Note that the original text can either be sourced from human or machine, which forms isolated perplexity distributions (Mitchell et al., 2023; Li et al., 2023b). Nevertheless, the perplexity of paraphrased text spans exhibits a distribution centered around a value of 12, indicating a strong writing pattern regardless of the data source. Therefore, sentence-level detection captures paraphrasing patterns more precisely compared to text-
level detection. We further compare the original and the para-
phrased text spans under different data sources and paraphrasing methods. Results show that both human-written and machine-generated sources yield similar word distribution divergences (0.083 v.s. 0.052). However, context-agnostic paraphrasing results in a significantly different word distribution than context-aware paraphrasing (0.22 v.s. 0.041), indicating that context-agnostic approaches are more lexically diverse but may largely deviate from the original style. Word clouds for both paraphrasing methods are shown in Appendix E, demonstrating the novel word distribution introduced by context-agnostic paraphrasing.

heading: 8 Results

heading: 8.1 In-distribution Performance
text: The detection performance on random-split testsets is presented in the upper part of Table 1. We consider two baselines: (1) Random which calculates BLEU scores of the paraphrased sentence with a random sentence in the original text and (2) Oracle which calculates BLEU scores of the paraphrased sentence with the aligned text span. All detection methods effectively distinguish paraphrased text spans, with AUROC exceeding 0.95. Although the classification model performs better in detection, it falls short compared to regression models in predicting paraphrasing degree due to improper calibration (discussed in Section 5.1). In contrast, regression models demonstrate stronger alignment with the reference differences between
original texts and paraphrases, both lexically and syntactically. In other words, regression models are more reliable indicators of the extent of difference present in a paraphrased text span. Regression models with grammatical or syntax supervision obtain the best lexical and syntactic correlation. The classification model and lexical regression model obtain the best accuracy (FPR 1%), effectively identifying paraphrases while maintaining a low false positive rate (1%). Overall, the aggregate regression achieves the best in-distribution performance.

heading: Effects of Data Source and Paraphrasing
text: Method. Consistent with the statistical results in Section 7, the data source (human-written or machine-generated) has minimal effect on detection performance, as indicated by an AUROC score of 0.97 for both. In contrast, detecting contextaware paraphrases proves to be more challenging, achieving an AUROC score of 0.94 compared to 0.98 for context-agnostic ones. The detection performance (lexical regression) on text from different domains and LLMs is presented in Figure 4. Paraphrased spans in technological news (TLDR) or scientific writings (Sci_Gen) are comparatively challenging to identify, followed by Wikipedia articles (SQuAD). On the other hand, paraphrasing texts produced by encoder-decoder LLMs (FLANT5) or larger LLMs (OpenAI, LLaMA, and GLM) pose significantly greater difficulties.
Effect of Number of Paraphrases. The impact of the number of paraphrased sentences in a text is illustrated in Figure 5. As depicted, the detection difficulty decreases as the text span includes more paraphrased sentences, as indicated by both AUROC and accuracy. This could be attributed to the fact that paraphrasing a longer text span (i.e., with
more sentences) can showcase more pronounced paraphrasing styles, encompassing both lexical usage and syntax structure. Consequently, it exhibits a more discernible contrast with its surrounding context. In contrast, when only one sentence is paraphrased, especially if it is short, there are limited features available for detection.

heading: 8.2 Out-of-distribution Performance
text: The lower part of Table 1 presents the out-ofdistribution performance on the generalization testset, which consists of paraphrased texts generated by novel LLMs. Despite a degradation in performance across all methods and metrics, they still achieve reasonably good results in terms of both paraphrase identification (ARUOC) and paraphrasing degree prediction (correlation). Similar to the in-distribution results, the classification model demonstrates the highest detection performance with an accuracy of 47.21%. Differently, the classification and lexical regression model are substantially stronger than all other models for detecting paraphrases, with a much higher accuracy. In predicting the degree of paraphrasing, the regression models perform better, particularly the aggregate regression model which achieves lexical and syntactic correlations of 0.66 and 0.70 respectively. We construct the generalization testset using various paraphrasing prompts (Appendix D). Empirical results demonstrate that the detector is resistant to prompt variance, with an AUROC exceeding 0.92 across all prompts. The performance of lexical regression against prompt variance is shown in Appendix G.

heading: 9 Analysis
text: In this section, we set the decision boundary to maintain an FPR of 1% on the validation set to
accommodate various testsets across all analytic experiments and report detection accuracy.
Effect of Surrounding Context. As discussed in Section 7, paraphrased texts exhibit strong writing patterns different from the surrounding texts. We conduct an ablation study on the effect of the surrounding context, with results shown in Table 2. The models in the table are trained merely on the paraphrased sentences, without considering the surrounding context. As shown, both classification and regression (lexical) models suffer substantial performance degradation when missing the context information, with over 100% performance drop on detection accuracy. To gain further insight, we analyze perplexity variations across token positions around the boundary of the paraphrased text span. The results are presented in Figure 6, where we observe perplexity impulses at both boundaries of the paraphrased text span. Context-agnostic paraphrases typically exhibit higher token perplexity from the beginning of the paraphrase. While context-aware paraphrases display lower token perplexity before reaching the end of the paraphrase, they encounter a substantial increase in perplexity afterwards. This indicates that paraphrased sentences cannot perfectly integrate into the original text, even if context is considered during paraphrasing.

heading: Generalization to Multiple Paraphrased Text
text: Spans. Our training data only considers para-
phrasing one text span in a text. In many application scenarios, users can paraphrase multiple text spans. To this end, we construct an additional testset where multiple text spans are paraphrased within each text. The testset consists of 500 randomly sampled in-distribution test instances. For each text, we randomly choose 2 to 5 non-adjacent text spans which consist of 1 to 3 sentences, and paraphrase these text spans using ChatGPT. The results are shown in Table 3. Although all detection methods experience a performance decline in terms of AUROC, they achieve a comparable detection accuracy, demonstrating the generalization ability to text with multiple paraphrased text spans. We present a case study of PTD in Appendix F.
Robustness to Minor Text Modification. A desirable characteristic of a paraphrase detector, in addition to detection accuracy, is the ability to distinguish texts with minor modifications from true paraphrases. We consider two types of minor modifications: sentence reordering and word replacement. For sentence reordering, we utilize Dipper with maximum ordering control and minimum lexical control. For text replacement, we randomly mask 10% of the text and use T5-3B (Raffel et al., 2020) to fill in the masked blank. Data details can be referred to in Appendix H. We evaluate these modifications on the in-distribution testset by considering all minor-perturbed texts as "nonparaphrased". As shown in Figure 7, all methods rarely misidentify texts with word replacements as paraphrased, with nearly perfect prediction accuracy. In contrast, texts with sentential reordering pose greater confusion and yield considerably
lower accuracy. Notably, regression models incorporating grammatical or syntax information exhibit more resistance to such confusions compared to the classification model which performs worst.
Defending AI-generation Detection. Previous work (Krishna et al., 2023; Sadasivan et al., 2023) has shown that paraphrasing attacks significantly degrade AI-generation detection systems. We propose a two-stage detection method, which utilizes paraphrased text span detectors as a pre-defense mechanism to block out paraphrased texts and employs traditional AI-generation detectors if no paraphrasing is detected. We calculate the text score by averaging the predicted scores of all sentences and evaluate the model on the paraphrasing attack testset proposed by Li et al. (2023b). We use an off-the-shelf AI-generation detector† and employ the aggregate regression model for defense. The results are presented in Table 4, where it can be observed that most machine-generated texts were misclassified by the AI-generation detector due to paraphrasing, resulting in low MachineRec (recall on machine-generated texts). The paraphrasing indicator successfully identifies most of the paraphrased texts and significantly improves averaged recall (AvgRec) scores while maintaining high recall scores on human texts (HumanRec).

heading: 10 Conclusion
text: In this work, we propose a detection framework, paraphrased text span detection (PTD), which aims to identify text spans paraphrased by AI, from a long text. We built a dedicated dataset, PASTED, based on which we train classification and regression models for PTD. Both in-distribution and outof-distribution results demonstrate that our methods can effectively detect paraphrased text spans. Although classification models achieve better detection accuracy, they fall behind regression models in predicting the paraphrasing degree. Statistical and model analysis showcases the importance
†https://github.com/yafuly/MAGE
of the context surrounding the paraphrased text spans for detection performance. Extensive experiments demonstrate the generalization to paraphrasing prompt types and multiple paraphrased text spans.

heading: Limitations
text: Although we extensively experiment and analyze the implementation of our newly proposed task PTD using the newly established dataset PASTED, there are several limitations: (1) We consider both context-agnostic and context-aware paraphrasing using limited paraphrasers and prompts. Future work should focus on constructing more challenging paraphrases. (2) We implement effective detection methods based on Longformer, but more advanced backbones like LLaMA can be explored. (3) To simulate real-life applications, we randomly paraphrase text spans in existing datasets. Future work should aim to construct more realistic data by involving crowdsourcing.

heading: Ethical Considerations
text: We honor the Code of Ethics. No private data or non-public information is used in this work. We adhere to the terms of companies offering commercial LLM APIs and express our gratitude to all global collaborators for their assistance in utilizing these APIs.

heading: Acknowledgement
text: We would like to thank all reviewers for their insightful comments and suggestions to help improve the paper. This work has emanated from research conducted with the financial support of the National Natural Science Foundation of China Key Program under Grant Number 62336006.

heading: A Aligning Paraphrased Sentences
text: We show a case where paraphrasing text involves sentence reordering and many-to-one sentence alignment in Figure 9. The detailed implementation for aligning paraphrased sentences with their reference original sentences is presented in Algorithm 1. Our approach involves greedily traversing each paraphrased sentence and identifying a span of consecutive original sentences that share high semantic similarity. If no suitable span is found, we fallback to finding the most semantically similar original sentence. To determine a similarity threshold, we sample 1,000 instances where the number of sentences remains unchanged during paraphrasing, where most cases exhibit trivial one-on-one alignment upon manual inspection. We compute the semantic similarity between all pairs of paraphrases and original sentences. Pairs with matching indices (e.g, 0 v.s. 0) are considered matched while others are unmatched (e.g, 0 v.s. 1 and 2). The distribution of similarities is depicted in Figure 8, suggesting that 0.75 is a promising threshold value.

heading: B Data Statistics
text: The data statistics of PASTED dataset is presented in Table 5.

heading: C Data Samples
text: We present several data samples in Table 6 and Table 7.

heading: D Prompt Design
text: The prompt templates used for constructing the data set is shown in Figure 11.

heading: E Word Cloud of Paraphrases
text: The word cloud of word distribution for original texts, context-agnostic paraphrases, and contextaware paraphrases are shown in Figure 12.
Algorithm 1 Paraphrase Alignment Setup: n: number of paraphrased sentences
m: number of reference sentences avg(L, i, j): calculate mean value of L[i,i+1,...j-1]
Input: mat: Similarity matrix between paraphrased sentences and reference sentences, Rn×m
τ : threshold 1: {initializing} 2: A← ∅ 3: i← 0 4: {align each paraphrased sentence individually.} 5: while i < n do 6: {get the similarity of the most similar reference sentence} 7: Vmax ← mat[i].max() 8: if Vmax ≤ τ then 9: {if the max similarity is less than or equal to the
threshold, we just align the paraphrased sentence i with the most similar reference.}
10: idx← argmax(mat[i]) 11: A.add((i, (idx, idx+ 1))) 12: else 13: {If the maximum similarity exceeds the threshold,
we align paraphrased sentence i with the longest reference sentence span that has an average similarity greater than the threshold.}
14: Wsize ← m 15: Flag ← 0 16: while Wsize ≥ 1 do 17: j ← 0 18: while j ≤ m−Wsize do 19: Vmean ← avg(mat[i], j, j +Wsize) 20: if Vmean > τ then 21: A.add((i, (j, j +Wsize))) 22: Flag ← 1 23: break 24: end if 25: j ← j + 1 26: end while 27: if Flag = 1 then 28: break 29: end if 30: Wsize ←Wsize − 1 31: end while 32: end if 33: i← i+ 1 34: end while 35: return A

heading: F Case Study
text: We present several cases of paraphrased text span detection (aggregate regression model) in Figure 13, Figure 14 and Figure 15.

heading: G Effect of Prompts
text: The effect of prompts for constructing paraphrases is shown in Figure 10, where “CG” and “CW” are abbreviations for “context-agnostic” and “contextaware” respectively.

heading: H Constructing Texts with Minor
text: Modification
For sentence reordering, we set the lexical control of Dipper to the lowest value (i.e., 20) and the order control value to the max (i.e., 100)‡. For word replacement, we randomly mask 10% of the tokens and use T5-3B model to fill in these blanks. We apply both methods on the in-distribution testset. To ensure minimal modifications, we calculate a similarity score between the original and para-
‡Please refer to https://github.com/martiansideofthemoon/aidetection-paraphrases for details.
phrased text. We then establish a threshold to exclude paraphrases that significantly differ from the original texts. We only retain paraphrases with BLEU scores above 70 when compared with their respective original texts. In this way, we construct 273 and 1,310 instances for sentence reordering and word replacement, respectively. We show several cases in Table 8.

heading: Context-aware

heading: 2. Contextual Coherence: The paraphrase should maintain coherence and consistency with the overall context of the larger text from which the excerpt is derived.

heading: 1. Fidelity to Original Meaning: The paraphrased text must preserve the complete meaning of the original excerpt without deviation.

heading: Prompt Type OOD Testsets

heading: Prompt Type Train/Dev/Test Prompt Type OOD Testsets

heading: Raw Text

heading: Paraphrased Text

heading: Raw Text

heading: Paraphrased Text

heading: Raw Text

heading: Paraphrased Text

heading: Future Work
text: Future work should focus on constructing more challenging paraphrases. (2) We implement effective detection methods based on Longformer, but more advanced backbones like LLaMA can be explored. (3) To simulate real-life applications, we randomly paraphrase text spans in existing datasets. Future work should aim to construct more realistic data by involving crowdsourcing