AI-generated text boundary detection with RoFT

Authors: laida  kushnareva
tatiana  gaintseva
german  magai
serguei  barannikov
dmitry  abulkhanov
kristian  kuznetsov
eduard  tulchinskii
irina  piontkovskaya
sergey  nikolenko
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-21 10:26:30

Abstract: Due to the rapid development of large language models, people increasingly often encounter texts that may start as written by a human but continue as machine-generated. Detecting the boundary between human-written and machine-generated parts of such texts is a challenging problem that has not received much attention in literature. We attempt to bridge this gap and examine several ways to adapt state of the art artificial text detection classifiers to the boundary detection setting. We push all detectors to their limits, using the Real or Fake text benchmark that contains short texts on several topics and includes generations of various language models. We use this diversity to deeply examine the robustness of all detectors in cross-domain and cross-model settings to provide baselines and insights for future research. In particular, we find that perplexity-based approaches to boundary detection tend to be more robust to peculiarities of domain-specific data than supervised fine-tuning of the RoBERTa model; we also find which features of the text confuse boundary detection algorithms and negatively influence their performance in cross-domain settings.

Main paper: 
 introduction: 
 Artificial text detection (ATD) is a very difficult problem in real life, where machinegenerated text may be intertwined with human-written text, lightly edited, or pad out human-generated prompts. However, in literature ATD is usually formulated in a simpler way, with a dataset of text samples labeled as either entirely human-written or entirely machine-written, so the detection problem can be safely treated as binary classification. Moreover, the models for this binary classification are often developed and trained to detect a particular type of generator, e.g., text produced by a specific large language model (LLM) (Uchendu et al. (2023) ). This is in stark contrast with how we may encounter artificially created text in real life, where documents partially written by humans and partially generated by LLMs already abound. This setting is much more complex and much less researched.In this work, we experiment with a lesser known dataset called RoFT (Real Or Fake Text), collected by Dugan et al. (2023) . Each text in this dataset consists of ten sentences, where the first several sentences are human-written and the rest are machine-generated starting from this prompt, mainly by models from the GPT family (Radford et al., 2019; Brown et al., 2020) . We consider several techniques developed for binary ATD, modifying them for this more complex boundary detection setting; e.g., following Tulchinskii et al. (2023a) we adapt intrinsic dimension estimation which is currently considered to be the most robust methodThe girl sat, eyes wet. "Get up. Don't give up now," her friend said, placing a comforting hand on her shoulder. The girl shook her head, tears streaming down her cheeks. "I can't do it. I'm just not strong enough." "You are," her friend insisted."Look at all the obstacles you've overcome already. You're stronger than you think." The girl took a deep breath and wiped her tears.The girl sat, eyes wet. "Get up. Don't give up now," her friend said, placing a comforting hand on her shoulder. The girl shook her head, tears streaming down her cheeks. "I can't do it. I'm just not strong enough." "You are," her friend insisted."Look at all the obstacles you've overcome already. You're stronger than you think." The girl took a deep breath and wiped her tears.The girl sat, eyes wet. "Get up. Don't give up now," her friend said, placing a comforting hand on her shoulder. The girl shook her head, tears streaming down her cheeks. "I can't do it. I'm just not strong enough." "You are," her friend insisted."Look at all the obstacles you've overcome already. You're stronger than you think." The girl took a deep breath and wiped her tears. shows the true bondary between human-written and machine-generated text. Other three parts are colored according to the features, extracted by various classifiers. Here, gpt3.5 predicts that only two last sentences are generated; TLE + TS binary predicts that generation starts from the second sentence; phi1.5 perplexity predicts that generation starts from the third sentence which is correct answer. See Section 3 for detailed explanation of all the methods and Appendix G for more examples from the dataset.for cross-domain and cross-model binary detection. Fig. 1 shows a sample input from RoFT and results of the considered models. Our primary contributions are as follows:(1) we evaluate five approaches to adapt perplexity-based detectors to the task of detecting the boundary between human-written and machine-generated text, discussing the differences in their behavior compared to binary ATD and providing a comprehensive analysis of how perplexity scores react to the machine-human transition in the text; we also compare several backbones for the perplexity evaluation to show which model works the best for this purpose;(2) we present evidence that perplexity scores obtained from small language model, trained on data generated by bigger LLM, provide strong features for boundary detection models, leading to relatively high accuracy of detection and cross-domain robustness related to other detection methods;(3) we introduce and evaluate two ways to adapt the classifiers based on intrinsic dimension estimation for this task, showing how the time series analysis can be used for extracting useful information from Transformer representations; (4) we show how the robustness of boundary detectors (including the fully-tuned RoBERTa baseline) to domain shift depends on the particular properties of the domains; we study the properties of the dataset itself and their effect on the performance of every approach; (5) we enrich the RoFT dataset with GPT-3.5-turbo1 (ChatGPT) generation samples; we share this new dataset with the community, establish baselines, and analyze the behavior of our detectors on it. 2We hope that this work will encourage further research both in boundary detection for texts that are partially human-written and partially generated and analyzing how inner representations of Transformer-based models react to such transitions; the latter direction may also help interpretability research. The rest of the paper is organized as follows. In Section 2 we survey related work, and Section 3 introduces the methods we have applied for artificial text boundary detection. Section 4 presents a comprehensive evaluation study on the RoFT and RoFT-chatgpt datasets. Section 5 presents a detailed discussion and analysis of our experimental results, and Section 6 concludes the paper. 
related work: 
 Artificial text detection (ATD) is a well-studied task with plenty of already considered approaches; see, e.g., the surveys by Yang et al. (2023) ; Uchendu et al. (2023) ; Wu et al. (2023a) . In our work, we mostly concentrate on adapting two groups of methods to artificial text boundary detection: methods based on perplexity estimation since they are the most widely used and methods based on topological data analysis (TDA) since they have shown promising robustness to domain shift and model shift (see below).Perplexity-based methods are based on the idea that human-written text tends to be more unpredictable and varied compared to AI-generated text which may follow more predictable patterns, i.e., has a lower perplexity score; among them we note GPTZero (Weber-Wulff et al., 2023) , Sniffer (Li et al., 2023a) , SeqXGPT (Wang et al., 2023) , and LLMDet (Wu et al., 2023b) . Topological data analysis (TDA) for ATD is inspired by the results of Kushnareva et al. (2021) and Tulchinskii et al. (2023a) . The latter has shown that for many artificial text generators the intrinsic dimensions (PHD) of RoBERTa embeddings of the texts created by these generators are typically smaller than those of the texts written by humans; also TDA has proven to be useful for closely related tasks of fake news detection (Tudoreanu) , authorship attribution (Elyasi & Moghadam, 2019) , and detection of synthetic speech (Tulchinskii et al., 2023b) .Style change detection and authorship attribution. ATD can be considered as a special case of authorship attribution (AA), where the LLM is one author and the human is another, and the task is to determine the authorship of different parts of the text; it is also similar to the style change detection problem in multi-author documents (Zangerle et al., 2021) . Jones et al. (2022) show that LLM models can successfully imitate human style and deceive existing popular online AA methods. However, Venkatraman et al. (2023) propose an approach based on the principle of uniform information density that can detect the authorship of LLM. State of the art style change detectors (Lin et al., 2022; Jiang et al., 2022; Lao et al., 2022; Iyer & Vosoughi, 2020) are based on Transformer-based encoders such as BERT Devlin et al. (2019) , RoBERTa Liu et al. (2019) , AlBERT Lan et al. (2020), and ELECTRA Clark et al. (2020) . Therefore, we use RoBERTa model as a baseline and a source of embeddings.RoFT (Real Or Fake Text). Our main dataset (Dugan et al., 2020; 2023) originates from a website called RoFT3 developed as a tool to analyze how humans detect generated text, including an extended study of whether and how they can explain their choice, when they say that a text sample is machine-generated, and how humans can learn to recognize machine-generated text better. Every user of this website can choose a topic ("Short Stories", "Recipes", "New York Times", or "Presidential Speeches") and start the following "game". The player sees ten sentences one by one. The first sentence is always written by a human, but for each subsequent sentence the player must determine whether it is machine-generated or human-written. If the player believes the text was AI-generated, they should explain why they think so. If they guess machine generation before the true boundary, they earn zero points. Otherwise, they earn 5x points, where x is the number of sentences after the correct boundary. This is actually a harder problem than just boundary detection since the player does not see the full text, and the scoring function is skewed. As a result, Dugan et al. (2023) created a dataset also known as RoFT, where every sample consists of ten sentences, starting with a human-written prompt and continuing by some language model; the data shows the true boundary, human prediction of it, the generator model, an explanation provided by the player, and information about the player. The original RoFT contains generations from GPT-2 (Radford et al., 2019) , GPT-2 XL, GPT-2 finetuned on the "Recipes" domain, GPT-3.5 (davinci), CTRL (Keskar et al., 2019) with control code "nocode" and with control code "Politics", and baseline, where instead of an LLM-generated continuation the passage transitions to a completely different news article selected at random. RoFT has been mostly used to investigate how humans detect artificially generated texts manually (Clark et al., 2021) ; Cutler et al. (2021) provided the first baselines on automatically solving RoFT, comparing several shallow classification and regression models based on RoBERTa and SRoBERTa (Reimers & Gurevych, 2019) embeddings collected from the last layer of these models. However, the cross-domain and cross-model settings in their research were very limited. In fact, Cutler et al. (2021)  
approach: 
 We consider several different approaches, including a multilabel classification framework, as proposed by Cutler et al. (2021) , where the label of a text corresponds to the number of the first generated sentence, time series analysis that slides a window over the text tokens, and regression methods that minimize the MSE between true and predicted boundaries. We design our classifiers based on features that have been successfully used in prior works on ATD (Solaiman et al., 2019; Mitchell et al., 2023) . We also introduce a new baseline based on sentence lengths. Below, we consider all of these approaches in detail.RoBERTa classifier. Unlike Cutler et al. (2021) who process each sentence separately, we fine-tune the RoBERTa model (Liu et al., 2019) to represent the entire text sample via the [CLS] vector. This is the only case in our work where we apply full model fine-tuning.For other methods, we use simpler classifiers such as logistic regression (LR) or gradient boosting (GB) (Friedman, 2001) trained on various features extracted directly from larger models (e.g., Transformer-based LLMs), with no updates to the larger model's weights.Perplexity from generative LLMs. In our perplexity experiments, we implement the black-box approach, which means that a single model is used for all data to compute sentencewise perplexity. This is more practical than the method of Cutler et al. (2021) who used perplexity scores from the original generator model, which may be infeasible if the generator is unknown, in particular in cross-model scenarios. Even for known generators, the exact version of the model is usually unknown if they are available via API, which harms the performance of white-box detectors. To derive perplexity features, we calculate the log likelihood of each token via LLM, and derive sentence-level mean and std values. As the perplexity estimator, we use several models such as GPT-2 (Radford et al., 2019) , Phi-1 (Gunasekar et al., 2023) , Phi-1.5 (Li et al., 2023b ), Phi-2 (Abdin et al., 2023) , and LLaMA-2-7B (Touvron et al., 2023) . Our underlying hypothesis here is that texts generated by models of similar architecture such as GPT-2 or GPT-3.5 might appear more "natural" to these models, as reflected by their likelihood scores. Our findings corroborate this assumption (see Appendix F). On top of sentence-wise perplexity features, we train either a classifier (logistic regression and gradient boosting) or a regressor that predicts boundary values (GB regressor); the regression setting takes advantage of the sequential nature of the task, minimizing the discrepancy in labels.DetectGPT. The DetectGPT framework (Mitchell et al., 2023) proposes a more nuanced perplexity-based scoring function. It involves perturbing a text passage and comparing log-probabilities between original and altered texts. This score then serves as an input to a classification model, with GPT-2 as the base model and T5-Large (Raffel et al., 2020) generating the perturbations.Length-Based Baseline. Since we have observed a statistical difference in sentence length distributions between human-written and generated texts (see Fig. 2 ), we leverage sentence lengths as a simple baseline feature. This baseline allows us to gauge the effectiveness of a classifier in identifying boundaries without semantic understanding.Topological Time Series (TTS). Inspired by Tulchinskii et al. (2023a) , we explore the potential of topological features based on intrinsic dimensionality (ID); we provide an introduction to topological data analysis (TDA), including the definitions of features, in Appendix A. We hypothesize that geometric variations in token sequences can help identify AI-generated text, so we introduce models that process TDA-based features treating them as time series.For every text, we slide a window of H = 100 tokens (step size S = 1) over RoBERTa token embeddings and find the intrinsic dimension (PHD) of the points within the window, as shown in Fig. 4 (Schweinhart, 2020) . Window size 100 was chosen as a minimal size where PHD estimator stayed stable. The time series are then classified with a multi-label SVM with the global alignment kernel (GAK) (Cuturi, 2011) , where label correspond to a number of the first generated sentence; this method is called "PHD + TS ML" in the tables.In this approach, we train a binary classifier to distinguish between fake and natural text atop a specific predictor. For the base predictor, we employ intrinsic dimensionality calculated over a sliding window of H = 20 tokens (step size S = 5)6 . The TLE (tight local) intrinsic dimension estimator was chosen due to its robust performance on small data samples; window size 20 was taken directly from Amsaleg et al. ( 2019), as the authors claim that their algorithm is stable enough "for 'tight' localities consisting of as few as 20 sample points". For the base classifier, we use gradient boosting trees (Friedman, 2000; 2002) . To translate probabilities predicted with a binary classifier into a specific boundary that separates real and fake text, we determine the final label by maximizing y → arg max I∈I s I (y, x), where the score function s I is defined as s I (y, x) = -∑ n j I j y log p(x j ), and p(x) are predictions from the base binary classifier. In this context, we use a binary indicator set I ∈ 1 n consisting of a chunk of k zeros followed by another chunk of nk ones. This method is called "TLE + TS Binary" in the tables.Zero-shot detection. As an additional baseline, we tried zero-shot boundary detection with two strongest black-box models: gpt-3.5-turbo and gpt-4. We describe the problem to them with a natural language prompt which is provided in Appendix B. 
experimental evaluation: 
 Dataset preparation and analysis. In all experiments, the task is to detect the exact boundary where a text passage that started as human-written transits to machine generation. In addition to RoFT (Dugan et al., 2020) , we created a new dataset called RoFT-chatgpt, where the same human prompts are continued with the gpt-3.5-turbo model. RoFT-chatgpt is designed to be more challenging for artificial text boundary detection while preserving the basic statistical properties of RoFT such as the label distribution. We removed duplicates that were different only in human-predicted labels; for RoFT-chatgpt, we also removed samples containing "As an AI language model..." and short failed generations. As a result, we retained 8943 samples from the original RoFT and 6940 samples in RoFT-chatgpt. Note that distributions of sentence lengths vary significantly across subdomains, as shown in Fig. 2 and Figs. 5, 6 in Appendix E (see also a discussion in Section 5).Artificial text boundary detection. Table 1 presents the main results of our experiments on the RoFT and RoFT-chatgpt datasets; the majority class prediction is the last (9th) class of fully human-written texts. On the original RoFT, we also include the human baseline and the best result reported by Cutler et al. (2021) , obtained by a classifier built upon concatenated SRoBERTa embeddings of each sentence. We report the accuracy (Acc) and two metrics that account for the sequential nature of boundary detection: soft accuracy, i.e., the percentage of predictions that differ from the correct label by at most one (SoftAcc1; indeed, Fig. 18 Table 1: Boundary detection results. Bold shows the best method, underlined -second best, italic for values outperforming the human baseline. in Appendix F shows that many misclassifications point to an adjacent class, which may be acceptable in real world applications), and mean squared error (MSE). More precisely,So f tAcc1 = 1 n ∑ n i=1 1 {|y i -ŷi |≤1} and MSE = 1 n ∑ n i=1 (y i -ŷi ) 2, where y i -predictions of the algorithm, ŷi -true labels, n -amount of the examples.First, on the original RoFT RoBERTa-based classifiers outperform others by a large margin (14% accuracy) and also significantly outperform the previously best reported SRoBERTa (Cutler et al., 2021) . This model also provides the lowest MSE (0.03) among all methods. We note, however, that our RoBERTa classifier has significantly more trainable parameters than any other method in the table because no other approaches require LM fine-tuning. Second, topological and perplexity features improve over the human baseline. Perplexitybased classifiers are the best in terms of accuracy, while the perplexity regressor provides good MSE values (recall from Section 2, however, that humans were solving a harder problem with a somewhat different objective). Third, RoBERTa's accuracy on RoFT-chatgpt drops by 6% compared to RoFT, while soft accuracy and MSE are roughly the same, but the opposite holds for perplexity-based methods: on RoFT-chatgpt their results are much better. The reason for this might be that we used GPT-like models for perplexity calculation and GPT-3.5-turbo to generate fake samples in RoFT-chatgpt. However, we used smaller models to detect text generated by a larger model (GPT-3.5) and got second-best results among other approaches, despite Mitchell et al. (2023) reporting that smaller models are not capable of detecting text generated by larger models. Moreover, when analyzing the performance of different perplexity estimators, we do not observe a clear correlation of the performance with the size of the estimator. The largest LLaMA model with 7B parameters works much worse than Phi-1.5 (1.3B) and Phi-2 (2.7B) on both datasets and slightly worse than GPT-2 (1.5B) on RoFT. For RoFT-chatgpt, the best perplexity estimator is a relatively small Phi-1.5 model, outperforming a GPT-2 estimator of comparable size by 15%. This impressive difference might be related to training data: Phi-1.5 was trained on GPT-3.5 generations. This suggests that small language models trained on data generated by LLMs, may appear a good perplexity-based detector of text generated by those particular LLMs. The baseline length-based classifier also improves its accuracy significantly (by 1.8x) when transferring to RoFT-chatgpt. We hypothesize that this kind of shallow feature emerges in ChatGPT generation and makes the task easier (see also Section 5). The other perplexitybased approach, DetectGPT, shows lower accuracy on both datasets. Mitchell et al. (2023) note that DetectGPT can detect whether the sample is generated by a specific base model, but we use several models in our setup, and text samples may be too short for this approach.On the other hand, DetectGPT has quite good MSE values, close to the regression approach that optimizes MSE directly.Cross-domain generalization. Supervised ATD methods with fine-tuning such as RoBERTa are more sensitive to spurious correlations in a dataset and often demonstrate poor crossdomain transfer, especially compared to topology-based approaches (Tulchinskii et al., 2023a) . Table 2 reports the results of cross-domain transfer between four text topics in the RoFT-chatgpt dataset. We report in-domain (IN) accuracy on domains seen during training and out-of-domain (OUT) accuracy on the unseen domain corresponding to this column; MSE scores are given in Table 5 (Appendix C). Each model was trained on three domains and tested on the fourth, unseen domain; we used 60% of these subsets mixed together as the training set, 20% as the validation set, and 20% as the test set for in-domain evaluation.Table 2 shows that RoBERTa's performance drops for all subsets very significantly, while perplexity-based classifiers demonstrate excellent cross-domain generalization for Presidential Speeches and Short Stories; for the Recipes domain, TTS classifiers prove to be the most stable, and the New York Times subset yields a large generalization gap for all classifiers. This means that every type of classifier can handle its own set of spurious features well, and no classifier is universally better than the others. Aggregation of different features can improve the results; for example, a classifier that returns the maximum of RoBERTa and perplexity-based predictions yields OOD scores on average 3% higher than the best result in Table 2 with comparable ID performance. As for classification accuracy, surprisingly, the perplexity-based classifier outperforms a fully fine-tuned RoBERTa on all subsets. Moreover, for Recipes the multilabel topological time series method places second; this happens because TDA-based methods are extremely stable under domain shift (despite being worse than others in absolute values on this dataset); see Section 5 for further discussion. In general, we conclude that perplexity-based classifiers outperform RoBERTa-based in cross-domain settings by a large margin. This is remarkable since the former is a simple classifier trained on ten features extracted by a LM with frozen weights, while the latter involves full LM fine-tuning. As for the length-based baseline, for in-domain data average sentence length provides a strong signal, leading to accuracy of 20-32% and even outperforming topological methods. But cross-domain generalization fails, which means that we should prefer classifiers that ignore this feature to achieve good generalization (see also Section 5).Cross-model generalization. We tune our classifiers on generation results produced by one model and test the performance for all other models. In general, this task is harder for all considered classifiers: there are models for which prediction accuracy drops down to virtually zero values (detailed experimental results are shown in Appendix D: Tables 3, 6, 7 ). But we observe an interesting result for the perplexity-based classifier: it achieves good generalization when transferring to very large models such as GPT3-davinci and GPT2-XL, while for other models the generalization is poor. In general, the strongest classifiers fail to generalize to simpler models; e.g., RoBERTa-based classifiers underperform on the generalization to GPT-2 and baseline; the baseline is also the hardest subset for perplexity-based classifiers (see Section 5 for a discussion). 
discussion and analysis of the results: 
 Detection models. We have found that although RoBERTa-based classifiers demonstrate excellent results for in-domain classification, they lose to perplexity-based methods when tested on texts with new styles and topics not present in the training set. For cross-model generalization, all methods demonstrate insufficient generalization abilities, especially when tested on generations of much larger or much smaller models. This suggests that AI content detectors can be fooled by either too good or too bad artificial texts. For perplexity-based classifiers, we have found that the size of the base model is not the main requirement for a good and robust detector. The training dataset for perplexity estimation is more important; e.g., ChatGPT generations were best detected by a much smaller model pretrained on synthetic data generated by the same model family. The above observations suggest that pre-training of the perplexity estimators on a large amount of synthetic data from different generators may become the best choice for AI-generated content detection. We leave it for future research. Topological classifiers demonstrated high robustness over domain transfer, but not very high overall accuracy. Geometric properties of the embeddings for both RoFT and RoFT-chatgpt, however, show the difference between PHD distributions of real and fake RoFT text for different models (Fig. 10 ) and topics (Figs. 11, 12 in Appendix E). TLE dimension distributions for sentences in human-written and AI-generated parts of the texts are different as well (Figs. 13, 14, 15, Appendix E) . This means that topological classifiers may help in robust artificial content detection in some particular domains and models.Data properties. Next, we summarize the observed properties of the data that influence detection quality and create difficulties for different types of artificial content detectors.1. First, the length of sentences seems to play an important role, deceiving our classifiers.There is indeed a significant difference between distributions of sentence lengths written by humans and generated by LLMs (see Fig. 2 and Figs. 5, 6 in Appendix E). This is supported by our experiments with the length-based classifier, which sometimes outperforms other methods in terms of accuracy (Tables 1 and 2 ), but fails to generalize.2. Label distributions (see Figs. 8 and 9 in Appendix E) vary significantly across models. GPT-2 is especially different from natural texts, and the behavior of our models in crossmodel transfer to GPT-2 (Appendix D), suggests that the only model stable under the label distribution shift is perplexity-based regression. Although its accuracy numbers are low, it outperforms the human level in terms of MSE even in out-of-domain evaluation.3. Special text structure can hurt ATD: e.g., recipes often contain numbered lists, and the first sentence tends to be comparatively long, and as a result the Recipes topic is hard for all classifiers to generalize to. The RoBERTa classifier loses the most here, so we investigated its confusion matrices; Fig. 3 shows that RoBERTa tends to classify all examples in the Recipes topic as fully generated. This kind of text structure can be viewed as an adversarial example for AI-generated texts, and indeed, ChatGPT is known for its tendency to generate structured output with bullet lists.4. Semantic and grammar inconsistencies. Surprisingly, cross-model generalization of RoBERTa-based and perplexity-based classifiers fails not only for the best OOD genera-tor (davinci), but also for the worst generators: small-size GPT-2 and baseline. Perplexity distributions for texts generated by baseline or GPT-2 are very different from the distributions of texts generated by other models (see Fig. 16 ). This might explain the poor performance of decision trees and linear classifiers when these generators are excluded from the training set. On the other hand, both generators are among the easiest for human raters, because of the high level of semantic inconsistencies and grammar issues. It seems that the supervised classifier discovers non-regularities in the text as a feature of human-generated texts; indeed, they present in our training set as a part of first-person narratives, especially in the Short Stories dataset.5. Discourse structure. Finally, analysing the poor generalization to narrative domains, we discover a subtle property of human-generated texts that influences boundary detection, namely the underlying discourse structure of the story that can be measured as peaks of perplexity along the story (Fig. 17 , Appendix E). Human-generated texts tend to have 2-3 such peaks, e.g. a peak in the middle of the story corresponding to a plot twist or change of narrative focus. This leads to the difficulties in perplexity-based boundary detection: the perplexity drop in the beginning of the fake text part is confused with the same drop due to the story discourse change. Strong perplexity-based classifiers can handle this issue due to the std feature reflecting token-level perplexity deviation (Fig. 18 , Appendix E). Fig. 2b shows that the RoBERTa-based classifier is prone to this discourse-related issue: it often incorrectly predicts the real-fake boundary in the middle of a human-generated story (5-6th sentence).A full analysis of feature distributions across domains and models is given in the Appendix. 
conclusion: 
 In this work, we address the task of boundary detection between human-written and generated parts in texts that combine both. We believe that this setting is increasingly important in real world applications and is a natural setting for recognizing artificial text since it is often mixed with and prompted by human text. We have considered the RoFT dataset, presented its modification RoFT-chatgpt generated with a more modern LLM, and investigated the performance of features that were useful for artificial text detection in previous works. In particular, we have shown that LLM fine-tuning works reasonably well for this task but tends to overfit to spurious features in the data, which leads to generalization failures in some settings. On the other hand, perplexity-based and topological features provide a signal that can help in these situations. We have demonstrated that perplexity features are the best overall on balance between accuracy, generalization, and training complexity.We analyze the base models for perplexity estimation and conclude, that reasonably small models work well, but pre-training data is important. Including synthetic data in the pretraining dataset improves the performance of the detector. As for the limitations of our approach, boundary detection is very hard in cross-domain and cross-model settings, both for short texts such as RoFT (due to lack of information) and longer texts (due to a large space of possibilities). Therefore, it is no wonder that none of the methods have achieved a really high quality in this setting, and the results suggest a large room for improvement.Besides, all methods we considered were based on Transformers with relatively small context window size, which limits the transferability of the proposed approaches onto longer text samples. Finally, our analysis has uncovered gaps in current approaches and discovered the more difficult aspects of the task, which we plan to address in future research. 
 
 Appendix: According to the manifold hypothesis (Narayanan & Mitter, 2010) , the data X lies on a low-dimensional submanifold: X ⊆ M n ⊆ R d , where d is the extrinsic dimension and n is the intrinsic dimension (ID). The geometric and topological properties of the manifold M are of particular interest. There are various methods for estimating the ID that can be divided into global and local methods.For the tight local intrinsic dimension estimator (TLE) proposed by Amsaleg et al. (2019) , we use the neighborhood center point x, a set of neighborhood samples V and a specially defined distance between points in a sufficiently small neighborhood of x:EQUATION)where r is the radius of the neighbourhood. For every three points x, v, w we can computeM(x, v, w) = ln d x,r (v, w) r + ln d x,r (2x -v, w) r .If V * = V ∪ {x}, then the intrinsic dimension can be found by averaging the estimates for all points x, as defined by the following formula:mr (x) = - 1 |V * | 2 ∑ v,w∈V * , v̸ =w M(x, v, w) -1 (2)Applied algebraic topology provides effective tools for analyzing the topological structure of data. The theoretical foundations of topological data analysis (TDA) have been described in detail by, e.g., Barannikov (1994) and Carlsson (2020) . TDA allows us to consider a dataset X ⊆ R d from the topological point of view. In order to move from point clouds X to topological spaces, it is necessary to approximate the data by a simplicial complex R. In our research, we use the Vietoris-Rips complex R(X; t). The method of constructing the complex R is as follows: simplexes are formed by subsets of points from X whose pairwise distances do not exceed t (a scaling parameter). An increasing sequence of simplicial complexes is called a filtration:{R t } (t≥0) = R t 0 ⊆ R t 1 ⊆ ... ⊆ R s .Homology groups H i (R) are a topological invariant that expresses the properties of a topological space R. We use β i (R) = dim H i (R), which is known as the ith Betti number, a topological feature equal to the dimension of the homology group; for i = 0, 1, 2 the Betti number corresponds to the number of connectivity components, cycles, and cavities respectively.Topological features appear and disappear at different values of t, which leads to the next core concept in TDA: the barcode. It summarises the dynamics of topological features in the filtration process. A bar is the lifetime of the nth homology feature I n = t birth n t death n . A long bar in the barcode means that the data contains a fairly persistent and informative topological feature. Schweinhart (2020) introduced the persistent homological fractal dimension (PHD) that generalized Steele (1988) for higher dimensions of the homology group and used the topological properties of the point cloud. The PHD has already been proven to be useful in the study of the properties of deep learning models (Birdal et al., 2021; Magai, 2023) .Let us denote the power-weighted sum of N bars for the ith degree of homology as follows:EQUATIONIt is interesting to note that E 0 1 (X n ) is equal to the length of the Euclidean minimum spanning tree (MST) of X n ⊆ R d (Skraba et al., 2017) .Then the persistent homological fractal dimension (PHD) can be defined as follows:EQUATIONwhereEQUATIONand x 1 , . . . , x n are sampled independently from X. That is, PHD i (X) = d if E i α (x 1 , ..., x n ) scales as n d-α d and α ≥ 0 (we take i = 0, α = 1). Persistent homological fractal dimension can be estimated by analyzing the asymptotical behavior at n → ∞ of E i α (x 1 ...x n ) for every i. In other words, to calculate PHD we must find a power law that shows how E i α (x 1 ...x n ) scales as n increases. See Adams et al. (2020) and Schweinhart (2020) for more details. B gpt-3.5-turbo and gpt-4 zero-shot experiment detailsBelow we show the prompt used for gpt-3.5-turbo and gpt-4 zero-shot experiments.You will be provided with a text that contains human-written sentences. Your task is to analyze the text and determine if any of the sentences were generated by an AI system. You should output a single number between 0 and 9 to indicate where the first AIgenerated sentence occurs in the text ( if any): For your output, you should provide a single number between 0 and 9, with the following meaning: 0 -The entire text was generated by an AI system. 1 to 8 -The first AI-generated sentence occurs at that position in the text (1 for the first sentence, 2 for the second sentence, and so on). 9 -The entire text was written by a human with no AI-generated sentences. Your output should consist solely of this single digit number, with no additional text or explanation: {content}Tables 5 and 4 supplement Table 2 from the main text, reporting the results of cross-domain transfer for our methods on the RoFT-chatgpt dataset across four text topics present in the data. We report in-domain and out-of-domain accuracy: the IN column shows results from domains seen during training, while the OUT column reflects the model's ability to detect artificial texts in the unseen domain corresponding to this column. For each model, training was done on three out of the four domains, and the resulting model was tested on the fourth, unseen domain; we used 60% of these subsets, mixed together, as the training set, 20% as the validation set, and 20% as the test set for in-domain evaluation. Tables 6 and 7 show our experimental results on cross-model transfer for all considered text generation models. The artificial text boundary detection models were trained on the parts of the dataset generated by all language models except one, which is held out for cross-model testing, and tested in-domain (ID) on the same parts and out-of-domain (OOD) on the remaining part generated by the held-out model.In this section we provide additional statistics and visualizations for the distributions of various features in the data. In particular, we note that on most diagrams, real texts have smaller PHD than fake texts, which is a very different result from the statistics presented by Tulchinskii et al. (2023a) , who noted that the PHD of real texts is larger than that of fake texts. We hypothesize that it can be due either to very short lengths of texts in our work compared to the texts considered by Tulchinskii et al. (2023a) or due to differences in the sampling strategy used by Dugan et al. (2020) and Tulchinskii et al. (2023a) when generating texts. Another observation is that the TLE dimension is very different for all generator models in the original RoFT dataset. This may be the reason for the bad generalization performance of intrinsic dimension-based algorithms across domains. For RoFT-chatgpt PHD and TLE, real and fake texts are close to each other.Table 7 : Original RoFT, cross-model transfer, part 2. The models were trained on all parts of the dataset except one and tested in-domain (ID) on the same parts and out-of-domain (OOD) on the remaining part. the fact that for the words of the first sentences the length of the text prefix is not enough for a stable calculation of perplexity. One solution to mitigate this effect and hence make perplexity-based classifiers more stable might be to generate new prefixes for the text using some generative model (e.g. gpt-3.5) and calculate perplexities of original text words using this generated prefix. We leave this idea for further research. Figure 18 visualizes the coefficients of a logistic regression model trained on sentence perplexities from the RoFT-chatgpt dataset (Perplexity + LogRegr rows in the tables). We can see a distinct pattern in this figure. For the label k, which means that the first fake sentence in the text is the (k + 1)st, the highest value of the coefficient are kth and k 10 th ones, and the lowest ones are often (k + 2)nd and (k + 12)th. This could mean that the model is "searching" for a sudden drop of mean and variance of perplexity at a point where the fake part is starting. This fits together well with the idea that language model (Phi-1.5) trained on data from another model (GPT-3.5 version) see text generated by this model as a more "natural" one than real human-produced text. Therefore, perplexity often drops at the point where fake text begins, and logistic regression can pick up this effect and use it as a decision rule.Finally, Figure 18b visualizes the confusion matrix on the test set of a logistic regression trained on the RoFT-ChatGPT dataset. We see that most of the errors concentrate around true labels, indicating that often model almost correctly finds a boundary, having a shift by +1/-1 label only. We provided more examples of correctly and incorrectly classified texts at the Figures 19 and 20 .Figure 19 : Correctly classified sample input from the ROFT-chatgpt dataset colored according to the perplexities of Phi-1.5 on each sentence; "greener" text corresponds to higher mean perplexity, "blue-er", to higher standard deviation of the perplexity, red text shows where both mean and std are low. The prompt is above the dotted line, the rest is generated.Figure 20 : Misclassified inputs from the ROFT-chatgpt dataset colored according to the perplexities of Phi-1.5 on each sentence. At the first picture, generated text starts from the fourth sentence, but perplexity-based classifier predicts that it starts from the fifth sentence; at the second picture, generated text starts from the second sentence, but perplexity-based classifier predicts that it starts from the fifth sentence.https://platform.openai.com/docs/model-index-for-researchersOur official repository: https://github.com/SilverSolver/ai boundary detection.githttp://www.roft.io/https://aclanthology.org/venues/semeval/https://github.com/mbzuai-nlp/SemEval2024-task8Stride size 5 was chosen empirically after the preliminary experiments. In these preliminary experiments we found out that for smaller stride sizes the series became too long for the Time Series SVM algorithm, and it wasn't able to converge. 


Decision: This paper examine which methods are most effective for solving the boundary detection problem---that is, identifying when text transitions from machine-generated to human-written. The reviewers emphasize that this is a relatively new and important problem for which there is little prior work. They also note that the paper's analysis is thorough and well-written.

Reviews: review: paper_summary: paper_summary: The paper examines a novel and interesting problem of detecting the transition between human-written and AI-generated text within documents that contain both. It uses updated datasets like RoFT and RoFT-chatgpt to test various detection methods, including RoBERTa-based classifiers, perplexity-based methods, and topological features. The findings reveal that while RoBERTa tends to overfit and struggle with generalization, perplexity-based methods offer a good balance of accuracy and generalization, especially when trained on synthetic data. Topological methods show promise in robustness across different domains but occasionally lack high accuracy. The study highlights significant challenges in cross-domain and cross-model generalization, particularly with short texts due to limited context and longer texts due to their complexity.
strength_weakness: strength_weakness: More explanations about the choice of parameter values may be helpful to understand the underlying methodology that has been adopted. For examples, the authors use a window size of 20 with strides of 5 when performing Topological Data Analysis (TDA). More explanation for the choice of 20 and 5 can be helpful. Are those values standard or dependent on text length and identified empirically?

Missing references can be corrected. For example, On page 5, the reference figure is missing in the text "(SoftAcc1; indeed, Fig. ?? in Appendix F shows that many misclassifications point to an adjacent class, which may be acceptable in real world applications)".
questions: questions: Why was the choice of datasets limited to only one? Is there a specific reason pertaining to the problem being solved?
paper_summary: paper_summary: The paper titled investigates the challenge of detecting boundaries between human-written and machine-generated text, an issue increasingly relevant due to the proliferation of LLMs. Utilizing the real or fake text dataset, which comprises texts with both human-written and machine-generated sections, the study tests several methodologies adapted for this complex detection scenario, moving beyond traditional binary classifications. Key approaches include perplexity-based methods that evaluate text unpredictability, adaptations of classifiers like RoBERTa that leverage text embeddings to detect shifts in text generation sources, and innovative applications of topological data analysis to determine the intrinsic dimensionality of text data, which provides robustness against domain and model shifts. The results suggest that PPL based methods and intrinsic dimension estimation offer promising results, outperforming traditional binary classifiers and showing better adaptation to the nuanced challenges of environments with mixed human and AI-generated text. This research contributes valuable insights into artificial text detection, highlighting the potential for further exploration and refinement of these techniques to improve our ability to discern human from machine-generated content effectively.
strength_weakness: strength_weakness: 1. Analysis is limited to only one datasets
paper_summary: paper_summary: This paper tackles the problem of detecting the boundary between human-written and machine-generated text in mixed texts, which is an increasingly common real-world scenario that has not been well-studied. The authors adapt several existing artificial text detection techniques to this boundary detection task and evaluate them on the RoFT (Real or Fake Text) dataset as well as a new RoFT-chatgpt dataset they created using GPT-3.5 generations.
strength_weakness: strength_weakness: 1.	Technically, this paper mainly conducts benchmarking on the RoFT dataset, primarily using evaluation metrics. It is more like a technical report than a research paper with novel contributions.
2.	To my understanding, detecting the boundary between human-written and machine-generated parts should go beyond sentence classification. It would be better to localize which parts of the paragraph are likely written by a human and which parts of the article are likely written by AI. However, the current evaluation in this paper is doing binary classification for each sentence in the RoFT dataset, which may not capture the nuances of the task.
3.	Some parts of the paper are not clear. For example, in the evaluation section on artificial text boundary detection, the terms "SoftAcc" and "mean squared error" are not explicitly defined. It would be better to include mathematical notations to clarify these metrics.
4.	While the paper mentions the limitations of the methods in general terms, it lacks a deep dive into specific instances where the models failed. Providing concrete examples and analyzing the reasons behind the failures could offer valuable insights for future improvements in boundary detection techniques.

Comments: title: Stride values
Maintaining original rating
More clarity on the choice of empirical values
Response to Area Chair 44QH and Reviewer Zghr
Please respond to the authors rebuttal
comment: In our preliminary experiments, we tried only stride values 1, 2, and 5. We didn't try values above 5.
Thank you for your comment to the review! I tend to maintain my original rating. Thanks
The response for choice of one dataset is reasonable and so is the mention about the window size of 20. Can you shed more light on the range of values selected for stride size. You did mention about the reason for rejecting smaller stride values but it could shed more light on your choice if you can mention the range of values initially considered for the preliminary experiments and also mention what did you observe for not only the smaller stride values but also the larger ones.
Dear Area Chair 44QH, dear Reviewer Zghr, we are grateful to the reviewer for the constructive feedback, we believe that we have addressed most of the reviewer's concerns, adding further improvements to the paper. We respectfully disagree with the  «technical report» remark. The distinction between a research paper and a «technical report» is rather subjective, 
our paper contains new ideas, novel experiments, and analysis, that we believe offer useful insights to the community.
Thanks for your response. I tend to maintain my original scores.
Hello reviewer,
As a reminder, the discussion period ends on Thursday, June 6. Please take a look at the author's rebuttal and acknowledge if they have adequately addressed the reasons you gave to reject.
Thanks,
AC