Robust AI -Generated Text Detection by Restricted Embeddings (2024)

Abstract: Growing amount and quality of AI-generated texts makes detecting such content more difficult. In most real-world scenarios, the domain (style and topic) of generated data and the generator model are not known in advance. In this work, we focus on the robustness of classifier-based detectors of AI-generated text, namely their ability to transfer to unseen generators or semantic domains. We investigate the geometry of the embedding space of Transformer-based text encoders and show that clearing out harmful linear subspaces helps to train a robust classifier, ignoring domain-specific spurious features. We investigate several subspace decomposition and feature selection strategies and achieve significant improvements over state of the art methods in cross-domain and cross-generator transfer. Our best approaches for head-wise and coordinate-based subspace removal increase the mean out-of-distribution (OOD) classification score by up to 9% and 14% in particular setups for RoBERTa and BERT embeddings respectively. We release our code and data: https://github.com/SilverSolver/RobustATD

## introduction the proliferation of generative ai leads to an explosion in ai-generated content. large language models (llms) can produce text that is very similar to human-written. however, ai-generated text can be used for malicious purposes, which leads to the artificial text detection (atd) problem: has a given text or image been created by an ai model or a human? existing approaches for artificial text detection can be divided into score-based and classifierbased. the former aim to identify and measure features that distinguish artificial text from real; e.g., generated text may exhibit statistical artifacts due to the specific generation process used by a language model @xcite , the difference may lie in perplexities measured by another 1 https://github.com/silversolver/robustatd language model @xcite , curvature of the probability function @xcite , or intrinsic dimensionality of contextualized representations @xcite . however, score-based methods often rely on prior knowledge about a specific generator and/or semantic domain, and known traces may be easy to remove, e.g., by paraphrasing the text @xcite . one notable exception is the intrinsic dimension feature for text content, shown to be robust to domain transfer and paraphrasing @xcite , but its overall detection quality is relatively modest. supervised classification methods show almost perfect in-domain detection quality, but fail to generalize to unseen text topics and writing styles @xcite . the choice of training data, both artificial and generated, is crucial for successful out-of-distribution (ood) transfer. in general, while usually there exist features that can distinguish between natural and artificial subsets of the training set, the classifier may lock into dataset-specific spurious differences and hence generalize poorly. it is hard to say in advance if a classifier trained on a given dataset will generalize well to new unseen generators and data sources. previous approaches to ood detection for atd include uid-based detectors @xcite and domain adversarial training @xcite , but most of these methods are very data-intensive @xcite . in this work, we aim to improve supervised classification by ignoring spurious features to enhance cross-distribution robustness, training on small number of domains or generator models. namely, we focus on methods of extracting residual subspaces and deleting information from embeddings. in many applications, retaining only important dimensions of high-dimensional data while treating projections onto less loaded subspaces as residual noise can benefit downstream tasks. however, for tasks such as ood detection the principal compo-nents of a dataset may be the least useful. @xcite found that nullifying the first (least important) principal components in the embedding space fine-tuned on in-distribution (id) data enhances ood detection quality; this is known as the partial mahalanobis distance. @xcite conducted similar analysis for transformerbased text classifiers and found that id data has orthogonal classes and lies on a unit sphere in a lowdimensional space. the main difference between id and ood data lies in the residual subspace, hence the partial mahalanobis distance performs well in ood detection. it is important to note that not all neural networks learn useful residual subspaces for a given dataset; e.g., @xcite and @xcite find that on text, cnn classifiers learn representations where components with low singular values contain too much information about id data, making it difficult to distinguish ood examples. in this work, we apply similar techniques to artificial text detection (atd). distribution shift, with variations in text styles, topics, and new generation models, is a major challenge for atd. supervised classifiers, even performing well on validation datasets, struggle in realistic settings, where the domain and model of the ai-written text are unknown. to address this, we first show that training a classifier on some residual subspace obtained by coordinate removal or layer pruning may significantly enhance atd robustness under domain and model shift. next, we show that controllable subspace removal can improve robustness, while also providing us with interpretable information about ai-written texts and domains. in particular, we use recent advances in concept erasure @xcite , experimenting with erasing semantic and syntactic concepts based on probing tasks by @xcite ; we show that some concepts are harmful for cross-domain and cross-model transfer. our primary contributions are: (i) a first application of the residual subspace approach for robust atd; we show that restricting the detector to a residual subspace increases cross-topic and cross-model robustness with especially significant improvements on the most difficult samples; (ii) analysis of different residual decomposition techniques, such as nullifying head-wise subspaces in intermediate data representations and concept erasure; (iii) analysis of applicability of our methods with different encoder-or decoder-based backbone models. besides, we create and release an exten-sion for one of the datasets with recent generating model gpt-4-o on three domains. below, section 2 surveys related work, section 3 describes the proposed methods, section 4 introduces the datasets, section 5 presents a comprehensive experimental evaluation, and section 6 concludes the paper. components of transformer-based embeddings can provide useful features via the geometry of their inner representations or parameter spaces. for instance, outlier dimensions in the embedding spaces of models such as bert @xcite or roberta @xcite , characterized by unusually high variance and/or mean values, have been studied in detail, including their emergence during training and effects of disabling them post-training @xcite , their relationship with positional embeddings and impact on word-in-context tasks @xcite , influence on the quality of representations @xcite , and relations to the shapes of attention maps and token frequencies @xcite . activations of transformer-based lms have been investigated for language structure information @xcite , semantic and syntactic features @xcite ; the latter work also introduces a comprehensive selection of probing tasks. however, only outlier dimensions have been studied in full detail; we aim to address this gap by studying how removing specific dimensions from roberta embeddings can improve detection of artificially generated text. semantics of attention heads in transformers have been studied for a long time: @xcite provided empirical research on bert attention heads, demonstrating overparameterization by pruning some of them, @xcite showed that most heads can be removed at test time without significant performance loss. @xcite studied the specialization of attention heads; @xcite , their functional roles. in bert-like models, important information is distributed across layers; e.g., @xcite showed that lower layers capture phrase-level information, which is partly lost in the upper layers; the model captures a hierarchy of linguistic information, with deeper layers required to capture long-distance dependencies. @xcite showed that attention maps are correlated across layers and organized into clusters. therefore, here we focus on groups of attention heads within a layer rather than individual heads. artificial text detection (atd) is a new field of study (up until recently, artificial content was mostly easy to distinguish), but there already exist many promising approaches. score-based methods include detectgpt, which measures the curvature of the probability function @xcite , and gptzero @xcite , which checks the perplexity and burstiness of a text; these methods, however, are limited to a single domain or generator. throughout recent work, it remains a reasonable baseline for general and cross-distribution atd to take embeddings from bert-like models as a feature space and train logistic regression (lr) over them. following @xcite and @xcite , we take the roberta model @xcite to extract text embeddings, use mean-pooling over embeddings, and train lr models for atd. the recent semeval-2024 competition @xcite proposed challenge in a multi-generator, multi-domain, and multi-language setting based on the new atd dataset that was introduced in @xcite . task 8 included problems such as binary classification, source identification, and fake/real text boundary detection. solutions used approaches such as llm fine-tuning (roberta, xlm-r), contrastive learning, and ensemble methods. however, while all these approaches are data-intensive, absolute classification quality is still poor. in this work, we use classifiers that perform well on in-domain data and aim to improve their performance on unseen domains. ## methods removing unnecessary features is often an effective method to improve the robustness of a machine learning model. the embedding space has linear substructures responsible for linguistic fea-tures such as token frequencies, word-in-context information etc. @xcite . we aim to detect and erase such substructures, which are harmful for atd generalization. ## results and analysis here we present results on baseline, heads pruning, concept erasure and selecting coordinates. pcabased results are reported in appendix h. baseline roberta. as a baseline we use logistic regression (lr) trained on mean-pooled roberta embeddings. results are shown in fig. 1 for semeval and fig. 3a for gpt-3d; the crossdomain and cross-model settings are challenging in both tasks. fig. 1 shows that in-domain classification is almost perfect for baseline lr on roberta embeddings, but the cross-domain part is very inconsistent: e.g., transfer from reddit to peerread works well across all models (91% avg accuracy) but transfer from arxiv to wikihow is uniformly bad (54%). in semeval, wikihow is the hardest domain to transfer to, while arxiv is the hardest domain to transfer from ( average transfer results. head pruning for transfer tasks. we adapt head pruning @xcite to remove a whole layer of attention heads. since layers of a model have rough linguistic meanings @xcite , thus we analyse the impact of structurallevel information on atd. fig. 2 and mains). pruning layers 3 and 4 is more stable and beneficial in both settings. cross-domain atd is more challenging; fig. 2 (top) shows that some domains (wikipedia and wikihow) exhibit similar patterns but others are unrelated. the best scores are in transfer from reddit, achieving 81% mean balanced accuracy with 0-th layer pruned (+5% to full roberta). the cross-model setting is easier and not greatly affected by pruning layers, with the exception of bloomz. here the best source model is gpt-3.5-davinci, with 92% cross-model accuracy after removing layer 4. concept erasure. generally, results on semeval show that the best concepts to erase are topconst and treedepth, improving up to 2.1% on crossdomain transfer and not hurting the cross-model transfer. erasing wc also performs well but is less stable. figs. 4 give more detailed information. although changes compared to we conclude that the ability to detect grammatically correct sentences is crucial for robust aigenerated text detection; the difference in global syntax between natural and generated texts is significant, but varies between models and domains, so erasing this information helps generalization, and individual word semantics is a source of spurious features. on the other hand, world-level grammatical categories are captured well by all generators and do not influence atd performance. ## conclusion in this work, we aim to improve the robustness of artificial text detectors via linear feature removal from text embeddings. we propose three ideas that are extremely easy to implement and achieve stable improvement in robustness averaged across domains and models, up to 14% depending on the text encoder. more importantly, we conclude with the following novel insights from our work. first, new generation models can completely break detectors; e.g., on the gpt-4 family previous detectors' perform below random, while on the same model classifiers demonstrate very high performance in the cross-domain setting. the reason could be the presence of watermarks in gpt-4 generations; if so, watermarks unknown for atd developers are dangerous, leading to unpredictable black-box behaviour. second, performance with respect to the training subset is often counterintuitive; e.g., a classifier trained on wikipedia may perform worse than on reddit, although wikipedia is considered a cleaner domain, better suited for general-purpose models. third, transformer encoders learn disentangled intrinsic features in coordinates and attention heads, and simple decompositions perform better for atd than more complex approaches. but this effect is less pronounced for decoder models. we plan to study differences in the geometry of encoder and decoder-based text representations in future work. finally, global syntax and sentence complexity is a key point for atd, but the exact differentiating features are domain-and model-specific, so this information should be ignored. local grammatical categories do not provide an important signal for atd. instead, the classifier should rely on features for detecting various types of inconsistencies.