Artificial Text Detection via Examining the Topology of Attention Maps (2021)

Abstract: The impressive capabilities of recent generative models to create texts that are challenging to distinguish from the human-written ones can be misused for generating fake news, product reviews, and even abusive content. Despite the prominent performance of existing methods for artificial text detection, they still lack interpretability and robustness towards unseen models. To this end, we propose three novel types of interpretable topological features for this task based on Topological Data Analysis (TDA) which is currently understudied in the field of NLP. We empirically show that the features derived from the BERT model outperform count- and neural-based baselines up to 10% on three common datasets, and tend to be the most robust towards unseen GPT-style generation models as opposed to existing methods. The probing analysis of the features reveals their sensitivity to the surface and syntactic properties. The results demonstrate that TDA is a promising line with respect to NLP tasks, specifically the ones that incorporate surface and structural information.

## introduction recent text generation models (tgms) based on the transformer architecture @xcite have demonstrated impressive capabilities of creating texts which are very close to human in terms of fluency, coherence, grammatical and factual correctness @xcite @xcite . extensive gpt-style tgms @xcite have achieved outstanding results over a great scope of nlp tasks employing zero-shot, one-shot, and few-shot techniques, even outperforming state-of-the-art finetuning approaches @xcite . however, such models can be misused for generating fake news @xcite , product reviews @xcite , and even many attempts have been made to develop artificial text detectors @xcite , ranging from classical ml methods over countbased features @xcite to advanced transformer-based models @xcite and unsupervised approaches @xcite . despite the prominent performance of these methods across various domains, they still lack interpretability and robustness towards unseen models. this paper introduces a novel method for artificial text detection based on topological data analysis (tda) which has been understudied in the field of nlp. the motivation behind this approach relies on the fact that (i) the attention maps generated by the transformer model can be represented as weighted bipartite graphs and thus can be efficiently investigated with tda, (ii) tda methods are known to capture well surface and structural patterns in data which, we believe, are crucial to the task. the contributions are summarized as follows. (i) to the best of our knowledge, this work is the first attempt to apply tda methods over the transformer model's attention maps and interpret topological features for the nlp field. (ii) we propose three types of interpretable topological features derived from the attention graphs for the task of artificial text detection. we empirically show that a simple linear classifier trained on the tda features produced over bert attentions @xcite outperforms count-and neural-based baselines up to 10%, and can perform on par with the fully finetuned bert model across three domains: social media, news articles and product reviews. (iii) testing the robustness towards unseen tgms, we find that the tda-based classifiers tend to be more robust as opposed to the existing detectors. (iv) the probing analysis of the features demonstrates their sensitivity to surface and syntactic properties. (v) finally, we are publicly releasing the code foot_0 , hoping to facilitate the applicability of the tda methods to other nlp tasks, specifically the ones that incorporate structural information. the research in the emerging scope of tda applications to neural networks and neural data representations has mainly focused on artificial datasets or common problems in computer vision. the desired topological properties of the data representation can be incorporated into the objective function during the training of a neural network, improving its robustness and performance on the downstream tasks such as human action recognition and image classification @xcite , image simplification @xcite , image segmentation @xcite or generation @xcite . another line aims to develop the topological criteria of the network's generalization properties @xcite @xcite or its robustness to adversarial attacks @xcite . ## persistent features of the attention graphs informal definition and interpretation we extract three groups of features from the attention graphs. topological features (section 4.1) include a set of standard graph properties: the number of connected components, the number of edges, and the number of cycles. these features are calculated for each pre-defined threshold separately and to give the linguistic interpretation of our features, recall that the graph structures are used in lexicology for describing semantic change laws @xcite @xcite . the evolution of the meaning of a word with time can be represented as a graph, in which edges represent a semantic shift to different word meanings. two typical patterns are distinguished in the graph structure: radiation -the "star" structure, where the primary meaning is connected to other connotations independently; concatenation, or chaining shift -the "chain" structure when the connotations are integrated one-by-one. note that the typical attention patterns @xcite have the same "radiation" and "concatenation" structure. in pretrained lms, the evolution goes through the layers of the model, changing the representation of each token, ending up with highly contextualized token representations, and the aggregated representation of the whole sentence (in the form of the [cls]token). we consider persistent features as the numerical characteristic of the semantic evolution processes in the attention heads. topological features deal with clusters of mutual influence of the tokens in the sentence and the local structures like chains and cycles. the barcode features characterize the severity and robustness of the semantic changes. the features with long persistence (large distance between "birth" and "death") correspond to the stable processes which dominate the others, while short segments in the barcode define processes highly influenced by noise. pattern features provide a straightforward measure of the presence of typical processes over the whole sentence. the so-called "vertical" pattern corresponds to the "radiation" around the single token when the meaning of the sentence or a part of the sentence is aggregated from all words equally. "diagonal" pattern represents consequent "concatenation" structure, going through all the sentence and thus reflecting the dependence of each token's meaning on its left context. ## discussion structural differences between natural and generated texts the tda-based classifiers rely on the structural differences in the topology of the attention maps to distinguish between natural and generated texts. figure 4 shows that the distributions of the sum of bars in h 0 differ for natural and generated texts. for the former, it is shifted to the left. we provide more examples of the distribution shift for different heads and layers in figure 5 , appendix b. the weights for natural texts are concentrated more on the edges of the maximum spanning tree (mst), so that the model focuses on the sentence structure, or on the "skeleton" of the mst. the weights for the artificially generated texts are distributed more evenly among all edges. as the tda-based classifiers appear to be robust towards unseen tgms, we may conclude that such structural properties are inherent to the models of different sizes, so that shifts in the distribution of the sum of bars in h 0 hold for texts generated by different tgms. this feature appears to be the key one as utilizing it alone for the prediction provides us with the 82% accuracy score on the webtext & gpt-2 dataset. semantics is limited the tda-based methods do not take the semantic word similarity into account, as they only capture inter-word relations derived from the attention graphs. the probing analysis supports the fact that the features do not encode the semantic properties, carrying only surface and structural information. however, this information appears to be sufficient for the considered task. ## conclusion this paper introduces a novel method for the task of artificial text detection based on tda. we propose three types of interpretable topological features that can be derived from the attention maps of any transformer-based lm. the experiments demonstrate that simple linear classifiers trained on these features can outperform count-and neuralbased baselines, and perform on par with a fully fine-tuned bert model on three common datasets across various domains. the experimental setup also highlights the applicability of the features towards the tgm architecture, tgm's size and the decoding method. notably, the tda-based classifiers tend to be more robust towards unseen gptstyle tgms as opposed to the considered baseline detectors. the probing analysis shows that the features capture surface and structural properties, lacking the semantic information. a fruitful direction for future work is to combine the topological features with those that encode the semantics of the input texts, and test the methods on a more diverse set of the tgm architectures, decoding methods and transformer lms to infer the attention graphs from. we are publicly releasing the code, hoping to stimulate the research on the tda-based investigation of the inner workings of the transformer-based models and the applicability of tda methods to other nlp tasks.