Cross-Domain Detection of GPT -2-Generated Technical Text (2022)

Abstract: Machine-generated text presents a potential threat not only to the public sphere, but also to the scientific enterprise, whereby genuine research is undermined by convincing, synthetic text. In this paper we examine the problem of detecting GPT-2-generated technical research text. We first consider the realistic scenario where the defender does not have full information about the adversary’s text generation pipeline, but is able to label small amounts of in-domain genuine and synthetic text in order to adapt to the target distribution. Even in the extreme scenario of adapting a physics-domain detector to a biomedical detector, we find that only a few hundred labels are sufficient for good performance. Finally, we show that paragraph-level detectors can be used to detect the tampering of full-length documents under a variety of threat models.

## introduction recent advances in techniques for generating realistic synthetic content (i.e., deepfakes) pose a diverse set of problems with significant societal consequences @xcite @xcite . the advent of large language models for text generation @xcite have made it easier than ever to create convincing synthetic foot_0 text @xcite . while much attention has been focused on the role of synthetic audio and video, it can be argued that synthetic text may give rise to some of the most serious threats to information integrity and the long-term preservation of archival knowledge @xcite . while there are currently no documented cases of published papers containing text from neural language models, non-neural machine-generated papers have already been published in peer-reviewed journals @xcite . with more convincing text generation techniques and a growing number of publications, this is a problem that could get much worse, particularly for non-peerreviewed technical text @xcite . in addition to hindering the scientific process, synthetic technical texts could be used to manipulate public opinion and sow discord around specific scientific topics @xcite . other consequences of machine-generated technical text include the contamination of nlp pipelines @xcite and poisoning of language models @xcite . the rise of realistic machine-generated text -and its possible misuse -has spurred the development of automated tools to distinguish between genuine and synthetic english text. 2 however, most work to date has focused on web text or news, rather than on domain-specific technical text @xcite . while synthetic technical text is likely to be distinguished by subject matter experts (smes), it is difficult for non-domain evaluators to do so. therefore, given the ease at which large amounts of synthetic text can be generated, automated countermeasures are necessary in order to alleviate the burden on smes. we focus on developing capabilities for detecting generated technical english text and delineating the contexts in which these detection approaches can be applied. 3 we assume an adversary generating technical text using gpt-2 @xcite , but do not know apriori in exactly which technical area. realistically, some domain shift will be inevitable, which leads us to investigate whether detectors can be applied across domains, e.g., can a detector of generated physics papers be adapted to also detect biomedical text? in addition, most work to date has focused on detecting short pieces of text. we show that automated detection of tampered full-length research papers is possible under various threat models. our work makes two main contributions: prior work closest to our study are @xcite , @xcite , @xcite , @xcite and @xcite , which look at cross-domain settings where the distribution of synthetic text used to train a detector differs from the target distribution. the shift could be due to different model architectures @xcite , different model sizes @xcite , different decoding strategies @xcite , or different fine-tuning datasets @xcite @xcite . @xcite show that energy-based models generalize poorly across corpora (wikipedia, news and books), but that training on the union of the source and target domains is effective. @xcite show that xlnet can accurately attribute synthetic text even when the gpt-2-generated portions of the training and test sets come from gpt-2 fine-tuned on different subreddits. unlike the previous papers, we evaluate our detectors on technical (biomedical) text, and vary the number of labeled samples available for training in the target domain, under the assumption that source labels are plentiful but target examples are more expensive to obtain (i.e., source samples can be generated at will, but target examples need to be discovered and verified by a human since the adversary's target text generator is generally unavailable). in addition, we evaluate the detection of full-length tampered documents consisting of a mix of real and machine-generated content. attribution for synthetic text in addition to distinguishing between real and generated text, one may also wish to determine which system generated a given text (e.g., model type, size, decoding strategy). variations of this "authorship attribution" problem have been explored by @xcite , @xcite and @xcite . these works have found that, in general, while the attribution problem is harder than the detection problem, details of the text generator can often be learned from the generated text. human detection of synthetic text human evaluations on the detection of generated text have been conducted for news @xcite @xcite , product reviews @xcite @xcite , web text @xcite , stories @xcite @xcite , peer reviews @xcite , cybersecurity text @xcite and submissions to federal public comment websites @xcite . these studies have shown that it is difficult for people to distinguish between real and neural-generated text. gehrmann et al. (2019) developed a tool, gltr, to help users visualize statistical artifacts in generated text. ## datasets here we describe the datasets used and other experimental details when simulating the adversary and defender's pipelines (figure 2 ). the experiments on synthetic abstract detection are described in §5.1; experiments on tampered document detection are described in §5.2. ## conclusion in this paper, we studied the problem of automatic detection of gpt-2-generated technical text. we found that roberta-based detectors can be successfully adapted from one scientific discipline (physics) to another (biomedicine), requiring relatively small amounts of in-domain labeled data. these could be provided by a subject matter expert (sme) in a reasonable amount of time. we also evaluated these paragraph-level detectors on a document tampering task, assuming that the adversary replaces randomly-selected paragraphs in a document with generated ones. future work should evaluate the extent to which this methodology would work on detecting text from newer generators such as gpt-3. other challenging scenarios include adding noisy labels (e.g., if a sme makes a certain fraction of mistakes), and class imbalance. our results on document tampering (i.e., that it is significantly harder to detect small amounts of generated text intermingled amongst real text) also point to the need to develop detection pipelines for other threat models such as single word or phrase substitutions @xcite . as text generation techniques continue to improve, it may be that more interpretable, fact-checking approaches are required to detect both human and machine-generated misinformation.