On the Zero-Shot Generalization of Machine-Generated Text Detectors (2023)

Abstract: The rampant proliferation of large language models, fluent enough to generate text indistinguishable from human-written language, gives unprecedented importance to the detection of machine-generated text. This work is motivated by an important research question: How will the detectors of machine-generated text perform on outputs of a new generator, that the detectors were not trained on? We begin by collecting generation data from a wide range of LLMs, and train neural detectors on data from each generator and test its performance on held-out generators. While none of the detectors can generalize to all generators, we observe a consistent and interesting pattern that the detectors trained on data from a medium-size LLM can zero-shot generalize to the larger version. As a concrete application, we demonstrate that robust detectors can be built on an ensemble of training data from medium-sized models.

## introduction thanks to large-scale pretraining and tuning with human feedback @xcite , large language models (llms) @xcite @xcite are now able to follow instructions and generate realistic and consistent texts. a prominent example is the recently developed chatgpt or gpt4 model (ope-nai, 2023), which when instructed, can write documents, create executable code, or answer questions that require world knowledge. in a lot of scenarios, the machine-generated texts have high quality and cannot easily be distinguished from genuine human texts @xcite . these trends give an unprecedented importance to the detection of machine-generated text @xcite @xcite . a lot of work has been devoted to proposing efficient detection models or algorithms @xcite @xcite . however, in most studies, the detector is tested on the same generator model that it is trained/tuned on. this study is motivated by an underexplored research question: how will the detector perform on a different generator that it is not trained on? this question is important due to multiple reasons: (1) llms are becoming increasingly large and expensive. some of the most recent models are either too large to fit into a common gpu (e.g., llama-65b) or require payment from the user (openai, 2023), making the collection of training samples difficult. (2) the number of released llms is growing rapidly. in a real application scenario, the detector needs to cover a wide range of llms (including the ones the detector is not trained on), instead of only one generator. in this work, we collect generation data from a wide range of llms. we then train neural detectors on data from each generator and test its performance on other generators. our primary findings include: (1) in many cases, detectors can zero-shot generalize to a held-out generator (figure 1 ). in particular, we observe an interesting pattern that the detector for the medium version of an llm can generalize to the larger version. (2) none of the detectors generalizes to all generators, implying that an ensemble of detectors/data is necessary for a wider coverage. (3) as a concrete application, we demonstrate that robust detectors can be built on an ensemble of training data from medium-sized models; excluding large-versions only leads to a minor drop in performance. ## methodology we begin by giving an overview of our experiment structure and establish some notations. this study includes detection of a range of popular llms (detailed in §3), and we construct train/dev/test sets for each generator. in §4.1, we train neural detectors on data from each generator and test its performance on other generators. in §4.2, we further consider an ensemble setting, where the detector is trained on data composed of multiple generators, and test its generalization ability on held-out generators. we denote the detector model trained on data from generator model m as d m . since we will test the accuracy of d m on data from different generators, we use acc n (d m ) to denote the accuracy of d m on the test set of generator n . finally, we define acc-gap d m n to measure the drop of performance when the detector is trained on generator m instead of n itself: we expect acc-gap to be larger than zero in general, and a large acc-gap means d m has poor generalization on generator n . ## experiment setup generators we include the detection of a total of 13 popular lms in our study, including gpt-1 @xcite , gpt-2 models (small, medium, large, and xl) @xcite , gpt-3 (text-davinci-003) @xcite , gpt-4 (openai, 2023), three gpt-neo models (125m, 1.3b and 2.7b) @xcite , gpt-j @xcite , and llama (7b and 13b) (touvron et al., 2023). 1 datasets we consider data from three domains: news, reviews and knowledge. for the news domain, we utilize the realnews dataset, which detectors for data from each generator, we train a electra-large model @xcite as a binary classifier. the detectors were trained for 1 epoch with a learning rate of 5e-6 (training for more epochs only gives minimal improvement on the dev set). for the data-mix baseline and pruned models in §4.2, 3 epochs of training is used. we use adam optimizer (kingma and ba, 2014) with β 1 = 0.9, β 2 = 0.999. the average accuracy (when tested on the same generator it is trained on) of all detectors in news, review and knowledge domains are 94.1%, 96.2% and 94.9%, separately. ## conclusion and discussion in this work, we observe a generalization relationship among detectors trained on different generators in three domains, where detectors for mediumversion models demonstrate the ability to effectively generalize to the larger-version. building upon this finding, we prune out data from largeversion generators in an ensembed training dataset and demonstrate that the performance loss is min-imal. our results indicate that practitioners with limited budget or computing resources can use data from medium-size llms as a good approximation for the large version. with the rapid release of various llms and generation apis, a detector needs to cover a wide range of generators. while our work makes some initial progress, our experiments show that the detection of an unseen (or non-public) generator is still a difficult and open question. we hope our work could motivate more research devoted to this important direction.