MAGE : Machine-generated Text Detection in the Wild (2024)

Abstract: Large language models (LLMs) have achieved human-level text generation, emphasizing the need for effective deepfake text detection to mitigate risks like the spread of fake news and plagiarism. Existing research has been constrained by evaluating detection methods o specific domains or particular language models. In practical scenarios, however, the detector faces texts from various domains or LLMs without knowing their sources. To this end, we build a comprehensive testbed by gathering texts from diverse human writings and deepfake texts generated by different LLMs. Empirical results on mainstream detection methods demonstrate the difficulties associated with detecting deepfake text in a wide-ranging testbed, particularly in out-of-distribution scenarios. Such difficulties align with the diminishing linguistic differences between the two text sources. Despite challenges, the top-performing detector can identify 84.12% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios.

## introduction with constant advancements in artificial intelligence generated content (aigc) technology @xcite @xcite @xcite , texts generated by large language models (llms) @xcite @xcite have reached a level comparable to that of human peers, enabling the generation of remarkably fluent and meaningful responses to various user queries. advanced llms have become prevalent in enhancing human life and productivity. nevertheless, they can also be employed for purposes such as * work was conducted during the internships of yafu li and qintong li at tencent ai lab. in practice, however, a machine-generated text detector may encounter fake news from various llms without knowing their sources, as depicted in figure 1 . the detector can also face chatgptgenerated student assignments across different tasks such as story generation, question answering, and scientific writing. as the detector encounters increasingly diverse texts from both human-written and machine-generated sources, it has fewer surface patterns or linguistic differences to rely on. in a more demanding scenario, the detector must identify texts from unfamiliar domains or those generated by unseen llms. in this study, we try to address the following research questions: (1) can existing detection methods effectively distinguish texts generated by diverse llms for various writing tasks in real-world scenarios? (2) are there inherent distinctions between human-written texts and machine-generated texts in an open-domain setting, irrespective of their topic or content? to this end, we build a large-scale testbed, mage, for machine-generated text detection, by collecting human-written texts from 7 distinct writing tasks (e.g., story generation, news writing and scientific writing) and generating corresponding machine-generated texts with 27 llms (e.g., chatgpt, llama, and bloom) under 3 representative prompt types. we categorize the data into 8 testbeds, each exhibiting progressively higher levels of "wildness" in terms of distributional variance and detection complexity. initially, we detect texts generated by a white-box llm within a specific domain. subsequently, we enhance the complexity by incorporating texts generated by additional llms across various writing tasks. the most challenging testbed necessitates the detector's ability to identify out-of-domain texts generated by newly developed llms and perform detection against paraphrasing attacks. we evaluate 4 commonly employed detection methods, encompassing both supervised and unsupervised approaches, on our proposed testbeds. empirical results indicate that all detection methods are effective in identifying machine-generated texts from a single domain or generated by a limited range of llms. however, as the diversity of domains and models increases, except for the plmbased detector, all other methods experience significant performance deterioration. the challenge intensifies with out-of-distribution (ood) testbeds, where even the best-performing detector misclassifies 61.95% of human-written texts from unseen domains. the suboptimal ood performance can be effectively mitigated by leveraging a mere 0.1% of in-domain data, resulting in over 80% recall for identifying out-of-domain texts generated by previously unencountered llms. this demonstrates the feasibility of machine-generated text detection in real-world scenarios. finally, we investigate potential differences between human texts and machine generations that can be utilized for detection. statistical findings demonstrate that while significant linguistic differences exist within a particular domain, they gradu-ally converge as more texts from diverse domains and language models are included. moreover, empirical results demonstrate that perplexity can serve as a fundamental feature for clustering the two sources of text. it is applicable to distinguishing between human and machine compositions in general, regardless of the text domain or the language model used for generation. we release our resources at https://github.com/yafuly/mage . ## related work a line of work explores the linguistic patterns to achieve automatic machine-writing detection, which has gone through n-gram frequencies @xcite , entropy @xcite , perplexity @xcite , and negative curvature regions of the model's log probability @xcite . one limitation of these statistics-based methods is the white-box assumption that we can access the model prediction distributions, hindering wider applications on models behind apis, such as chatgpt. another alternative paradigm is training neural-based detectors @xcite @xcite . some works @xcite @xcite explore the potential of watermarks in language models, making model-generated texts easier to detect. @xcite indicate that texts by non-native speakers are more likely to be incorrectly identified as ai-generated. our work does not assume language models are enhanced with watermarks, instead considering a more common detection setting where we do not know the sources of detected texts. current ai text detection has not achieved significant success, as evidenced by the successful exploits of paraphrasers that expose weaknesses in existing detectors @xcite , raising concerns about the robustness of current detection methods. on the other hand, most of the detectors focus on specific domains, such as news @xcite and reviews @xcite , or specific models @xcite @xcite . the transferability of detection capabilities to out-of-distribution scenarios, involving texts from unseen domains or models, remains uncertain and represents a crucial practi-cal challenge. to address this issue, we examine a scenario where texts from various domains generated by different language models are combined and extended to out-of-distribution settings with consideration for paraphrasing attacks. model sets. we aim to adopt a wide spectrum of representative large language models (llms) to construct machine-generated texts. in particular, we consider 27 llms in this work: openai gpt (text-davinci-002/textdavinci-003/gpt-turbo-3.5) @xcite , llama (6b/13b/30b/65b) @xcite , glm-130b @xcite , flan-t5 (small/base/large/xl/xxl) @xcite , opt (125m/350m/1.3b/2.7b/6.7b/13b/30b/iml-1.3b/iml-30b) @xcite , bigscience (t0-3b/t0-11b/bloom-7b1) @xcite and eleutherai (gpt-j-6b and gpt-neox-20b) @xcite . prompts. to generate machine-generated text for each instance in the collected data, we use three types of prompts to feed the llms: (1) continuation prompts: ask llms to continue generation based on the previous 30 words of the original human-written text; (2) topical prompts: as * https://huggingface.co/datasets/julesbelveze/tldr_news llms to generate texts based on a topic (e.g., argument, news title, story topic, etc.) and (3) specified prompts: topical prompts with specified information about the text sources (e.g., bbc news, reddit post, etc.). the topical and specified topical prompts are designed for openai models, as they can respond to such prompts robustly. we present several prompt examples in appendix a. in summary, for each human-written text, we generate a set of machine-generated texts using 27 llms with 3 different prompts. data construction details and statistics are presented in appendix b. ## detection methods a detection system labels a text as either machinegenerated or human-written, or outputs a probability distribution. in this work, we consider a set of commonly used detection methods. to showcase detection difficulty, we first consider naive baselines, i.e., human detection and ask chat-gpt, by asking human and query chatgpt to identify the text source. for supervised methods, we choose the plm-based classifier, which is commonly used in text detection @xcite . we report the performance of longformer @xcite in the remainder of the paper, as it outperforms other commonly used plms, such as bert @xcite , roberta @xcite , and gpt-2 @xcite . detailed comparisons can be found in appendix e. gltr @xcite is also included to represent methods that leverage modelbased features. in addition, we include fasttext @xcite , which uses linguistic statistics as features. for unsupervised detection, we consider detectgpt @xcite to study the robustness of zero-shot detectors, which can also serve as a representative method that requires access to the text-generation llm. implementation details are shown in appendix c. ## conclusion we proposed a comprehensive testbed for machinegenerated text detection, by gathering texts from various writing tasks and machine-generated texts generated by different llms. empirical results on commonly used detection methods demonstrated the challenge of ai-generated text detection. outof-distribution posed a greater challenge for detectors to be employed in application scenarios. with the boundary refined, the best-performing detector on our testbeds (i.e., longformer detector) achieved 86.54% avgrec on out-of-domain texts generated by a new llm, i.e., gpt4. by studying differences between human and machine compositions, we find that perplexity can serve as a fundamental feature for classification regardless of text domain or generation llm. to the best of our knowledge, this is the first study to investigate the challenges and feasibility of ai-generated text detection in a "wild" testbed.