M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection (2024)

Abstract: Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries. However, this has also raised concerns about the potential misuse of such texts in journalism, education, and academia. In this study, we strive to create automated systems that can detect machine-generated texts and pinpoint potential misuse. We first introduce a large-scale benchmark M4, which is a multi-generator, multi-domain, and multi-lingual corpus for machine-generated text detection. Through an extensive empirical study of this dataset, we show that it is challenging for detectors to generalize well on instances from unseen domains or LLMs. In such cases, detectors tend to misclassify machine-generated text as human-written. These results show that the problem is far from solved and that there is a lot of room for improvement. We believe that our dataset will enable future research towards more robust approaches to this pressing societal problem. The dataset is available at https://github.com/mbzuai-nlp/M4

## introduction large language models (llms) are becoming mainstream and easily accessible, ushering in an explosion of machine-generated content over various channels, such as news, social media, questionanswering forums, educational, and even academic contexts. recently introduced llms, such as chat-gpt, gpt-4, llama 2 @xcite , and jais @xcite , generate remarkably fluent responses to a wide variety of user queries. the high quality of the generated texts makes them attractive for replacing human labor in many scenarios. however, this raises concerns regarding their potential misuse, e.g., to spread disinformation or to cause disruptions in the education system @xcite . since humans perform only slightly better than chance when classifying machine-generated vs. human-written texts @xcite , we aim to facilitate the development of automatic detectors to mitigate the potential misuse of llms. in particular, we construct a diverse resource that could be used for training and testing various models for detecting machine-generated text (mgt). previous efforts in detecting mgt (i) focused on only one or two particular languages, typically only on english, (ii) used a single generator, e.g., just chatgpt @xcite , (iii) leveraged fine-tuned llms for specific tasks, e.g., machine translation or text summarization @xcite , or (iv) considered only one specific domain e.g., news @xcite . in contrast, here we encompass multiple languages, various llms, and several diverse domains, aiming to enable more general machine-generated text detection. our dataset serves as the basis for semeval-2024 task 8 @xcite . our contributions are as follows: in this study, we focus on black-box techniques because they aim to solve the task for the more practical and general use case. however, we note that their effectiveness heavily depends on the quality and the diversity of the training corpus. related corpora recently, a growing body of research has concentrated on amassing responses generated by llms. turingbench @xcite comprises 200k human-and machinegenerated pieces of text from 19 generative models. however, it is outdated, as the most advanced model used in this research is gpt-3. @xcite collected the hc3 dataset, which consists of nearly 40k questions and their corresponding answers from human experts and chatgpt (english and chinese), covering a wide range of domains (computer science, finance, medicine, law, psychology, and open-domain). @xcite gathered toefl essays written by examined people and such generated by chatgpt (126 essays for each). the ruatd shared task 2022 involved artificial texts in russian generated by various language models fine-tuned for specific domains or tasks such as machine translation, paraphrase generation, text summarization, and text simplification @xcite . we pay more attention to zero-shot generations of llms, such as the subset of ruatd generated by rugpt-3. in general, previous studies have concentrated on detecting machine-generated texts in one or two languages, for a specific llm such as chatgpt, or within a single domain such as news @xcite . our work broadens this scope to include multiple languages and a variety of widely-used llms across different domains. black-box detectors are usually binary classifiers based on three types of features: statistical distributions @xcite , e.g., gltr-like word rankings @xcite , linguistic patterns (such as vocabulary, part-of-speech tags, dependency parsing, sentiment analysis, and stylistic features), and factverification features @xcite . classification models involve deep neural networks, such as roberta @xcite , or more traditional algorithms, such as logistic regression, support vector machines, naïve bayes, and decision trees. there are also widely-used off-the-shelf mgt detectors, such as the openai detector, foot_0 gptzero, foot_1 and zerogpt. 3 according to the limited public information about them, these detectors are trained on collections of human-written texts and texts generated by various llms. for example, the training data of the openai detector contains generations from 34 llms from various organizations, including openai itself. for our m4 dataset, we selected a diverse set of state-of-the-art black-box methods and features, including one off-the-shelf detector. ## the m4 dataset we gather human-written texts from a diverse range of sources across various domains and languages. for english we have wikipedia (the march 2022 version), wikihow @xcite , reddit (eli5), arxiv, and peerread @xcite , for chinese we have baike/web qa question answering (qa), for russian we have ruatd @xcite , for arabic wikipedia, and we use news for urdu, indonesian, and bulgarian. details about the data sources are provided in appendix a.1 and a.2. for machine generation, we prompt the following multilingual llms: gpt-4, chatgpt, gpt-3.5 (text-davinci-003), cohere, dolly-v2 @xcite , and bloomz 176b @xcite . the models are asked to write articles given a title (wikipedia), abstracts given a paper title (arxiv), peer reviews based on the title and the abstract of a paper (peerread), news briefs based on a title (news), also to summarize wikipedia articles (arabic), and to answer questions (e.g., reddit and baike/web qa). 4 3.1 collection prompt diversity for each generator, we carefully designed multiple (2-8) prompts in various styles, aiming to produce diverse outputs that are more aligned to divergent generations in real-world application scenarios. for example, on simple domains of wikipedia and wikihow, two prompts are applied. for arxiv and reddit, as well as for chatgpt, we use five prompts and four prompts for peerread. we generate varying tones of responses with prompts such as answer the question (1) "like i am five years old"; (2) "in an expert confident voice"; (3) "in a formal academic and scientific writing voice"; etc. data cleaning simple artifacts in mgts, such as multiple newlines and bullet points, could assist detectors, as teir presence in the training data may discourage detectors from learning more generalized signals. therefore, we performed minimal cleaning of the human-written and the machine-generated texts: (i) in a human-written wikihow text, we removed multiple commas at the beginning of a new line (like """""", we believe that ...") and repeating newlines ("\n\n\n\n\n text begin \n\n\n\n\n"); quality control unlike other tasks, where the data quality can be evaluated through the agreement between annotators over gold labels, we naturally obtain gold labels along with the collection of machine-generated texts. therefore, we checked the data quality by randomly sampling 10-20 cases for each domain/generator and manually assessing the plausibility of generated texts. this can effectively circumvent incoherent, disorganized, and illogical generations that are easy to distinguish from human-written ones due to improper prompts or hyper-parameter settings of the generators (e.g., some generators repeat newly generated snippets to satisfy the minimum setup of new tokens). moreover, in order to mimic human-written texts, we control the length of mgts. it should be highlighted that we did not pick examples. the quality control we exercised was model-level rather than example-level. we checked for cases where a model fundamentally failed, e.g., by generating visibly very bad output (e.g., very repetitive, english instead of foreign language output, etc.). this was very high-level checking (whether to keep a certain model in m4 or not); at the individual example level, we just checked whether the output had at least 1000 characters in length. thus, we believe any biases that we might have introduced are minimal. ## detectors we evaluated seven detectors; see roberta this detector is based on the pretrained roberta model @xcite , which we fine-tuned to detect machine-generated texts. electra we further fine-tuned elec-tra @xcite . its pre-training objective is more aligned with our mgt task: it was pretrained to predict whether a token in a corrupted input was replaced by a plausible alternative sampled from a small generator network. xlm-r we fine-tuned xlm-roberta, a multilingual variant of roberta @xcite . ## experiments and results in this section, we first describe our experiments, which come in three settings: (i) same generator, cross-domain evaluation, (ii) same domain, crossgenerator evaluation, and (iii) cross-lingual, crossgenerator evaluation. as mentioned in the previous section, we also experiment with gptzero in a zero-shot setting, as it has not seen our data (even though it might have been trained on some domains involved in our data). we further discuss the evaluation results of these experiments. ## conclusion and future work we presented m4, a large-scale multi-generator, multi-domain, and multi-lingual dataset for machine-generated text detection. we further experimented with this dataset performing a number of cross-domain, cross-generator, cross-lingual, and zero-shot experiments using seven detectors. we found that detectors struggle to differentiate between machine-generated and human-written texts if the texts come from a domain, a generator, or a language that the model has not seen during training. our results show that the problem is far from solved and that there is a lot of room for improvement. we hope that our release of m4, which we make freely available to the community, will enable future research towards more robust approaches to the pressing societal problem of fighting malicious machine-generated text. we have already created an extension of m4 for semeval-2024 task 8 (wang et al., 2024), foot_5 which features additional languages, domains, and three new task (re)formulations. in future work, we plan to expand our m4 dataset continuously by introducing new llm generators, by exploring different domains, by incorporating new languages, and by diversifying the range of tasks and prompts used. we believe that this is a good, practical way to keep the dataset up-to-date in response to the ongoing progress in llms. our aim is to maintain a dataset that remains relevant as llms continue to evolve.