M 4 GT -Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection (2024)

Abstract: The advent of Large Language Models (LLMs) has brought an unprecedented surge in machine-generated text (MGT) across diverse channels. This raises legitimate concerns about its potential misuse and societal implications. The need to identify and differentiate such content from genuine human-generated text is critical in combating disinformation, preserving the integrity of education and scientific fields, and maintaining trust in communication. In this work, we address this problem by introducing a new benchmark based on a multilingual, multi-domain and multi-generator corpus of MGTs — M4GT-Bench. The benchmark is compiled of three tasks: (1) mono-lingual and multi-lingual binary MGT detection; (2) multi-way detection where one need to identify, which particular model generated the text; and (3) mixed human-machine text detection, where a word boundary delimiting MGT from human-written content should be determined. On the developed benchmark, we have tested several MGT detection baselines and also conducted an evaluation of human performance. We see that obtaining good performance in MGT detection usually requires an access to the training data from the same domain and generators. The benchmark is available at https://github.com/mbzuai-nlp/M4GT-Bench.

## introduction the advent of large language models (llms) such as chatgpt 1 and gpt-4 marks a transformative era in text generation. these models are able to generate a coherent text that is very similar to human-written content. they are also easily accessible and becoming more and more widespread. while such tools significantly boost productivity across various fields such as journalism, social media, education, and academic writing, they also 1 we refer to gpt-3.5-turbo throughout this paper. llms can generate immense amounts of deceptive fake news cluttering the information space. in social media, they have the potential to automatically generate fake accounts and increase their influence in social media communities without much effort from their owners. in the academic context, these technologies may advance beyond mere replication of existing content, potentially diminishing the effectiveness of established plagiarism detection methodologies and, consequently, posing a threat to the fundamental tenets of scholarly integrity. as a result, it could impact academic writing by leading to an increase in publications of automatically generated papers lacking scientific merit in respected journals and conferences. finally, the scientific community has started reporting on cases of potentially automatically generated reviews in some well-known conferences. studies have shown that humans perform only marginally better than random chance in distinguishing human-written from machine-generated texts (mgts) @xcite . therefore, automatic mgt detection becomes essential in tackling misinformation, preserving the integrity of digital platforms and the scientific community, and ensuring trust in communications. previous work typically frames mgt detection as binary classification @xcite @xcite , focusing primarily on english. moreover, the majority of studies also overlook the fact that the content can be a mixture of an mgt and human-written text. these limitations make the previous experimental setups significantly different from practice. in this work, we address these issues by presenting a new benchmark for mgt detection that consists of three tasks, which were not previously explored in this exact formulation, each shedding light on different facets of this challenge. machine-generated text classification. the objective is to categorize a given text as either human-written or machine-generated. this task is similar to the problem formulations in the previous work. however, our dataset offers annotated resources of new domains and generators in multiple languages. task 1 sets two distinct tracks, one focusing exclusively on english sources (monolingual), and the other embracing a multilingual scope and introducing greater diversity compared to the previous work. our contributions are summarized below: ## datasets and metrics this section describes the datasets for each task and the details of their creation. the data statistics is shown in ## human evaluation we investigate the complexity of differentiating the language model used for creating machinegenerated texts for human readers. considering that annotators may lose patience to learn and summarize the patterns of seven distinct generators, we control the number of unique models they could identify and simplify to four-class classification with five demonstration examples (five-shot) as practice to learn before the formal annotation. data for human evaluation there are 35 unique domain-generator combinations -five domains × seven generators (six llms plus humanwritten texts) for task 2 data. with the concern of annotators' workload, we sample 4 examples for each combination, in total of 140 examples, split into four groups as figure 1 . without consideration of out-of-domain classification, we follow the best-performing setup: see all, classify all. if each annotator needs to check all domains, they need to learn 20 demonstration examples, which is a time consuming and laborious undertaking. let us consider that each annotator annotates 3 domains. they would need to check 4 models × 3 domains × 4 @xmath0, which is five demonstration examples are randomly selected, to ensure that four classes are all included. annotator background. group 1, 2, 3, and 4 are annotated by native italian, chinese, english, and russian speakers, respectively; they are either postdoc or phd students in nlp. results intuitively, it would be challenging to distinguish between generators based on 1-shot per class. annotators may make decisions depending on the observations that generations by cohere tend to be repetitive, dolly-v2 has high-quality and coherent texts, while bloomz texts are more likely to contain a bunch of numbers or a bunch of "yes" or "no" characters repetitively at the end of the texts, compared to other models. for gpt series, annotators distinguish them depending on their quality level assumption that gpt-4 > @xmath1. they spot human-written texts based on formatting patterns (e.g., initial double newlines, initial space, completely missing new lines in the paragraph), typos, inconsistencies within texts, or specific references and urls. as shown in ## experiments we conduct an experimental evaluation with several common baselines representing a diverse set of detection techniques: standard fine-tuning of an encoder-based transformer model, elaborated feature engineering for the similar task, ranking-based features which form patterns for different llms. for classification tasks 1 and 2, we use five baseline classifiers suggested by @xcite . they include fine-tuned roberta and xlm-r classifiers, logistic regression with gltr features (lr-gltr) @xcite , svm with stylistic (stylistic-svm), and svm with nela features (nela-svm) @xcite . for the boundary detection task 3, we applied two detectors based on semantic features. gltr features these features are based on the assumption that in order to generate fluent and natural-looking text, most llm decoding strategies sample high-probability tokens from the head of the distribution @xcite . thus, word ranking information of an lm can be used to distinguish machine-generated texts from humanwritten. following @xcite , we select two categories of these features: (1) the number of tokens in the top-10, top-100, top-1000, and 1000+ ranks from the lm predicted probability distributions (4 features); and (2) the frac(p) distribution over 10 bins ranging from 0.0 to 1.0 (10 features). frac(p) describes the fraction of probability for the actual word divided by the maximum probability of any word at this position. we train a logistic regression model based on these 14 features to perform the binary classification task with the parameter of the maximum @xmath2,000. stylistic and nela features we use stylistic features including (1) character-based features, e.g., the number of characters, letters, special characters, etc., (2) syntactic features, e.g., a number of punctuation and function words, (3) structural features, e.g., a total number of sentences, and (4) word-based features, e.g., a total number of words, average word length, average sentence length, etc. news landscape (nela) features (horne et al., 2019) involves six aspects: (1) style: the style and the structure of the article; (2) complexity: how complex the writing is; (3) bias: overall bias and subjectivity; (4) affect: sentiment and emotional patterns; (5) moral based on the moral foundation theory @xcite , and (6) event: time and location. an svm classifier is applied to perform the binary distinction. ## conclusion and future direction in this work, we extend m4 into a new benchmark m4gt-bench for mgt detection, providing a wide coverage of nine languages, six domains, and nine state-of-the-art generators, including gpt-4 and llama-2 series. we formulate three tasks, with a novel formulation of detecting the changing point from human-written to machine-generated. we observe that human readers are unable to consistently differentiate between generators and operate merely at the level of making random guesses. for all tasks, we find that the detector usually suffers a severe penalty in their performance when they are challenged with unseen domains or generators. some generators such as bloomz and domains like arxiv appear harder than others. overall, transformer-based detectors such as roberta and xlm-r are usually the best on average. in future work, we plan to develop a demonstration system assisting human to detect mgt, especially in some high-stake domains such as clinical, financial and legal domains, and distinguish the authorship of different llms, promoting the protection of intelligence property. in addition to text, we would extend to other modalities detection including speech, image and videos. from the perspective of adversarial attacks such as paraphrasing, it is worthwhile to explore mixed texts with multiple changing points.