RAID : A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors (2024)

Abstract: Many commercial and open-source models claim to detect machine-generated text with extremely high accuracy (99% or more). However, very few of these detectors are evaluated on shared benchmark datasets and even when they are, the datasets used for evaluation are insufficiently challenging—lacking variations in sampling strategy, adversarial attacks, and open-source generative models. In this work we present RAID: the largest and most challenging benchmark dataset for machine-generated text detection. RAID includes over 6 million generations spanning 11 models, 8 domains, 11 adversarial attacks and 4 decoding strategies. Using RAID, we evaluate the out-of-domain and adversarial robustness of 8 open- and 4 closed-source detectors and find that current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models. We release our data along with a leaderboard to encourage future research.

## introduction large language models (llms) have been able to fool humans into thinking their outputs are humanwritten for roughly four years @xcite . in that short span of time we have seen llm-generated text be used for targeted phishing attacks @xcite , mass spam and harassment @xcite , disinformation campaigns @xcite , and spurious scientific publication @xcite . in order to document and eventually mitigate such harms, we must develop robust automatic detectors of machine-generated text. many exciting and inventive methods have been proposed in recent years for detecting generated text @xcite . however, when evaluating these methods, authors typically generate detect ai: ai-generated ## related work in ## conclusion as the generation capabilities of language models have continued to increase, accurately and automatically detecting machine-generated text has become an important priority. detection efforts have even surpassed the bounds of natural language processing research, spurring discussions by social media companies and governments on possibly mandating labels for machine-generated content. despite the protective intentions of these mandates, our work shows that such regulations would be difficult to enforce even if they were implemented. detectors are not yet robust enough for widespread deployment or high-stakes use: many detectors we tested are nearly inoperable at low false positive rates, fail to generalize to alternative decoding strategies or repetition penalties, show clear bias towards certain models and domains, and quickly degrade with simple black-box adversarial attacks. the bulk of our findings may sound bleak, but we did uncover promising signs of improvement. binoculars, for example, performed impressively well across models even at extremely low false positive rates, originality achieved high precision in some constrained scenarios, and gptzero was unusually robust to adversarial attacks. we believe that openly evaluating detectors on large, diverse, shared resources is critical to accelerating progress-and trust-in detection. evaluating robustness is particularly important for detection, and it only increases in importance and the scale of public deployment grows. we also need to remember that detection is just one tool for a larger, even more valuable motivation: preventing harm by the mass distribution of text. detecting machine-generated text was a useful proxy for identifying harmful text for a long time, but language models have improved to the point that generated text is frequently legitimate and not harmful @xcite . therefore, detecting specific harmful elements-like misinformation, hate speech, and abuse-should take precedence over whether or not the text was authored by a machine. knowing if a text was machinegenerated, however, does still offer insights on the types of errors we can expect or the recency of the facts cited within. we hope that our analyses and the raid dataset are a step toward a future in which ai detection tools are safely integrated into society as a multi-pronged approach to reducing harm. we encourage future work to build on this by including more models, languages, and generation settings in future shared resources.