A Survey on Detection of LLM s-Generated Content (2024)

Abstract: The burgeoning capabilities of advanced large language models (LLMs) such as ChatGPT have led to an increase in synthetic content generation with implications across a variety of sectors, including media, cybersecurity, public discourse, and education. As such, the ability to detect LLMs-generated content has become of paramount importance. We aim to provide a detailed overview of existing detection strategies and benchmarks, scrutinizing their differences and identifying key challenges and prospects in the field, advocating for more adaptable and robust models to enhance detection accuracy. We also posit the necessity for a multi-faceted approach to defend against various attacks to counter the rapidly advancing capabilities of LLMs. To the best of our knowledge, this work is the first comprehensive survey on the detection in the era of LLMs. We hope it will provide a broad understanding of the current landscape of LLMs-generated content detection, and we have maintained a website to consistently update the latest research as a guiding reference for researchers and practitioners.

## introduction with the rapid development of powerful ai tools, the risk of llms-generated content has raised considerable concerns, such as misinformation spread @xcite @xcite , fake news @xcite @xcite , gender bias @xcite , education @xcite , and social harm @xcite . we also find on the google search trend, that the concerns about ai-written text have witnessed a significant increase since the release of the latest powerful large langue models (llms) such as chatgpt @xcite and gpt-4 * work done partly during the internship at nec laboratories america. @xcite . humans are already unable to directly distinguish between llms-and humanwritten text, with the fast advancement of the model size, data scale, and ai-human alignment @xcite . concurrently, growing interests are shown to detectors, like the commercial tool gptzero @xcite , or ope-nai's own detector (openai, 2023a) since humans can be easily fooled by improvements in decoding methods @xcite . however, the misuse of detectors also raises protests from students on the unfair judgment on their homework and essays @xcite and popular detectors perform poorly on code detection @xcite . alongside these advancements, there has been a proliferation of detection algorithms aimed at identifying llms-generated content. however, there remains a dearth of comprehensive surveys encompassing the latest methodologies, benchmarks, and attacks on llms-based detection systems. earlier work on text detection dates back to feature engineering @xcite . for example, gtlr @xcite assumes the generated word comes from the top distribution on small lms like @xcite or gpt-2 @xcite . recently, there has been an increasing focus on detecting chatgpt @xcite @xcite , to mitigate chatgpt misuse or abuse @xcite . in particular, it has recently been called for regulation 1 on powerful ai like chatgpt usage @xcite . therefore, we firmly believe that the timing is ideal for a comprehensive survey on the detection of llms-generated content. it would serve to invite further exploration of detection approaches, offer valuable insights into the strengths and weaknesses of previous research, and highlight potential challenges and opportunities for the research community to address. our paper is organized as follows: we first briefly describe the problem formulation, including the task definition, metrics, and datasets in section 2. in section 3, we classify detection by their working mechanism and scope of application. in section 4, we summarize the three popular detection methods: training-based, zero-shot and watermarking. we also investigate various attacks in section a.2 since defending against attacks is of increasing importance and point out some challenges in section a.3. finally, in section 5 we provide additional insights into this topic on potential future directions, as well as the conclusion in section 6. ## detection scenarios the findings of previous research, such as @xcite and @xcite , highlight the general difficulty faced by humans in distinguishing between human-and machinegenerated text, motivating the development of automatic solutions. the detection process can be classified into black-box or white-box detection based on whether the detector has access to the source model 2 https://sharegpt.com/ black-box known sourcing white-box full access black-box unknown sourcing white-box partial access increasing transparency increasing transparency increasing difficulty increasing difficulty watermark training-based zero-shot three categories of detectors output logits. in black-box detection, there are two distinct cases: 1). when the source model name is known, such as gpt-4; 2). when the source model name is unknown, and the content might have been generated by models like gpt-4, bard, or other undisclosed models. on the other hand, white-box detection also encompasses two cases: 1). the detector only has access to the model's output logits or partial logits, such as the top-5 token log probability in text-davinci-003; 2). the detector has access to the entire model weights. ## detection methodologies in this section, we delve into further details about the detection algorithms. based on their distinguishing characteristics, existing detection methods can be categorized into three classes: 1) training-based classifiers, which typically fine-tune a pre-trained language model on collected binary data -both human and ai-generated text distributions. 2) zeroshot detectors leverage the intrinsic properties of typical llms, such as probability curves or representation spaces, to perform self-detection. 3) watermarking involves hiding identifying information within the generated text that can later be used to determine if the text came from a specific language model, rather than detecting ai-generated text in general. we summarize the representative approaches in figure 1 as classified by the scenarios listed in section 3. ## attack, challenges, future outlook the detection of llm-generated content is an evolving field. detection attacks can be found in appendix a.2 and we also summarize the challenges in appendix a.3. additionally, we list some potential avenues for future work (details are included in appendix a.8): 1). robust and scalable detection techniques; 2). rigorous and standard evaluation; 3). fine-grained detection; 4). user education and awareness; 5). transparency and explainability. ## conclusion we comprehensively survey llms-generated content detection over existing task formulation, benchmark datasets, evaluation metrics, and different detection methods to help the research community quickly learn the progress in this field.