Detecting C hat GPT : A Survey of the State of Detecting C hat GPT -Generated Text (2023)

Abstract: While recent advancements in the capabilities and widespread accessibility of generative language models, such as ChatGPT (OpenAI, 2022), have brought about various benefits by generating fluent human-like text, the task of distinguishing between human- and large language model (LLM) generated text has emerged as a crucial problem. These models can potentially deceive by generating artificial text that appears to be human-generated. This issue is particularly significant in domains such as law, education, and science, where ensuring the integrity of text is of the utmost importance. This survey provides an overview of the current approaches employed to differentiate between texts generated by humans and ChatGPT. We present an account of the different datasets constructed for detecting ChatGPT-generated text, the various methods utilized, what qualitative analyses into the characteristics of human versus ChatGPT-generated text have been performed, and finally, summarize our findings into general insights.

## introduction llms have been showing remarkable abilities in generating fluent, grammatical, and convincing text. the introduction of @xcite has been widely regarded as a significant and controversial milestone for llms. models such as gpt-3 @xcite and palm @xcite already demonstrated the power of llms in many natural language processing (nlp) tasks. chatgpt is the first model that has seen widespread adoption outside nlp research. the increased performance of llms raises important questions regarding their potential societal impact. the risks of llms are numerous, from confidently presenting false information to generating fake news on a large scale @xcite . chatgpt is no exception in this regard @xcite . instances of the misuse of chatgpt have already been documented in various domains, including education @xcite , scientific writing @xcite , and the medical field @xcite . given this context, the detection of machine-generated text is gaining considerable attention. this detection is part of a larger push towards responsible and appropriate usage of generative language models @xcite . in addition to academic interest, a growing number of commercial parties are trying to solve this task. recent work from @xcite gives an overview of commercial and freely available online tools. they come close to the current work. however, we limit our scope to academic work and provide additional background information on methods, datasets, and qualitative insights. many approaches, datasets and shared tasks foot_0 have been put forth recently to tackle the general (i.e., not specific for chatgpt) task of detecting machine-generated text @xcite . given the enormous use and cultural impact of chatgpt, we limit our review to datasets and methods developed directly for chatgpt. we discuss these methods in the context of the controversial position chatgpt is in, namely that it is a closedsource system with very little information available regarding its training setup or model architecture at the time of writing. we outline what general methods exists for this task and review recent work that directly focuses on datasets and methods for chatgpt. given the peculiar place chatgpt is in, we also consolidate qualitative insights and findings from the works we discuss that might help humans to detect chatgpt-generated text. these include lin-guistic features or writing styles to look out for. lastly, we present outstanding challenges for this detection task and possible future directions. ## related work on detecting machine-generated text llms have become a driving force in many language processing-related benchmarks and tasks @xcite @xcite . llms can solve complex nlp tasks and generate convincing and trustworthylooking text. however, they are also prone to generating false and misleading information, generally referred to as hallucinating @xcite . additionally, misuse of these models can pose significant risks in academia, journalism, and many other areas. currently, human judges are decent at spotting machine-generated text from older llms such as gpt-2 @xcite @xcite . still, the increasing abilities of llms give rise to the need for more sophisticated detection tools and models. a recent survey by @xcite provides a thorough overview of risks, approaches, and detection methods. they discuss interesting aspects such as the effect of domains on the detection task, adversarial attacks, and societal impacts of generated texts. work done by @xcite inspects the field of machine-generated text detection. it outlines three main detection methods: a classifier trained from scratch, zero-shot detection by a language model, and a fine-tuned language model as a classifier. recently, detection methods from computer vision have also been tried on language models, such as watermarking @xcite or trying to find model-identifying artifacts in generated content @xcite . to use and evaluate these methods, fine-grained access to the source model is required in training and inference time. both these preconditions are not the case with chatgpt at the time of writing. when discussing detection methods, an essential factor to consider is access to the log probability output of a model. this is the probability distribution over the vocabulary of a model for the next token to be generated. numerous successful detection methods evaluate the average log probability per token combined with a threshold in a zero-shot setting @xcite @xcite . this method is model agnostic and generally performs quite well. at the time of writing, users of chatgpt do not have access to these probabilities. without this access or knowledge about model internals, detection methods are limited to using just the generated text in a binary classification setting, with the options being human or machine. these methods use simple classifiers trained on n-grams @xcite or fine-tuned pre-trained language models @xcite @xcite . another group of detection tools we want to mention are the human-machine collaboration systems, as jawahar et al. (2020) labels them. these tools do not necessarily classify a passage directly but assist a human in making that decision. the previously mentioned work by @xcite visualizes tokens in different colors, depending on where a given token ends up in the top-k most probable tokens from the model. this can also assist a human judge in spotting which part of a larger text might be machine-generated, such as possibly rephrased or copied sections for example. as mentioned, this method requires access to output probabilities, so it is not usable for chatgpt. another tool to help humans in the detection task is to outline the linguistic properties and characteristics of machine-generated text. this was one of the main goals of the real or fake text? (roft) game created by @xcite . this game asked players to decide if a machine partially wrote a piece of text, and if yes, where the transition point from human to machine is in the text. this resulted in a considerable dataset of annotations and indicators humans look for in detecting machine-generated text. another area of research that might help humans to make this decision is explainable ai. as we will see, some papers we discuss use explainability methods, such as shap (lundberg and lee, 2017), in their approaches. these methods help to better understand how detectors make their predictions. such methods can help provide insights on the input features that most contribute to a prediction, which, in turn, can facilitate analyses of the differences between human and chatgpt writing styles. as far as we know, the previously mentioned work by @xcite and @xcite come closest to ours. they discuss detection methods and datasets but not chatgpt. the work from @xcite does mention chatgpt, among other models, but focuses mainly on online detection tools. our contributions are the following: ## conclusion and future work the impressive capabilities of chatgpt in producing high-quality and convincing text have brought attention to the risks associated with its improper usage across different domains. consequently, the reliable detection of chatgpt-generated text has become an important task. to address this concern, numerous datasets and detection methods have been proposed. in this paper, we provided a concise overview of the diverse datasets created, proposed methods, and qualitative insights of comparing human-written text with text generated by chatgpt. we see a wide variety of approaches and datasets in the papers we discussed. on the one hand, this is good to see since many factors, such as the domain, language, or format, influence the detection task. on the other hand, we also see a big diversity in experimental and dataset setups. some works use adversarial examples, and others do not. some allow the rephrasing of human text by chatgpt, while others use purely human versus machinegenerated text. some works include the prompts and chatgpt versions they used to generate the data; others do not. these, among other differences, make comparisons difficult, which is one reason we do not include scores in this survey. this also highlights important future work, namely to test methods across datasets and datasets across methods. another factor to consider is the domain of the text. the datasets we have discussed are in diverse domains and cover at least two important ones affected by chatgpt's risks: health and education. one notable domain we did not encounter is (fake) news. although this is a big nlp field on its own, we expected more attention for it in the context of chatgpt. future work can definitely help in this area. the format of the text is related to the domain and is another important factor to consider. for example, the shared tasks we mentioned provide tweets, news articles, or reviews as their formats. a systematic look at format and domain influence concerning chatgpt could be valuable future work. multilinguality is another open problem. as with virtually all nlp tasks, we have seen that english is, unfortunately, the dominant language in the datasets. experiments and gathering datasets across different languages are important future directions. the current task could also draw inspiration from the field of machine translation. it has a long and ongoing history of trying to detect (badly) translated text, so-called translationese @xcite , which could be used or adapted to detect general machine-generated text. lastly, an important factor we have not seen discussed much is the temporal aspect of chat-gpt. outputs might change over time, especially since it is a closed-source system. this calls for repeated tests over time to ensure detection methods are not regressing in their performance. machinegenerated text detection is also a cat-and-mouse game; since models are optimized to mimic human language, detection becomes harder and harder.