Towards a Robust Detection of Language Model-Generated Text: Is C hat GPT that easy to detect? (2023)

Abstract: Recent advances in natural language processing (NLP) have led to the development of large language models (LLMs) such as ChatGPT. This paper proposes a methodology for developing and evaluating ChatGPT detectors for French text, with a focus on investigating their robustness on out-of-domain data and against common attack schemes. The proposed method involves translating an English dataset into French and training a classifier on the translated data. Results show that the detectors can effectively detect ChatGPT-generated text, with a degree of robustness against basic attack techniques in in-domain settings. However, vulnerabilities are evident in out-of-domain contexts, highlighting the challenge of detecting adversarial text. The study emphasizes caution when applying in-domain testing results to a wider variety of content. We provide our translated datasets and models as open-source resources.

## introduction advances in natural language processing (nlp) have been driven mainly by scaling up the size of pre-trained language models, along with the amount of data and compute required for training @xcite @xcite @xcite . openai recently released chatgpt, a text generation model with conversational capabilities. the model is based on gpt3.5 which is a version of gpt3 @xcite first fine-tuned on code then fine-tuned using reinforcement learning from human feedback (rlhf) @xcite , a method previously demonstrated by openai with instructgpt @xcite . this fine-tuning process contributes not only to the model's knowledge but also simplifies the model's interface compared to gpt3, which necessitated substantial prompt engineering to achieve satisfactory outcomes, and hence facilitating the extraction and application of that built-in knowledge. as a result of these significant performance improvements, chatgpt and other large language models have gained much popularity in the media and in the social context, often without fully understanding the underlying limitations of the models -e.g., the possibility of generating hateful, hateful, toxic, or disrespectful content @xcite @xcite . another potential misuse of llms or chatgpt is industrializing radicalization and harmful propaganda which poses a significant and unconventional threat to civil society. in response to the mounting concerns surrounding potential misuse, numerous researchers are now exploring various strategies to mitigate associated risks. for example, some have proposed watermarking techniques to trace the origin of generated text foot_0 , while others are developing methods to detect and flag text generated by these models. of particular interest, a recent study by @xcite investigated the text generation capabilities of chatgpt and its proximity to human-generated text. to create a dataset of chatgpt-generated text, the authors leveraged pre-existing questionanswering datasets in both english and chinese, using the questions as prompts to generate responses from the model. in addition, the authors conducted a linguistic analysis to compare the output generated by chatgpt with human-written text, and they also developed a detector to distinguish between chatgpt-generated text and human-written text by fine-tuning a separate language model on a dataset containing both types of text. the aim of this research is to explore the development of chatgpt detectors in multiple languages, along with evaluating their robustness on out-of-domain text, we selected french as the language of interest. therefore, we propose a methodology that involves translating the english dataset into french and subsequently training a classifier on the translated data. we conducted a series of evaluations in a monolingual and multilingual setting on both in-domain and out-of-domain data. the in-domain data consisted of text generated by chatgpt using prompts related to the topics covered in the training dataset. the out-of-domain data included text generated in french by chatgpt and bing, a search engine powered by chatgpt foot_1 , which has access to a broader range of internet content and may generate text on a wider range of topics than chatgpt. given that @xcite demonstrated the vulnerability of bert-based detectors for gpt-2 against basic attack schemes, such as substituting characters with homoglyphs or misspelled words, we also evaluated the robustness of our models against these types of attacks. furthermore, we hypothesize that the detector models we trained rely heavily on the didactic response style of chatgpt to distinguish between human-generated content. the contributions of this study can be summarized as follows: overall, our work contributes to the growing body of research on developing and evaluating chatgpt detectors, with a focus on multilinguality, generalizability, and robustness to attacks. our findings have practical implications for the use of chatgpt detectors in various settings, including social media platforms, online forums, and chatbots, where the detection of harmful content is critical for maintaining a safe and respectful online environment. ## methodology 3.1 data collection to train and evaluate our chatgpt detectors, we leveraged the human chatgpt comparison corpus (hc3) created by @xcite which contains both human-written and chatgpt-generated text in english and chinese. we primarily focus on the english portion of the dataset as machine translation performs optimally on it. the dataset consists of 24,322 human-written questions and 58546 answers sourced primarily from eli5 (fan et al., 2019), wikiqa @xcite , crawled wikipedia, medical dialog dataset (chen et al., 2020), and fiqa @xcite . the authors generated chatgpt responses using openai's web application, foot_2 automating the input of questions and scraping the answers with the help of automation testing tools, for a total of 26903 chatgpt-generated answers. to create a french dataset, we translated the english dataset using the google cloud translation api. we then split the dataset into three splits train, validation, and test, by first selecting 710 balanced question and answer pairs to be validated, manually annotated foot_3 and to serve as our test set. we split the rest in an 80/20 split to get the training and validation set. furthermore, to assess the chatgpt detectors' ability to generalize, we manually compiled out-ofdomain test data by means of: ## conclusion in conclusion, this paper proposed a methodology for developing and evaluating chatgpt detectors in multiple languages, focusing on french as a case study. the proposed method involved translating an english dataset into french and training a classifier on the translated data. the results demonstrate that the proposed method can effectively detect chatgpt-generated text, with a certain degree of robustness against basic attack techniques, albeit exclusively within the in-domain setting. however, the detectors display evident vulnerabilities in out-of-domain contexts, emphasizing the importance of considering different writing styles in training language models. additionally, the study highlights the significant challenge of detecting adversarial text, which even openai's detection methods have difficulties with. the key takeaway is that caution should be exercised when applying in-domain testing results to a wider variety of content. we provide open-source resources to further advance research in this and are currently working to extend the adversarial dataset to better understand the limitations of these models.