LLM -as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected? (2024)

Abstract: With the rapid development and widespread application of Large Language Models (LLMs), the use of Machine-Generated Text (MGT) has become increasingly common, bringing with it potential risks, especially in terms of quality and integrity in fields like news, education, and science. Current research mainly focuses on purely MGT detection, without adequately addressing mixed scenarios including AI-revised Human-Written Text (HWT) or human-revised MGT. To tackle this challenge, we define mixtext, a form of mixed text involving both AI and human-generated content. Then we introduce MixSet, the first dataset dedicated to studying these mixtext scenarios. Leveraging MixSet, we executed comprehensive experiments to assess the efficacy of prevalent MGT detectors in handling mixtext situations, evaluating their performance in terms of effectiveness, robustness, and generalization. Our findings reveal that existing detectors struggle to identify mixtext, particularly in dealing with subtle modifications and style adaptability. This research underscores the urgent need for more fine-grain detectors tailored for mixtext, offering valuable insights for future research. Code and Models are available at https://github.com/Dongping-Chen/MixSet.

## introduction the remarkable advancement of large language models (llm) has sparked global discussions on the effective utilization of ai assistants @xcite . given that llms can correctly follow human instructions and produce useful texts efficiently, more and more people prefer to integrate these powerful tools into their workflow by revising machine generated text (mgt) or using llms to polish their human written text (hwt), such as fact-checking revising in journalism @xcite and enhancing storytelling in the game industry 1 . despite its various usages, the application of llms also causes the potential risk of mgt usage, raising public concerns on various misuse, as seen in the undermining of journalistic integrity and quality @xcite , reproducing and amplifying biases @xcite , plagiarism among students (heavenarchive, 2023), and leading disruptions in trust towards scientific knowledge @xcite . the misuse of machine-generated text has been a serious concern that is also raised by experts in different fields of work foot_1 . previous studies proposed many methods to detect mgt, including metric-based and model-based methods, where they have only tried to enhance the detection ability on binary classification, i.e., pure mgt or hwt. however, they did not pay much attention to revised texts (i.e., mixtext), but considered these cases as an attack on the detection system @xcite or complex cases for detection @xcite . however, the mixture of mgt and hwt is an essential scenario in our daily lives when using llm assistants. for instance, thousands of non-native english speakers utilize llms to polish their drafts to avoid grammar problems. moreover, llms can follow human instructions to produce new stories and interactive dialogue in game design 3 . authors can also use llms to complete stories, providing them with new ideas and inspiration with llm assistants like @xcite and sparks, thereby generating metaphorical and 3 https://aicontentfy.com/en/blog/chatgpt-in-gamingindustry-enhancing-storytelling-and-interaction science writing suggestions and supporting creative writing tasks @xcite . hence, there is a pressing demand to comprehensively analyze mixture cases and give a formal definition of them. given that mixtext is a very common case in daily life and its amount continuously increases in nlp areas, it holds significant importance, especially in education. to end this, we propose a new dataset mixset, which is the first dataset that aims at the mixture of hwt and mgt, including both ai-revised hwt and humanrevised mgt scenarios as illustrated in figure 1 , which addresses gaps in previous research. further details of the dataset and definitions can be seen in section 3. we also examine our dataset on mainstream detectors in binary and three-class settings to further analyze and raise concerns about these common but hard-to-detect cases. to summarize, our work provides three main contributions: ## mixset dataset in this section, we present mixset (mixcase dataset), the first dataset featuring a blend of hwt and mgt. distinguished from earlier datasets exclusively composed of pure hwt and mgt, mixset comprises a total of 3.6k mixtext instances, and the pipeline of its construction is shown in figure 3 . these operations are grounded in real-world application scenarios, each altered by a single llm or through manual intervention, contributing 300 instances in our mixset. for our base data, we meticulously select pure hwt and mgt datasets. in the case of hwt, we gather datasets proposed before the widespread use of llms to mitigate potential contamination by mgt, as detailed in text type original resources email content enron email dataset (cmu, 2015) news content bbc news (greene, 2006) game review steam reviews (najzeko, 2021) paper abstract arxiv-10 (farhangi et al., 2022) speech content ted talk (thedatabeast, 2021) blog content blog (schler et al., 2006) ## empirical findings there is no obvious classification preference in current detectors on mixtext. in other words, the detectors do not exhibit a strong tendency to classify mixtext as either hwt or mgt. as we can observe from figure 2 and gpt-sentinel, distillbert, and finally, the chatgpt detector. among various operations, 'humanize' exhibits the poorest transfer performance in almost all scenarios. additionally, other operations also experience significant declines when dealing with 'humanize' mixtexts. this suggests that 'humanize' falls outside the current detectors' distribution of mgt, a gap that could be addressed by retraining on these specific cases. as shown in 9 it is also noteworthy that texts generated by llama2-70b demonstrate stronger transfer abilities than those generated by gpt4. increasing the number of mixtext samples in the training set effectively enhances the success rate of mixtext detection. however, adding pure text samples does not yield significant improvements and may even have a negative impact on detector performance, especially for metric-based methods. this may be attributed to subtle distribution shifts between mixtext and pure text. the current detector still faces significant challenges in capturing these subtle shifts. for mixtext scenarios, a more powerful and fine-grained detection method is needed. ## conclusion in this paper, we defined mixtext, the mixed text of human and llm-generated content. then, we proposed a dataset mixset to address the research gap in studying the mixed scenarios of machine-generated text (mgt) and human-written text (hwt). a thorough evaluation of the dataset is conducted, performing binary, three-class, and transfer experiments on mainstream detectors. the results underscore the complexities inherent in identifying mixtext, indicating the challenge of distinguishing the subtle differences in mixtext. as a result, there is a need for more robust and finegrained detection methods.