Hidding the Ghostwriters: An Adversarial Evaluation of AI -Generated Student Essay Detection (2023)

Abstract: Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks. However, the utilization of these models carries inherent risks, including but not limited to plagiarism, the dissemination of fake news, and issues in educational exercises. Although several detectors have been proposed to address these concerns, their effectiveness against adversarial perturbations, specifically in the context of student essay writing, remains largely unexplored. This paper aims to bridge this gap by constructing AIG-ASAP, an AI-generated student essay dataset, employing a range of text perturbation methods that are expected to generate high-quality essays while evading detection. Through empirical experiments, we assess the performance of current AIGC detectors on the AIG-ASAP dataset. The results reveal that the existing detectors can be easily circumvented using straightforward automatic adversarial attacks. Specifically, we explore word substitution and sentence substitution perturbation methods that effectively evade detection while maintaining the quality of the generated essays. This highlights the urgent need for more accurate and robust methods to detect AI-generated student essays in the education domain. Code and data are released for public use.

## introduction large language models (llms) have demonstrated exceptional capabilities in various text generation tasks @xcite @xcite , raising both excitement and concerns within the research community and society at large. while these models offer tremendous potential for advancing natural language processing and understanding, their utilization also introduces inherent risks @xcite @xcite . among these risks, there are particular challenges related to educational exercises. for instance, students may rely on llms for essay writing practice, achieving high grades without developing essential writing skills. these risks highlight the importance of addressing the ethical and practical implications of llm usage in order to mitigate potential negative consequences. efforts have been made to address these concerns by developing detectors for identifying aigenerated content (aigc). the majority of existing studies in this area have focused on straightforward detection approaches @xcite . there has been limited exploration of the effect of adversarial measures on detection accuracy @xcite , especially within the education domain. for example, @xcite evaluate the robustness of detectors against character-level perturbations or misspelled words focusing on french as a case study. @xcite train a generative model (dipper) to paraphrase paragraphs to evade detection. however, none of these studies are specific to the context of student essay writing in consideration of the quality of the generated text, as well as potential adversarial measures. this paper aims to bridge this gap by presenting an adversarial evaluation of the effectiveness of aigc detectors specifically on student essays. to facilitate this evaluation, we construct the aig-asap dataset, a collection of ai-generated student essays based on the popular asap dataset foot_0 for automated essay scoring, using a variety of text perturbation methods. in addition to the existing paraphrasing perturbation method @xcite , we address the observed tendency of llms to excessively use topical words in the generated essays by developing word substitution and sentence substitution methods that perturb the ai-generated essays at the word and sentence levels, respectively. the results demonstrate that while paraphrasing a human-written essay can help evade detection to some extent, the word and sentence substitution perturbation methods significantly degrade the detection performance. remarkably, these methods achieve this while preserving the high writing quality of the generated essays. the major contributions of this research are threefold: 1) proposal of the word substitution and sentence substitution perturbation methods that can evade aigc detection while ensuring the quality of the generated essay. 2) we introduce the aig-asap dataset, which serves as a benchmark for evaluating the performance of aigc detectors in the education domain. by employing text perturbation techniques, we simulate real-world scenarios where ghostwriters or ai-powered tools are employed to produce essays unnoticeable by current detection methods. 3) we conduct empirical experiments to evaluate the performance of existing aigc detectors on the aig-asap dataset. our findings shed light on the vulnerabilities of current detection methods, revealing that they can be easily circumvented through uncomplicated automatic adversarial attacks. this highlights the pressing need for more accurate and robust detection methods tailored to the unique challenges posed by ai-generated student essays. ## problem statement artificial intelligence generated content (aigc) is defined as content that is generated by machines utilizing artificial intelligence (ai) systems, often in response to user-input prompts @xcite . in the upcoming sections, we conceptualize aigc as a sequence-to-sequence generation process. each input, denoted as @xmath0; c], comprises an instruction i and contextual demonstrations c. ## methodology this section delves into the detailed process of constructing our aig-asap dataset. our study aims to investigate the vulnerability of aigc detectors and their instability to perturbation for student essays, resulting in manipulated detection performance that decreased far from the original context. ## experiments in this section, we evaluate the performance of detectors on aig-asap to assess their accuracy and auroc in detecting ai-generated essays. ## conclusion we note that basic paraphrasing techniques like rewriting essays can decrease the effectiveness of detection, while fully automated machinegenerated content can be easily detected by current detectors. this implies that human writing plays a crucial role in evading detection in the traditional process. to address this, we introduced two distinct perturbation strategies: word substitution and sentence substitution. these strategies have shown substantial potential in deceiving aigenerated content detection while ensuring content quality in our experiments. the results highlight the inherent vulnerabilities in existing detection methods and emphasize the need for the development of more resilient and precise approaches for detecting ai-generated student essays. yilun zhao, haowei zhang, shengyun si, linyong nan, xiangru tang, and arman cohan. 2023. large language models are effective table-to-text generators, evaluators, and feedback providers. corr, abs/2305.14987.