Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Authors: Xinlin Peng
Ying Zhou
Ben He
Le Sun
Yingfei Sun
Venue: EMNLP 2023 Main (2023)
Keywords: AIGC detection
AI-generated student essay
education

Abstract: Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks. However, the utilization of these models carries inherent risks, including but not limited to plagiarism, the dissemination of fake news, and issues in educational exercises. Although several detectors have been proposed to address these concerns, their effectiveness against adversarial perturbations, specifically in the context of student essay writing, remains largely unexplored. This paper aims to bridge this gap by constructing AIG-ASAP, an AI-generated student essay dataset, employing a range of text perturbation methods that are expected to generate high-quality essays while evading detection. Through empirical experiments, we assess the performance of current AIGC detectors on the AIG-ASAP dataset. The results reveal that the existing detectors can be easily circumvented using straightforward automatic adversarial attacks. Specifically, we explore word substitution and sentence substitution perturbation methods that effectively evade detection while maintaining the quality of the generated essays. This highlights the urgent need for more accurate and robust methods to detect AI-generated student essays in the education domain. Code and data are released for public use.

Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Xinlin Peng Ying Zhoutl.2 Ben He* Le Sun and Yingfei Sun*!
'University of Chinese Academy of Sciences, Beijing, China 2 Institute of Software, Chinese Academy of Sciences, Beijing, China pengxinlin22, zhouying20@ mails.ucas.ace ucas.ac.cn, benhe, yfsun @ ucas.ac.on, sunle @ iscas.ac.cn
Abstract
For instance, students may rely on LLMs for es- say writing practice, achieving high grades without developing essential writing skills. These risks highlight the importance of addressing the ethical and practical implications of LLM usage in order to mitigate potential negative consequences.
Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks. However, the utilization of these models car- ries inherent risks, including but not limited to plagiarism, the dissemination of fake news, and issues educational exercises. Although sev- eral detectors have been proposed to address these concerns, their effectiveness against ad- versarial perturbations, specifically in the con- text of student essay writing, remains largely unexplored. This paper aims bridge this gap by constructing AIG-ASAP, student essay dataset, employing a range of text perturbation methods that are expected gener- ate high-quality essays while evading detection. Through empirical experiments, we assess the performance of current AIGC detectors the AIG-ASAP dataset. The results reveal that the existing detectors can be easily sircumvented using straightforward automatic adversarial at- tacks. Specifically, we explore word substi- tution and sentence substitution perturbation methods that effectively evade detection while maintaining the quality of the generated essays. This highlights the urgent need for more accu- rate and robust methods to detect Al- Al-generated student essays the education domain. Code and data are released for public use!.
Efforts have been made to address these con- cerns by developing detectors for identifying AI- generated content (AIGC). The majority of existing studies in this area have focused on straightforward detection approaches (Liu et al., 2023b,a; Guo al., 2023). There has been limited exploration the effect of adversarial measures on detection accu- racy (Susnjak, 2022; Liang et al., 2023), especially within the education domain. For example, Antoun et al. (2023) evaluate the robustness of detectors against character-level perturbations misspelled words focusing on French as case study, Krishna et (2023) train a generative model (DIPPER) paraphrase paragraphs to evade detection. How- ever, none of these studies are specific to the con- text of student essay writing in consideration of the quality of the generated text, as well as potential adversarial measures.
Al-generated
This paper aims to bridge this gap by present- ing adversarial evaluation of the effectiveness of AIGC detectors specifically on student essays. To facilitate this evaluation, we construct the AIG- ASAP dataset, a collection Al-generated student essays based on the popular ASAP dataset? for automated essay scoring, using variety of text perturbation methods. In addition to the existing paraphrasing perturbation method (Sadasivan et al., 2023), we address the observed tendency of LLMs to excessively use topical words in the generated es- says by developing word substitution and sentence substitution methods that perturb the Al-generated essays at the word and sentence levels, respectively. The results demonstrate that while paraphrasing a essay can help evade detection to
1 Introduction
Large language models (LLMs) have demonstrated exceptional capabilities in various text generation tasks (Zhao et al., 2023; Wu et al., 2023; Li et al., 2023), raising both excitement and concerns within the research community and society large. While these models offer tremendous potential for advanc- ing natural language processing and understand- ing, their utilization also introduces inherent risks (Chen et al., 2023; Liu et al., 2023a; Sadasivan et al., 2023). Among these risks, there are par- ticular challenges related to educational exercises.
Equal contribution. hhttp::
thttps///www.. con/ c/asapaas//

---

some extent, the word and sentence substitution perturbation methods significantly degrade the de- tection performance. Remarkably, these methods achieve this while preserving the high writing qual- ity of the generated essays.
into account various linguistic and statistical fea- tures of the input text.
AIGC Detection Attack. AIGC detection attack refers to deceiving or evading AIGC detection through carefully crafted prompts (Lu et al., 2023) or paraphrasing attacks (Sadasivan et al., 2023), which aims exploit vulnerabilities or limitations in the detection algorithms evade detection mislead the system into classifying Al-generated content as human-generated or vice versa. Given an AIGC detection model P with parameters 6, an at- tack on the AIGC detector aims to find an adversar- ial input X that minimizes the detection probabil- ity of AIGC content P(AIGC X'; 0) while max- imizing the probability of the content being classi- fied human-generated P uman X);
The major contributions of this research threefold: 1) Proposal of the word substitution and sentence substitution perturbation methods that can evade AIGC detection while ensuring the quality of the generated essay. 2) We introduce the AIG- ASAP dataset, which serves as a benchmark for evaluating the performance of AIGC detectors in the education domain. By employing text perturba- tion techniques, simulate real-world scenarios where ghostwriters Al-powered tools em- ployed produce essays unnoticeable by current detection methods. 3) We conduct empirical ex- periments to evaluate the performance of existing AIGC detectors on the AIG-ASAP dataset. Our findings shed light on the vulnerabilities of cur- rent detection methods, revealing that they can be easily circumvented through uncomplicated auto- matic adversarial attacks. This highlights the press- ing need for more accurate and robust detection methods tailored to the unique challenges posed by AI- Al-generated student essays.
3 Methodology
This section delves into the detailed process of con- structing our AIG-ASAP dataset. Our study aims to investigate the vulnerability of ATGC detectors and their instability to perturbation for student essays, resulting in manipulated detection performance that decreased far from the original context.
3.1 Data Construction
2 Problem Statement
Similar to the approach by Liu et al. (2023b), we create AIG-ASAP, a dataset of machine-generated essays, based on the existing ASAP dataset. ASAP is a well-established benchmark dataset utilized for evaluating automated essay scoring systems. It consists of essays written by high school students in the United States, covering eight different essay topics/prompts. Each essay in the dataset is accom- panied by a human-assigned holistic score. We leverage the ASAP dataset and provide the LLM with the essay prompt to generate machine-writte essays. The prompts serve as instructions or guide- lines for the LLMs to generate essays in trolled manner, as introduced in the following. Instruction-based Writing. In recent research (Shyr et al., 2023; Qasem et al., 2023; Huang et al., 2023), has been demonstrated that LLMs like GPa exhibit strong language and logic abili- ties. To explore the potential of LLMS in content generation, employ the original essay prompts in the ASAP dataset as input for the language model, considering the entirety of the LLM's out- put as the generated essay. In other words, we utilize the direct machine writing approach, where we solely provide the original essay topic require-
Artificial Intelligence Generated Content (AIGC) is defined as content that is generated by machines uti- lizing artificial intelligence (AI) systems, often in response to user-inpur user-input prompts (Wu et al., 2023). In the upcoming sections, we conceptualize AIGC as a seatuence-t--equeeee generation process. Each input, denoted as = [T: C). comprises an instruc- tion / and contextual demonstrations C'.
Detection of AIGC. With the recent advances of LLMs, AIGC has become increasingly proficient in mimicking human-generated content. Conse- quently, the detection of AIGC, which involves the task of differentiating between content generated by machines and content created by humans, has emerged as a significant area of research. Formally, the detection of AIGC can be defined as follows: Given piece of textual content, represented as a se- quence of tokens denoted by X [21, where i; represents the i-th token, the task of AIGC detection aims to determine the probability that the content was generated by an AT system, denoted as P(AGGC X; 6). The detection model parame- ters are trained to predict this probability, taking
42, 12, J.,,,

---

ments the instruction without including any contextual information for C.
pairs be 0.677 in the Al-generated dataset and 0.617 in the human-written dataset. These values indicate a reasonable level of diversity among the AI Aeggenerte essays. Moreover, it is also observed that the machine-gentn -eeerrrte text after refining stu- dent essays exhibits higher similarity compared to collection of human essays. This suggests that LLMs tend to utilize similar structures expres sions more frequently than humans.
Refined Writing. In this data construction sce- nario, we provide the student-written essay as the reference for the LLMs to polish. By exposing the model human-crafted essays, we strive to main- tain control over factors such length, format, and wording in the output of the LLM. This enables us to generate articles that are more in line with the writing style of students. We utilize the following sentence as instruction and the original full essay (referred to as ex) as the context C.
Polish and optimize the following essay: Ez-
7
Instruction-bisee bucciiio-bbsedd Generated
Humon Writton
Refined Generated
Normal Distrbution
Continuation Writing, Drawing inspiration from the pre-training paradigm of LLMs, we also adopt a left-to-right generation approach for the essay- generating task. To facilitate this, we offer the LLMs the initial first sentence of the ASAP es- say (denoted Cinit) along with the composition requirements the prompt, allowing the model generate coherent and contextually appropriate essays based on the given input. Specifically, we consider the following prompt design:
Probabiiity Donsty
3
1-
0.3
0.4
0.3
0.6
0.7
0.8
0.9
1.0
Similarity Malue
Figure The SimHash probability distribution writtew and ChaiGPT-generated essays, with normal distribution curve fitted to the data points of the human-written data.
Follow the first couple of sentences from an essay to write follow-up paragraph: einit-
Examples of the prompts used in this work can be found in Appendix B.
LLMs have a tendency to excessive use of topical words. An empirical observation we made is that the LLMs often exhibit a tendency to repeatedly utilize words prompted in the essay topic while generating essays. For instance, when examining set of 1,000 essays generated by ChatGPT using instruction-writing for ASAP topic 1, we find that the average frequency of occurrences of the topical word "technology" in each generated essay 1.927. which is significantly higher than the average fre- quency of occurrences of 0.496 each original human-written essay. This indicates that the LLMs tend to excessively employ topical words in their generated output. This excessive use of topical words opens up the possibility of evading detection by substituting these frequently used words with similar words synonyms, without significantly degrading the quality of the generated essays. In the subsequent section, we will introduce the word and sentence substitution methods that explore this possibility and its impact on detection.
3.2 Dataset Analssis
The generated essays exhibit a decent level of di- versity. To assess the level of diversity in the gener- ated essays, we conduct an analysis to measure the text overlap between pairs of essays. Specifically, we randomly select 100 essays for each topic and compute the SimHash similarity (Charikar, 2002) between each essay and all remaining essays for that topic. The histogram presented Figure provides insights into the distribution of SimHash similarities for essay pairs across the topics, com- paring machine-geneen generated and human- human-written written es- says. The histogram shows that the similarity val- ues of both types of essays roughly follow normal distribution, spanning range from 0.38 to 0.94, There is a moderate shift in the mode of the distribu- tions, suggesting some differences in text overlap between the two types of essays. Furthermore, we calculate the average SimHash similarity of essay

---

3.3 Perturbation Methods
ASAP perturbation, the output text may exhibit greater coherence, mimicking the natural writing style of human authors and making it less distin- guishable from genuine numan-authorre content. To guide the rewriting process of LLMs on human- written essays, given an essay e, from the ASAP dataset which is originally written by a human, we have designed prompt templates follows:
Recent research (Krishna et al., 2023; Sadasivan et al., 2023) demonstrate that simple paraphrasing Al-generated text, regardless of whether it origi- nates from the same model or different models, can remarkably evade the detector. Therefore, to eval- uate the stability of the current Al-generated text detector, we design and employ a range of text per- turbation methods
constructed AIG-ASAP dataset. Our study encompasses the following three methods: Essay Paraphrasing. Word Substitution, and Sentence Substitution. Within this set of meth- ods, paraphrasing has been previously applied to the open-ended generation and long-form question answering. Empirical results indicate that the para- phrasing perturbation can be readily identified by detector (Krishna al., 2023). However, its applicability student essay writing is limited due to the requirement of large set of machine-generated and human-written essays for comparison for each essay topic. In addi- tion to paraphrasing, we design Al-generated text perturbation methods with two different granulari- ties: word, and sentence. In essence, our perturba- tion method primarily concentrates on altering the distribution of the original generative model's out- put while preserving the quality of the essay, which aims to decrease the final detection performance to evade current detectors.
Please rewrite the essay and imitate its word using habits: ex. Try to be different from the original text.
It is worth highlighting the intuitive similarity between refined writing and essay paraphrasing, they both involve altering the distribution of the original oulput affect the detection results. Nev- ertheless, our evaluation results reveal significant distinctions between the two approaches, which we will elaborate on in the experimental section.
BM25 vetrieve-then-compee
3.3.2 Sentence Substitution
At the sentence level, we utilize a different and smaller generative model FLAN-TS (Chung et al., 2022) to perform text replacement, in order to in- troduce diversity in the LLM-generated essays. For each LLM-generated essay, randomly select set of sentences and replace them with the mask token. Afterward, we employ the generative model to generate replacements for the masked segments. By incorporating random sentence replacements, we simulate a scenario where users manually mod- ify certain parts of the Al-generated text with the assistance of generative model. This enables to assess the effectiveness of the generative model in reducing detection accuracy and emulating the writing process of human users. Formally, given an Al-generated essay Eg. a random masker Rt and a smaller generative model Myens we obtain the perturbed essay as follows:
3.3.1 Essay Paraphrasing
According to Sadasivan et al. (2023), leveraging LLMs to rewrite the human-written text pre- existing Al-generated text has the potential to dis- rupt the distribution of LLM outputs and the syn- tactic structure the essay. Consequently, this can lead to a decreased probability of detection.
Previous research (Lu al., 2023) has explored the use of LLMs to paraphrase content generated by other LLMs. However, our preliminary inves- tigation suggests that this approach has minimal impact on both the writing quality and the detec- tion rate of the generated essays. As result, in this study, we shift our focus to rewriting genuine human-written essays instead, in order to more ac- curately replicate the writing process that students typically undergo. Essentially, to avoid detection, users often manually complete low-quality draft composition with the assistance of the language model and then utilize the model to enhance the ar- ticle. By incorporating these aspects into the AIG-
ei.ez 2 .... of MMgn (R(eg))
(1)
3.3.3 Word Substitution
As aforementioned, we observe that LLMs tend to rely more on the information provided in the instructions compared to human writers when gen- erating essays. Notably, the content generated by ChatGPT exhibits significantly higher usage of words mentioned in the prompts compared human written essays. This discrepancy in writ- ing behavior between LLMs and humans could

---

Algorithm 1 Word Substitution
candidate, we select the top-1 word based on the BERT prediction score.
Essay topic instruction Z, Al-generated essay 6, knowledge base for synonyms iCs, smaller perturbation model M
Input:
We have introduced three methods for perturb- ing essays, each contributing to perturbation ap- proach that ranges from meare-ggainned to fine- grained techniques. The first method we employ op- erates coarse-grained level, involving full-text replacement through paraphrasing. By rewriting the entire content, we are able introduce substan- tial changes to the original essay while maintain- ing its overall structure and coherence. Moreover, we take a finer-grained approach by focusing on the manipulation of individual sentences. By se- lecting and modifying specific sentences, we can introduce subtle changes to the article's content and meaning. Lastly, based of the distinct writing styles between humans and LLMs, we introduce to operate at a more granular level, targeting word substitution. By analyzing high-frequency words and utilizing a perturbation model, we selectively replace specific words in the essay with suitable alternatives. This approach en- ables more nuanced perturbations while preserving the original structure and context of the essay.
Parameter: number of words to be substituted, p top predictions generated by M,, and n maximal synonyms queried from Kc
Output: Essay Csub with at most h substituted words
1: Stage 1: High-frequency Words Sampling
2: Initialize Wh. to store high-fiu eigh-frequency words:
Wh. < 0
3: while [Wal= < k do
4: w Sample high-f feghffreueency word from the instruction I or the essay e.
5: if w not Wj, then
observations
6: Add w to Wh.
7: end if
8: end while
9: Stage 2: Word Substitution with KC, and M
for w in W/, do
11: u Generate and sample top p candi-
dates by replacing occurrences of w with the [MASK] token using the perturba- tion model M
4 Experiments
12: UK, + Query the knowledge base I's to
retrieve up to synonyms for w
In this section, we evaluate the performance of detectors AIG-ASAP to assess their accuracy and AUROC in detecting Al-generated essays.
13: end for
14: Replace each us in Wi with corresponding words in WM n waub generate esub
15:
return Caub
4.1 Experimental Setup
Datasets. Based on ASAP, we have created several sets of Al-generated essays: AIG-ASAP- cearuutioo-baaee-wiiiiie AIG-ASAP-refined and MIG-ASAP-continuule based on the different gen- eration stages stated in Section 3.1. Addition- ally, by introducing perturbations to the AIG-ASAP- cnstruction-base-wreee dataset using BERT-base (Devlin al., 2019) for word substitution and FLAN-T5-base (Chung al., 2022) for sentence substitution, we generate three sets of challeng- ing AIG-essass AIG-ASAP-paraphreesnng, AIG- ASAP-word-substtttiin and A/G-ASAP-seentence substitution. For word substitution, we choose the top-10 10 frequent words to be replaced. For sen- tence substitution, randomly select 20% of the sentences within each essay to be masked and com- pleted using FLAN-T5-base. For brevity, we will omit the prefix AIG-ASAP from the dataset names in the subsequent sections. For training purposes, we use a mixture of the randomly selected 90%
potentially account for this observed phenomenon. Based on these insights and drawing from the re- search robustness in natural language processing (Antoun et al., 2023), we devise a perturbation strategy that substitutes the frequent words in the prompts, An overview of our word substitution pro- cedure is outlined in Algorithm 1. The algorithm starts by identifying the high-frequency words in each topic's prompt. These words are then replaced with the [MASK] token, and the BERT-base model (Devlin et al., 2019) is utilized to predict the most suitable substitution for each masked word. To ensure the diversity and quality of the generated essays while maintaining semantic coherence and relevance to the given topic, leverage Word- Net 3 to find similar words as candidates for each masked word. If WordNet does not provide any
http://wwoodneet..iinttooom

---

of the generated essays for each of the machine- generated essay sets, The remaining 10% of the generated essays are used for testing. As for the human-written essays, we also randomly select 90% for training, and use the rest for testing. Statis- tics of the dataset can be found in Appendix A. Generators. In order to better simulate the usage scenarios of student users in educational applica- tions, we employ several commonly used open- source or commercial LLMs for essay generation and perturbation, including ChatGPT+, GPT-4 and Vicuna-7b (Chiang et al., 2023).
2023a.b), we report the detection accuracy of both the Al-generated and the human-written essays evaluate the performance of the detector in distin- guishing between genuine numan--ritter content and Al-generated content. In consideration of eval- uating the confidence of the detectors, we also offer the area under the receiver operating characteris- tic (AUROC) (Fawcett, 2006) results. Furthermore, incorporate the AES score as additional crite- rion to evaluate the quality of the generated essays.
4.2 Results
Detectors. We evaluate the performance of open- source detectors over our dataset, such as ArguGPT (Liu et al., 2023a) and CheckGPT (Liu et al., 2023b). In particular, the recent RoBERTa- QA and RoBERTa-Single models trained on the HC3 dataset (Guo et al., 2023) serve as statc-of-the- art detectors as they show strong detection perfor- mance in previous studies (Liu et al., 2023a; Wang al., 2023b). Furthermore, insights into the performance variations by fine- tuning the RoBERTa- CAATTa-- detectors using the training set of our constructed data.
A Al-generated essays without perturbation are highly identifiable. The detection accuracy and AUROC of essays generated by different language models under various detectors are summarized in Table 1. As shown in Table 1, the main con- clusion drawn is that machine-generater essays without perturbation can be easily detected by all detectors, regardless of whether they are written directly based instructions, refined by human articles, or continued from existing text. As for the best-performing detector, RoBERTa -A,, it achieves detection accuracy and AUROC of over 90% for nearly all machine-generann
can gain further
Essay Scorer. We incorporate text quality of the generated essays in evaluation. In reference to recent work AES (Jin et al., 2018; Wang et al., 2022), we train& a straightforward yet highly effec- tive scorer. We initially normalize the score range of various essay topics to values between and 10. Subsequently, we employ RoBERTa-base (Liu et al., 2019) to train scorer on the original ASAP training data. On the reserved validation data, the fine-tuned scorer achieves quadratic weighted kappa (QWK) of 0..77.. 0,770, which is close to the aver- age inter-human QWK agreement of 0.760 reported in (Doewes and Pechenizkiy, 2021).
data. No- tably, it also demonstrates a decent accuracy of 89.3% when detecting human- essays.
Rewriting and substitution greatly reduce de- tection accuracy. As expected, the application of perturbed methods to machine- text ef- fectively reduces the possibility of detection. We observe that as the granularity of perturbation de- creases from full-text to word, the impact on the detector's performance becomes more evident. Re- markably, using the word level perturbation, in many cases, the detection accuracy reduced approximately 50%, which is almost equivalent to a random classifier. Furthermore, as discussed in Section 3. both refined and paraphrasing per- turbations require a given human-written essay to generate. However, in terms of detection effec- tiveness, paraphrasing demonstrates better attack performance. We speculate that this performance difference comes from the prompt design. In para- phrasing. the given article serves more as a refer- ence demonstration rather than a strict template. Consequently, the language model has the flexibil- ity to incorporate more diversity and creativity in its writing, leading to improved evasion of detectors. Finetuned RoBERTa detectors can identify rewritten essays but fail to detect essays that
Human Evaluation. We further conduct human evaluation tests by presenting two annotators (mas- ter candidates in computer science) with 162 sets of essay pairs (an average of 9 essay pairs are eval- uated in each subset of AIG-ASAP) and record- ing their pairwise preferences, without disclosing which essays were written by humans or AI. Then statistical analyses are performed to assess inter- annotator agreement and draw conclusions about the quality of essays generated by humans and AI. Metrics. In line with previous studies (Antoun et al., 2023; Chakraborty et al., 2023; Liu al.,
hhttps://chat.opemmt "https://openaa http://pppnii.cco/ggtoottoo

---

Generator/APerturrb.
ArguGPT
CheckGPT
RoBERTa-Siie Singlee
RoBERTa-QA
Quality
U-shot
O-shot
O-shot
fine-tune
O-shot
fine-tune
score
ACC
AUC
ACC
AUC
ACC
AUC
ACC
AUC
ACC
AUC
ACC
AUC
(0-10)
Human
69.2
68.8
99.5
90.7
90.9
88.4
90.5
87.2
89.8
85.3
89.3
86.8
6.25
Vicuna-Inste writing
93.5
92.3
92.8
92.4
76.2
74.3
78.0
75.7
94.2
91.2
94.8
91.9
6.08
rrefined
88.3
87.2
88.5
87.3
72.1
70.1
76.5
74.3
91.9
89.6
87.4
-continuation
90.7
88.5
93.4
89.6
69.7
67.4
73.1
71.5
92.5
90.8
92.5
91.5
5.90
*paraphrasing
91.0
90.0
89.9
87.9
74.3
73.8
77.2
76.9
89.1
88.2
88.6
88.6
5.79
+sentence-suff sub
74.7
72.6
60.9
60.5
59.2
58.5
62.0
63.3
63.6
64.6
66.8
68.5
6.12
+word-sub
60.8
57.2
56.9
47.4
52.6
54.3
57.2
ChaGGPT-nstt-wittng
95,4
94.8
90.6
89.3
69.6
68.7
81.5
82.4
93.3
93.8
94.1
94.8
8.20
-refined
94.6
93.5
89.6
88.9
64.3
63.0
80.9
82.7
90.6
90.9
92.0
92.4
8.31
-continuation
89.1
87.3
79.1
82.8
81.1
95.5
94.6
96.1
7.27
+paraphrasing
84.6
82.8
72.1
69.7
53.1
52.8
74.6
73.3
71.8
73.9
82.7
84.0
8.21
+sentence-sub
61.0
60.3
57.3
57.6
62.9
63.5
65.8
69.2
64.7
68.8
68.1
72.3
7.36
+word-sub
53.8
53.4
49.9
51.3
55.0
53.6
58.2
60.3
57.4
60.8
61.6
64.7
8.12
GPT-4-Instt-wrriting
96.9
94.8
92.4
91.5
71.3
70.4
80.6
82.5
93.1
93.0
93.7
94.2
8.46
-refined
92.0
90.1
85.3
86.0
68.2
67.7
72.2
74.7
89.1
89.6
88.7
89.2
8.63
-continuation
93.7
88.9
87.5
73.0
78.1
78.7
91.7
95.1
8.35
*paraphrasing
82.3
80.5
71.8
72.6
66.4
67.1
75.3
76.3
69.7
70.5
78.5
79.7
8.17
+sentence-sub
64.9
62.7
58.5
59.2
61.3
63.1
63.3
64.7
58.0
59.4
63.8
65.3
7.82
+word-sub
55.6
54.3
53.6
53.8
54,9
55.7
58.1
59.4
56.2
57.3
59.5
61.7
8.21
Table Accuracy(ACC) and AUROC(AUC) of various detectors using different essay generation strategies. As the perturbation granularity becomes finer, there is decrease in the probability of AT-generated text being detected. "O-shot" represents the performance of the off the-s detector, "fine-tune" corresponds to our model further fine-tuned with training data, and "quality" of generated essays is measured using our trained essay scorer. lowest detection rate generator of each detector is in bold. The lowest quality AES score of each generator is underlined.
Metric
Evaluator
Instr-writing
Refined
Cont.
Paraph.
Sent-sub
Word-sub
AIGC
scoreTues
70.4
77.8
63.0
74.1
66.7
63.0
Pref.
humanj
63.0
74.1
59.3
66.7
55.6
44.4
Ratio
humanz
66.7
70.4
59.3
74.1
63.0
55.5
human & scoreTaes
66.9
70.0
76.7
82.4
46.2
35.2
Evaluators'
humanz & scoreraes
74.3
80.9
76.7
61.4
59.5
54.2
Kappa
humanj & humang
91.9
72.4
84.7
82.4
84.7
78.0
Table 2: Results of human evaluation. AIGC Pref. Ratio refers to the ratio of Al-generated text being preferred by the evaluator. The kappa value that measures the inter-rater agreement is also given.
underwent substitution. To better identify the perturbed essays, proceed to further fine-tune the RoBERTa-Single and RoBERTa--Q QA using the reserved training data, as the method described in the original paper. As shown in the "fine-tune" columns of Table 1, the detectors have successfully improved their ability to detect machine-generated content while maintaining their proficiency in identifying human-written essays. Notably, for the ChatGPT paraphrasing data, RoBERTa-Sing' achieves notable improvement, increasing from 54.1% 74.6% in terms of detection accuracy. However, should be noted that the performance improvement achieved through fine-tuning varies across the perturbation methods. In the case of
the substitution methods, the fine-tuned model has resulted in relatively modest improvement, with an increase of only up to 5.2% in the performance which is evident from the Vicuna word substitution data on the RoBERTa-Single model. particular, the accuracy of the fine-tuned RoBERTa models is still around or below 60% of the generated essays that underwent word substitutions, which signals an unreliable detection performance for practical use.
Human quality evaluation. Table presents the results of the human evaluation conducted to assess the quality of the generated essays. The results from the AIGC Pref. Ratio show that, in general, AI- Al-generated essays are more likely to be preferred

---

Essay Type
Instr-writing
Refined
Cont.
Paraph.
Sent-sub
Word-sub
Argumentative 92.7
91.3
93.8
80.7
63.4
38.4
Source-dependent 95.1
94.5
97.6
84.1
72.6
74.7
Narrative
94.5
90.2
96.3
83.2
68.3
58.6
Table 3: The detection accuracy of RoBERTa-QA (fine-tuned) varies across different essay types. All perturbation methods demonstrate moderate fluctuations across various essay categories, while word substitution has notable impact argumentative essays.
tinct topics that are further categorized into 3 es- say types: argumentative, source-dependent, and narrative. Detecting these essay topics reveals vari- ations in the difficulty of AIGC detection and the impact of perturbations. Table 3 shows the detec- tion accuracy across essay categories, focusing on the pest-performing detector, fine-tuned RoBERTa- QA, when applied to ChatGPT-gennrated data. The results indicate that, on average, source-dependent essays are more easily detectable by the detector. This observation could be attributed to the source article that is provided to ChatGPT during source- dependent content generation. Such souree articles may impose limitations the diversity of words used by the language model, making the generated content more distinguishable. Additionally, is observed that the detection performance over essay paraphrasing and sentence substitution has mod- erate fluctuations over different essay categories. However, word substitution has significant in- fluence on argumentative essays, which can be at- tributed to the fact that the words selected for sub- stitution in argumentative essays are more likely to involve argumentative keywords, resulting in a more noticeable effect on the overall content.
Argumentative Essay Source dependent Essay Narrative Essay
Roboets 0A Accuarty
0.6
15
20
Parturbation Wards Dapth
Figure 2: Detection performance curves for different es- say categories using RoBERTa-QA QA (fine-tuned), where the X-axis represents perturbation depth.
by both human and machine evaluators, particu- larly for essays generated through refined writing and paraphrasing techniques. However, when comes to word substitution, only half chance of preference is observed. This can be attributed to the complete replacement frequent words in the essays, which may be inappropriate in certain cases. Although the word substitution experiments demonstrate the potential for circumventing AIGC detectors, the human evaluation results suggest that further improvements can be made enhance the quality of the generated essays. For instance, a partial replacement of frequent words could be con- sidered to preserve the writing quality of language models while introducing perturbations. Further- more, the results indicate substantial degree of consistency between human evaluation and the au- tomatic scorer. However, on the word sentence substitution datasets, the consistency is limited, suggesting a certain level of discrepancy between the viewpoints of humans and algorithms.
Perturbation Depth. We further investigate the influence of increasing perturbation depth the de- tection accuracy of Al-generated essays with word substitutions, where perturbation depth refers the number of most frequent words replaced with perturbation methods while generating A/G-ASAP. word- word-subbstitutinn set. Figure 2 presents the rela- tionship between perturbation depth and detection accuracy. The curves clearly demonstrate an in- verse relationship hetween perturbation depth and detection probability, confirming the intuitive ex- pectation that the more perturbed words present, the lower the likelihood of detection by the detector. Additionally, it is worth noting that the quality of the perturbed content experiences a slight decrease, as detailed in Appendix D.
4.3 Analysis
Detection difficulty varies to different essay types. The AIG-ASAP dataset encompasses dis-

---

5 Related Work
detecting Al Al-generated student essays.
With the advances of large language models (LLMs), there have been a plethora of research on detecting Al-generated content (Becker et al., 2023; Crothers et al.. 2022; Jawahar et al., 2020; Mireshghallah et al., 2023; Su et al., 2023; Wang al., 2023a). Current Al-generated text detection methods can be in general categorized into three categories (Lu et al., 2023): statistical (Gehrmann et al., 2019), watermarking (Kirchenbauer et al., 2023), and training-based (Gallé et al., 2021).
Acknowledgments
This work is supported by the National Natural Science Foundation of China (62272439), and the Fundamental Research Funds for the Central Uni- versities.
The statistical approach analyzes measures like entropy, perplexity, and n-gram frequency to iden- tify statistical irregularities (Gchrmann al., 2019; Lu et al., 2023), which rely statistical charac- teristics of the text differentiate between Al- generated and human-written content. The wa- termarking methods (Wilson et al., 2014) imprint specific patterns on generated text. Soft water- marking (Kirchenbauer et al., 2023) presents novel statistical test designed to detect the pres- ence of watermark, providing interpretable p- values evidence, and additionally, establish- ing information--theortii framework that ables the analysis of the watermark's sensitivity. Early training-based detection studies focused on identifying fake reviews (Hovy, 2016) and fake news (Zellers et al., 2019). Recent research em- ployed LLM-generated texts for training classifiers, such as ArguGPT (Liu et al., 2023a), CheckGPT (Liu et al., 2023b), and G3Detector (Zhan et al., 2023). Notably, Guo et al. (2023) fine-tuned the RoBERTa-QA and RoBERTa-single models on the HC3 dataset and achieved high detection accuracy.
6 Conclusion
We note that basic paraphrasing techniques like rewriting essays decrease the effectiveness of detection, while fully automated machine- generated content can be easily detected by cur- rent detectors. This implies that human writing plays a crucial role evading detection in the tra- ditional process. To address this, we introduced two distinct perturbation strategies: word substi- tution and sentence substitution. These strategies have shown substantial potential in deceiving Al- generated content detection while ensuring content quality in our experiments. The results highlight the inherent vulnerabilities in existing detection methods and emphasize the need for the develop- ment of more resilient and precise approaches for

---

Limitations
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang. Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
This paper addresses important and timely topic regarding the detection of Al-generated student es- says. However, the AIG-ASAP dataset constructed for this research is built upon the ASAP dataset composed of English essays written by high-school students in the United States. The dataset's effec- tiveness in representing the broader landscape of Al-generated essays in different educational con- texts and different languages could impact the gen- eralizability of the findings.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, Albert Web- son, Shixiang Shane Gu, Zhuyun Dai, Mirac Suz- gun, Xinyun Chen, Aakanksha Chowdhery. Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe Le, and Jason Wei. 2022. Scaling instruction-fine finetuned language models, CoRR, abs/2210.11416.
While the paper emphasizes the need for more accurate and robust methods to detect Al-generated student essays, it does not delve into the practical implementation and deployment of such methods. Considering the potential impact on educational in- stitutions and the challenges of real-time detection, further exploration of the feasibility and scalability of proposed detection methods would enhance the practical implications of the research.
Evan Crothers, Nathalic Japkowicz, and Herna L. Viktor. 2022. Machine generated text: comprehensive sur- vey of threat models and detection methods. CoRR, 10.00321..
Jacob Devlin. Ming-Was Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, NAACL-HLT' 2019, Minneupolis, MN. USA, June 2-7, 2019, Volume (Long and Short Papers), pages 4171-4186, Association for Computational Linguistics.
Ethics Statement
In this paper, investigate the potential vulnera- bility concerns of the current AIGC detection meth- ods and explore several adversarial measures that generate student essays that are not easily detected. We hope that this study could inspire further explo- ration and design of AIGC detection methods and aid in the development of robust real-world AIGC detectors.
A ffrizal Doewes and Mykola Pechenizkiy. 2021, On the limitations of mated essay scoring. In EDM. International Educa- tional Data Mining Society.
agreement in auto-
Tom Fawcett. 2006. An introduction to ROC analysis. Pattern Recognit. Lett., 27(8):661-874,
References
Matthias Gallé, Jos Rozen, Germán Kruszewski, and Hady Elsahar. 2021. Unsupervised and distribu- tional detection machine-generated text. CoRR, 11.0278..
Wissam Antoun, Virginie Mouilleron, Benoit Sagot, and Djamé Seddah. 2023. Towards robust detection of language model generated text: Is chatgpt that easy to detect? CoRR, abw/2306.05871.
Sebastian Gehrmann, Hendrik Strobelt, and Alexan- der M. Rush. 2019. GLTR: statistical detection and visualization of generated text. In ACL (3), pages 111-116. Association for Computational Linguistics.
Jonas Becker, Jan Philip Wahle, Terry Ruas, and Bela Gipp. 2023. Paraphrase detection: Human vs. ma- chine content. CoRR, Abs/2303.13989
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt human experts? comparison corpus, evaluation, and detection. CORR, abs/2301.07597.
Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. 2023, On the possibilities of ai- ai-geeerated text detection. CoRR, abs/2304.047366
Moses Charikar. 2002. Similarity estimation techniques from rounding algorithms. In STOC, pages 380-388. ACM.
Dirk Hovy. 2016. The enemy in your own camp: How well can detect statistically geneerated fake views an adversarial study. In Proceedings of the 54th Annual Meeting of the Association for Compu- tational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers. The Asso- ciation for Computer Linguistics.
Chen Chen, Jie Fu, and Lingjuan Lyu. 2023. A pathway towards responsible Al generated content. CoRR, abs/2303.01325.

---

Rongjie Huang, Mingze Li, Dongchao Yang, Jia- long Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong. Jiawei Huang, Jinglin Liu, Yi Ren, Zhou Zhao, and Shinji Watanabe. 2023. Audiogpt: Understanding and generating speech, music, sound, and talking head. CoRR, 1ew/2344.1999,
2023. Smaller language models are better black- box machine gennerttd text detectors. CoRR. abs/2305.00 09859.
Rabee Qasem, Banan Tantour, and Mohammed Maree. 2023. Towards the exploitation of un-based chatbot for providing legal support to palestinian coopera- tives. CoRR, abs/2306.05827.
Ganesh Jawahar, Muhammad Abdul- MaguedM and Laks V. S. Lakshmanan. 2020, Automatic detection of ma- chine generated text: A critical survey. In Proceed- ings of the 28th International Conference Com- putational Linguistics, COLING 2020, Barcelona, Spein (Online), December 8-13, 2020, pages 2296- 2309. International Committee Computational Linguistics.
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Bala- subramanian, Wenxiao Wang. and Soheil Feizi. 2023, Can ai-generated text be reliably detected? CoRR, abs/2303.11155
Cathy Shyr, Yan Hu, Paul A. Harris, and Hua Xu. 2023, Identifying and extracting rare disease phenotypes with large language models.
Cancan Jin, Ben He, Kai Hui, and Le Sun. 2018. TDNN: A two-stage deep neural network for prompt- independent automated essay scoring. In ACL (1). pages 1097. Association for Computational Linguistics.
Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. 2023. Detectllm: Leveraging log rank information for zero-shot detection of machine- generrttd text. CoRR, abs/2306.05540.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. watermark for large language models. CoRR,
Teo Susnjak. 2022. Chatgpt: The end of online exam integrity? CoRR, abs/2212.09292..
abs/2301.10226.
Hong Wang. Xuan Luo, Weizhi Wang. and Xifeng Yan. 2023a. Bot or human? detecting chatgpt imposters with A single question. CoRR, abs/2305.06422
Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Tyyer. 2023. Paraphras- ing evades detectors of ai-generated text, but retrieval is an effective defense. CoRR, abs/2303, 13408.
Yongjie Wang. Chuang Wang, Ruobing Li, and Hui Lin. 2022. On the use of bert for automated essay scor- ing: Joint learning of multi-scale essay representation. In NAACL-HLZ, pages 3416-3425. Association for Computational Linguistics.
Yongqi Li, Mayi Xu, Xin Miao, Shen Zhou, and Tieyun Qian. 2023. Large language models as counterfac- tual generator: Strengths and weaknesses. CoRR, abs/2305.1477 14791..
Zecong Wang, Jiaxi Cheng, Chen Cui, and Chenhao Yu. 2023b. Implementing BERT and fine-tuned roberta to detect Al generated news by chatgpt. CORR. abs/2306.07401.
Weixin Liang, Mert Yüksekgönül, Yining Mao, Eric Wu, and James Zou. 2023. GPT detectors are biased against non-native english writers. CoRR, abs/2304.02819.
Alex Wilson, Phil Blunsom, and Andrew D. Ker. 2014. Linguistic steganography on twitter: hierarchical lan- guage modeling with manual interaction. In Media Watermarking, Security, and Forensics 2014, San Francisco, CA, USA, February 2, 2014, Proceedings. volume 9028 of SPIE Proceedings, page 902803. SPIE.
Yikang Liu, Ziyin Zhang. Wanyang Zhang, Shisen Yue. Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang, and Hai Hu. 2023a. Argugpt: evaluating, understanding and identifying argumentative essays generated by GPT models. CoRR, abs/2304.07666.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1997. 16692.
Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and Hong Lin. 2023. Ai-generated content (AIGC); A survey, CoRR, abs/2304.06632.
Rowan Zellers, Ari Holtzman, Hannah Rashkin. Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- tion Processing Systems 2019, NewrIPS 2019, De- cember 8- 8-14, 2019, Vancouver, BC, Canada, pages 9051-9062.
Zeyan Liu, Zijun Yao, Fengjun Li. and Bo Luo. 2023b. Check me if you can: Detecting chatgpt- generated academic writing using checkgpt. CoRR, abs/2306.05524.
Ning Lu, Shengcai Liu, Rui He, Qi Wang. and Ke Tang. 2023. Large language models can be guided to evade ai-generated text detection. CoKR, 10044705
Haolan Zhan, Xuanli He, Qiongkai Xu, Yuxiang Wu, and Pontus Stenetorp. 2023. G3detector: General ept-generated text detector. CoRR, 122800
Fatemehsadat Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri, and Taylor Berg- Kirk-iiik....ic..

---

Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, and Arman Cohan. 2023. Large language models are effective table-to-text gener- ators, evaluators, and feedback providers. CoRR, 0bs/2305.14987.
Narrative prompt Topic 7 Act as middle school student. please read the below prompt and write an essay:
Write about patience. Being patient means that you are understanding and tolerant. A patient per- son experience difficulties without complaining, Do only one of the following: write story about a time when you were patient OR write story about time when someone you know patient OR write story in your own way about patience.
A Dataset Statistics
Statistics of our constructed AIG-ASAP dataset can be found in Table 4.
B AIG-ASAP Data Construction Prompts
B.1 Instruction-base Writing
B.2 Refined Writing
A Argumentative prompt Topic 1 Act mid- dle school student, please read the below prompt and write an essay:
General Prompt ing essay:
Polish and optimize the follow-
More and more people use computers, but not everyone agrees that this benefits society. Those who support advances in technology believe that computers have positive effect on people. They teach hand-eye coordination, give people the abil- ity to learn about faraway places and people, and even allow people to talk online with other people. Others have different idcas. Some experts are con- cerned that people are spending too much time on their computers and less time exercising, enjoying nature, and interacting with family and friends.
Based on the story it all started when they had build the empire state building and the workers should have knowned that the building for the wind climate, up in the sky and plus when you make a building like that you have to make sure it's strong enough to keep balance. They faced problem's 's like when the U.S. tried to make a deal with them they would let them use the top of the building for dock to let civilians on the deck, and after they made a law that no more blimps can land on the dock blimp attempted to land on the dock.
too big
Write letter to your local newspaper in which you state your opinion on the effects computers have on people. Persuade the readers to agree with you.
B.3 Continuation Writing
General prompt Follow the first couple of sen- tences from an essay to write a follow-up para- graph:
Source-dependent prompt- Topic 3 Act as a middle school student, please read the source essay and write an essay based on given topic prompt.
Dear @LOCATIONI, I know having computers has positive effect on peoplc...
Source essay:
Besides, consider the essay writing prompt:
"ROUGH ROAD AHEAD: Do Not Exceed Posted Speed Limit by Joe Kurmaskie FORGET THAT OLD SAYING ABOUT NEVER taking candy from strangers. No, better piece of advice for the solo cyclist would be, "Never accept travel advice from collection of old-timers who haven't t left the confines of their porches since Carter was in office." It's not that group of old guys doesn't know the terrain. With age comes wisdom and all that, but the world a fluid place. Things change..."
More and more people use computers, but not everyone agrees that this benefits society. Those who support advances in technology believe that computers have positive effect on people. They teach hand-eye coordination, give people the abil- ity to learn about faraway places and people, and even allow people to talk online with other people. Others have different ideas. Some experts are con- cerned that people are spending too much time on their computers and less time exereising, enjoying nature, and interacting with family and friends.
Prompt: Write response that explains how the features of the setting affect the cyclist. In your response, include examples from the essay that support your conclusion.
Write a letter to your local newspaper in which you state your opinion on the effects computers have on people. Persuade the readers to agree with you.

---

Perturbation
Human
Vicuna
ChatGPT
GPT-4
train
5114
3600
7200
3600
Instr-writing
test
571
400
800
400
train
5114
10230
5114
Refined
test
571
1140
571
Cont. train
5114
10230
5114
test
571
1140
571
train
5114
5114
5114
Paraph.
test
571
571
571
train
3600
7200
3600
Sent-sub
test
400
800
400
train
3600
7200
3600
Word-sub
test
400
800
400
Table
Statistics of the AIG-ASAP dataset.
Substitution examples
speaking, and it was taking him a long time to get his words out. My grandma sat beside him, listening intently and waiting patiently for him to finish his sentences. She didn't interrupt him finish his thoughts for him; she simply waited until he was done speaking.
We give an example of the result essays under word substitution and sentence substitution. The essay after word substitution is detected as 'Human' with the 0.67 probability and the essay after sentence substitution is detected as 'Human' with the 0.81 probability while the instruction-based writing re- sult is detected "ChatGPT" with the 0.99 prob- ability. The quality score received from our essay scorer 8.33 for instruction-based writing, 8. 5.18 for word substitution, and 8.05 for sentence sub- stitution. The substitutions may not be the most appropriate alternatives as shown the examples, leading to degraded quality as noticed by the essay scorer, but drastically reduces the chance of being caught by the detector.
Her patience not only helped my grandpa feel heard and understood, but it also helped our entire family stay calm and positive during difficult time. Whenever I feel myself getting impatient, think back that moment in the hospital room and try to channel my grandma's patience.
Being patient may not always be easy, but it is incredibly valuable trait to have. It allows us to be more understanding and tolerant of others and helps us experience difficulties without complaining.
Instruction-based writing result Patience is something that have always struggled with. I am someone who always wants things to happen quickly and tend to get frustrated when they don't t. However, witnessing the patience of someone else has helped me to understand the importance of be- ing patient.
Word substitution result Patience is something that have always struggled with. I somebody who always wants things happen quickly and tend gain frustrated when they don't 't. However, witnessing the tolerance of somebody else has take me to understand the importance of being patient.
Last summer, my grandpa suffered stroke and was admitted to the hospital. was a difficult task my family as we watched him struggle to do things that were once easy for him. However, my grandma was the epitome taught. She never once complained or got frustrated with my grandpa, no matter how difficutt things are.
Last summer, my grandpa suffered a stroke and was admitted to the hospital. It was a difficult time for my family as we watched him struggle to do things that were once easy for him. However, my grandma the epitome of patience. She never once complained got frustrated with my grandpa, no matter how difficult things became.
remember night when we were visiting him the hospital, my grandpa was having a hard strong- less speaking, and it was taking him a long effect
I remember one day when we were visiting him in the hospital, my grandpa was having hard time

---

gain his words out. My grandma sat beside him, listening intently and awaiting patiently for him to finish his sentences. She didn't interrupt him or complete his thoughts for him; she simply timeed until he was dtime speaking.
82
8.1
Her tolerance not only make my grandpa feel heard and understood, but it also make our entire family to stay calm and effective during difficult time. Whenever I feel myself become impatience, think back that moment in the hospital room and try channel my grandma's patience.
Contert Quality
8.0
7.9
188
1.7
Being patience may not always be easy, but is an incredibly valuable trait to have. It allows us to be more understanding and tolerant of oth- ers and make us to experience difficulties without complaining.
Depth
Figure 3: Our proposed perturbation methods result in slight decrease content quality. Even after replacing 20 words, the overall essay scores only drop by 0.365.
Sentence substitution result Patience some- thing that have always struggled with. I am some- one who always wants things to happen quickly and tend to get frustrated when they don't. And my grandma was the epitome of patience.
by the detector. To explore the extent this quality degradation, we present the corresponding curve in Figure 3. It can be observed that due to the compre- hensive consideration of synonyms and language model output predictions during perturbation, the score the perturbed text does not decrease rapidly as the perturbation depth increases. This indicates that the quality of the perturbed text is relatively preserved, despite the increasing level of perturba- tion applied.
Last summer, my grandpa suffered stroke and admitted to the hospital. As my grandma's patience has helped me understand the importance of being patient. However, my grandma was the epitome patience. She never once complained or got frustrated with my grandpa, no matter how difficult things became.
I'm always frustrated when things don't ton't happen quickly. I'm always frustrated when things don't 't go planned. I remember visiting him in the hospital, my grandpa hard time speaking, and it was taking him a long time to get his words out. My grandma sat beside him, listening intently and waiting patiently for him to finish his sentences. My grandma was a patient and understanding person. Whenever feel myself getting impatient, think back to that moment in the hospital room and try to channel my grandma's 's patience.
day when we were
having
Being patient may not always be easy, but it incredibly valuable trait to have. It allows to be more understanding and tolerant of others and helps us to experience difficulties without complaining.
Detailed Analysis
Context Quality with Perturbations. Accord- ing the previous examples, the substitutions may introduce spelling or grammatical errors, leading to degraded quality as noticed by the essay scorer, but drastically reduce the chance of being caught


Reviews:
[{"review_id": "WKp8Q5rnae", "reviewer": "Reviewer_9VU8", "summary": "", "questions": "Q1. I’m curious why you used two different models (FLAN-T5 and BERT) for sentence and word substitution. \nQ2. Have you tried evaluating the detectors’ performance in few shot manner? This could significantly influence models’ detection accuracy.\nQ3. Sometimes LLMs are not faithful to the prompt. Did you put some efforts in checking the quality of generated texts? This may also impact the results (if generators were prone to be unfaithful to given prompts, the distinction between three different writing types will be blurred.)", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive comments. Please see below for our responses.\n1. As discussed in Sec 3.3, the present research primarily centers around employing paraphrasers to counteract the detector's capabilities. For instance, in [1], content from paraphrasers is harmonized with human-generated sentences, while in [2], off-the-shelf models are harnessed for the rewriting process. Notably, these procedures share similarities to our \"Essay Paraphrasing\" methodology, leading us to believe that our perturbation method encompasses techniques from these preceding studies.\n2. We agree that there is potential for expanding the dataset by incorporating additional LLMs or perturbation methods. Our current approach to data generation and perturbation was refined based on insights from multiple preliminary experiments. Nevertheless, it's worth noting that the AIG-ASAP dataset already stands at a scale ten times larger than the original ASAP dataset.  Also, compared to the corpus of ArguGPT, which consists of 4,038 argumentative generated essays, our dataset has a total of 110,535 generated essays.\n3. Given the substantial volume of generated essays, we opted for an automated scoring model. However, as indicated in our response to Reviewer WQAB, we have a future plan to introduce a pairwise human evaluation approach.\n\nQ1: FLAN-T5 boasts greater potency and is relatively well-suited for generating lengthier sequences. Conversely, BERT, due to its compact size, exhibits limitations in text generation prowess. Employing BERT for word completion strikes a balance between effectiveness and efficiency.\n\nQ2: The detectors' performance in a few-shot manner has only marginal benefit over zero-shot detection in our preliminary experiments, so only the zero-shot and finetuned detection results are given. We are considering adding a few-shot evaluation in a future version.\n\nQ3: We did not systematically check the relevance of the generated essays to the prompts. However, our initial observation is that the LLMs are good at generating essays faithful to the prompts, and GPT-3.5/4 are in general better than Vicuna. We are considering adding more analysis. For instance, we will sample essays from the AIG-ASAP dataset to check, especially those that received low scores from the automatic scorer.\n\n[1] Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. \"Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense\"\n\n[2] Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. \"Can ai-generated text be reliably detected?\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Here are reasons to accept this paper:\n1. This paper discusses a very timely and important topic given the surge in LLM usage in all writing domains.\n2. AIGC detection in student essay writing is relatively under-explored compared to other writing domains.\n3. It is well-structured and well-written, with few grammatical mistakes. \n4. Its methodology and experiment setup with detector selection are reasonable.\n5. It demonstrates several interesting findings that have not been introduced in other papers, such as the observations that detection difficulty varies to different essay types.", "reasons_to_reject": "Here are weaknesses of this paper:\n1. As stated in the paper, many existing papers have experimented with different text perturbation techniques. The existing works can serve baselines to gauge the effectiveness of proposed perturbation strategies more accurately.\n2. The dataset seems limited. \n3. For AIGC quality check, including humans’ evaluation sounds more reasonable and suitable than simply relying on the automated approach.", "questions_to_authors": "Q1. I’m curious why you used two different models (FLAN-T5 and BERT) for sentence and word substitution. \nQ2. Have you tried evaluating the detectors’ performance in few shot manner? This could significantly influence models’ detection accuracy.\nQ3. Sometimes LLMs are not faithful to the prompt. Did you put some efforts in checking the quality of generated texts? This may also impact the results (if generators were prone to be unfaithful to given prompts, the distinction between three different writing types will be blurred.)", "paper_topic_and_main_contributions": "This paper studies several AIGC detectors’ performance on their new dataset, AIG-ASAP, which is specifically covering student essay writing. In the process of AIG-ASAP construction, authors employed 3 different automatic text perturbation techniques given human-written essay writings. Proposed text perturbation techniques range from coarse-grained modification (full-text level) to fine-grained modification levels (sentence/word-level), enabling the authors to investigate AIGC detectors’ vulnerabilities depending on perturbation depth. They also report that these perturbation depth affects AIGC detectors’ capabilities when they are finetuned on the curated dataset. \n\nThis paper’s contributions are as follows:\n1. Curation of a new AIGC dataset (AIG-ASAP) in education domain\n2. Introduction of three perturbation techniques that can effectively fool AIGC detectors\n3. Providing the current state of state-of-the-art AIGC detectors on the curated dataset and highlighting their vulnerability", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Not that I know of."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Few grammatical mistakes I found: \nIn line 454, the reference text contains a parenthesis. \nIn line 492, both refined and paraphrasing perturbations requires -> require\nln line 513, ChatGPT paragraphing data -> paraphrasing data"}}, {"review_id": "CklRCTWMib", "reviewer": "Reviewer_4N4g", "summary": "", "questions": "Out of pure curiosity, is there any connection between the initial score of the essay from the ASAP dataset and the score after refinement or continuation by the model? ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive comments. Please see below for our responses.\n\nQuestion：\n\nAs in reply to point 1 of Reviewer rqmL, it is essential to note that the original ASAP student essays were written by high-school students. Our empirical observations are that the GPT-generated text generally surpasses the original student essays regarding coherence, structure, and fluency. This is supported by the quality scores in Table 1, illustrating the refined essays consistently achieve higher quality scores compared to the continuation essays, which could be attributed to the fact that the continuation might have constrained the machine's creative writing style and direction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is very well-written and presents a valuable contribution for detection of AIGC texts in educational domain. Moreover, the authors of the paper presented extensive evaluation of existing methods. They used both commercial and open-source models. ", "reasons_to_reject": "I can't think of any, it is a very good paper :)", "questions_to_authors": "Out of pure curiosity, is there any connection between the initial score of the essay from the ASAP dataset and the score after refinement or continuation by the model? ", "paper_topic_and_main_contributions": "The paper focuses on the task of detecting AI generated essays. The main contributions are as follows:\n- dataset of AI generated (written, refined or continued by AI) essays, being a new benchmark for evaluating performance of AI generated text detectors in educational domain,\n- attacks for detection of AI generated essays (paraphrase, sentence substitution, word substitution),\n- broad evaluation of existing AIGC detectors on the created dataset.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4udCBCvGiS", "reviewer": "Reviewer_rqmL", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive comments. Please see below for our responses.\n\n1. High quality. We used automatic quality evaluation due to the large number of essays. Essays in ASAP were written by high-school students which have highly diverse human ratings. Our trained automatic scorer shows a strong correlation (kappa=0.77) with human raters on the validation set. Meanwhile, according to the [Paper with Code](https://paperswithcode.com/sota/automated-essay-scoring-on-asap), the SOTA ASAP-AES model currently achieves kappa=0.791. Given that the average inter-human agreement is kappa=0.760 [1], we believe that our evaluation score is adequate to assess the quality of the generated text. Table 1 demonstrates that ChatGPT and GPT-4 generated essays receive higher quality ratings. Our empirical observations are that GPT-generated text generally surpasses the original STUDENT essays in terms of coherence, structure, and fluency. We plan to incorporate pairwise human evaluation in a future version (as per our reply to point 2 of Reviewer WQAB) and acknowledge the potential benefit of adding grammar checkers to further enhance quality.\n2. Selection of topical words. As explained in Sec 3.2, LLMs have the tendency to excessively use topical words in the prompt. Therefore, replacing topical words could mislead the detectors by changing the word usage style of the generated essays. As explained in Sec 3.3.3 and \"Algorithm 1 Word Substitun\", we sort the key topic words based on their concurrency in the essay requirement (instruction) and replace them considering both LM and WordNet synonyms. Moreover, we provide experiments on the effect of word perturbation depth on detection accuracy, which is discussed in Sec 4.3.\n3. Quality metric. As explained in Sec 4.1, our fine-tuned scorer achieves a kappa of 0.770, which closely aligns with the average inter-human agreement of kappa=0.760. Hence, it is deemed to provide a fair assessment of essay quality. We will consider more metrics, such as perplexity and grammar checker ratings for essay evaluation in a revision. \n4. Dataset analysis. Thanks for the suggestion. As explained in lines 531-534, there are 8 topics with 3 different types. We should clarify that each LLM generates no less than 4,000 essays for each prompt with each perturbation method. Consequently, the collective result comprises a total of 110,535 generated essays with an average length of 247 words. We intend to incorporate a table of detailed dataset statistics.\n5. Writing. Thanks for the suggestion. We will proofread and improve the writing.\n\n[1] Afrizal Doewes and Mykola Pechenizkiy. 2021. On the limitations of human-computer agreement in automated essay scoring. In EDM. International Educational Data Mining Society", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper introduces a novel application of adversarial attacks in the student essay domain that opens a new research direction across the education and AI field. It also attempts to resolve the concerns on ethical and practical usage of LLMs in other domains. Also, the paper is logically sound and includes comprehensive analysis to support its argument. ", "reasons_to_reject": "High quality. \nThe authors state that the generated essays using three adversarial techniques maintain the quality of the original essays. Were there any qualitative results that confirm the quality of the original and generated essays are comparable?  Also, GPT-instructed/created sentences are known to have textual --grammatical, syntactical--noise (which is obvious to humans). Was there any process to filter these out? \n\nSelection of Topical Words.\nThe authors state that topical words were replaced in the generated essays to create new essays. The paper does not mention how these topical words were selected and replaced. To my understanding, only the overlapping words with the prompts were replaced. How does removing topical words affect the detection accuracy?  \n\nQuality Metric. \nIt seems like the metric from asap-aes was only employed to measure the quality of the essays. It might be worthwhile to adopt other metrics for essay evaluation, as adversarial samples created from LLMs could be biased in a way that could lead to lexical hallucinations, and lower the detection accuracy.\n\nDataset Analysis.\nThe authors do not provide elementary distribution of the introduced dataset, such as topics, types, sentence length of essays, and number of words or sentences perturbed in each essay. Dataset papers should include basic information about the dataset. \n\nThe wording could be improved to explain the concept better and repetitive information throughout the paper could be removed. \n", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper introduces an adversarial dataset to evade model detection accuracy in student essay writing domain. They use basic adversarial attacks such as word and sentence perturbation techniques on GPT models while preserving the quality. Their analyses show that models generally fail in the detection performance when using these attacks. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "No typos or grammar mistakes, but the organization of the paper could be improved. It was a bit confusing to understand the task intuitively. The paragraphs could be more self-contained. "}, "extra_text": {}}, {"review_id": "rC9HB0F6T9", "reviewer": "Reviewer_WQAB", "summary": "", "questions": "In the results, you list the accuracy of binary classification, right? But do the detectors always provide only binary results(i.e. human written or Ai-generated)? Have you considered evaluating the confidence of the detectors? For example, the probability of one essay being AI-generated is  xx%. With such confidence-related metrics, it would be interesting to see how much the performance was dropped through adversarial attacks.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive comments. Please see below for our responses.\n\n1. Robustness. While our paper primarily focuses on the adversarial evaluation of AIGC detectors, we recognize the significance of addressing practical implementation and robustness. We believe that utilizing reinforcement learning and adversarial generation techniques to address perturbation challenges are viable future research directions.\n2. Quality evaluation. Given the substantial number of AI-generated essays, we employed an automatic scorer for quality evaluation, which demonstrated a high kappa agreement (0.77) with human annotators. In a future version of our work, we plan to add human-evaluation results such as pairwise preferences. This involves presenting annotators with sets of essay pairs, without disclosing which essays were written by humans or AI.\n3. Threshold. Following [1], we selected the default confidence threshold setting (0.5) for our detectors for binary classification. We plan to include additional experiments in a revised version to evaluate other confidence-related metrics.\n\n[1] Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. \"How close is chatgpt to human experts? comparison corpus, evaluation, and detection \"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Solid work with details of the generation process. Not only the dataset could be reused, but also the methods are highly reproducible. Important findings highlight the vulnerabilities in existing detection methods.", "reasons_to_reject": "No discussion on how to improve the robustness of detectors to this dataset. The quality of the AI-generated essays was automatically scored.", "questions_to_authors": "In the results, you list the accuracy of binary classification, right? But do the detectors always provide only binary results(i.e. human written or Ai-generated)? Have you considered evaluating the confidence of the detectors? For example, the probability of one essay being AI-generated is  xx%. With such confidence-related metrics, it would be interesting to see how much the performance was dropped through adversarial attacks.", "paper_topic_and_main_contributions": "This paper constructs a new data resource AIG-ASAP, which employs a range of text perturbation methods and can be used for evaluating AI-generated content detectors. It also assesses the performance of current AIGC detectors on this dataset through empirical experiments.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 418 purposes, we..."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers agreed that the paper addresses an important new topic by creating a new dataset of perturbed student essays and highlighting the vulnerability of current AIGC detectors. They found the paper mainly to be sound and well-executed, although there were suggestions that including a human evaluation would be helpful and more baselines from previous work could be included. Other criticism concerned improvements in the presentation of the content that could easily be fixed in a final version.
Decision: Accept-Main