Are AI-Generated Text Detectors Robust to Adversarial Perturbations? (2024)

Abstract: The widespread use of large language models (LLMs) has sparked concerns about the potential misuse of AI-generated text, as these models can produce content that closely resembles human-generated text. Current detectors for AI-generated text (AIGT) lack robustness against adversarial perturbations, with even minor changes in characters or words causing a reversal in distinguishing between humancreated and AI-generated text. This paper investigates the robustness of existing AIGT detection methods and introduces a novel detector, the Siamese Calibrated Reconstruction Network (SCRN). The SCRN employs a reconstruction network to add and remove noise from text, extracting a semantic representation that is robust to local perturbations. We also propose a siamese calibration technique to train the model to make equally confident predictions under different noise, which improves the model’s robustness against adversarial perturbations. Experiments on four publicly available datasets show that the SCRN outperforms all baseline methods, achieving 6.5%-18.25% absolute accuracy improvement over the best baseline method under adversarial attacks. Moreover, it exhibits superior generalizability in crossdomain, cross-genre, and mixed-source scenarios. The code is available at https://github. com/CarlanLark/Robust-AIGC-Detector.

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6005–6024 August 11-16, 2024 ©2024 Association for Computational Linguistics
The widespread use of large language models (LLMs) has sparked concerns about the potential misuse of AI-generated text, as these models can produce content that closely resembles human-generated text. Current detectors for AI-generated text (AIGT) lack robustness against adversarial perturbations, with even minor changes in characters or words causing a reversal in distinguishing between humancreated and AI-generated text. This paper investigates the robustness of existing AIGT detection methods and introduces a novel detector, the Siamese Calibrated Reconstruction Network (SCRN). The SCRN employs a reconstruction network to add and remove noise from text, extracting a semantic representation that is robust to local perturbations. We also propose a siamese calibration technique to train the model to make equally confident predictions under different noise, which improves the model’s robustness against adversarial perturbations. Experiments on four publicly available datasets show that the SCRN outperforms all baseline methods, achieving 6.5%-18.25% absolute accuracy improvement over the best baseline method under adversarial attacks. Moreover, it exhibits superior generalizability in crossdomain, cross-genre, and mixed-source scenarios. The code is available at https://github. com/CarlanLark/Robust-AIGC-Detector.

heading: 1 Introduction
text: Large Language Models (LLMs) such as GPT-4 (Achiam et al., 2023) have shown great promise in producing text that closely mimics human language (Wang et al., 2023b; Chiang and Lee, 2023; Park et al., 2023). However, concerns about the misuse of AI-generated text (AIGT) have arisen in various areas, including the spread of fake news (Hanley and Durumeric, 2023), academic dishonesty
*Work done during ByteDance Research internship. †Corresponding author.
(Perkins et al., 2023), and gender bias (Wan et al., 2023). To tackle these issues, various AIGT detection methods have been developed, using statistical features from language models and text features from different model architectures and training approaches (Solaiman et al., 2019; Gehrmann et al., 2019; Mitchell et al., 2023; Guo et al., 2023).
Current AI-generated text (AIGT) detectors can effectively identify AI-generated text but struggle with minor adversarial perturbations, such as word substitutions or character swapping (Peng et al., 2023; Cai and Cui, 2023). Small changes that do not change the original text’s meaning can cause these detectors to fail. Figure 1 shows a concrete example: a RoBERTa-based AIGT detector can be fooled into classifying AI-generated text as human-written by simply abbreviating "California" to "Calif." This example underscores the limitations of relying solely on token-level features. Therefore, developing robust AIGT detection methods that rely on high-level features is crucial to counteract adversarial perturbation attacks.
To address these challenges, we introduce the Siamese Calibrated Reconstruction Network (SCRN), which consists of an encoder, a reconstruction network, and a classification head. The model first converts input texts into token represen-
6005
tations, then introduces random Gaussian noise to simulate a perturbation attack. The reconstruction network, acting as a denoising auto-encoder (Vincent et al., 2008), aims to remove this noise and recover the original representations. The classification head processes these denoised features to produce the final result. During training, we optimize both classification and reconstruction losses, encouraging the model to learn representations that are resilient to random input perturbations.
Empirically, we observe that a model trained for robustness against random perturbations may not necessarily be robust against adversarial perturbations. To address this issue, we introduce a training technique called siamese calibration. During training, the model generates two classification results using two independent sets of random noise. The training procedure aims to minimize the symmetric Kullback–Leibler (KL) divergence between the two output probability distributions. Since KL divergence is sensitive to changes in probabilities at all confidence levels, the model can incur a significant loss even when it makes consistently correct predictions but with varying confidence levels due to different noise. This stronger constraint forces the model to make equally confident predictions regardless of the noise. In experiments, we find that this approach encourages the model to rely more on high-level contextual features, thereby significantly enhancing its robustness against adversarial attacks.
Our contributions are as follows: (1) We introduce a reconstruction network that
enhances the model’s robustness by promoting the learning of resilient representations under tokenlevel perturbations.
(2) We propose a siamese calibration technique that trains the model to make predictions with consistent confidence levels for various random perturbations, which improves its robustness against adversarial attacks.
(3) We establish a comprehensive benchmark for assessing the robustness of AIGT detection methods against a range of adversarial perturbations, including word-level and character-level substitution, deletion, insertion, and swapping. This benchmark encompasses a wide variety of detectors, such as metric-based and model-based detectors, trained using different methods. We evaluate these detectors on four publicly available datasets to test their robustness in in-domain, cross-domain, cross-genre, and mixed-source scenarios.
(4) Our experiments on the benchmark show that SCRN significantly outperforms all baselines in terms of robustness, achieving higher accuracy under adversarial perturbation attacks. Specifically, our method improves over the best baseline method by 11.25, 18.25, 14.5, and 15.75 absolute points of accuracy under attack in in-domain, cross-domain, cross-genre, and mixed-source scenarios, respectively.

heading: 2 Related Work
text: In recent years, Large Language Models (LLMs) like GPT-2 (Radford et al., 2019) and GPT-3 (Brown et al., 2020) have shown impressive performance in various natural language generation tasks (Kamalloo et al., 2023; Wang et al., 2023b; Cheng et al., 2023b; Chiang and Lee, 2023; Park et al., 2023; Qin et al., 2023). However, the advent of more advanced models such as GPT-4 (Achiam et al., 2023) has raised concerns about the potential misuse of AI-generated texts (AIGT) in areas like fake news (Hanley and Durumeric, 2023; Zhou et al., 2023), academic cheating (Perkins et al., 2023; Foltynek et al., 2023), and ingroup bias (Wan et al., 2023; Gallegos et al., 2023). This highlights the importance of robust detection mechanisms to ensure the security and reliability of applications using LLMs.
To differentiate between human-written and AIgenerated texts, various AIGT detection methods have been developed (Gehrmann et al., 2019; Ippolito et al., 2020; Uchendu et al., 2021; Guo et al., 2023; Mitchell et al., 2023). These methods fall into two categories: metric-based and model-based. Metric-based methods use a language model to generate scores for the text and create statistical features from them, such as probability score (Solaiman et al., 2019), rank score (Mitchell et al., 2023), and entropy score (Gehrmann et al., 2019). Model-based methods, on the other hand, employ neural network architectures and supervised learning to train detectors end-to-end using labeled text. For example, OpenAI trained a RoBERTa model to detect GPT-2-generated text (Solaiman et al., 2019), while (Guo et al., 2023) developed a ChatGPT detector based on question-answer text from various domains.
However, research shows that both metric-based and model-based detectors are susceptible to adversarial perturbations (Cai and Cui, 2023; Peng et al., 2023), such as synonym replacement (Ren et al.,
2019; Jin et al., 2020). This means that minor word changes can lead to incorrect classification of AIgenerated texts in various fields, including news, education, and finance.(Peng et al., 2023) While robust methods have been introduced in related areas like sentiment analysis (Wang et al., 2023c) and speech recognition (Cheng et al., 2023a), a comprehensive analysis of the resilience of AIGT detectors against adversarial perturbations remains lacking. In this work, we aim to explore the robustness of existing detectors against adversarial perturbations, both in-domain and out-of-domain.

heading: 3 Siamese Calibrated Reconstruction Network

heading: 3.1 Model Architecture
text: Encoder Given a dataset D = {(x, y)}, where x = (w1, w2, ..., wn) is the input text with n tokens and y denotes the binary label (human or AI), we use a pre-trained RoBERTa (Liu et al., 2019) as the encoder to obtain the text representation:
[h1, h2, ..., hn] = RoBERTa(w1, w2, ..., wn)
where hi ∈ Rd are the encoded tokens. Reconstruction Network To make the detector more robust to text perturbations, we simulate an actual perturbation by adding random noise to each hi, then utilize a reconstruction network to remove the noise. To inject noise, we split the token representation into a semantic term and a perturbation term, where the former retains the semantic meaning and the latter contains noise. This approach is inspired by (John et al., 2019), which separates the text representation into semantic and style terms to control text style.
Specifically, for the i-th token representation hi ∈ Rd, we use an MLP encoder to map it to a lower-dimensional latent space:
zi = MLP (enc)(hi)
where zi ∈ Rdz is the i-th latent representation. We define a semantic term z(s)i and a perturbation term z(p)i based on zi as follows:
z (s) i = W (s)zi + b (s)
z (p) i = W (p)zi + b (p)
Here, W (s) ∈ Rdz×dz ,W (p) ∈ Rdz×1, b(s) ∈ Rdz , b(p) ∈ R are trainable parameters.
Then we combine the two terms to define a noisy latent representation:
z̃i = z (s) i + ϵ · z (p) i · I
where ϵ ∼ N (0, 1) is a standard Gaussian noise and I ∈ Rdz is a scalar vector with each entry equal to 1.
To control the numerical scale of latent representations, we introduce a regularization term:
Lreg(x)= 1
n
n∑
i=1
( ∥z(s)i ∥22+|z (p) i | 2 −α·log(|z(p)i |) )
Here, the first two terms serve to penalize the large numerical scale of the latent representations, while the third term aims to prevent the noise scale from decreasing to zero. The hyperparameter α allows for adjusting the degree of penalty applied.
Finally, an MLP decoder is applied to reconstruct the original token representation based on the noisy latents:
h (re) i = MLP (dec)(z̃i)
where h(re)i ∈ Rd represents the reconstructed i-th token representation. The reconstruction error is the mean square error between the reconstructed representation and the original one:
Lmse(x) = 1
n
n∑
i=1
||h(re)i − hi||22
The final reconstruction loss is the sum of the reconstruction error and the regularization term, where β is a hyperparameter.
Lre = − 1 |D| ∑
x∈D (Lmse(x) + β · Lreg(x))
Classification Head After obtaining the reconstructed token representations, we use a max pooling layer to extract the feature of the final classification:
h(cls) = MaxPooling([h(re)1 , ..., h (re) n ]).
Then we predict the label ŷ using a MLP classifier. The classification loss is the standard cross-entropy loss, denoted by Lcls.

heading: 3.2 Siamese Calibration
text: The reconstruction loss helps the detector learn robust representations against random perturbations, but it does not guarantee robustness against targeted adversarial perturbations. Empirically, we find that the model, optimized with both reconstruction and classification losses, remains vulnerable to adversarial attacks.
To address this issue, we propose a siamese calibration training strategy. This strategy aims to minimize the symmetric Kullback-Leibler (KL) divergence between the outputs of two inference branches, each subjected to independent random noises, given the same input. Specifically, let P (x, ϵ) be the predicted class distribution for input x with noise ϵ. The symmetric divergence is then defined as the average of DKL(P (x, ϵ)||P (x, ϵ′)) and DKL(P (x, ϵ′)||P (x, ϵ)), where ϵ and ϵ′ are independent copies of random noise, and DKL represents the Kullback–Leibler (KL) divergence. Unlike the cross-entropy loss, which is negligible for correct predictions with high confidence, the symmetric divergence is sensitive to all confidence levels. For instance, consider the correct label is AI. If the first branch predicts AI with a probability of 0.99 and the second branch predicts AI with a probability of 1− δ perturbated by a different random noise, where δ < 0.01, the cross-entropy loss for both branches is less than 0.01, providing little incentive for further optimization. However, the
symmetric divergence between these two distributions can approach infinity as δ → 0. We define the siamese calibration loss, denoted as Lsc, as the symmetric divergence across all training inputs x ∈ D. This loss specifically penalizes inconsistent confidence levels, imposing a stricter requirement than the cross-entropy loss. By minimizing this loss, the detector is encouraged to focus on highlevel contextual features that are less susceptible to token perturbations.
For training the SCRN model, we make a weighted summing over the above three losses:
Lall = λ1(Lcls + L′cls) + λ2(Lre + L′re) + λ3Lsc
where Lcls,L′cls are classification losses of two branches, Lre,L′re are reconstruction losses, and Lsc is siamese calibration loss. λ1, λ2, λ3 are hyperparameters.
Siamese calibration is only applicable during training. During inference, the detector generates a single copy of random noise and produces a single prediction. Training with siamese calibration significantly enhances the model’s prediction consistency during inference. Analysis in Appendix A.7 shows that the fluctuation in inference robustness between two independent branches becomes negligible after implementing siamese calibration.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Datasets To assess the robustness of AIGT detectors to adversarial perturbations, we conduct experiments on four public datasets: HC3(Guo et al., 2023), TruthfulQA(He et al., 2023), Ghostbuster(Verma et al., 2023), and SeqXGPT-Bench(Wang et al., 2023a). These datasets encompass a wide range of AIGT scenarios, including in-domain, cross-domain, cross-genre, and mixed-source. The datasets and number of samples used for training and evaluating AIGT detectors are listed in Table 1. For dataset details, see Appendix A.1.
Metrics Following the approach of (Wang et al., 2023c), we assess model accuracy and robustness using four metrics: (1) Original Accuracy (OA% ↑) measures the model’s raw accuracy without adversarial perturbations. (2) Accuracy Under Attack (AUA% ↑) quantifies the model’s accuracy on adversarially perturbed text. (3) Attack Success Rate (ASR% ↓) indicates the percentage of test samples successfully fooled by the attacker. (4) Average Number of Queries (ANQ ↑) represents the average number of adversarial attack queries on test samples, with higher values indicating a more robust model. The symbols ↑ and ↓ denote that higher and lower values are better, respectively. Additionally, we report traditional Precision, Recall, and micro-F1 scores when no attack is performed.
Adversarial Perturbations In our experiments, adversarial perturbations are conducted by three attack methods: PWWS (Ren et al., 2019), DeepWord-Bug (Gao et al., 2018), and Deep-WordBug (Gao et al., 2018). These methods involve various types of adversarial perturbations including character-level and word-level substitution, deletion, and insertion. More details can be found in Appendix A.2
Baselines To create a comprehensive benchmark, we do our best to select a wide range of de-
tector methods. These include metric-based detectors such as Log-Likelihood(Solaiman et al., 2019), Log-Rank(Mitchell et al., 2023), Entropy(Gehrmann et al., 2019), GLTR(Gehrmann et al., 2019), and SeqXGPT(Wang et al., 2023a), along with model-based detectors like Bert(Devlin et al., 2019), Roberta(Liu et al., 2019), Deberta(He et al., 2020), and ChatGPTDetector(Guo et al., 2023). We also implemented recent methods for enhancing classification robustness in other NLP tasks, such as Flooding(Ishida et al., 2020), RDrop(Wu et al., 2021), RanMASK(Zeng et al., 2023), and RMLM (Wang et al., 2023c). Details can be found in Appendix

heading: A.3.
text: Experiment Settings To ensure a fair comparison of the AIGT detectors, all models were trained on 8 * 32GB NVIDIA V100 GPUs and evaluated on a single 32GB NVIDIA V100 GPU, using the same environment. We utilized the base versions of pre-trained Bert, RoBERTa, and DeBERTa models as employed in the compared detectors. The implemented methods were based on their officially released code, and for the ChatGPT-Detector, we utilized the provided model weight. The hyperparameters of SCRN and more information can be found in Appendix A.4.

heading: 4.2 In-domain Robustness
text: In the in-domain scenario, where the test data comes from the same domain as the training data (HC3), our SCRN model demonstrates superior robustness against various types of adversarial perturbations from three attack methods. As illustrated in Table 3, SCRN achieves a notable improvement in Accuracy Under Attack (AUA), with a 6.5-11.25 absolute increase compared to the best baseline detector. The AUA values for SCRN remain high, exceeding 91.25 under all three attack methods, only marginally lower than its original accuracy without any attacks. Table 3 also reveals that evasion attacks (tricking the model into classifying AI-generated answers as human, de-
noted by AI→Human) are easier than obfuscation attacks (tricking the model into classifying humangenerated answers as AI, denoted by Human→AI) on HC3. This is likely because human-generated answers are diverse (Ma et al., 2023), making it challenging to perturb them all to resemble AIgenerated responses.
In the absence of adversarial perturbation, all detectors except RanMASK demonstrate high performance, as illustrated in Table 2. RanMASK’s lower accuracy can be attributed to its masking of 30% of the text during both training and inference, resulting in significant information loss.

heading: 4.3 Cross-domain Robustness
text: In the cross-domain setting, where the test data (TruthfulQA) and training data (HC3) come from different domains but share the same genre (question answering), SCRN significantly outperforms other baseline models. As Table 4 and Table 6 demonstrates, SCRN achieves a notable improvement, with at least a +18.25 increase in absolute Accuracy Under Attack (AUA). Although absolute accuracy tends to be lower in cross-domain scenarios, SCRN’s margin of lead is even more pronounced.

heading: 4.4 Cross-genre Robustness
text: In the cross-genre setting, the test data (Ghostbuster) is from a different genre than the training data (HC3). Specifically, Ghostbuster comprises news articles, essays, and creative writings, while HC3 is a question-answering dataset. As shown in Table 5 and Table 7, SCRN outperforms
all baseline models in overall Accuracy Under Attack (AUA) and Attack Success Rate (ASR).
We observe that SCRN’s AUA is lower than some baselines under obfuscation attacks (Human→AI). However, SCRN is substantially more robust under evasion attacks (AI→Human), achieving 71% AUA, while baseline models’ scores are near zero. This indicates that in the cross-genre setting, minor perturbations on AI-generated content can easily lead to a “Human” prediction by baseline models, as they are not trained on AIgenerated content like news articles, essays, and creative writings. As a by-product, obfuscation attack against these models become very hard. Practically speaking, defending against evasion attacks is more important. SCRN demonstrates balanced robustness, underscoring its generalizability.
It is also noteworthy that RMLM, a model obtained through adversarial training, shows good robustness in the in-domain setting but fails in crossdomain and cross-genre settings. This suggests that merely augmenting the training set with adversarial data is insufficient to enhance the model’s robustness against out-of-distribution samples (Wang et al., 2022).

heading: 4.5 Mixed-source Robustness
text: Tables 8 and 9 present the results on the SeqXGPT-Bench dataset, which comprises AIgenerated content from various LLMs in mixedsource scenarios. Notably, SCRN shows excellent performance both with and without attacks.
Specifically, model-based detectors like SCRN significantly outperform metric-based detectors in the absence of an attack. However, the accuracy of all models decreases with the PWWS attack. Compared to the best-performing baseline, RMLM, our SCRN achieves a notable improvement of 15.75 in Accuracy Under Attack (AUA), consistent with other settings.

heading: 4.6 Ablation Study
text: In this section, we perform an ablation study to evaluate the effectiveness of our key design choices. To assess the impact of siamese calibration, we train the SCRN model without it, denoted as SCRNSC. Table 10 shows that the detector’s robustness significantly worsens across all four AIGT scenarios. Notably, the accuracy under attack (AUA) of SCRN-SC is lower than the baseline RoBERTa detector (SCRN-SC-R) in the in-domain setting. This decline may be attributed to SCRN-SC’s inclusion
of random noise during inference, which, without
siamese calibration, leads to unstable predictions
under adversarial attacks. These findings underscore the importance of siamese calibration.
To examine the role of the reconstruction network, we replace it with a simple dropout layer, resulting in the SCRN-R model. As depicted in Table 10, SCRN-R experiences a decrease in Accuracy Under Attack ranging from 18.0 to 33.25 across the four scenarios. This decline occurs be-
cause the dropout layer merely omits text information without simulating adversarial perturbations, which involve more complex editing actions such as substitution, insertion, and deletion.
Regarding the noise of the reconstruction network, completely removing the noise ϵ (SCRN-ϵ) leads to a significant decrease in the detector’s robustness, particularly in cross-domain scenarios of
29.5 AUA drop. Furthermore, when we dropped the regularization loss Lreg, although the noise still existed, it degenerated during the optimization of Lmse. The AUA scores of SCRN-Lreg demonstrated that maintaining a degenerated noise showed better robustness compared to completely removing the noise (SCRN-ϵ). However, SCRNLreg performed significantly worse than SCRN, which highlights the necessity of the regularization loss Lreg in preserving the noise’s effectiveness and maintaining the desired robustness.

heading: 4.7 More Analysis
text: We also provide a threshold analysis in Appendix A.5, a comparison of inference speeds in Appendix A.6, an analysis of inference fluctuations in Appendix A.7, and case studies in Appendix A.8.

heading: 5 Conclusion
text: While AI-generated text (AIGT) detection is promising for various applications, it struggles with the robustness of current methods against adversarial perturbations. To tackle this, we introduce the Siamese Calibrated Reconstruction Network (SCRN). SCRN uses a reconstruction network to model text perturbations and employs siamese calibrated training to improve inference robustness. Experiments demonstrate SCRN’s effectiveness in defending against adversarial perturbations in four AIGT scenarios, highlighting its practical utility in addressing real-world AIGT detection challenges.

heading: Limitations
text: Although SCRN demonstrates excellent robust performance across all four scenarios, including indomain, cross-domain, cross-genre, and mixedsource AIGT detections, it still has several limitations:
(1) We did not consider the text paraphrasing attack (Tulchinskii et al., 2024; Macko et al., 2024) as a form of text perturbation. Our focus was primarily on adversarial perturbations with minor modifications, and we regarded paraphrased text as completely modified. Strictly, if a paraphrased AI-generated text becomes the same as an existing human-created text, it should be assigned a humancreated label by the AIGT detector. Future work may conduct a further more detailed analysis of the paraphrased text.
(2) Our experiments mainly focused on English corpora, and while our proposed method is general, we did not explore its performance on multilingual corpora. We leave the detailed analysis of multilingual datasets in future work.

heading: Acknowledgements
text: The work is supported by the National Key R&D Program of China (Nos. 2022YFA1005201, 2022YFA1005202, 2022YFA1005203) and the NSFC Major Research Plan - Interpretable and General Purpose Next-generation Artificial Intelligence (No. 92270205).

heading: A Appendix

heading: A.1 Datasets
text: HC3 (Guo et al., 2023): We conducted our indomain experiments using the HC3 dataset, which is a compilation of human and ChatGPT answers from QA answers across four fields, including media, wiki, medicine, and finance. The dataset contains both English and Chinese data, but for our experiments, we focused on the English corpus. This corpus comprises 26,903 human texts and 58,546 ChatGPT texts. Following the methodology described in (Guo et al., 2023), we randomly partitioned the dataset, allocating 90% for training and 10% for testing. To evaluate the robustness of the AIGT detectors against adversarial perturbation attacks, we randomly selected 200 human-created samples and 200 AI-generated samples from the test set. TruthfulQA (He et al., 2023): To evaluate the
detector robustness in cross-domain scenarios, we utilized the TruthfulQA dataset. This dataset comprises human-created answers and AI-generated answers for 817 questions spanning 38 diverse fields, such as law, finance, health, and politics. It provides a suitable environment for assessing the crossdomain robustness of AI-generated text methods. In our experiments, we trained all the compared detectors on the HC3 train set and tested them on 817 human-created text and 817 ChatGPT-generated text of the TruthfulQA dataset. To evaluate the robustness of the detectors, we randomly selected 200 human-created samples and 200 ChatGPTgenerated samples from TruthfulQA as test samples. Ghostbuster (Verma et al., 2023): For evaluat-
ing the detector robustness in cross-genre scenarios, we employed the Ghostbuster dataset. This dataset consists of 3,000 parallel human-created articles and AI-generated articles from student essays, news articles, and creative writing. Similarly, we trained all the compared detectors on the HC3 train set, which compiles QA answers. We then assessed the detectors on articles from the Ghostbuster dataset. The robustness of detectors against adversarial attacks is evaluated by randomly selecting 200 human-created articles and 200 ChatGPTgenerated articles from Ghostbuster dataset. SeqXGPT-Bench (Wang et al., 2023a): To assess
the detector robustness under mixed-source AIgenerated texts, we utilized the SeqXGPT-Bench dataset. This dataset consists of parallel human-
created articles and AI-generated articles from various sources, including GPT-2, GPT-Neo, GPT-J, LLaMa, and GPT-3. It encompasses different domains such as news, social media, the web, scientific articles, and technical documents. For our experiments, we incorporated all 6,000 humancreated texts and randomly selected one parallel AI-generated text from each of the five different AI sources, resulting in 6,000 parallel AI-generated texts from diverse sources. We allocated 90% of the data for training and reserved the remaining 10% as the test set. Again, for the adversarial perturbation evaluation, we randomly chose 200 human-created samples and 200 AI-generated samples from the test set.

heading: A.2 Adversarial Perturbations
text: To evaluate the impact of adversarial perturbations on the AIGT detectors, we leverage three adversarial attack methods that encompass character-level and word-level substitution, deletion, and insertion.
PWWS (Ren et al., 2019), a widely utilized adversarial attack method, efficiently performs synonym substitution based on word saliency scores and maximum word-swap variance.
Deep-Word-Bug (Gao et al., 2018) incorporates random word-level substitution, swapping, deletion, and insertion, mimicking real-world human activities.
Pruthi (Pruthi et al., 2019) introduces adversarial perturbations by altering a small number of characters, resembling common typos. It encompasses character substitution, deletion, and insertion.

heading: A.3 Baselines
text: To evaluate the performance of our proposed model, we compared it against several baseline models that are commonly used in AIGT detection. These baseline models include:
Log-Likelihood (Solaiman et al., 2019) employs the average token-level log probability generated by a language model as a feature. It trains a machine-learning classifier to determine whether a human or machine generates the text. A higher log probability indicates a higher likelihood that the text is AI-generated.
Log-Rank (Mitchell et al., 2023) utilizes the logarithm of the probability rank of each token generated by a decoder-only language model. A lower rank suggests a higher likelihood that the text is AI-generated.
Entropy (Gehrmann et al., 2019) computes the entropy value for each token based on the preceding text and then averages these scores to create the final feature.
GLTR (Gehrmann et al., 2019) constructs features based on the number of tokens in the top-10, top-100, top-1000, and top-1000+ ranks according to the token-level probability rank generated by the language models. The idea is that AI-generated tokens are more likely to belong to the head of the distribution during decoding.
SeqXGPT (Wang et al., 2023a) ensembles log probability scores from multiple language models and trains a CNN-based transformer model to detect the AI-generated sentences.
Bert (Devlin et al., 2019) is a widely used language model that demonstrates significant advancements in various NLP tasks, including many classification tasks.
Roberta (Liu et al., 2019) and Deberta (He et al., 2020) exhibit improved generalization ability compared to Bert, benefiting from additional training enhancements.
ChatGPT-Detector (Guo et al., 2023) is a RoBERTa-based detector tuned on the HC3 dataset, which achieved state-of-the-art performance on ChatGPT text detection.
Additionally, we compared our method with recent robust methods that enhance classification robustness in other NLP tasks:
Flooding (Ishida et al., 2020) alleviates the model’s overfitting by introducing an additional threshold into the training loss. This prevents the loss from decreasing further when it is already lower than the threshold, resulting in improved robustness.
RDrop (Wu et al., 2021) leverages dropout layers to generate similar text representations and then aligns the outputs to be the same. It reveals better generalization in several NLP tasks. In our experiments, we use a RoBERTa-base encoder as the base model.
RanMASK (Zeng et al., 2023) utilizes ensemble inference on several randomly masked text sequences copied from the input text. This approach enhances the model’s robustness, particularly when encountering input word changes. In our experiments, we follow (Zeng et al., 2023) to use a RoBERTa-base encoder as the base model and set the mask percentage to 30%.
RMLM (Wang et al., 2023c) is an adversarial training method that adds extra adversarial samples
to the training data. It aims to defend the adversarial text attack by corrupting the adversarial text and then correcting the abnormal contexts. We follow the settings in their paper to train a BERT-based AIGT detector in our experiments.

heading: A.4 Experiment Settings
text: We employed the RoBERTa-base encoder as the foundation of our model. To ensure a fair comparison, we selected the base versions of the pre-trained Bert, RoBERTa, and DeBERTa encoders used in the models under comparison. The detector training was conducted on 8 * 32GB NVIDIA V100 GPUs within the same environment. Subsequently, we evaluated the detectors under an adversarial perturbation attack on a single 32GB NVIDIA V100 GPU within the same environment.
Regarding the hyperparameters, we did not tune any of them in our experiment. For model training, we used a linear decay schedule with an initial learning rate of 1e-4. Consistent with (Guo et al., 2023), we trained all the compared detectors for 2 epochs on both the HC3 dataset and the SeqXGPTBench dataset. More detailed hyperparameter values can be found in Table 11.
To implement the adversarial attack methods, we utilized the open-source Textattack package (Morris et al., 2020). We implemented the compared models using their officially released code. For the ChatGPT-Detector, we directly utilized the model weight they released 1.
1https://huggingface.co/Hello-SimpleAI/ chatgpt-detector-roberta

heading: A.5 Threshold Analysis
text: In Section 4.2 - 4.5, we set the binary classification threshold to 0.5 to demonstrate the performance of AIGT detectors in general. However, the consequences of misclassifying human-created and AIgenerated text in real-world scenarios may differ. Mislabeling human-created text as AI-generated text can have severe implications, such as affecting education exams and causing harm to innocent individuals. On the other hand, misclassifying AIgenerated content as human-created can lead to the dissemination of harmful misinformation, resulting in public confusion or even social unrest.
To evaluate the robustness of AIGT detectors under different misclassification cost variations, we conducted experiments following the 1% false positive rate (FPR) setting proposed in (Hans et al., 2024; Krishna et al., 2024; Soto et al., 2024). Specifically, we evaluated the detectors under a fixed 1% FPR, where either AI-generated or human-created text is considered the positive sample. The training and testing were performed on SeqXGPT-Bench, which involves the generation of AI-generated text by mixed-source language models, better simulating the challenges encountered in real-world applications.
As shown in Table 12, when considering AIgenerated text as positive and using the threshold under the 1% FPR setting, our SCRN detector achieves the highest AUA score, signifying superior robustness, while remaining competitive in terms of the OA score. Similarly, as illustrated in Table 13, when setting the threshold for 1% FPR with human-created text as positive samples, SCRN consistently demonstrates the best robustness, as indicated by the AUA score, and maintains competitive accuracy without attack, as measured by the OA score.
A.6 Inference Speed Comparison
Since the inference speed is an essential metric to evaluate the AIGT detector, we compared the inference speed of our proposed SCRN model with three other models: the RoBERTa backbone model, the previous state-of-the-art robust method RanMASK, and the adversarial training method RMLM.
Our SCRN model demonstrates highly effective inference speed when compared to the other AIGT models. As presented in Figure 3, in real-world inference service scenarios with a batch size of one, our model achieves an inference speed that is
1.91× faster than RMLM. This speed advantage is primarily because RMLM involves result selection from a twice-forward process. Additionally, our model achieves an inference speed that is 28.5× faster than RanMASK, as RanMASK requires en-
sembling hundreds of results. When compared to the RoBERTa backbone detector, our model maintains 92.7% of RoBERTa’s inference speed while surpassing up to 25.75 of accuracy under attack (AUA) achieved by RoBERTa as shown in Table 14.
For larger batch sizes, our SCRN model also demonstrates significantly faster inference speed compared to RMLM and RanMASK, with only a slight decrease in speed when compared to RoBERTa.
A.7 Inference Fluctuation Analysis
To assess the impact of inference fluctuation between two sub-model branches of SCRN, particularly due to randomness introduced by the reconstruction layer, we conducted a comparison of the detectors’ accuracy under attack (AUA). Fig-
ure 4 presents the AUA fluctuation on HC3 dataset, considering five different random seeds. Notably, the AUA fluctuation remains below a 2.5 accuracy score across all scenarios. This observation indicates that the randomness introduced by the reconstruction process does not significantly affect the robustness of the SCRN detector. Thus, the gap between training and inference can be neglected.
This resilience can be attributed to the implementation of siamese calibration during the training of SCRN. The siamese calibration strategy aims to minimize the discrepancy in output distributions between the two branches of the model. Consequently, the classifier layer of SCRN is encouraged to prioritize high-level features over token-level features. This preference for high-level features enhances the model’s robustness against token-level noise originating from the reconstruction process.
In contrast, when SCRN is trained without siamese calibration, the AUA fluctuation increases significantly, reaching up to a 14.75 accuracy score. This ablation experiment serves as further evidence of the effectiveness of siamese calibration in enhancing the robustness of the detector.

heading: A.8 Case Study
text: As part of our research, we conducted a case study involving four cases from the Ghostbuster dataset, representing a cross-genre scenario. In this study, we compared our model with the previous stateof-the-art robust model, RMLM, which has been trained using adversarial techniques.
Figure 5 illustrates Case #1 and Case #2, which demonstrate word perturbations and character perturbations, respectively. While RMLM improves its robustness against adversarial perturbations by incorporating additional adversarial data during training, it fails to maintain robust detection in the cross-genre scenario. This observation highlights that simply augmenting the training set with adversarial data is not sufficient to effectively enhance the model’s robustness against out-of-distribution samples (Wang et al., 2022). In contrast, our proposed SCRN model does not rely on extra prior information from the training set. As a result, SCRN successfully defends against cross-genre attacks, achieving superior robustness performance, as demonstrated in Table 16.
Moving on to Case #3 and Case #4, these cases explore the effects of word perturbations and character perturbations in human-created text, respectively. In comparison to AI-generated text, humancreated text poses a greater challenge for attacks. It requires a higher proportion of manipulated words or characters to deceive the AIGT detector successfully.

heading: A.9 Details of Cross-domain AIGT Detection under Adversarial Perturbations
text: Table 15 shows the robustness performance of all compared AIGT detectors on TruthfulQA dataset under three adversarial attack methods. The results consistently demonstrate the superior robustness of SCRN in cross-domain AIGT detection.

heading: A.10 Details of Cross-genre AIGT Detection under Adversarial Perturbations
text: Table 16 shows the robustness performance of all compared AIGT detectors on Ghostbuster dataset under three adversarial attack methods. The results consistently demonstrate the superior robustness of SCRN in cross-genre AIGT detection.

heading: A.11 Details of Mixed-source AIGT Detection under Adversarial Perturbations
text: Table 17 shows the robustness performance of all compared AIGT detectors on SeqXGPT-Bench dataset under three adversarial attack methods. The results consistently demonstrate the superior robust-
ness of SCRN in mixed-source AIGT detection.

heading: A.12 AI Assistant Statement
text: Following the ACL 2023 Policy on AI Writing Assistance, we use AI assistant purely for the language of the paper, containing spell-checking, grammar-checking, and polishing our original content without suggesting new content. We affirm that all words refined by the AI assistant have been carefully reviewed and either rechecked or modified by us.

heading: Future Work
text: Future work may conduct a further more detailed analysis of the paraphrased text. (2) Our experiments mainly focused on English corpora, and while our proposed method is general, we did not explore its performance on multilingual corpora. We leave the detailed analysis of multilingual datasets in future work