Generative Adversarial Training with Perturbed Token Detection for Model Robustness
Authors: Jiahao Zhao
Wenji Mao
Venue: EMNLP 2023 Main (2023)
Keywords: generative adversarial training
adversarial defense
adversarial detection
discriminative pre-trained model

Abstract: Adversarial training is the dominant strategy towards model robustness. Current adversarial training methods typically apply perturbations to embedding representations, whereas actual text-based attacks introduce perturbations as discrete tokens. Thus there exists a gap between the continuous embedding representations and discrete text tokens that hampers the effectiveness of adversarial training. Moreover, the continuous representations of perturbations cannot be further utilized, resulting in the suboptimal performance. To bridge this gap for adversarial robustness, in this paper, we devise a novel generative adversarial training framework that integrates gradient-based learning, adversarial example generation and perturbed token detection. Our proposed framework consists of generative adversarial attack and adversarial training process. Specifically, in generative adversarial attack, the embeddings are shared between the classifier and the generative model, which enables the generative model to leverage the gradients from the classifier for generating perturbed tokens. Then, adversarial training process combines adversarial regularization with perturbed token detection to provide token-level supervision and improve the efficiency of sample utilization. Extensive experiments on five datasets from the AdvGLUE benchmark demonstrate that our framework significantly enhances the model robustness, surpassing the state-of-the-art results of ChatGPT by 10% in average accuracy.

Generative Adversarial Training with Perturbed Token Detection for Model Robustness
Jiahao Zhao, Wenji Mao"
Institute of Automation, Chinese Academy of Sciences
School of Artificial Intelligence, University of Chinese Academy of Sciences [zhaojiahao201 wenji maajji.maaa... mat.mamaaaiaa.... cn
Abstract
The adversarial vulnerability of deep learning models long-standing problem (Goodfellow et al., 2015). Various attack methods have demon- strated that even LLMs can be deceived with small, intentionally crafted perturbations (e.g.. typos and synonym substitution) (Jin et al., 2020; Li et al., 2020; Chen et al., 2021; Wang et al., 2022; Liu et al., 2022a; Wang et al., 2023b). In response adversarial attacks, many adversarial defense meth- ods have been proposed to enhance model robust- ness. Among them, adversarial training (Miyato et al., 2019; Wang and Bansal, 2018; Zhu et al., 2020; Wang et al., 2021c; Ni et al., 2022; Xi et al., 2022) is widely recognized as the most effective approach, which involves continuous perturbations. In adversarial training, the gradient-based perturba- tions are introduced the embedding layer while solving a min-max problem. Another form of ad- versarial training is adversarial augmentation (Jin et al., 2020; Li et al., 2020; Si et al., 2021; Ivgi and Berant, 2021; 1; Maheshwary et al., 2021; Liu et al., 2022a), which utilizes discrete perturbations. In ad- versarial augmentation, the search for adversarial examples is treated tion problem and the model is trained using the augmented adversarial examples.
Adversarial training is the dominant strategy towards model robustness. Current adversarial training methods typically apply perturbations to embedding representations, whereas actual text-based attacks introduce perturbations as discrete tokens. Thus there exists gap be- tween the continuous embedding representa- tions and discrete text tokens that hampers the effectiveness of adversarial training. Moreover, the continuous representations perturbations cannot be further utilized, resulting in the sub- optimal performance. To bridge this for adversarial robustness, in this paper, we devise novel generative adversarial training frame- work that integrates gradient-based learning, adversarial example generation and perturbed token detection. Our proposed framework con- sists of generative adversarial attack and ad- versarial training process. Specifically, in gen- erative adversarial attack, the embeddings are shared between the classifier and the generative model, which enables the generative model to leverage the gradients from the classifier for generating perturbed tokens. Then, adversarial training process combines adversarial regular- ization with perturbed token detection to pro- vide token-level supervision and improve the efficiency of sample utilization. Extensive ex- periments on five datasets from the AdvGLUE henchmark demonstrate that framework sig- nificantly enhances the model robustness, sur- passing the of-tc-of-the-art results of ChatGPT by 10% in average accuracy.
combinatorial optimiza-
However, existing adversarial training methods have encountered non-trivial challenges in real- world applications. In adversarial training, the per- turbations applied at the embedding layer create a gap between the continuous perturbations used during training and the discrete perturbations that real attack applies in the testing phase (Xu et al., 2020). On the other hand, adversarial augmenta- tion demands high computational cost, typically with hundreds to thousands of queries per example (Maheshwary al., 2021). Consequently, the con- tinuous representations of perturbations cannot be further utilized in adversarial training, leading to the suboptimal performance.
1 Introduction
Pre-trained language models (PLMs) and large language models (LLMs) (Ouyang et al., 2022) have made remarkable advancements in natural lan- guage processing (NLP). Given their increasing im- pact science, socicty and individuals, adversarial robustness has become crucial task for building trustworthy NLP systems (Kaur al., 2022).
"Corresponding author
In this work, we aim to address the afore-

---

mentioned issues and make substantial step towards adversarial robustmess. We propose novel Generative Adversarial Training (GenerAT) framework that consists of generative adversarial attack and adversarial training process. To bridge the gap between continuous perturbations and dis- crete text tokens, generative adversarial attack gen- erates discrete perturbed tokens based on gradients. Specifically, our gradient-based attack calculates adversarial gradients with the classifier through for- ward and backward propagations, which is more efficient than Search-basee adversarial augmenta- tion. The accumulated gradients are then shared at the embedding layer between the classifier and the generative model, so that they can be leveraged to guide the perturbed token generation. Besides, generative
The adversarial training process in our frame- work further combines adversarial regulariza- tion with perturbed token detection, which improves the efficiency of sample utilization.
Extensive experiments five datasets demon- strate the effectiveness of our generative adver- sarial training framework, which consistently improves robustness by a large margin.
Related Works
2.1 Adversarial Defense and Detection
Adversarial training is acknowledged as the most effective defense method. In text domain, adversar- ial training (Sato et al., 2018; Zhu et al., 2020; Li and Qiu, 2021; Wang et al., 2021a; Li et al., 2021; Ni al., 2021; Pan et al., 2022) solves the min- max optimization problem and injects continuous adversarial perturbations in the embedding layer, which leaves gap between real discrete text per- turbations. Adversarial training in continuous rep- resentation is an approximation of the real discrete adversarial perturbations in text. Previously devel- oped adversarial training methods try improve the approximation (Li et al., 2021; Pan al., 2022). However, this is hard problem and the quality of the approximation significantly affects model ro- bustness. Our generative adversarial training takes the approach to directly bridge the gap between continuous embedding representations and discrete text tokens, and generates adversarial replaced to- kens based gradients.
to get more robust representations, adversarial attack is built upon the discriminative PLM (He al., 2023), which is capable of dis- tinguishing subtle semantic differences between similar words compared to other PLMs like BERT that most defense methods concentrate on.
Adversarial training process utilizes adversarial regularization to further improve robustness, by restricting the representations between the origi- nal and corresponding adversarial examples. As the generated perturbed tokens can provide fine- grained teken-level supervision, adversarial regu- larization integrated with perturbed token detec- tion the training process to improve sample us- age efficiency. We conduct robustness experiments on the challenging AdvGLUE benchmark (Wang et al., 2021b), which applies multiple types of ad- versarial attacks on five datasets from GLUE (Wang al., 2018). Extensive experiments show that framework significantly improves adversarial ro- bustness on all the datasets, surpassing ChatGPT (Wang et al., 2023b) by 10% in average accuracy and establishing new state-of-the-art results.
Adversarial augmentation (Jin al., 2020; Li et al., 2020; Si et al., 2021: Ivgi and Berant, 2021; Maheshwary et al., 2021; Liu et al., 2022a) searches the most effective replacement of tokens to generate adversarial examples by solving com- binational problem and then using the augmented examples to retrain the model. However, these methods need over hundreds of iterations per ex- ample find the corresponding replacement which brings high computational cost. To tackle these challenges, we propose a novel generative adver- sarial training framework that leverages gradients for the generation of discrete perturbed tokens.
The main contributions are as follows:
We propose the first generative adversarial training framework for adversarial robustness. Based on discriminative PLM, our framework provides a comprehensive means to integrate gradient-based learning, adversarial example generation and perturbed token detection.
Another line of work is adversarial detection (Zhou al., 2019; Mozes et al., 2021; Nguyen-Son et al., 2022; Li al., 2023) which focuses on de- tecting perturbed tokens. These methods typically detect replaced tokens and subsequently restore them to their original form, allowing the model
The generative adversarial attack in frame- work exploits gradient propagation through sharing embeddings between the classifier and the generator, which enables the generator to effectively generate perturbed tokens.

---

make predictions on clean, restored data. In contrast to these methods, in our framework, we incorporate perturbed token detection to provide token-level supervision and enhance efficiency in the training process.
3.1 Preliminaries
Problem Formulation We consider a general text classification problem with dataset 'D = {0, }i= consisting of inputs text æq E X and labels yi E V. Adversarial attack aims to deceive the victim model by generating adversarial exam- ples. Our goal is train the model on 'D to max- imize adversarial robustness, that is, increase model accuracy on adversarial examples.
2.2 Pre-trained Language Models
BERT (Devlin et al., 2019) is an encoder-only lan- guage model that was trained with the masked lan- guage modeling (MLM) task. It represents sig- nificant milestone in transformer-basee PLMs and reveals the great potential of PI.Ms. After that, diverse forms of PLMs have emerged. One cate- gory is decoder-only PLMs, such as GPT-3 (Brown et 2020), OPT (Zhang et al., 2022) and BLOOM (Scao et al., 2022). These auto-aggressive language models are trained with causal language model- ing (CLM). Another category is encoder-decoder PLMs like T5 (Raffel et al., 2020) and FLAN-T5 (Chung et al., 2022). These models convert a va- riety text-based language problems into text-1o- text format and train the model as a sequence-lo- sequence generation problem.
Discriminative Pre-trained Models Discrimina- tive PLMs such as ELECTRA (Clark et al., 2020) and DeBERTaV3 (He et al., 2023) differ from BERT (Devlin et al., 2019) in their pre-training task. Instead of masked language modeling (MLM), dis- criminative PLMs employ a replaced token detec- tion (RTD) approach. In this setup, a generator and a discriminator are jointly trained. The generator, which is a transformer encoder, is trained on MLM to generate ambiguous tokens that substitute the original tokens in the input sequence. On the other hand, the discriminator, larger transformer en- coder, is trained token-level binary classifier determine whether the tokens have been replaced by the generator. The discriminator is then used for fine-tuning in downstream tasks.
In contrast, discriminative PLMs, such ELEC- TRA (Clark et al., 2020) and DeBERTa-v3 (He al., 2023) have received comparatively less at- tention. discriminative PLM contains two trans- former encoders. One is the generator trained with MLM, and the other the discriminator trained with replaced token detection (RTD) task, that de- termines whether the tokens are replaced by the generator. We argue that discriminative language models learn the representations by distinguish- ing the subtle semantic difference between simi- lar words, leading to more robust representations compared to BERT-style context-based cloze com- pletion tasks. Therefore, our framework is built upon discriminative PLMs. A detailed comparison between these models is provided in Appendix A.
3.2 Generative Adversarial Attack
Adversarial attacks deceive the victim model by ap- plying small but intentionally worst-case perturba- tions to the original input. For example, in FGSM (Goodfellow et al., 20155) attack, the adversarial gradients are used as the perturbation:
7 - sign [ U))
(1)
where r is the perturbation, is the hyperparameter that controls the perturbation size, and J denotes the loss function used for training the victim model. While pixels in image are continuous values that can be directly added with perturbations, text data are discrete tokens in nature. As a result, exist- ing gradient- based methods (Sato et al., 2018; Zhu et al., 2020; Li and Qiu, 2021; Li et al., 2021; Pan et al., 2022) apply adversarial gradients to embed- ding representation of text:
3 Proposed Method
Adversarial training typically applies perturbations to embedding representations, whereas real-world adversarial examples introduce perturbations at dis- crete text tokens. To bridge the gap between contin- uous embeddings and discrete tokens, we propose novel generative adversarial training framework GenerAT that consists of generative adversarial at- tack and adversarial training process. We present the overall design of our framework as well as each component in this section.
Badv F F(x)) +n
(2)
where a addv is the virtual adversarial example of æ. Fi(x)) is the embedding of æ.
However, actual text perturbations introduced by adversarial attacks, such as synonym replace-

---

Step 1: Calculate Adversarial Gradients
Step 2: Generate dvvesarial Examples
positive negative
Merge
enjoy
this
Funny
film
A Adversrrial Gradient
Classifter
MLM Head
Discriminator Encoder
Generator Encoder
(Transformer)
(Transformer)
Shared Embedding
Embedding Layer
Embedding Layer
Apply Mask
like
this
Funny movie
MAAK
this
(MASK] [MASK]]
Figure 1: Overview of generative adversarial attack. The adversarial gradients are first calculated on the discriminator and accumulated to the embedding layer. Through the shared embedding with the generator, the generator leverages the gradients and generates perturbed tokens.
ment, are applied to discrete tokens. The gap be- tween continuous perturbations and actual discrete tokens limits the effectiveness of adversarial train- ing. Therefore, to bridge the gap, introduce novel generative adversarial attack.
Generate Adversarial Examples A generator is employed to generate perturbed tokens. The generator is pre-trained language model with an MLL header that is able to generate semantically closed tokens of the masked tokens.
As shown in Figure 1, we integrate one discrim- inator the victim model and one generator to generate adversarial perturbed tokens. We first calculate the gradients the victim model. The adversarial gradients are passed from the discrim- inator to the generator through the shared embed- ding layer. Then generator generates adversarial perturbed tokens with adversarial gradients.
Similar to MLM, we first mask some percentage of the input tokens at random.
a' fusk (x)
(4)
where fmask(() is the mask strategy and x' is the masked sample.
Then the generator predicts the masked tokens. The probability for a particular masked token T; is:
Calculate Adversarial Gradients We calculate gradients on the discriminator. Input text x is passed through the discriminator with a classifier header to calculate the loss. Using binary classifica- tion an example, the loss function of the victim model is cross entropy:
Pda (xa, x + n) = softmax (Ho (x' + m))
(5)
where 77 is the perturbation and 0G denotes the parameter of the generator.
As mentioned before, we share the embedding layer between the discriminator and the generator. Consequently, gradients calculated on the discrimi- nator are accumulated on the embedding layer and can propagate through the masked token predic- tion of the generator. In this way, fill the gap between continuous gradients and discrete text to- kens.
[zx y) --y y log (xx) (1 y) log(1 (xx)
g = (xr
(3)
where o--) denotes the sigmoid function, is the gradient respect to (x, y).
Lastly, we merge the generated token with the original tokens:
By integrating (3) with (1) and (2), the gradients are accumulated at the embedding layer. To enhance the training stability, we apply the per- turbations to the normalized word embeddings.
4; { { 4; p0c (2i x +n),
i C
(6)
i i C

---

Task
I
positive negative
like
Discriminator Encoder (Transformer)
Embedding Layer
Classifier Layer
this
SKI
funny
movie
Shared Parameters
L LPDD
original riporni
Classifier Layer
enjoy
Discriminator
Embedding Layer
replaced
this
Encoder (Transformer)
replaie
funny
Perturbed Token Detection Layer
original replaced
film
replaced
Figure 2: Overview of adversarial training process. By utilizing the generated adversarial examples in attack, the training process detects the perturbed tokens in the adversarial examples, and also regularizes the model behavior between clean and adversarial examples.
where C. is set of masked token positions, a [ 2...... (n] is the generated adversarial exam- ple corresponding to
With LPTD. the model learns to distinguish in- tentionally misleading tokens. This teven-level supervision facilitates robustness and enhances the efficiency of sample utilization.
3.3 Adversarial Training Process
Adversarial Regularization To enhance adver- sarial robustness, we employ the symmetric KL- divergence as the adversarial regularization term. which promotes consistency in model performance between original æ and adversarial examples a.
Figure 2 illustrates the adversarial training process. After generating adversarial examples, the discrim- inator trained using adversarial regularization and perturbed token detection techniques, which improves the robustness of the model.
LSKL "DKL(PeD (.) pep (x))
DDL (pe,, (x) pe, (.))
Firstly, same the standard fine-tuning, the dis- criminator trained with cross-entropy loss per- form the classification task:
(9)
where (..) is the KL-divergence between distributions.
C
Dask = CE (x:()) (x), = Σ
yi log (pop (x))
Learning and Optimization The trainable pa- rameters the discriminator include the embed- ding layer, discriminator encoder, classifier layer, and perturbed token detection layer. The classifier layer is implemented as shallow MLP, while the perturbed token detection layer is also implemented as shallow MLP. The discriminator encoder sists of stack of transformers.
(7)
where d denotes the parameter of the discrimina- tor.
Perturbed Token Detection With the adversarial example & is fed into the discriminator, perturbed token detection layer is trained to classify each token in & as either original replaced.
Towards adversarial robustness, the overall loss is obtained by integrating (7), (8) and (q.
C = L + APTDDCPr + ASKI (10)
- log (1 (+; = x;) i)
LPTD -
where XPTD and ÀSKL are hyperparameters that control the balance between perturbed token detec- tion and adversarial regularization. The hyperpa- rameter search is provided in Appendix E.
(8)
where 11() is the indicator function, & is the adver- sarial example constructed with (6)

---

The training objective of the discriminator is to minimize the overall loss L. During the adversarial generative attack, the parameters are not updated. The attack responsible for generating adversarial examples. And both the attack and training pro- cesses are jointly applied within each batch. To enhance training efficiency, parameters in GenerAT are initialized using pre-trained weights from the discriminative PLM.
2022) adds regularization between examples in the same batch.
LLMs: BART-L (Lewis et al., 2020), GPT- J-6B (Wang, 2021), Flan-T5-L (Chung et al.,
2022), GPT-NEOX 20B (Black et al., 2022). OPT-66B (Zhang et al., 2022), BLOOM (Scao et al., 2022), GPT-3 (text-davinci-002 and text- davinci-003) and ChatGPT.
4 Experiments
Following the convention (Zhu et al., 2020; Li and Qiu, 2021; Li al., 2021; Pan al., 2022; Wu et al., 2023), the accuracy on adversarial examples the metric for robustness, with higher accuracy indicates better robustness.
To evaluate the effectiveness of our framework, compare GenerAT with the SoTA methods on five tasks. We also conduct additional experiments on model parameter analysis and provide case study.
4.3 Implementation Details
4.1 Tasks and Datasets
We use DeBERTa-v3-large as the pre-trained dis- criminative language model. We train our model single V100 GPU. Details of hyperparame- ters and training costs are provided in Appendix E, The results of comparative methods are based on the results reported in Tong et al. (2022); Wu et al. (2023); Wang et al. (2023a).
We conduct our experiments on AdvGLUE (Wang et al., 2021b), the most representative and widely used robustness evaluation benchmark. It con- sists of five challenging tasks in GLUE (Wang al., 2018): Sentiment Analysis (SST-2), Dupli- cate Question Detection (QQP), and Natural Lan- guage Inference (NLI, including MNLI, RTE, and QNLI). In the construction of adversarial exam- ples, AdvGLUE applies 14 textual adversarial at- tacks including various word-level, sentence-level perturbations, and human-crafted examples. The constructed adversarial examples are validated by human annotators. In experiments, we employ the development set of AdvGLUE since its test is not publicly available.
Our codes are provided in https://gis thub. com/Oppdoop/GenerA...
4.4 Main Results
Table gives the adversarial robustness results of our framework and the baselines the AdvGLUE benchmark. We can see from the table that across all tasks, GenerAT significantly outperforms com- parative basclines by a large margin.
In comparison to adversarial training and robust fine-tuming approaches, GenerAT exhibits signifi- cant enhancement in adversarial robustness, nearly doubling the accuracy For instance, the SoTA of these methods (Match- Tuning) only achieves an average accuracy of 45.7, while GenerAT achieves an average accuracy of 80. This illustrates that adding perturbations to continuous embedding layer is sub-optimal in text domain, as the real attack adds perturbations at dis- crete tokens. GenerAT bridges this gap between continuous embedding representation and discrete tokens, which significantly improves the adversar- ial robustness of the model.
4.2 Baselines and Evaluation Metric
We evaluate the performances of GenerAT by com- paring it with the state-of of thethe art (SoTA) adversar- ial training methods, robust fine-tuning methods and large language models (LLMs).
adversarial examples.
Adversarial training: FreeLB (Zhu et al., 2020) adds adversarial perturbations to em- bedding and minimizing the risk in different regions. CreAT (Wu et al., 2023) is an ad- versarial training that finds perturbations in contextual representation.
Robust fine-tuning methods: R3F (Agha- janyan et al., 2021) applies noise the orig- inal pre-trained representations with regular- ization. ChildTuning (Xu al., 2021) masks subset of parameters and only updates the child network. Match-Tune Tuning (Tong et al.,
Compared autoregressive language models, is observed that increasing the parameter size slightly improves the model's robustness. For in- stance, transitioning from GPT-J-6B to GPT-3 (text-

---

Method
advSST-2
advQQP
advMNLI--m advQNLI
advRTE
Avg
Adversarial Training Methods with BERT-base Model (Wu et al., 2023)
Vanilla Fine-tuning (110 M)
32.3
50,8
32.6
40.1
37.0
38.6
FreeLB (110 M)
31.6
51.0
33.5
43.4
42.0
40.7
BERT MLM (110 M)
32.0
48.5
27.6
43.4
45.9
39.5
BERT CreAT (110 M)
35.3
51.5
36.0
44.8
45.2
42.6
Robust Fine-tuning Methods with BERT-large Model (Tong et al., 2022)
Vanilla Fine-tuning (340 M)
47.6
38.5
35.0
46.4
41.7
41.8
R3F (340 M)
38.5
40.6
35.8
47.5
50.1
42.5
ChildTuningr (340 M)
34.5
40.4
33.9
47.5
42.0
39.6
ChildTuningD (340 M)
39.2
40.7
34.1
49.6
46.2
41.9
Match-Tuning (340 M)
51.4
41.5
35.5
47.5
52.5
45.7
State-of-the-art Large Language Models (Wang et al., 2023a)
BART-L (407 M)
43.9
37.2
41.3
48.0
43.2
42.7
GPT-J-6B (6 B)
51.3
41.0
26.4
50.0
43.2
42.4
Flan-T5-L. B)
59.5
41.0
51.2
50.0
43.2
49.0
GPT-NEOX-20B (20
47.3
43.6
40.5
46.0
51.9
45.9
ОРТ-66B (66 B)
52.4
46.1
39.7
47.3
42.0
45.5
BLOOM (176 B)
51.3
41.0
26.4
50.0
43.2
42.4
text-davinci-O00 (175 B)
54.0
71.8
45.4
54.7
64.2
58.0
text-davinci-003 (175 B)
55.4
44.9
55.4
61.5
65.4
56.5
ChalGPT (175 B)
60.1
82.0
67.8
65.5
75.3
70.1
Robust Training Methods with DeBERTa-v3-large Model
GenerAT (436 M)
69.6
89.7
78.5
73.6
88.9
80.1
Table 1: Adversarial robustness results the AdvGLUE benchmark. We report the accuracy values adversarial examples. The best performing scores are in bold. Models are ranked by parameter size. Avg stands for the average accuracy on AdvGLUE. Our GenerAT surpasses the baselines by large margin.
davinci-002) leads to an average accuracy increase from 42.4 to 58.0. However, models with the same parameter size exhibit considerable performance variation, with BLOOM only achieving 42.4 in average accuracy. Notably, the Encoder-decoder structure of Flan- T5-L, incorporating instruction tuning, outperforms other autoregressive language models of similar scale, such as GPT-NEOX- 20B, underscoring the importance of model architecture for adversarial robustness. Among LLMs, Chat- GPT achieves the highest average accuracy of 70.1. GenerAT utilizes DeBERTa-v3-large as its base PLM. With less than 1% of parameters compared ChatGPT, GenerAT surpasses ChatGPT by 10% in average accuracy. This result demonstrates the effectiveness of our approach. Table gives the results of GenerAT on clean GLUE dev datasets. can be seen from the table that, in contrast to clas- sical adversarial training, our GenerAT framework does not sacrifice the performance on clean data.
4.5 Results on Ablation Study
To evaluate the effectiveness of GenerAT, we con- duct the ablation study on its different variances:
PTD: removing the perturbed token detection
(PTD) module from GenerAT.
GAA: removing the generative adversarial at-
tack (GAA) and LSKL from GenerAT.
GAA PTD: removing both GAA and PTD modules from GenerAT.
Table 3 gives ablation results on different com- ponents of GenerAT in advGLUE datasets. It can be seen from the table that the individual modules of GAA and PTD each play pivotal role in en- hancing adversarial robustness. Specifically, when GAA removed, the average accuracy declines 73.7. Similarly, in the absence of PTD, the aver- age accuracy declines to 71.1. When both GAA and PTD are removed simultaneously. the model

---

GLUE dev
SST-2
QQP
MNLI-m
MNLI-mm
QNLI
RTE
Avg.
GenerAT
96.3
89.7
91.4
91.5
95.5
89.9
92.4
Table Accuracy results of GenerAT on clean GLUE dev datasets. MNLI-m is the matched version of MNLI and
MNLI-mm is the mismatched version of MNLI.
Finetuning
GAA
PTD
advSST-2
advQQP
advMNLI-m
advQNLI
advRTE
Avg
x
X
59,2
69.3
64.2
63.2
79.0
66.9
/
X
62.1
75.6
68.5
65.5
83.9
71.1
V
65.4
78.2
71.9
67.8
85.2
73.7
V
69.6
89.7
78.5
73.6
88.9
80.1
Table 3: Ablation results on different components of GenerAT advRTE dataset.
Base Model
Layers
Parameters
advSST-2
advQQP
advMNLI-m
advQNLI
advRTE
Avg
DeBeRTa-v3-large
436M
69.6
78.5
73.6
88.9
80.1
DeBIRTa-v3-xsmall
52.0
56.8
61.5
56.5
Table 4: Effect of parameter scales GenerAT, where Avg stands for the average accuracy AdvGLUE datasets.
degrades to vanilla finetuning and the average ac- curacy declines to 66.9. This indicates that both GAA and PTD are important for enhancing model robustness. The results of the ablation study further verify the effectiveness of each component in our framework.
80
bert-base
bert-large
Average Accurscy on AdvGLUE
deberts--3-xemall
deberta-v3-large
GPT-J-6B
Flan-T5-L
GPT-NEOX-208
OPT-66B
4.6 The Effect of Parameter Size
60
BLOOM
text-davinci-002
In order to assess the impact of parameter size on the performance of GenerAT, we conducted an evaluation utilizing reduced version of DeBERTa. The average performance AdvGLUE of Gen- erAT based on DeBERTa-v3- and DeBERTa- v3-large are shown in Table 4. We observe that downscale the parameter size from 436M 71M, the average accuracy drops from 80.1 to 56.5, which suggests a larger base model is important for robustness. The experimental findings reveal that employing larger parameter sizes in conjunc- tion with deeper network architectures markedly improves the model's performance in the presence of adversarial examples and highlight the impor- tance of having sufficiently large parameter space to learn robust representations. And DeBERTa-v3- large a suitable choice in this regard.
text-davinc--033
40
ChatGPT
10
10'
10"
Model Parameters (Billions)
Figure 3: Illustration of model robustness results.
ChatGPT. These findings suggest that simply in- creasing the model size does not inherently mitigate the issue of adversarial vulnerability. Instead, the structure and training strategy of the model em- ployed in downstream tasks play crucial role in adversarial robustness.
4.7
Case Study: Perturbed Token Detection
We further visualize the perturbed token detection results on adversarial examples in Table 5. the table, perturbed positions are marked in the origi- nal example with gray shadows and the detection results of the corresponding adversarial example marked with yellow shadows. The shade of the color represents the normalized perturbed proba- bility. For instance, in the advSST-2 example, the word 'wildly' in the original example replaced with 'strangely" in the adversarial example. The
We also visualize the relationship between model size and performance in Figure 3. We can that GenerAT based on DeBERTa-v3-xsmin achieves comparable robustness GPT-3. Gen- eTAT based on DeBERTa-large achieves nearly twice the robustness compared to models with sim- ilar parameter scales, such as BERT-large. More- over, GenerAT also surpasses other LLMs like

---

Dataset
Type
Text Content
s too much forced drama in this wildly uneven movic about a young battle with his inescapable past and uncertain future in a very sha pable but largely un fulfilling present
Original
Example
advSST-2
8 so much foreed drama in this strangely uneven movie, about a young man battles with his inescapable past and uncertain future the very sha pable but often un fulfilling present
Adversarial
Example
What would happen an astronaut dies in space while on ISS SEEP
Original Example
What would it be like if an astronaut died aboard the International Space Station
advQQP
What would happen if one astronaut dies in space while on Mars [SEP] What would feel like if an astronaut died within the International Flight Station ?
Adversarial
Example
CLS) Because of the limited money available the first grants were restricted funding for civil legal services and hotline 8 [SEP] Due to limited money available companies had to cut spending
Original
Example
adv.MNII-m
[CLS] Because of scarce limited money available. the first efforts were restricted in funding for civil legal services and hotline support [SEP] Due inadequate money available companies had to cut spending A large scale solar distillation project was first constructed in 1872 in the Chilean mining town of Las Salinas [SEP] When was the first large solar distillation plant created
Adversarial
Example
Original Example
advQNLI
A large- scale water distillation project was first established in 1872 near the Chilean mining town of Las Salinas [SEP] When was the first large solar distillation plant constructed
Adversarial Example
Companies are working to reduce the interval between drug discovery and marketing... [SEP] Companies are working to shorten the new drug development period to an average of eight to nine years in the US
Original Example
advRTE
Companics are working to shorten the roadblocks between drug discovery and marketing [SEP] Investigators working to shorten the new drug development period from an average of eight to nine years in the US
Adversarial Example
Table 5: Qualitative results of perturbed token detection. In the original example, the token position of perturbations are marked gray. the adversarial example, the detection results are marked in yellow. The darker the color, the more likely the token is replaced.
darker the color, the more likely the token is re- placed. We can see from the table that perturbed token detection is skeptical of most tokens and discovers the real perturbed tokens with high con- fidence. This indicates that further enhancing the accuracy of perturbed token detection may lead the increase of robustness. Thus the ability to detect perturbed tokens can provide valuable token- level supervision signals, making it crucial task for improving model robustness.
in existing adversarial training methods. Our pro- posed framework integrates gradient-bart based learn- ing, adversarial example generation, and perturbed token detection for improving adversarial robust- ness. The generative adversarial attack shares the embeddings between the classifier and the gener- ative model, which allows the generative model to utilize gradients from the classifier for generat- ing perturbed tokens. Then the adversarial training process incorporates perturbed token detection into adversarial regularization, which provides token- level supervision and enhances sample usage effi- ciency. Extensive experiments on five datasets in AdvGLUE benchmark demonstrate that our frame- work significantly boosts the model robustness.
5 Conclusion
This paper presents novel generative adversarial training framework to bridge the gap between em- bedding representations and discrete text tokens

---

Limitations
Samuel Weinbach. 2022. GPT-NeoX 20B:: An open- source autoregressive language model. In Proceed- ings of the ACI. Workshop Challenges & Perspec- rives Creating Large Language Models.
In our generative adversarial attack, we omit se- mantic constraints for the sake of efficiency. Al- though some prior research suggests that semantic constraints are important for preserving the origi- nal semantic meaning when generating adversarial perturbations, we observed that the automatic con- straint metric is sensitive across different datasets. And establishing appropriate threshold for se- mantic constraints often necessitates significant hu- man involvement to ensure rationality. However, our experimental results show that even without explicit semantic constraints, our approach still effective. The impact of semantic constraints is left for future research.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in neural information process- ing systems, pages 1877-1901.
Yangyi Chen, Jin Su, and Wei Wei. 2021. Multi- granularity textual adversarial attack with behavior cloning. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pages 4511-4520
Hyung Won Chung, Le Hou, Shayne Longpre, Bar- ret Zoph, Yi Tay. William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling misttuetion-iinetunee language models. arXiv preprint arXiv:2200.11 11411.
As we focus on adversarial robustness in down- stream tasks, the evaluation of our GenerAT frame- work has been limited robustness benchmarks. The potential of this framework in scenarios in- volving data shift, out-of-distribution samples, and other situations remains a topic for future research.
Kevin Clark, Minh-Thaas Thang Luong, Quoe V. Lc, and Christopher D. Manning. 2020. ELECTRA: Pre- training text encoders as discriminators rather than generators. In 8th International Conference on Learning Representations.
Ethics Statement
Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics, pages 4171-4186.
Our work focuses on enhancing the adversarial robustness of models in downstream tasks. The proposed GenerAT framework has demonstrated significant improvements in adversarial robustness. Careful investigations are needed to understand the impact of enhancing robustness on existing biases and fairness issues in machine learning systems.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adver- sarial examples. In 3rd International Conference Learning Representations.
Acknowledgments
Pengcheng He. Jianfeng Gao, and Weizhu Chen. 2023, DeBERTav3: Improving deBERTa using ELECTRA- style pre-training with drietannddseentangledd embed- ding sharing. In / 11th International Conference on Learning Representations.
This work is supported in part by the Ministry of Science and Technology of China under Grant #2020AAA0108401, and the National Natural Sci- ence Foundation of China under Grants #62206287, #1 1832000 and #72293575.
Maor Ivgi and Jonathan Berant. 2021. Achieving model robustness through discrete adversarial training. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1529-1544.
References
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. 2021. Better fine-tuning by reducing representational collapse. In International Conference on Learning Representations.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is BERT really robust? A strong baseline for natural language attack on text classifi- cation and entailment. In The Thirty- Fourth AAAI Conference Artificial Intelligence, pages 8018- 8025.
Sid Black, Stella Biderman. Eric Hallahan, Quentin An- thony, Leo Gao, Laurence Golding, Horace He, Con- nor Leaby, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and
Davinder Kaur, Suleyman Uslu, Kaley Rittichier, and Arjan Durresi. 2022. Trustworthy artificial intelli- gence: a review. ACM Computing Surveys (CSUR). 55(2):1-38.

---

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Länguistics, pages 7871-7880.
Maximilian Mozes, Pontus Stenetorp, Bennett Klein- berg, and Lewis Griffin. 2021. ceqquency-guidess word substitutions for detecting textual adversarial examples. In Proceedings of the 16th Conference of the European Chapter of the Association for Compu- tational Linguisties: Main Volume, pages 171-186,
BART: Denoising
Hoang. -uuncg-Quccuuu Nguyen- Huy Quang Ung. Seira Hi- dano, Kazuhide Fukushima, and Shinsaku Kiyomoto. 2022. CheckHARI Checking hard labels for ad- versarial text detection, prediction correction, and perturbed word suggestion. In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2022, pages 2903-2913.
Linyang Li. Ruotian Ma. Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. BERT-ATTACK, Adversarial attack against BERT using BERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6193-6202.
Shiwen Ni, Jiawen Li, and Hung-Yu Kao. 2021. Dropat- tack: A masked weight adversarial training method to improve generalization of neural networks. arXiv preprint arXiv:2108. 12805.
Linyang Li and Xipeng Qiu. 2021. Token-aware vir- tual adversarial training in natural language under- standing. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, pages 8410-8418.
Shiwen Ni, Jiawen Li, and Hung- Yu Kao. 2022. R- AT: Regularized adversarial training for natural lan- guage understanding. In Findings of the Association for Computational Linguisties: EMNLP 2022, page 1427-6440
Linyang Li, Demin Song, and Xipeng Qiu. 2023. Text adversarial purification as defense against adversarial attacks. In Proceedings of the 61 Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroli Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. In Advances in Neu- ral Information Processing Systems, pages 27730- 27744.
Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiao- qing Zheng. Qi Zhang, Kai- Chang, and Cho-Jui Hsich. 2021. Scarching for an effective defender: Benchmarking defense against adversarial word sub- stitution. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3137-3147.
Lin Pan, Chung- Wei Hang. Avirup Sil, and Saloni Potdar. 2022. Improved text classification via con- trastive adversarial training. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, pages 11130-11138.
Aiwei Liu, Honghai Yu, Xuming Hu, Shu' ang Li, Li Lin, Fukun Ma, Yawen Yang, and Lijie Wen. 2022a. Character-level white-bor box adversarial attacks against wransfommers via attachable subwords substitution. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 7664-7676.
Colin Raffel, Noam Shazeer, Adam Roberts. Katherine Lee. Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, :185555455-555.
Qin Liu, Rui Zheng. Bao Rong. Jingyi Liu, ZhiHua Liu, Zhanzhan Cheng, Liang Qiao. Tao Gui, Qi Zhang, and Xuanjing Huang. 2022b. Flooding X: Improv- ing BERT's s resistance to adversarial attacks via loss- restricted fine-tuning. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5634- 5644.
Motoki Sato, Jun Suzuki, Hiroyuki Shindo, and Yuji Matsumoto. 2018. Interpretable adversarial pertur- bation in input embedding space for text. In Pro- ceedings of the -Sventh International Joint Conference on Artificial Intelligence.
Teven Le Scao, Angela Fan, Christopher Akiki, EI- lie Pavlick, Suzana Ilié, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: 176b- parameter open-access multilingual language model. arXiv preprint arXiv:2211.055. 1051000.
Rishabh Maheshwary. Saket Maheshwary, and Vikram Pudi. 2021. A strong baseline for query efficient attacks in black box setting. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1396-8409,
Takeru Miyato, Shin- Ichi Maeda, Masanori Koyama, and Shin Ishii. 2019. Virtual adversarial training: A regularization method for supervised and semi- supervised learning. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 41(8):1979- 1993.
Chenglei Si, Zhengyan Zhang, Fanchao Qi. Zhiyuan Liu, Yasheng Wang, Qun Liu, and Maosong Sun. 2021. Better robustness by more coverage: Adver- sarial and mixup data augmentation for robust fine- tuning. In Findings of the Association for Computa- tional Linguistics: ACL-IJCNLP 2021.

---

Shoujie Tong, Qingxiu Dong, Damai Dai, Tianyu Liu, Baobao Chang. Zhifang Sui, et al. 2022. Robust fine-tuning via perturbation and interpolation from in-batch instances. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelli- gence.
Hongqiu Wu, Yongxiang Liu, Hanwen Shi, hai zhao, and Min Zhang. 2023. Toward adversarial training contextualized language representation. In The Eleventh International Conference on Learning Rep- resentations.
Zhiheng Xi, Rui Zheng. Tao Gui, Qi Zhang, and Xu- anjing Huang. 2022. Efffcientt adversarial training with robust early-bird tickets. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 83 8318-8331.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy. and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for nat- ural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353-355.
Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and Anil K Jain. 2020. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing. 17:151-178.
Ben Wang. 2021. GPT-J-6B: Billion Parameter Autoregressive Language Model. https: thub. com//kingoflolz/meshh transformer- jax.
Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang, 2021. Raise child in large language model: To- wards effective and generalizable fine- mun-tunig. In Pro- ceedings of the 202 Conference on Empirical Meth- ods in Natural Language Processing, pages 9514- 9528.
Boxin Wang, Shuohang Wang, Yu Cheng. Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. 2021a. Info bert]: Improving robustness of language models from an information theoretic perspective. In International Conference on Learning Representations.
Boxin Wang, Chejian Xu, Xiangyu Liu, Yu Cheng, and Bo Li. 2022. SemAttack: Natural textual attacks via different semantic spaces. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 176-205.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre- trained transformer language models. arXiv preprint orXiv:2205.0106k
Boxin Wang, Chejian Xu, Shuohang Wang. Zhe Gan, Yu Cheng. Jianfeng Gao, Ahmed Hassan Awadal- lah, and Bo Li. 2021b. Adversarial GLUE: A multi- task benchmark for robustness evaluation of language models. In Thirty-fifth Conference on Neural In- formation Processing Systems Datasets and Bench- marks Track.
Yichao Zhou, Jyun- Yu Jiang, Kai- Wei Chang, and Wei Wang. 2019. Learning to discriminate perturbations for blocking adversarial attacks in text classification. In Proceedings of the 2019 Conference on Empiri- cal Methods in Natural Language Processing, pages 4913-4913.
Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang. Haojun Huang, Wei Yc, Xiubo Geng, Binxin Jiao, Yue Zhang. and Xing Xie. 2023a. On the rohustness of chatgpt: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2302. 12095.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Gold- stein. and Jingjing Liu. 2020. Freelb: Enhanced adversarial training for natural language understand- ing. In 8th International Conference on Learning Representations.
Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Hao- jun Huang. Wei Ye. Xiubo Geng, et al. 2023b. On the robustness of chatgpt: An adversarial and -iitt--iiitiiitiio perspective. arXiv preprint arXiv:2302. 12095.
Xiaosen Wang, Yichen Yang, Yihe Deng, and Kun He. 2021c. Adversarial training with fast gradient projec- tion method against synonym substitution based text attacks. AAAI Conference on Artificial Intelligence, pages 13997-14005.
Yicheng Wang and Mohit Bansal. 2018. Robust ma- chine comprehension models via adversarial training. In Proceddings of the 2018 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. Volume 2 (Short Pupers), pages 575-581.

---

Architecture
Pre-training Task
Model
Encoder
Encoder- Decoder
Discriniiator
Decoder
MLM
SD
RID
NSP CLM
BERT
FLAN-T5
ELECTRA
/
DeBERTa VI
DeBERTa V3
/
BLOOM
Table 6: Comparison of different language models.
Model
advSST-2
udvQQP
advMNLI-m
advQNLI
advRTE
Avg
3pt-3.5-turbo (zero-shot)
66.9
29.5
63.6
43.9
39.5
48.7
3pt-3.5-turbo (five-shot)
56.9
43.4
35.5
49.4
gpt-3.5-turbo (ten-shot)
76.8
39.7
55.9
44.9
40.9
51.6
GenerAT
69.6
89.7
78.5
73.6
88.9
80.1
Table 7: Robustness results AdvGLUE of GPT-3.5
Comparison between LLMs
the robustness of GPT-3.5.
Table 6 provides a comprehensive comparison of different language models, with primary focus the analysis of architecture and pre-train pre-training tasks, both of which have significant impact on the ac- quired representations. In terms of architecture, dif- ferent designs are considered, including Encoder- only, Decoder-only, Encoder-decoder and Encoder with an additional Discriminator. The choice of pre-training task is closely aligned with the se- lected architecture. Specifically, the pre-training task options encompass Masked Language Model- ing (MLM), Span-Mask Denoising (SD), Replaced Token Prediction (RTD), Next Sentence Predic- tion (NSP), Causal Language Modeling (CLM) and Instruction Finetuning (IF). Our proposed gen- erative adversarial training framework Ge builds on the DeBERTa-v3, where the discrimina- tor is trained using the RTD and the generator is trained using MLM. In contrast MLM or CLM approaches that rely on contextual information for representation learning, we argue that the discrim- inator serves a superior backbone network for adversarial robustness, as acquires representa- tions by discerning similar words. Therefore, we select DeBERTa-v3 as the backbone network of our framework.
C
Robustness under Textfooler Attack
Method
IMDB
AG NEWS
BERT
2.8
19.4
RoBERTa
25.2
25.2
Adv-HodFlip (BERT)
8.0
18.2
FreeLB (BERT)
7.3
20.1
FreeLB++ (BERT)
45.3
RanMASK (RoBERTs)
23.7
TAVAT
27.6
39.7
InfoBERT
27.4
29.2
Flooding
39.5
38.8
Flooding-X
40.5
42.4
Text PurificationBBERT)
51.0
34.9
Text (Rrffic
54.3
34.2
GenerAT
44.0
Table 8: Robustness Results under Textfooler Attack on IMDB and AG NEWS datasets.
Table gives the results under Textfooler attack. We focus on IMDB and AGNEW'S for experimen- tation (as SST-2 and QNLI have been tested under multiple attacks in AdvGLUE). The results of base- lines are derived from (Li et al., 2023) and (Liu et al., 2022b). It can be seen that our generative ad- versarial training method GenerAT still surpasses the compared baselines by large margin.
D Additional Results
B Robustness of GPT-3.5
Table 7 shows the results of GPT-3.5. We the prompt provided (Wang al., 2023b) with ran- domly selected examples. can be seen from the table that sho- examples marginally improve
To ensure fairness and consistency in comparison, Table only presents those results that all the base- lines have reported. Here, we provide additional experiment results in Table

---

advQQP
advMNLI-mm
Method
(FI)
(accuracy)
Adversarial Training Methods with BaRT-base Model
Vanilla Fine-tuning (110 M)
19.3
FreeLB (110 M)
21.9
BERT MLM (110 M)
20.8
BERT CreAT (110 M)
22.0
Robust Fine-ruming Methods with BERT-large Model
Vanilla Fine-tuning (340 M)
27.59
R3F (340 M)
35.23
30.26
Child'Tuiigg (340 M)
35.82
26.53
ChildTuningo (340 M)
39.80
27.84
Match-Tuning (340 M)
32.62
31.07
Stote-of-fhe-or Large Language Models
BART-L (407 M)
GPT-J-6B
Flan-TS-L. B)
GPT-NEOX-222 (20
OPT-66B (66 B)
BLOOM (176 B)
text-davinci-002 (175 B)
lext-davinci-003 (175 B)
ChaiGPT (175 B)
Robust Training Methods with DeBERTa-v3-lary Model GenerAT
86.95
70.52
Table 9: Results on advQQP and advMNLI-mm.
E Model Hyperparameters
Table 10 provides the hyper-parameten for the best performance of GenerAT on each advGLUE dataset. In hyper-parameter search process, the select space of λPTD is (0.2.1,115,2.2.55) and the select space of ÄSKL is (1.5.3554444555.6666..777 APTD and ÄSKL are selected on the advRTD dataset, and the best hyperparameter values subsequently applied the other datasets. The learning rate set within the range of { (1e-5,2e- 6.56-6,7e-6). The total training time is calculated based on a single V100 (32GB) GPU. For the com- plete parameter setup, please refer to our code.
Hypes perameter
advSST
adxQQe 21
advMNLI-m
adN@NLI
ad+ETE
Number of Layers
24
24
24
Hidden Size
1024
1021
1024
1024
1024
Hosk
16
Amention Head Size
64
64
64
Dropout
1.1
0.1
0.1
011
01.1
Learning
Batch Size
16
16
Epech
2
2
Arro
Agn
(Total, blomm)
Table 10: Hyper-parameters GenerAT on AdvGLUE.


Reviews:
[{"review_id": "EdH9AJn77z", "reviewer": "Reviewer_WUQd", "summary": "", "questions": "1. In line 305 and 317, the authors should explain the mean of eta.\n2. Adversarial training has other method training in embedding representations, such as TAVAT[1], InfoBERT[2], Flooding-X[3]. The authors should compare their method with these methods as well.\n3. The authors should evaluate their methods in BERT-base and BERT-large model instead of DeBERTa model, which is more fairer and more convincing.\n[1] Token-aware virtual adversarial training in natural language understanding.\n[2] Infobert: Improving robustness of language models from an information theoretic perspective.\n[3] Flooding-X: Improving BERT’s resistance to adversarial attacks via loss restricted fine-tuning.\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper proposes a novel generative adversarial training framework to boost the robust performance of model.\n2. The proposed method outperform the baseline with a clear margin and it can detect perturbed token accurately.\n", "reasons_to_reject": "1. The authors do not analysis the weakness of training with continuous representations of perturbations and the strength of the proposed method.\n2. The comparison of LLMs and proposed method is not fair. The LLMs are tested in zero-shot setting while this proposed method are trained by adversarial examples before evaluating. \n3. DeBERTa model is different from BERT-base model and the former has better performance than the latter. The author should compare the results with same BERT model.\n", "questions_to_authors": "1. In line 305 and 317, the authors should explain the mean of eta.\n2. Adversarial training has other method training in embedding representations, such as TAVAT[1], InfoBERT[2], Flooding-X[3]. The authors should compare their method with these methods as well.\n3. The authors should evaluate their methods in BERT-base and BERT-large model instead of DeBERTa model, which is more fairer and more convincing.\n[1] Token-aware virtual adversarial training in natural language understanding.\n[2] Infobert: Improving robustness of language models from an information theoretic perspective.\n[3] Flooding-X: Improving BERT’s resistance to adversarial attacks via loss restricted fine-tuning.\n", "paper_topic_and_main_contributions": "This paper first proposes a novel generative adversarial training framework, which bridges the robustness gap of continuous embedding representations discrete text tokens, boosting the robustness of model. In generative adversarial attack, the embeddings are shared to leverage the gradients from the classifier for generating perturbed tokens. Then, adversarial training combines adversarial regularization with perturbed token detection to provide token-level supervision.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Linyang Li and Xipeng Qiu. 2021. Token-aware virtual adversarial training in natural language understanding.\n[2] Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. 2021a Infobert: Improving robustness of language models from an information theoretic perspective.\n[3] Qin Liu, Rui Zheng, Bao Rong, Jingyi Liu, ZhiHua Liu, Zhanzhan Cheng, Liang Qiao, Tao Gui, Qi Zhang, and Xuanjing Huang. 2022. Flooding-X: Improving BERT’s resistance to adversarial attacks via loss restricted fine-tuning.\n"}}, {"review_id": "noGEsES6TR", "reviewer": "Reviewer_LmDX", "summary": "", "questions": "please see the reasons above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. the motivation is strong. it is very intuitive to consider using generative models to detect/purify adversaries in improving model robustness. this paper proposes a framework to accomplish this goal\n2. the method is reasonable: the adversarial training can guide the generative process so that the model can detect the adversaries and therefore improve model robustness against adversaries\n3. the experiments show the effectiveness of the method", "reasons_to_reject": "1. some baseline methods should be compared with this work. e.g.  [1] Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution [2] Certified robustness to text adversarial attacks by randomized [mask] [3] Text Adversarial Purification as Defense against Adversarial Attacks. the purification process used in the image domain is somehow similar to the proposed idea, therefore, it is important to consider these lines of works.\n2. the tests use static datasets while it is somewhat important to see how the proposed framework can actually help resist adversarial attack process such as textfooler.\n3. how do you instruct the ChatGPT models is important, I'm concerned that with some carefully designed instructs, it is possible that ChatGPT can obtain significant improvements.", "questions_to_authors": "please see the reasons above.", "paper_topic_and_main_contributions": "This paper proposes a generative framework to detect adversarial text against adversarial attacks. They propose a framework that utilizes gradient-based adv. training and uses the gradients to guide the generation process sothat the model robustness is improved.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "please see the reasons above"}, "extra_text": {}}, {"review_id": "aR8WtJK1ls", "reviewer": "Reviewer_1YBd", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper proposes a novel generative adversarial training framework to improve model robustness by integrating gradient learning, adversarial sample generation, and perturbation marker detection.\n\n2. This paper addresses the gap between continuous perturbation and discrete text labeling in existing adversarial training methods, which has long been a limiting factor.\n\n3. Extensive experiments on five datasets demonstrate the effectiveness of the proposed generative algorithm training framework, which significantly improves robustness.\n", "reasons_to_reject": "The proposed method is almost the same as the training method for the model ELECTRA.\n\nThe authors may also need to test whether the performance of the proposed method is degraded on non-adversarial datasets.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The topic of this thesis is Generative Adversarial Training with Perturbation Labeling Detection for Model Robustness. The main contribution of the paper is to propose a novel generative adversarial training framework to improve model robustness by integrating gradient learning, adversarial sample generation and perturbation labeling detection. The framework significantly improves the robustness of the model on multiple datasets and exceeds the recent results of ChatGPT. This research fills the gap between continuous perturbation and discrete text markers in existing adversarial training methods and generates discrete perturbation markers via gradient propagation, combined with perturbation marker detection and adversarial regularization to improve sample utilization efficiency.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Ni, Shiwen, Jiawen Li, and Hung-Yu Kao. \"Dropattack: A masked weight adversarial training method to improve generalization of neural networks.\" arXiv preprint arXiv:2108.12805 (2021)."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a generative adversarial training framework that combines three components: gradient-based learning, adversarial example generation, and perturbed token detection. The experiments demonstrate the effectiveness of the proposed method on five benchmark datasets. The reviewers acknowledged the contribution and merit of this work, including the motivation of the research, the proposed method, and empirical evidence from experiments. However, there are also issues with the experiment design, especially the baselines and additional result analysis.
Decision: Accept-Main