Toward Adversarial Training on Contextualized Language Representation
Authors: Hongqiu Wu
Yongxiang Liu
Hanwen Shi
hai zhao
Min Zhang
Venue: ICLR 2023 poster (2023)
Keywords: pre-trained language model
adversarial training

Abstract: Beyond the success story of adversarial training (AT) in the recent text domain on top of pre-trained language models (PLMs), our empirical study showcases the inconsistent gains from AT on some tasks, e.g. commonsense reasoning, named entity recognition. This paper investigates AT from the perspective of the contextualized language representation outputted by PLM encoders. We find the current AT attacks lean to generate sub-optimal adversarial examples that can fool the decoder part but have a minor effect on the encoder. However, we find it necessary to effectively deviate the latter one to allow AT to gain. Based on the observation, we propose simple yet effective \textit{Contextualized representation-Adversarial Training} (CreAT), in which the attack is explicitly optimized to deviate the contextualized representation of the encoder. It allows a global optimization of adversarial examples that can fool the entire model. We also find CreAT gives rise to a better direction to optimize the adversarial examples, to let them less sensitive to hyperparameters. Compared to AT, CreAT produces consistent performance gains on a wider range of tasks and is proven to be more effective for language pre-training where only the encoder part is kept for downstream tasks. We achieve the new state-of-the-art performances on a series of challenging benchmarks, e.g. AdvGLUE (59.1 $ \rightarrow $ 61.1), HellaSWAG (93.0  $ \rightarrow $ 94.9), ANLI (68.1  $ \rightarrow $ 69.3).

Published as a conference paper at ICLR 2023
TOWARD ADVERSARIAL TRAINING ON CONTEXTUAL- IZED LANGUAGE REPRESENTATION
Hongqiu Wu & Yongxiang Liu & Hanwen Shi & Hai Zhao! 1. Min Zhang*
Department of Computer Science and Engineering, Shanghai Jiao Tong University
Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China School of Computer Science and Technology, Soochow University, Suzhou, China (wuhongqiu u, sam. 1 fiu, shi shihan ) @ jtu. edu. en, zhaoha êcs sjtu, edu. cn, mi minzhange suda. edu. en
ABSTRACT
Beyond the success story of adversarial training (AT) in the recent text domain top of pre-trained language models (PLMs), our empirical study showcases the inconsistent gains from AT on some tasks, e.g. commonsense reasoning, named entity recognition. This paper investigates AT from the perspective of the contex- tualized language representation outputted by PLM encoders. We find the current AT attacks lean to generate sub-optimal adversarial examples that can fool the de- coder part but have a minor effect on the encoder. However, we find necessary to effectively deviate the latter to allow AT to gain. Based on the observation, we propose simple yet effective Contextualized (CreAT), in which the attack is explicitly optimized to deviate the contextualized representation of the encoder. It allows a global optimization of adversarial ex- amples that can fool the entire model. We also find CreAT gives rise to a better direction to optimize the adversarial examples, to let them less sensitive to hy- perparameters. Compared
Training
AT, CreAT produces consistent performance gains on a wider range of tasks and is proven to be more effective for language pre- training where only the encoder part kept for downstream tasks. We achieve the new state-of-the-art performances on a series of challenging benchmarks, e.g. AdyGLUE (59.1 + 61.1), HellaSWAG (93.0 94.9), ANLI (68.1 - 69.3).
INTRODUCTION
Adversarial training (AT) GGooffello et al. 2015) is designed to improve network robustness, in which the network is trained to withstand small but malicious perturbations while making correct predictions. In the text domain, recent studies (Zhu et al. 2020 Jiang et al. 2020) show that AT can be well-deployed on pre-trained language models (PLMs) (e.g. BERT (Devlin et al., 2019). RoBERTa (Liu et al.) 2019h) and produce impressive performance gains on number of natural language understanding (NLU) benchmarks (e.g. sentiment analysis, QA).
However, there remains concern as to whether it is the adversarial examples that facilitate the model training. Some studies (Moyer et al.) 2018 AAghaanyynn et al., 202T) point out that a similar performance gain be achieved when imposing random perturbations. To answer the question, we present comprehensive empirical results of AT on wider types of NLP tasks (e.g. reading com- prehension, dialogue, commonsense reasoning, NER). It turns out the performances under AT inconsistent across tasks. On some tasks, AT can appear mediocre or even harmful.
This paper investigates AT from the perspective of the contextualized language representation, ob- tained by the Transformer encoder (Vaswani et al. 2017). The background that PL.M is typically composed of two parts, Transformer-haar based encoder and decoder. The decoder can vary from tasks (sometimes a linear classifier). Our study showcases that, the AT attack excels at fooling the
* *Corresponding author; This paper was partially supported by Key Projects of National Natural Science Foundation of China (U1836222 and 6173301 (ttt0lti/itttt////// thtt:///////uuuuuee/e//p/ com/ san/CAA

---

Published as a conference paper at ICLR 2023
decoder, thus generating greater training risk, while may yield minor impact the encoder part. When this happens, AT can lead to poor results, or degenerate to random perturbation training (RPT) [Bishop 1995).
Based on the observations, we are motivated that AT facilitates model training through robust rep- resentation learning. Those adversarial examples that effectively deviate the contextualized rep- resentation are the necessary driver for its success. To this end, we propose simple yet effective Contextualized representation-Addeersriii Training (CreAT), to remedy the potential defect of AT. The CreAT attack is explicitly optimized to deviate the contextualized representation. The obtained "global" worst-case adversarial examples can fool the entire model.
CreAT contributes to consistent fine-tuning improvement on a wider range of downstream tasks. We find that this new optimization direction of the attack causes more converged hyperparame- ters compared to AT. That means CreAT is less sensitive to the inner ascent steps and step sizes. Additionally, we always re-train the decoder from scratch and keep the PLM encoder weights for fine-tuning, As direct result of that, CreAT is shown to be more effective for language pre-training. We apply CreAT to MLM-style pre-training and achieve the new state-of-the-art performances on series of challenging benchmarks (e.g. AdvGLUE, HellaSWAG, ANLI).
PRELIMINARIES
This section starts by reviewing the background of adversarial training and contextualized language representation, and then experiments are made to investigate the impact of adversarial perturbations on BERT from two perspectives: (1) output predictions and (2) contextualized language represen- tation. The visualization results lead us potential correlation between them and the adversarial training gain.
2.1 ADVERSARIAL TRAINING
Adversarial training (AT) Goodfellow et al. 2013) improves model robustness by pulling close the perturbed model prediction and the target distribution (i.e. ground truth). We denote the output label as y and model parameters as 0, so that AT seeks to minimize the divergence (i.e. Kullback-Leible divergence):
min D (g()))), p(y)xx + 8", 8",
(1)
where q(y x) refers to the target distribution while p(v x + 8*, 0) refers to the output distribution under particular adversarial perturbation 8 The adversarial perturbation defined as the worst- case perturbation which can be evaluated by maximizing the empirical risk of training:
8 arg max D (g(ylx), + 8, 0)]
(2)
where || 5 || e refers to the decision boundary (the Frobenius norm) restricting the magnitude of the perturbation.
In the text domain, a conventional philosophy is to impose adversarial perturbations to word em- beddings instead of discrete input text (Miyato et al. 2017). brings down the adversary from the high-dimensional word space to -ww-imensional representation to facilitate optimization. As op- posed to the image domain, where AT can greatly impair model performances (Madry et al. 2018 Xie et al.] 2219), such bounded embedding perturbations are proven to be positive for text models (Jiang et al., 22200 Zhu et al., 2020; Wang et al., 2021a). However, there is no conclusion as to where such a gain comes from.
2.2 CONTEXTUALIZED REPRESENTATION AND TRANSFORMER ENCODER
Transformer ( /Vaswani al. 2017) has been broadly chosen the fundamental encoder of PLMs like BERT (Devlin et al. 2019). RoBERTa (Liu et al. 2019). XL-Net (Yang et al. 2019). DeBERTa (He al. 2021). The Transformer encoder is stacked-up language understanding system with number of self-attention and werwwrd sub-laye sub-layers, which continuously place each token into the context and achieve its corresponding contextualized representation its output.
2

---

Published as a conference paper at ICLR 2023
.2.5
Right
N-AT
204 aid
cd
DREAM
Just
Avcuracy
Similarity
Similarity
440
Linner
sits
AT
RPT
RPT
CrAA
CreAT
HellasWAG
DREAM
MNLI
551-2
Loss
Loss
Figure I: Left: Performances across different training methods, where N-AT refers to the regular situation where no adversarial training is applied. Middle & right: The relationship between the embedding similarity (contextualized representation) and training loss on different types of tasks. Note that shift the training loss along the horizontal axis for better view, the magnitude of which increases along the axis. We also investigate the attack success rate of AT in Appendix B.
One PLM will be deployed in two stages. For pre-training, the encoder will be trained on large- scale corpus for long period. For fine-tuning, the pre-trained encoder will be attached with task- specific classifier as the decoder, which is initialized from scratch (we do distinguish between the classifier and decoder in what follows). Eventually, the contextualized representation will be re-updated toward the downstream task.
2.3 IMPACT OF ADVERSARIAL TRAINING ON PLMS
To access the impact of AT, we choose four types tasks and fine-tune BERT (Devlin et 2019)
on them respectively: (1) sentiment analysis (SST-2), (2) natural language inference (MNLI), (3) dialogue comprehension (DREAM), (4) commonsense reasoning (HellaSWAG). Among them, the first two tasks are relatively simple, while the last two are much more challenging. We summarize the test accuracies across different training methods in Figure (left).
We calculate two statistical indicators for each training process. The first is the training loss. A higher training loss indicates that the model prediction is more deviated from the ground truth after the perturbation. The second is the cosine similarity of the output embeddings before and after the perturbation. Researchers always take the final output of the encoder (i.c. output embeddings) as the representative of contextual language representation (Li al., 2020a Gao et al. (021). Specifically, we consider the lower bound of all the tokens in the sentence. A lower similarity indicates that the encoder is more disturbed by the perturbation. Note that the indicators here are averaged over the first 20% of the training steps, as the model will become more robust under AT. We depict the above indicators in Figure [ (middle and right). The discussion is below. We also depict random perturbation training (RPT) and CreAT (will be presented later) as references here for better comparison with AT. We will mention this figure again in the next section.
The gain from AT is inconsistent on the four tasks. From Figure I (left), we can see that AT achieves nice results on two sentence- classification tasks (0.9 and 1.2 points absolute gain on MNLI and SST-2). On the other two tasks, AT performs badly, even leading to 3.4 points drop on HellaSWAG.
Observation 1:
Observation 2: AT contributes the most to deviating model predictions. From F (middle and right), we can see that the training loss caused by AT' is always the highest (always on the rightmost side of the graph).
Observation 3: AT weakly influences the contextualized representation on DREAM and HellaSWAG. We can that the resultant similarities of the output embeddings caused by AT are also both high (equivalent to RPT on HellaS and slightly lower than RPT on DREAM). It suggests that the adversarial examples enough to fool the classifier. However, we see an opposite situation on SST-2 and MNLI, where the similarities caused by AT are much lower.
gentle for the encoder, although they malicious
3

---

Published as a conference paper at ICLR 2023
The above observations raise question AT in the text domain: whether fooling model predic- tions is purely correlated with training performances of PLMs? Grouping observation 1 and observation 2, racies). Grouping observation and observation 3, we can see that AT mainly fools the classifier on some of the tasks, but takes little impact the encoder.
that high training loss does not always lead to better performances (i.e. accu-
Summing all observations up together, we can derive that the performance gain from AT on PLMs necessarily driven by the fact that the model needs keep its contextualized representation robust from any adversarial perturbation so that its performance is enhanced. However, this favour can be buried on tasks such as reading comprehension and reasoning, where AT is weakly- effective in deviating the contextualized representation (comparable to RPT). The resultant classifier becomes robust, while the encoder is barely enhanced.
3 CONTEXTUALIZED RERRESENTATION-AAVVERR
TRAINING
In this section, we propose Contextualized repressenaation-Adveesertt Training (CreAT) to effec- tively remedy the potential defect of AT.
3.1 DEVIATE CONTEXTUALIZEH REPRESENTATION
We let h(x, 8) be the contextualized representation (i.e. the final output of the encoder), where refers to the model parameters and refers to the input embeddings. Similarly, we let h(x + 5, 0) be the contextualized representation after x is perturbed by 8.
Our desired direction of the perturbation is to push away the two output states, i.e. decreasing their similarity. We leverage the cosine similarity to measure the angle of deviation between two word vectors Miikolov al. 2013 Gao al.) 2021). Thus, deviating the contextualized representation is the same as:
min (x h(x 5,8)]
(3)
where S(a, b] = |alll
3.2 CREAT
CreAT seeks to find the worst-case perturbation 8" which deviates both the output distribution and contextualized representation, which can be formulated as:
8 arg max D [g(y x), p(y|x + 5, 0) - S (1( h(( + 8)0
(4)
6: Se
where 7 is the temperature coefficient to control the strength of the attacker on the contextual rep- resentation and larger means that the attacker will focus more fooling the encoder. CreAT is identical to AT when T = O.
Why CreAT? In the previous discussion, AT is found to lead to the local worst case that partially fools the decoder part of the model. CreAT can be regarded as the "global" form of AT, which solves the global worst case for the entire model (both the encoder and decoder). The encoder is trained to be robust with its contextualized representation that the model can perform better.
Training with CreAT Different from commonly-used AT methods (Goodfellow et al. 2015 Zhang et al. 2019b] Wang et al. 2020), where the adversarial risk treated as regularizer to en- hance the alignment between robustness and generalization, in this paper, adopt straight- forward method to combine the benign risk and adversarial risk (Laine & Aila 2017 Wang & Wang
2022 Given a task with labels, the training objective is follows:
min XC y, 0) + (1 A)L(x + 8", 0)
(5)
where L is the task-specific loss function and the two terms refer to the training loss under the respective benign example and adversarial example 8. We find that À = 0.5 performs just well
4

---

Published as a conference paper at ICLR 2023
Algorithm Contextualized represeetatioo-AAdverratte Training
Input: Model 0, training model step size 18, ascent step size a, decision boundary c, number of ascent steps k, temperature coefficient r
1: while not converged do
2: (x,9) t -SamleBBatch(T)
3: da f Init()
4:
Forward(x, 0)
D. Go benign forward pass
5:
for
to do Forward(x + 03-1,
6:
Go adversarial forward pass Update the perturbation following Eq.
7:
6, + B==akaaaUUllaaee0:1irrr end for
8:
9:
Forward(x 8)
10: / + BackwardUpdaterr v.
> Update the model parameters following Eq.
end while
Table 1: Results across different tasks over five runs. The average numbers are calculated based on the right side of the table (5 tasks). The variances for all tasks except WNUT and DREAM are low (< 0.3), we omit them. Our CreAT-trained DeBERTa achieved the state-of-the-ar result (94.9) on HellaSWAG (H-SWAG) on May 2022!.
MNLI-m (Acc)
QQP
WNUT
DREAM
H-SWAG
AlphaNLI (Acc)
RACE (Acc)
Avg
Gain
(F1)
(F1)
(Ace)
(Acc)
BERTbase base
71.6
48.60.s
63.0p.9
39.4
65.2
65.3
56.3
FreeLB
85.5
73.1
48 48.21.3
64.10.9
39.8
65.3
62.8
56.4
1 0.1
SMART
85.6
72.7
48.8p.s
64.50.a
39.2
65.1
63.3
56.2
4.11
AT
85.2
72.9
49.71.2
63.8o.a
36.0
64.9
63.0
55.5
J 0.8
CreAT
85.3
73.0
49.90.9
66.0p.7
40.5
67.0
68.0
58.3
1 2.0
in our experiments. Eq. 5 agnostic both pre-training (e.g. masked language modeling Lmmt) and fine-tuning (e.g, named entity recognition Cner) of PLMs.
Algorithm D summarizes the pseudocode of CreAT. The inner optimization based on projected gradient descent (PGD) (Madry et al. 2018). At each training step, which corresponds to the outer loop (line 2 line 10), we fetch the training examples and initialize the perturbation ög- In the following inner loop (line 5 ~ line 7), we iterate to evaluate 8* by taking multiple projected gradient steps. the end of the inner loop, we obtain the adversarial perturbation 8 Ō₁. Eventually, train and optimize the model parameters with the adversarial examples (line 10).
4 EMPIRICAL RESULTS
Our empirical results include both general and robust-learning tasks. The implementation is based on transformers (Wolf et al. 2020).
4.1 SETUP
We conduct fine-tuning experiments on BERT base (Devlin al., 2019). We impose the same bounded perturbation for all adversarial training methods (fix to le-1). We tune the ascent step size a from 11e-1, 1e-2, 1e-3) and the number of ascent steps from {1, 2) following the settings
in previous papers (Jiang al. 2020 Liu et al. 2020).
We conduct MLM-style continual pre-training and obtain two language models: BreTCreAD based base
on BERT base (Devlin al. 2019) DeBERTaCreat based on DeBERTa a (He al. 2021). For the training corpus, we use subset (nearly 100GB) of C4 (Raffel al. 2020). Every single model is trained with a batch size of 512 for 100K steps. For adversarial training, we fix a and to le-1,
https:://eaderrbaark 1enai wogghell waggghhllsswwg/sslle hawwagsubmm ons/public

---

Published as a conference paper at ICLR 2023
Table 2: Results AdvGLUE. For dev sets (upper), report the results five and report the mean and variance for each. For test sets (bottom), the results are taken from the official leader- board, where CreAT achieved the new state-oftthe-art on March 16, 2022
SST-2
QQP
QNLI
MNLI-m
MNLI-mm
RTE
Avg
(Ace)
(Acc/F1)
(Acc)
(Acc)
(Acc)
(Ace)
BERT base
32.31.4
50.82.1/
40.10.6
32.61.3
19.3u.s
37.02.s
35.4
FreeLB-BERG
31.60.3
51.02.a/-
45.40.a
33.5u.4
21.90.9
42.01.a
37.6
BERTOLL
32.0u.6
48.51.3/-
43.41.4
27.60.4
20.8u.s
45.92.5
36.4
BERT BrR.C
35.31.0
21.5
44.80.6.
36.0o.s
22.0o.a
45.22
39.1 12.7
base
T5 targe
60.6
63.0/55.7
57.6
48.4
39.0
62.8
55.3
MHART-ROBERTaaar
50.9
64.2/44.3
52.2
36.1
70.4
52.0
DeBERTaJarge
57.9
60.4/48.0
57.9
58.4
52.5
78.9
59.1
DeBERTap Baage
CreAT
63.5
67.5/54.5
61.8
56.7
51.7
72.0
61.1
12.0
and & to 1. Training base/large-size model takes about 30/100 hours on 16 V100 GPUs with FP16. Readers may refer to Appendix A for hyperparameters details.
To distinguish different training settings, for example, CreAT-BERT base means we fine-tune the original BERT model with CreAT; BERTCreAT means we pre-train the model and then regularly base
fine-tune it without adversarial training; BERTMLM means we regularly pre-train and fine-tune the model.
base
We describe a number of adversarial training counterparts below.
2020) is state-of-the-ar fast adversarial training algorithm, which incorpo- rates every intermediate example into the backward pass.
FreeLB (Zhu et al.
SMART (Jiang et al. 2020) is another state-of-the-ard adversarial training algorithm, which keeps the local smoothness of model outputs.
AT (i.e. vanilla AT) refers to the special case of CreAT when 0. Both AT and CreAT are traditional PGD attackers (Madry et al. 2018).
4.2 RESULTS ON VARIOUS DOWNSTREAM TASKS
We experiment MNLI (natural language inference). QQP (semantic similarity), two represen- tative tasks in GLUE (Wang et al. 2019a); WNUT-17 (named entity recognition) (Aguilar et al.)
2017; DREAM (dialogue comprehension) (Sun et al., 2019: HellaSWAG, AlphaNLI (common-
sense reasoning) (Zellers et al. 2019 Bhagavatula et al. 2020); RACE (reading comprehension)
Lai et al. 2017).
Table [ summarizes the results on BERT base. First, each AT method works well on MNLI and QQP (two sentence classification tasks), and the improvement quite similar. However, on the right side of the table, we can see AT appears harmful on certain tasks (average drop over BERT: 0.1 for SMART, 0.8 for AT), while CreAT consistently outweighs all the counterparts (absolute gain over BERT: 3.0 points on DREAM, 1.8 on AlphaNLI, 2.7 on RACE) and raises the average score by 2.0 points. Table also verifies our previous results Figure From Figure we can also see that the embedding similarities under CreAT are the lowest all four tasks.
AT is sensitive to hyperparameters. For example, FreeLB obtains 36.6 and 39.8 on H-SWAG under different a, le-1 and le-3. However, we find that CreAT is less sensitive the ascent step size a, and le-1 performs well on all datasets. In other words, the direction of the CreAT attack makes easier to find the global worst case.
4.3 ADVERSARIAL GLUE
Adversarial GLUE (AdvGLUE) (Wagg et al. 202 is a robustness evaluation benchmark derived from number of GLUE (Wang et al. 2019a) sub-tasks, SST-2, QQP, MNLI, QNLI, and RTE.
httpss://adversaaiiaall/
ic/

---

Published as a conference paper at ICLR 2023
Different from GLUE, it incorporates large number of adversarial samples in its dev and test sets, covering word-level, sentence-level, and human-craftee transformations.
Table 2 summarizes the results all AdvGLUE sub-tasks. We see that CreAT-trained models consistently outperform fine-tuned ones by large margin. In addition, we see that directly fine- tuning BERTCreAT works better than fine-tuning with FreeLB. To verify the effectiveness of CreAT- base based adversarial pre-training, we pre-train MERT based MLM with the same number of steps (100K steps). From Table 2 we see that simply training longer leads to a limited robustness gain, while the CreAT-trained one is much more powerful (over BERTMLM. 3.3 on SST-2, 6.2 QQP, 8.4 on MNLI-m). Compared with other strong PLMs (Liu et al.) 2019 Raffel et al. (0020), the further CreAT-based pre-training let DeBERTafarge CreAT achieves the new state-of-the-an result.
4.4 ADVERSARIAL SQUAD
AdvSQuAD (Jia & Liang 2017) is a puzzling machine reading comprehension (MRC) dataset derived from SQuAD-1.0 (Ra-) jpurkar al.) 2016), by inserting sentences into the paragraphs, replacing nouns and adjectives in the questions, generating fake answers similar to the original ones.
Table 3: Results AdvSQuAD dev sets over three runs.
AddSent (EM/FI)
AddOneSent (EM/FI)
BERT base
59.3.//65.7
67.3.s/74.344
FreeLB-BERT base
60.9.s/67.4.s
11.6/1.7
67.9.2/74.8.6 10.6/0.5
BERT CTeA
6/.3.g/69.5
13.0/3.8
69.5,
12.2/2.2
base
DeBERTa
80.2,178655
88333.789.II1
DeBERTA
CreATT large
81.6,1/87.333
11.4/0.8
84.3.3/90.2, 2 11011111
From Table we see CreAT leads to impressive performance gains by 1 to points on all metrics.
4.5 ADVERSARIAL NLI
Adversarial Natural Lan- guage Inference (ANLI) (Nie et al. 2020) is an iteratively- strengthened robustness bench- mark. In each round, human annotators are asked craft harder samples to fool the model. In our experiment, each model is trained on the concatenated training samples of ANLI and MNI.I.
Table Test results all rounds on ANLI over five runs.
Round I
Round 2
Round 3
Avg
BERT base
55.
44.5.s
42.5.y
47.4
BERT large
57.6.4
48.0.;
43.2.
49.6
CreAT
59.2.
19.1
14.1
45.9,7
1.4
44.0
11.5 11.5
49.7
1211
DeBERTapre
77.8.7
66.2.2
60.4.8
68.1
ReBERTaCreaA
78.7.7
10.9
66.9.1
10.7
62.3,
1.9
69.3 1.1
InfoBERT Targe
773.5
31.4
49.8
38.9
ALUMaarge
72.3
52.1
48.4
57.6
Table summarizes the test results of all rounds. BERT CreAT is powerful and even surpasses base
BERT large- outperforming by 1.6 points and 0.8 point Round 1 and Round 3. Besides, DeBERTa significantly outperforms the previous state-of-the-art models InfoBERT [Wang et al. (202 and ALUM (Liu et al. 2020). However, DeBERTaCreA* further pushes the average score from 68.1 to 69.3, and especially on the most challenging Round 3, it leads to a 1.9 points gain on DeBERTalare Taagg--
ANALYSIS
5.1 ABLATION STUDY
To access the gain from CreAT more clearly, in this part, we disentangle the CreAT perturbations level by level. To ensure the fairness of experiments, all the mentioned training methods follow the same optimization objectives in Eq. 4 and Eq. 55] Specifically, we first apply random perturbation training (RPT) on top of BERT fine-tuning. Then we apply AT, where the random perturbations become adversarial. CreAT refers to the case where the attack is solely optimized to deviate the contextualized representation (i.e. removing the left term in Eq.
7

---

Published as a conference paper at ICLR 2023
Hidden states
Attention probability
AT
CreAT
84-
CreAT
08
0.
02
Figure 2: Impact of CreAT attack on the intermediate layers of BERT base (12 layers).
From 5
that AT is superior to RPT on all three tasks. The improvement from
Table Results ablation study over three runs.
CreAT is greater, outperforming AT by large margin on DREAM and AlPhaNLI. In addition, can see CreAT still improves DREAM and AlphaNLI without the adversarial objective. It turns both parts of the CreAT attack are nec- essary. CreAT acts as a supplement to AT to find the optimal adversarial examples. The in- teresting thing is that removing the adversarial objective causes large drop on MNLI It implies that the gain from AT sometimes does not come from increasing the training risk, which can vary from tasks or datasets. We will leave this part for the future work.
MNLI-m
DREAM
AlphaNLI
BERT RPT
84.3
63.0
65.2
84.1
63.6
64.9
+ +AT
85.2
64.0
64.9
+ CreAT
83.5
65.0
65.7
CreAT
85.3
66.4
67.1
Table 6 demonstrates a number of ad- versarial pre-training methods. We see that CreAT obtains stronger perfor- mances than simply training MLM for longer period on all three tasks. How- ever, the BreseB one is almost parable to MLM, while the AT-based one slightly better than it. To explain, we only keep the encoder and drop the MLM decoder when fine-tuning the PLMs on downstream tasks, while CreAT can effectively allow more robust encoder.
Table 6: Continual pre-training results on BERT base in different settings over fine runs. Each model is trained for the same number of steps.
HellaSWAG
PAWS-QQP
PAWS- PAks-Wiki
MLM
42.6
88.5
92.0
FreeLB
42.7
88.2
92.1
+T
43.1
89.3
92.8
CreAT
43.8
90.2
93.1
5.2 IMPACT OF CREAT ATTACK
We take a closer look into the learned contextualized representation each intermediate BERT layer, including the hidden states and attention probabilities, compare the impact of CreAT and AT. We calculate the cosine similarity the hidden states and the KL divergence of the attention probabilities before and after perturbations.
From Figure 2 (left), CreAT and AT look similar on lower layers. For the last few layers (10 ~ 12), CreAT causes stronger deviation the model (lower similarity). From Figure (right), (right,, observe that CreAT has a strong capability to confuse the attention maps. It turns out AT is not strong enough to deviation the learned attentions of the model. This helps explain why CreAT can fool the PLM encoder more effectively.
RELATED WORK
Language modeling: Our work acts the complement for AT on pre-trained language models
(PLMs) (Vaswani et al. 2017: Devlin et al. 2019: Liu et al. 2019 Lan et al. 2020 Clark et al.)
2020 Raffel et al. 2020 Brown al. 2020 He et al.) 2021). From the perspective of language
pre-training, it is essentially a common improvement of the conventional pre-training paradigm,
e.g. MLL (Devlin al. 2019
Clark al. 2020 Yang et al. 2019 Wu et al. 2022). From the
8

---

Published as a conference paper at ICLR 2023
perspective of model architecture, it is parallel to strengthening encoder performances Vaswani et al. 2017).
Adversarial attack: Adversarial training (AT) (Goodfellow et al.
2015 Athalye et al. 2018
Miyato et al. 2019) is a common machine learning approach to create robust neural networks. The technique has been widely used in the image domain (Madry et al. 2018 Xie et al. 2019). In the text domain, the AT attack is different from the black-box text attack (Gao et al., 2018) where the users have access to the model. Researchers typically substitute and reorganize the composition of the original sentences, while ensuring the semantic similarity, e.g. word substitution (Jin et al.)
2020 Dong et 2021). scrambling (Ebrahimi al., 2018 Zhang et al., 2019c). back translation 41yyer et a 2018, Belinkov & Bisk 2018), language model generation (Li et al. [020b] Garg & [Ramakrishnan, 2020; Li et al. 2002). However, a convention philosophy of the AT' attack to
impose bounded perturbations to word embeddings (Miyato et al., 2017 Wang et al. 2019b). which recently proven be well-deployed on PL.Ms and facilitate fine-tuning on downstream tasks, e.g-
202 ASA (Wu & Zhao 2022). ALUM (Liu et al. 2020) first demonstrates the promise of AT to language pre-training. Our work practices AT on more types of NLP tasks and re-investigates the source of gain brought
SMART (Jiang et al. 2020), FreeLB (Zhu et al. 22200), InfoBERT (Wang et al.
by these AT algorithms.
Adversarial defense: Our work act novel AT attack to obtain the global worst-case adversarial examples. It is agnostic to adversarial defense for the alignment between accuracy and robustness, e.g. TRADES (Zhang al. 2019b), MART (Wang al. 2020), SSL (Carmon et al. 2019).
Self-supervising: AT is closely related to self-supervised learning (Madras et al. 2018 Hendrycks 2019), since it corrupts the model inputs. On the other hand, is parallel to weight perturba-
al.
tions
Wen et al.) 2018 Khan et al.. 2018: Wu et al. 2020), and contrastive learning (Hadsell et al.,
2006
which relies on the corruption of model structures (Chen al. 2020 Gao et al. [0011).
AT is still expensive at the moment, though there are works for acceleration (Shafahi et al. 2019 Zhang al. 2019a Zhu al., 2020 Wong et al. 22020). This work is agnostic to these implementa- lions, Another important line is to rationalize the behaviour of the attacker (Sato et al. 2018 Chen 2022). In our work, we explain the impact of AT on contextualized language representation.
7 CONCLUSION
This paper investigates adversarial training (AT) on PLMs and proposes Contextualized irarsentatioonAAverrrariiil Training (CreAT). This is motivated by the observation that the AT gain necessarily derives from deviating the contextualized language representation of PLM encoders. Comprehensive experiments demonstrate the effectiveness of CreAT on top of PL.Ms, which obtains the state-of-the-art performances on series of challenging benchmarks. Our work is limited to Transformer-base PLMs and other architectures, vision models are not studied.
REFERENCES
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. Better fine-tuning by reducing representational collapse. 9th International Confer- on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net, 2021. htthtttts::///pe. http://pppppeeeee.....eeee
net. wttp:/oruuummuooomo fet. 7MMV
Gustavo Aguilar, Suraj Maharjan, Adrián Pastor López-Monroy, and Thamar Solorio. A multi-task approach for named entity recognition in social media data. In Leon Derczynski, Wei Xu, Alan Ritter, and Tim Baldwin (eds.), Proceedings of the 3rd Workshop on Noisy User-generated Text, NUT@ EMNLP 2017, Copenhagen, Denmark, September 7, 2017, pp. 148-153. Association for Computational Linguistics, 2017. doi: 10.18655/v11ww17444111 URL https: /oi /rg/ 100.8653/w1/ww 1449
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässnn, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of

---

Published as a conference paper at ICLR 2023
Machine Learning Research, pp. 274.283. 2018. URL URLnhtp://proeeeciggg.. html
Yonatan Belinkov and Yonatan Bisk. Synthetic and natural noise both break neural machine trans- lation. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver BC, Canada, April 30 May 2018, Conference Track Proceedings. OpenReview.net, 2018. URL. ftttp:://ppenreeee
net/ wttptteete///eeee BBJ6veJJec
Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Han- nah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. Abductive commonsense reasoning. In 8th International Conference on Learning Representations,, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL httph: http::::///ppeeeeiiiice /toppppps////eeeeeeeee..... net forum?ics
Byg vl HKDB
Christopher M. Bishop. Training with noise is equivalent to tikhonov regularization. Neural Com- put., 701):088-166 116, 1995. doi: 10.1162/ne.. 1995.711.10.. URL https://doi orr//00..
1162/nee
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Iya Sutskever, and Dario Amodei. Language models few-shot learners. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tieun Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL http:s::/pppcceeddiieiee wew.eeddiiinnnuuuueeeeeeeee neur cccppppper/222/haatt 74
html
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C. Duchi, and Percy Liang. Unla- beled data improves adversarial robustness. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché-Buc, Buch Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 11190-11201, 2019. URL http:::/ppoocceeetteeess www.nnngnnnneeeueeeeeeeeeeeeeewe neur rirrrips.... wwweriipppccpppeeeeeee wctppppr///119999 hast:/99ebbal19777/4422777777773
Abbut 1ttttarattttttll...... html
Hanjie Chen and Yangfeng Ji. Adversarial training for improving model robustness? look at both
prediction and interpretation. CoRR, aabs/2203.112000 2022. doi: 10448500/rxiv..203..777
URL https: http://oo...og111...855//////11 2203. 221
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In Proceedings the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 1597-1607, PMLR, 2020. httphttt:/pprcceeeii mlr. w...ppres/vvi 19/ 9/chh//ccccccccmmmom/..
Kevin Clark, Minh-Thann Thang Luong, Quoc V. Le, and Christopher D. Manning. ELECTRA: pre- training text encoders as discriminators rather than generators. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- net. wtr.fformmmideem 1 BtVE
view.net, 2020. URL http:///......tttptp////////eeee
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. NAACL-HLT 2019, Minneapolis, MN, USA. June 2-7, 2019, Volume 1 (Long and Short Papers), pp. 4171- 4186. Association for Computational Linguistics, 2019. doi: 10.18653/ 1/n53//l/n191... 1423. URL
https::/doi.ooz http:/.1..115331/1111131
10

---

Published as a conference paper at ICLR 2023
Xinshuai Dong. Anh Tuan Luu, Rongrong Ji, and Hong Liu. Towards robustness against natu- ral language word substitutions. In 9th International Conference on Learning Representations, ICLR 2021. Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https: //openevveww net./ form/form? d=ks
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics, ACI. 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers, pp. 31-36. Association for Computational Linguistics, 2018. doi: 10.18653/v 1/P18/////11111111 2006.. Uttpttps: httpt::s///aaaatoooooooooooooooe pttpa//aaatool....rggaa..../ -00
Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops, SP Workshops 2018, San Francisco, CA, USA, May 24, 2018, pp. 50-56. IEEE Computer Society, 2018. doi: 10.1109/SP.2201..000 00016 6
URL https: /doi
2018.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence embeddings. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 6894-6910. Association for Computational Linguistics, 2021. doi: 10. 10.18653/v12201. emnlp-main.552
org 10. 1ww536116110000/ emn coammlpmmainn www..aiin....mmmmmmmm 552,
Siddhant Garg and Goutham Ramakrishnan. BAF: bert-based adversarial examples for text clas- sification. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, On- line, November 16-20, 2020, pp. 6174-6181. Association for Computational Linguistics, 2020. doi: 10.18653/v 20/163/////220..oooo 1t21220220022p22 emr.pp http:/0e000.mmmaiinn...ooo 498. BRLLhtts::////
498)
Ian Goodfellow. Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceed- ings, 2015. URL http: org. 1412. 6572
Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recog- nition (CVPR 2006). 17-22 June 2006, New York, NY. USA, pp. 1735-1742, IEEE Computer Society, 2006. doi: 10. 10.1109//VPR.2000.100 100. URL httttp:://oi..o0/10.11111 1109///P1.. 2006.100
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: decodiingeehhanced bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. | https:
/ //openreview.nettr
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learn- ing can improve model robustness and uncertainty. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett (cds.), Ad- vances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- tion Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 15637-15648 2019. URL htttp:://p.pceeeee/eeoo
newriieerrppp... ccc/paper/2019/ hash/ fassh/2bb550 2rt. -ee. Abstract httml
Mohit lyyer, John Wieting, Kevin Gimpel, and Luke Zettlemayer. Adversarial example gener- ation with syntactically controlled paraphrase networks. In Marilyn A. Walker, Heng Ji, and Amanda Stent (eds.). Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pp. 1875- 1885. Association for Computational Linguistics, 2018 doi: 10.18653/v1/118- 1170. URL https://doi http:///ooooooo//.///// 1et...165///111111
11

---

Published as a conference paper at ICLR 2023
Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), Proceedings of the 2017 Conference Empirical Methods in Natural Language Processing, EMNLP 2017. Copenhagen, Denmark, September 9-11, 2017, pp. 2021-2031. Association for Computational Linguistics, 2017. doi: 1/1771/1/117-12155 URL ptttttss:///o..oo 1ttp://011111111111/111111-11111
Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. SMART: robust and efficient iinetuning for pre-trained natural language models through principled reg- ularized optimization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguis- fics, ACI. 2020, Online, July 5-10, 2020, pp. 2177-2190. Association for Computational Linguis- tics, 2020, doi: 10.18653//1/202..aa acw.0000..cllmiiinn. aal-main..l. 197. 2020 acl- 19..n.197
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applica- tions of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educa- tional Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 8018 8025. AAAI Press, 2020. URL https: / aaai. org/ wiw/inndxx..///ee phpp AAAI/ article/view/655
Mohammad Emtiyaz Khan, Didrik Nielsen, Voot Tangkaratt, Wu Lin, Yarin Gal, and Akash Sri- vastava. Fast and scalable bayesian deep learning by weight-perturbation in adam. In Jen- nifer G. Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference on Machine Learning. ICML 2018, Stockholmsmüssan, Stockholm, Sweden, July 10-15, 2018, vol- ume 80 of Proceedings of Machine Learning Research, pp. 2616-2625. PMLR, 2018. URL httt::/proceedie
.ml. prw.sressseeeeee / 1ate0//haal8..haa...
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard H. Hovy. RACE: large-scale
reading comprehension dataset from examinations. In Martha Palmer, Rebecca Hwa, and Se- bastian Riedel (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pp. 785- 794. Association for Computational Linguistics, 2017. doi: 10. 10.18653/1/411-1082. URL htttp:://bi..rr// 1ttp///..1663//1/1111111
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning- In 5th Interna- tional Conference on Learning Representations, ICLR 20/7, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL htttps://epenreeieew n
forum?idd-BJooffqqe
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. ALBERT: lite BERT for self-supervised learning of language representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020, URL httph::tt////tt /tttpp:///penreview net/ forum?id
H1 ee 7ABttvS
Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. On the sentence embeddings from pre-trained language models. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 9119-9130, Asso- ciation for Computational Linguistics, 2020a. doi: 10.18653//1/20200.o 1:003//112200.mmmmmmma..... http://ti...o../////// 181330..11653/11222222 httpppp//ooo..oor//000.633331112 em. 1p ma main. 733
URL
Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, and Bill Dolan. Contextualized perturbation for textual adversarial attack. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilck Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cot- terell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pp. 5053-5069 Association for Computational Linguistics, 2021. doi: 10.18653/v 1/2021 .ww.cl-main.44.ee0001 400.. URL https: 1ttp://1...65331122200000 naac. -mai 400
12

---

Published as a conference paper at ICLR 2023
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. BERT-ATTACK: ad- versarial attack against BERT using BERT. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 6193-6202. Associa- tion for Computational Linguistics, 2020b. doi: 1tt5:////020.emmmpm.mm caanenppp-annnn mainn.00
URL
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. CoRR, abs/2004.08994, 2020. URL
https://arxx: org/ abs 2004 08994
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11699 1692, 2019, URL htt.:ttp:// htttp:/ http::/axxivv..rg.... abs 1907 11692
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning. ICML 2018, Stockholmsmüssan. Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 3381 3390. PMLR, 2018. URL htRR:htt::/// http:::///oocceenngg.... ml r. pressrssssss.... v80 12580/mmdrass Ba html
peress/v8//e
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Camada, April 30 May 3, 2018, Conference Truck Proceedings. OpenReview.net, 2018. URL. hnttps://spenreeeee nett/
Forum?id-rJzIBFF
Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word represen- tations in vector space. In Yoshua Bengio and Yann LeCun (eds.), 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings, 2013. URL httt:://tttii..t//
abaa 3781
Takeru Miyato, Andrew M. Dai, and Ian Goodfellow. Adversarial training methods for semi- supervised text classification. In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https::/
net forum? forum?id=r1x332 x1
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: A regularization method for supervised and cemi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell., 4188):1979-1993, 2019, doi: 10. 2.0. URL Ittpp:://
doi. doi.org/10. 10. 1,000.1100 11w.111999910 TPAMI 22188822888111
Daniel Moyer, Shuyang Gao, Rob Brekelmans, Aram Galstyan, and Greg Ver Steeg. In- variant representations without adversarial training. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Gamett (eds.), Ad- vances in Neural Information Processing Systems 31: Annual Conference on Neural Informa- tion Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp- 9102 9111, 2018 URL
neur ips. httppp/rp/////aaaaeeeeppp html
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adver- sarial NLI: A new benchmark for natural language understanding. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 4885-4901. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020. aci.22200...mmmamm444444 441. URL acl
httpps://do doi wrg./ogggl0 1ttp//.0.1663///1 18e.1166333/88
441
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to- text transformer. J. Mach. Learn. Res., 21:140:1-140:67, 2020. URL http:/ //jmrr. orq/
paperss/21/20-000 html
13

---

Published as a conference paper at ICLR 2023
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for machine comprehension of text. In Jian Su, Xavier Carreras, and Kevin Duh (eds.), Proceedings Empirical Methods in Natural Language Processing. EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pp. 2383-2392. The Association for Computational Linguistics, 2016. doi: 10. 10.18653/v1/d16- 1264. URL https://doil wtt.://i..rg111............ 10.1//11...1651v1111 d16 6164-264
of the 2016 Conference
Motoki Sato, Jun Suzuki, Hiroyuki Shindo, and Yuji Matsumoto. Interpretable adversarial per-
turbation imput embedding space for text. Jérôme Lang (ed.). Proceedings of the Twenty-
Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pp. 4323 4330. ijeai.org, 2018. doi: 10.24963/ijcai.20188 2018///01. URL
https://doi. /00 24963/ //aiiiicca.... jcai 2018/601
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John P. Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Hanna M. Wal- lach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alcch-BBuc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 1353-3366- 2019. URL htt.ttpps///rrceeeinngee 71t.:/55003ccc05553 ott.://cc.......0eeeeeeeeeeeee20 9wwpeebcebeeeeeeeeeee
wwcplpss.ccpppppp cc/paper/
Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. DREAM: chal- lenge dataset and models for dialogue- based reading comprehension. Trans. Assoc. Comput.
Linguistics, 7:217-231, 2019. URL petttss:/trrasseee wwt.oo///////.ooo http:////ooo/////o/////////////e phpptacclpppppttcc
article/view/153
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 2ateer///1////h/////
5998-6000 2017. URL http::ttt//ssttt htttt::://proccedd.ngg neuri www.rrips.... 3f5ee24354 7dee Fbd053cicc48555aaaae Abstrcaaaaaaaaaaaaa.tt....... html
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Leurning Representations, ICLR 2019, New Orleans, LA, USA, net/ wott:/foumm?aa iid=
May 6-9, 2019. OpenReview.net, 2019a. URL
PJ4 km225tt22
Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. In- fobert: Improving robustness of language models from an information theoretic perspective. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021a. URL https:://openreiit wiww.w net/ / forumm?rm?? di 5ukkk
Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Has- san Awadallah, and Bo Li. Adversarial GLUE: A multi-task benchmark for robustness evaluation of language models. In Joaquin Vanschoren and Sii-Kit Yeung (eds.), Pro- ceedings of the Neural Information Processing Systems Track on Datasets and Bench- marks 1, NeurlPS Datasets and Benchmarks 2021, December 2021, virtual, 2021b. URL. neuriess cc 222er/221/ Seecc--bsstacc roontt-oondd..... html
Dilin Wang, Cheng Yue Gong, and Qiang Liu. Improving neural language modeling via adversarial training. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th Inter- national Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 6555-65665 PMLR, 2019b. URL htt.:httppppprocceeeiii htt.://eeee.nn............ssss/e wttp//gg//g//// wwwg//agggggg..aao 9f. html
.mlr.
Hongjun Wang and Yisen Wang. Self-ensemble adversarial training for improved robustness. In The Tenth International Conference on Learning Representations, ICL. 2022, Virtual Event,
14

---

Published as a conference paper at ICLR 2023
April 25-29, 2022. OpenReview.net, 2022. URL https:://ppeneeveee..eetttte GU3aTsmeROV,
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving ad- versarial robustness requires revisiting misclassified examples. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenRe- view.net, 2020. URL https:: /apenzeveeeee net forum? rk1ggg
Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, and Roger B. Grosse. Flipout: Efficient pseudo- independent weight perturbations on mini-batches. In 6th International Conference on Learn- ing Representations. ICLR 2018, Vancouver. BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL httpthtt:///ppeeeeeee
net fetr/forrmmmdd
EJNpi FWAb
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. As- sociation for Computational Linguistics. URL https::///// /wwww www..w..........c............... org/ anthooooooooy /gy/
2020. emnlp-demss 6
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 30,30,22000 2020. OpenReview.net, 2020. URL htttts://pppnrreeee BJX040EFVH]
net/ forum?
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gen- eralization. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tice Tien Lin (cds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL /ttp://tss/////roedd.g....../../ neurips cc/ppaperr/22//haat/ Lef91c212e30e
Hongqiu Wu and Hai Zhao Adversarial self-attention for language understanding. CoRR, abs/2206. 12608, 2022. doi: 10448550WaXX..2226.22660 URL attts:://do..rrg/11
48550/arXiv 2206 12608
Hongqiu Wu, Ruixue Ding, Hai Zhao, Boli Chen, Pengjun Xie, Fei Huang, and Min Zhang. Forging multiple training objectives for pre-trained language models via meta-learning. In Yoav Gold- berg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 6454 6466. Association for Computational Linguistics, 2022. URL httts:://ccaatthon org/2002. findi was-gmmme 482
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L. Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 501- 509. Computer Vision Foundation / IEFE, 2019. doi: 10.1109/CVPP.221990000 htttp://openacceess. thecvf. www.cooto/etteeeeoeeoo __________________ 20t.20009ttmmxieeeee Feature_ Denoising for L Robustness_CVP
URI.
2019 paper.
html
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. Xlnet: Generalized autoregressive pretraining for language understanding. Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' A'Ach--Buu, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancouver; 2019. URL ht pp. 23
neur.ps............... /pppr//
15

---

Published as a conference paper at ICLR 2023
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can ma- chine really finish your sentence? Anna Korhonen, David R. Traum, and Lluís Märquez (eds.), Proceedings of the 57th Conference of the Association for Computational Linguisties, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume I: Long Papers, pp. 4791- 4800. Association for Computational Linguistics, 2019. doi: 10.18653/w1/p19--4472 URL
1472
Dinghuai Zhang, Tianyuan Zhang. Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propa- gate once: Accelerating adversarial training via maximal principle. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances Neural Information Processing Systems 32: Annual Conference on Neural Infor- mation Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 228-238, 2019a. URL 812b4ba28 112o4a22776ee00css45bbbbbbeeeee -bs-ttt7---bbbtaaaaaa html
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Confer- ence on Machine Learning. ICML 2019, 9-15 June 2019, Long Beach, California, USA, vol- ume 97 of Proceedings of Machine Learning Research, pp. 7472-7482, PMLR, 2019b. URL http: http://rroceeeeee ngs ml press / v97 2t///h/ggll 9p. htm
Yuan Zhang, Jason Baldridge, and Luheng He. PAWS: paraphrase adversaries from word scram- bling. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Con- ference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACI-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers), pp. 1298- 1308. Association for Computational Linguistics, 2019c. doi: 10.18653/v 1ttp:////11//11111 1131.
URL https: /15
1131
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adver- sarial training for natural language understanding. 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL htttps://ppenree ew net/ dd=ygzbyyFVV
16

---

Published as a conference paper at ICLR 2023
A TRAINING DETAILS
Table 7: Hyperparameters for pre-training.
BERTbase
DeBERTatare
Number of hidden layers
24
Hidden size
1024
Intermediate size
3072
4096
Number of attention heads
12
16
Dropout
0.1
0.1
Batch size
512
512
Learning rate
5c-5
6c-6
Weight Decay
0.01
0.01
Max sequence length
256
256
Warmup proportion
0.06
0.06
Max steps
100K
Gradient clipping
1.0
1.0
Ascent step size
le-1
le-1
Decision boundary
lc-1
le-1
Ascent steps
FP16
Number of GPUs
16
Training period
30 hours
100 hours
Table 8: Hyperparameters for fine-tuning BERT. For DeBERTa, fix all hyperparameters the same, but set the learning rate to le-5 for all tasks. (dp: dropout rate, bsz: batch size, Ir: learning rate, wd: weight decay, msl: max sequence length, wp: warmup. ep: epochs).
(AJMNLI
QQP
WNUT
DREAM
H-SWAG
AlphaNLI
RACE
SQuAD
dp
0.1
0.1
0.1
bsz.
128
128
16
16
16
64
8
16
3e-5
5e-5
Se-5
2e-5
2e-5
Se-5
2e-5
5e-5
wd
0.01
0.01
0
0
0
msl
128
64
128
384
wp
0.06
0.06
0.1
0.06
0.06
0.06
0,06
0.06
cp
3
5
3
2
2
2
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
le-1
2
17


Reviews:
[{"review_id": "kQGsmcxMBI", "reviewer": "Reviewer_ANTZ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new adversarial training approach in which perturbation is added to the inner layers of a deep transformer and thus affects contextualized representations. Empirical studies are conducted over a number of benchmarks with remarkable improvements to the existing AT methods. ", "summary_of_the_review": "The paper discloses that existing AT methods fail to manipulate the contextual representations, and thus render inconsistent performance over different tasks. Hence, the authors consider add perturbations to inner layers of a deep transformer. \n\nThe observations are interesting, and the proposed method is simple yet effective. The descent improvements over a variety of tasks indicate that it is worth to try the method in practice. It would be better if the authors could release their code and thus other can reproduce the results more easily. \n\n---------------------------------------\nAfter reading the comments from other reviewers and the response from the authors, I agree that the improvements on BERT over the baselines are sometimes limited (e.g., on MNLI and QQP). I still feel that the idea is interesting, but also raise concerns on when or on what kind of tasks and models, the proposed method is really useful. Therefore, I slightly downgrade my rating. ", "strength_and_weaknesses": "Strength:\na new adversarial training approach that is effective over a number of tasks. \n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jTAh-Roq8zF", "reviewer": "Reviewer_bCxQ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the adversarial training problem in the NLP field.\nSpecifically, they focus on the contextualized representations and propose an AT algorithm that calculates the perturbations based on the contextualized representations, not just the classification losses.\nThey first provide an analysis exploring the loss and the similarity to back their intuition to construct representation-based adversarial training.\nThey conduct experiments on both NLU benchmarks and robustness-concerned datasets and prove the effectiveness of their method.\n", "summary_of_the_review": "As mentioned in the strength and weaknesses, the paper is clear yet the experimental results seem less convincing especially since the key hyper-parameters are all fixed in all experiments.\nI suggest a more detailed hyper-parameter search and analysis which could significantly improve the soundness of the proposed method.", "strength_and_weaknesses": "Strength:\nA. the paper is clearly written and easy to understand\n\nWeakness:\nA. the experimental results are not convincing enough. \nGenerally, methods such as FreeLB and SMART, as well as a bunch of following adversarial training methods such as Token-Aware Virtual Adversarial Training (Li et al. 2021), adversarial training for large neural language models (Liu et al. 2020), Weighted Token-Level Virtual Adversarial Training in Text Classification (Sae-Lim et al. 2022) are exploring the effectiveness of adversarial training on general NLU tasks.\nThe proposed method seems only to obtain improvements on adversarial datasets as illustrated in Table 1.\nThe performances of methods like freeLB are lower than BERT-base results which are rather strange to me. \nConsidering that the hyper-parameter setting is actually very important in the adversarial training process, I am wondering if the hyper-parameters are well-searched in all the baseline methods and the proposed methods.\n\nB. a hyper-parameter concern.\nAs mentioned, the hyper-parameter setup is important in the adversarial training process, which is also well-studied in Searching for an effective defender: benchmarking defense against adversarial word substitution (Li et al. 2021) where a freeLB++ method is proposed with the perturbation range much wider than the FreeLB method.\nAccording to the FreeLB++ method, the perturbation set to 1e-1 compared with some figures like 3e-5 or 3e-4 in the original FreeLB setup, the defense performances are significantly improved.\nAs the proposed CreAT method uses 1e-1 for all their experiments, I wonder whether the robustness improvement is not influenced by the contextualized representation-related term in the AT process.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "3qvZuncX0g", "reviewer": "Reviewer_Vm6d", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies adversarial training for pretrained NLP models. In adversarial training for NLP, perturbations are applied to the input word embeddings -- such that the prediction changes. The authors investigate this and find that while it sometimes helps, it hurts many tasks, most notably a 3.4 points drop on HellaSWAG.\n\nInvestigating this further, the authors propose a new method called CreAT, where the idea is that a perturbation must not only change the model's prediction, but also push away the contextualized representation from the unperturbed state. Experiments show that it enables BERT-like models to be finetuned better.", "summary_of_the_review": "I'm slightly leaning towards accepting this paper, but I'd like to see a bit more contextualization of the cost of adversarial training, and how important the novel contextualized-representation-perturbation is. If those concerns are addressed I'd be happy to increase my score in revision.\n\n---\nUpdate post-author response: updating my score from 6->8.", "strength_and_weaknesses": "Strengths:\n* The analysis about how simple adversarial training doesn't work on all NLP tasks seems important for the community to know about. It seems like a good corollary to Vision results that say there is an explicit tradeoff between robustness and performance (Madry et al 2018), and NLP results showing that it is effective on some tasks (Jiang et al 2020, Zhu et al 2020, Wang et al 2021 as in the paper). The NLP results might be cherry-picked and perhaps misleading to readers.\n* The approach seems to work well on a variety of different tasks.\n\nWeaknesses:\n* Conventional wisdom is that BERT was an undertrained model. I'm glad DeBERTa large was tried too, but I'm curious as to whether this same result would hold on e.g. RoBERTa Base.\n* I'd like to see some analysis as to the cost of running CreAT / AT for a finetuning task (e.g. HellaSwag.) For instance, if CreAT is K times more expensive than conventional finetuning (I'm not sure I know the exact K) -- is it better to do adversarial training or ensemble together K models? Future work might make these approaches more efficient, but I think this analysis would be nonetheless important for practitioners.\n* I'd be curious as to how well the model does if only the new contextualized-representation perturbation is factored in. A reasonable question would be whether this is what's really important for pretrained language models during finetuning.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster