RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Training
Authors: Jaehyung Kim
Yuning Mao
Rui Hou
Hanchao Yu
Davis Liang
Pascale Fung
Qifan Wang
Fuli Feng
Lifu Huang
Madian Khabsa
Venue: EMNLP 2023 Findings (2023)
Keywords: Language model robustness
adversarial training

Abstract: Fine-tuning pre-trained language models (LMs) has become the de facto standard in many NLP tasks. Nevertheless, fine-tuned LMs are still prone to robustness issues, such as adversarial robustness and model calibration. Several perspectives of robustness for LMs have been studied independently, but lacking a unified consideration in multiple perspectives. In this paper, we propose Robustifying LMs via Adversarial perturbation with Selective Training (RoAST), a simple yet effective fine-tuning technique to enhance the multi-perspective robustness of LMs in a unified way. RoAST effectively incorporates two important sources for the model robustness, robustness on the perturbed inputs and generalizable knowledge in pre-trained LMs. To be specific, RoAST introduces adversarial perturbation during fine-tuning while the model parameters are selectively updated upon their relative importance to minimize unnecessary deviation. Under a unified evaluation of fine-tuned LMs by incorporating four representative perspectives of model robustness, we demonstrate the effectiveness of RoAST compared to state-of-the-art fine-tuning methods on six different types of LMs, which indicates its usefulness in practice.

RoAST: Robustifying Language Models via
Adversarial Perturbation with Selective Training
Jaehyung Kim
Yuning Mao
Rui Hou
Hanchao Yu
Davis Liang
Pascale Fung
Qifan Wang
Fuli Feng*
Lifu Huang" Madian Khabsa*
TKAIST,
+Meta AI,
HKUST, USTC, Viirinnia Tech
jaehyungkimekaist. .w..eaaaiit..............
Abstract
perturbations (Jin et al.. 2020: Li et al., 2020), and also can be biased by the superficial cues, e.g., key- word or negation (McCoy et al., 2019; Niven and Kao, 2019). As such, it is critical to additionally account for robustness during fine-tuning and eval- uation of LMs.
Fine-tuning pre-trained language models (LMs) has become the de facto standard in many NLP tasks. Nevertheless, fine-tuned LMs are still prone robustness issues, such as adversar- ial robustness and model calibration. Several perspectives of robustness for LMs have been studied independently, but lacking a unified consideration in multiple perspectives. In this paper, we propose Robustifying LMs via Ad- versarial perturbation with Selective Training (RoAST), a simple yet effective fine-tuning technique to enhance bustness of LMs in unified way. RoAST effectively incorporates two important sources for the model robustness, robustness on the perturbed inputs and generalizable knowledge pre-trained LMs. To be specific. RoAST introduces adversarial perturbation during fine- tuning while the model parameters are selec- tively updated upon their relative importance to minimize unnecessary deviation. Under uni- fied evaluation of fine-tuned LMs by incorpo- rating four representative perspectives of model robustness, we demonstrate the effectiveness of ROAST compared to state-of-the-art fine- tuning methods on six different types of LMs, which indicates its usefulness in practice.
The robustness of fine-tuned LMs has been in- vestigated in various perspectives, such adver- sarial robustness or distribution-shift generaliza- tion (Wang et al., 2021a; a; Zhou al., 2021; Nam et al.. 2022; Hendrycks al., 2020). However, ex- isting research exhibits certain limitations as these studies primarily focus on a single perspective model robustness, rather than considering multi- ple perspectives simultaneously. Since real-world applications necessitate models to simultaneously exhibit robustness across multiple dimensions, unified framework is crucial build a reliable sys- tem. Moreover, as diverse and distinct methods are available for enhancing the robustness of each perspective, it hard for practitioners to find an ef- ficient approach for fine-tuning LMs to ensure such comprehensive reliability. Consequently, these lim- itations inspire us to explore single effective way for fine-tuning LMs to improve its robustness in multiple perspectives.
multi-perspective ro-
1 Introduction
In this paper, we propose a simple yet effec- tive fine-tuning technique (Figure 1) that aims improve the multi-perspective robustness of LMs. coined Robustifying LMs with Adversarial per- turbation with Selective Training (RoAST). The high-level idea of ROAST is effectively incorporat- ing two important sources for the model robustness during the fine-tuning: robustness on the perturbed input and generalizable knowledge learned during pre-training of LMs. While both factors have been separately demonstrated enhance the various facets of robustness, the collaborative framework pursuing unified robustness has been less explored yet. Specifically, ROAST first generates adver- sarial perturbation and adds it to the training in-
Fine-tuning pre-trained language models (Jing and Tian, 2020; Brown et al., 2020) has now become the de facto standard in many NLP tasks (Ken- ton and Toutanova, 2019; Liu et al., 2019; Wang et al., 2019). The typical practice for evaluating fine-tuned LMs is to measure the task performance (e.g., accuracy) on a fixed (validation) set of labeled samples. However, this evaluation approach may fall short in ensuring the reliability of fine-tuned LMs, as they are still prone robustness issues in real-world usage. For example, their predictions are known to be still vulnerable to small wevd-level
Work done during Meta Al internship.

---

Model Update with Masked Gradients
① Ltrain
Original Input
0
Adversarial Input
Language Model fe
Collecting Gradient Information Dataset
Acquisition of Relative Importance
Sampling for Gradient Masking
Figure I:
Illustration of ROAST: Robustifying LMs via Adversarial perturbation with Selective Training.
put. Then, to prevent large deviation from the pre-trained model while learning new task-relevant knowledge, RoAST updates model parameters se- lectively; after measuring their relative importance for solving a given task on the fly, only key respon- sible parameters are updated. We further justify this technique with a theoretical unalysis, providing insight into the proposed selective training method.
Distribution (i.e., out-of-domain) generaliza- tion evaluates how well the models can generalize to various forms of distribution shift from trained one at inference time (Ng et al., 2020; Liu et al.. 2022). For example, a classifier trained on a sen- timent classification dataset expected to also perform well on another sentiment classification dataset, as both datasets the same task. On the other hand, it is well known that the predictions of deep neural networks can be arbitrarily wrong, even with human imperceptible adversarial perturbations (Goodfellow et al., 2015; Zhang et al., 2019; Wu ct al., 2020). Since this problem not exceptional for LMs with word- or sentence-level adversarial perturbation (Jin et al.. 2020; Li et al., 2020), resiliency to the adversarial examples is also an important perspective of model robustness. In addition, model calibration consid- ers the alignment between the model's predicted probability and the true correctness likelihood (Guo et al.. 2017; Desai and Durrett, 2020), and hence is essential for the reliability and interpretabil- ity of model prediction. Lastly, anomaly detection performance measures the model's capability to distinguish whether given input is drawn out-of- distribution (OOD) of the training set (Hendrycks et al., 2020; Zhou al., 2021) not. In the vision domain, Hendrycks et al. (2022) recently inves- tigates the effectiveness of existing data augmen- tation methods to enhance robustness multiple perspectives; however, such investigation has not yet been explored in NLP.
constructed to solve
To perform unified evaluation of the multi- perspective robustness of LMs, we construct new evaluation benchmark using two popular NLP tasks, sentiment classification and entailment tasks; alongside the performance on the validation set, we integrate four distinct aspects of model robustness: distribution-shiie generalization (Ng et al., 2020), adversarial robustness (Wang al., 2021b), model calibration (Desai and Durrett, 2020), and anomaly detection (Tack al., 2020). Under this robust- ness benchmark, we demonstrate the effectiveness of ROAST for enhancing the robustness of fine- tuned LMs. Notably, across the four robustness aspects along with standard validation accuracy, RoAST yields 18.39% and 7.63% average rela- tive improvement compared to the traditional fine- tuning methodology sentiment classification and entailment tasks, respectively. Furthermore, discover that RoAST significantly improves the robustness of six state-of-ie oftt--of-the-rt LMs, which fur- ther indicates its practical usefulness as a simple yet effective solution for robustiying LMs.
2 Background
Multi-perspective of model robustness. For re- liable deployment in real-world scenarios (Shen al., 2017; Gupta al., 2021), the models should be robust in various perspectives, not just be ac- curate on given validation set, drawn from the trained distribution. To this end, various perspec- tives of model robustness have been investigated.
Enhancing robustness of language models. To enhance the robustness of fine-tuned LMs upon (limited) training the downstream tasks (Kim et al., 2022), various fine-tuning techniques have been explored. One line of work has explored perturbation based regularization, which enhances model robustness by simulating the specific types

---

of perturbation the inputs during fine-tuning, For example, Wu et al. (2021); Chen et al. (2021a) utilize Dropout to impose a stochastic perturba- tion to consider a different view of input, and Ng et al. (2020) substitutes the words using pre-trained masked LMs (e.g., BERT (Kenton and Toutanova, 2019)). In addition, Zhu et al. (2020); Li et al. (2021) impose adversarial perturbation word em- beddings to generate challenging view of inputs. More interestingly, Kireev al. (2022) reveals that training with adversarial perturbation can be effec- tive for improving model calibration. On the other hand, another line of work has focused on preserv- ing the generalizable knowledge learned during pre-training of LMs; Aghajanyan et al. (2021) iden- tifies a risk of losing the generalizable knowledge during fine-tuning and proposes noise-based reg- ularization to prevent it. Chen et al. (2020) directly minimizes the distance between the parameters of fine-tuned and pre-trained models as regularization, and Xu et al. (2021) proposes to only update a fixed subset of model parameters during the entire fine- tuming. A two-step strategy of linear probing and then full fine-tuning has recently been shown to be effective for distribution-shift generalization by reducing the deviation from the pre-trained model (Kumar al., 2022). In addition, the recent result (Uppaal et al., 2023) indicates the importance of generalizable knowledge in pre-trained LMs for better anomaly detection. As both approaches en- hance the different perspectives of model robust- ness, the effective framework for their collaborative utilization is expected to can serve as a unified way for robusifying LMs. However, such direction is under-explored from now on, and we try to fill this gap in this work.
Adversarial training. To improve the robustness of LM fe, we first incorporate adversarial perturba- tion during fine-tuning. Specifically, at each train- ing iteration, we construct an adversarial example x for training example x. Instead of discrete token- level adversarial perturbation (Jin et al., 2020), we consider embedding- continuous perturbation (Zhu et al., 2020) which adds noise to the word embedding of each token. Specifically, to con- struct X, we use single-step gradient ascent (Jiang et al., 2020) with a step size 6 under E.3 norm: x := x + 6 (aLtaak/ (10Lta//a /a0//laaa /xx|loo We then train fe with a following training loss:
Lirain = Lxaex(X, v) + Lraek (Z, y)
+ X.cons (x, X)),
(1)
where Lraek is a task-specific loss (e.g., cross- entropy) with given label y and Lcone (x, X) == (fe(F((x) (x)) + (ex) is bidi- rectional KI. divergence (Wu et al., 2021).
Selective model training via gradient masking, However, fine-tuning with adversarial perturbation could be suboptimal in some perspectíves of model robustness, as it incurs a relatively high training loss compared to naive fine-tuning (Dong et al., 2021) and hence may lose useful pre-trained knowl- edge due to large updates. Hence, to reduce such a potential risk, we reduce an unnecessary deviation by explicitly constraining the update. Specifically, during each iteration of the model update, we selec- tively update the model parameters by masking out the gradient of less important ones to solve a given task. To measure the relative importance score s()) of each model parameter E O we use the sum of the square of gradients
$ <(0)
Σ lo(0)) g(0) 3.trasn (2) (x.y)ED
3 Robustifying LMS via Adversarial Perturbation and Selective Training
Note that sff) is highly connected to Fisher's information matrix, which provides a point esti- mate of the task-relevant importance of the parame- ters (Kirkpatrick et al., 2017; Xuhong et al., 2018). However, the calculation of gradients for all sam- ples in D significantly increases training cost. To alleviate this, we approximate them by using the gradients calculated during the backward pass of model training, i.e., which can be obtained for free. Although different values of f are used to calculate the gradients at each training step, we empirically
Overview. In this section, we present our method, ROAST, that aims to enhance the multi-perspective robustness of LMs in a unified way. Our main idea is collaboratively incorporating two important sources of model robustness during fine-tuming; we improve the generalization of LMs on the per- turbed input using adversarial perturbation and preserve the generalizable knowledge within pre- trained LMS via selective training with gradient masking. Figure| shows an overview of ROAST. Next, we describe in detail our techniques to im- prove the robustness of fine-tuned LMs.
For solving the given task with D and training loss Cerson

---

verify that such an approximation is effective and remark that similar approaches have been adopted in other domains (Berthelot et al., 2019).
true gradient, thus safeguarding the convergence properties of the optimization process.
In practice, we accumulate the gradients during each training epoch to calculate the importance scores, and then generate the gradient masks for the next epoch. In addition, we empirically ob- serve that RoAST can work without scaling term and it sometimes outperforms the original. Hence, we consider whether to apply scaling as an addi- tional hyper-parameter. The overall procedure of RoAST is described in Algorithm 1. More details are presented in Appendix A.3.
Using the importance scores {s(0) E }}. ROAST decides whether to update each model parameter 9 or not. Specifically, we first ob- tain a normalized score 8(0) from the rela- tive order between |s(0] such that s(Omin) 0 < 8(0)
5(Onax) where Ouin arg mingee s()) and faxx :- arg maxace s(0). Then, we set a sampling probability p(0) using a smooth approximation of the Heaviside step func- tion with the logistic function (Davies, 2002):
Experiments
p(0) - 1/(1 + (xp(3(()))))) o))): (3)
In this section, we evaluate the effectiveness of RoAST to enhance the robustness of fine-tuned LMs in multi-perspectives. We first describe the experimental setups, including how the robustness evaluation benchmark is constructed, in Section 4.1. In Section 4.2, we present experimental results of RoAST and other baselines on the constructed benchmark. In Section 4.3, provide more anal- ysis of RoAST including (a) ablation study, (b) comparison with different methods sharing similar intuition, and (c) qualitative analysis.
where and 3 are hyper-parameters that control the masking ratio and smoothness, respectively. Remarkably, p(0) becomes the hard thresholding that has been utilized in prior studies (Zhang et al., 2021; Xu et al., 2021) as |3- - 2o. Compared to this, our smooth approximation enables a more cal- ibrated use of the importance score. Then, gradient mask m²))) is sampled from Bernoulli distribution with a probability p(0), and ROAST selectively updates model parameters by masking the gradient using m(f) with scaling term 1/p(0):
4.1 Experimental Setup
f 0 17. g ((0) = (m(0)/p(0) o (0),
Tasks and metrics. We select two popular NLP tasks, sentiment classification and entailment, to measure the robustness of fine-tuned LMs in a uni- fied way. We take training sets of SST-2 (Socher et al., 2013) and MNLI (Williams et al., 2018) training data for each task, respectively.
(4)
where y7 denotes learning rate. To demonstrate the soundness of proposed selective training with a masked gradient g(0), we derive a theoretical corollary based on previous result (Xu et al., 2021).
In-distribution performance (Accιa): We first evaluate model performance with respect to train- ing distribution: we measure the accuracy on the validation sets following the common practice.
Corollary. Under mild assumptions, the masked gradient ī(0) becomes an unbiased estimator of the original gradient g(0), i.e., B[r(O)] 9()). and the norm of covariance is upper bounded.
Distribution-shif generalization (AcCshart): To evaluate model capability out-of-domain gen- eralization, we measure the average accuracy on multiple distribution shifted datasets, i.e., different datasets with the same task. For entailment, we follow the setups in Liu et al. (2022) - 7 different entailment datasets are used to evaluate the entail- ment classifier, For sentiment classification, we use 5 different datasets following Potts et al. (2021). Adversarial robustness (Accaav): To measure the adversarial robustness of LMs, first construct text-level adversarial examples using TextFooler (Jin et al., 2020) on vanilla fine-tuned BERT and RoBERTa models following Wang et al. (2021a).
We present a formal derivation of Corollary in Appendix The corollary establishes that, under certain conditions, the masked gradient g()) is an unbiased estimator of the original gradient g(f), which indicates that the masked gradient correctly identifies the true gradient on average; therefore, the variance introduced by masking doesn't 't intro- duce systematic bias that could deviate the model from converging its optimal parameters. In ad- dition, the upper bound on the norm of covariance provides confidence in the stability of this estima- tor. In practical terms, assures us that the masked gradient won't 't be too volatile or far off from the

---

We also consider the datasets constructed via human-in-the-loop dynamic adversarial data col- lection (Nie et al., 2020; Potts et al., 2.21). In addition, we use the datasets from a recent bench- mark for adversarial robustness, AdvGLUE (Wang et al., 2021b), which incorporate various types of adversarial noises. We report the average perfor- mance on these datasets.
Baselines. We compare ROAST with various line-tuning algorithms; we first consider a naive
fine-tuning method, denoted by Vanilla. We then consider a wide range of perturbation- based fine- tuning algorithms: (la) WordDrop (Guo et al., 2020): dropping input words; (lb) HiddenCut (Chen et al., 2021a): dropping spanned hidden fea- tures; (Ic) AdvWeight (Bahri et al., 2022): adding adversarial perturbation the model parameters; (Id) AdvEmbed (Madry et al., 2018): adding adver- sarial perturbation on the word embeddings. (Ie) FreeLB (Zhu et al., 2020): introducing efficient adversarial training with gradient accumulation. (If) SMART (Jiang et al., 2020): in addition to adversarial perturbation, introducing EMA-based Bregman proximal point regularization. (/g) RIFT (Dong al., 2021): introducing adversarial fine- tuning method from an ciformation-thooretiin per- spective. Furthermore, we consider recent state-of- the-art algorithms that preserve the generalizable knowledge of pre- praitra language models during fine-tuning: (2a) R3F (Aghajanyan et al., 2021); noise- based consistency regularization to prevent representation collapse; (2b) Weight Consolidation (WCons) (Chen et al., 2020): incorporation of & distance between trained and pre-trained models regularization; (2c) Child-tuning (C-Tune) (Xu et al., 2021): selective update of model parameters with a sampled child model; (2d) LP-FT (Kumar et al., 2022): two-step strategy of linear probing and then full fine-tuning, More details of baselines are presented in Appendix A.2.
Model calibration (ECE): To measure the model's calibration performance, we report the average Expected Calibration Error (Guo et al., 2017), calculated during the evaluations on differ- ent datasets including in-distribution, distribution- shifted, and adversarial datasets. As a lower ECE indicates a better calibration unlike other consid- ered robustness metrics, we denote it with (J).
Anomaly detection (AUROC): To measure model performance on anomaly detection, we use multi- ple external datasets as anomaly samples following the setups in recent studies (Hendrycks et al., 2020; Zhou et al., 2021). We use the maximum softmax probability score (Hendrycks et al., 2020) and AU- ROC for the evaluation method and metric.
To evaluate multiple robustness aspects in a unified way, we first report average relative im- provement aavg compared to the vanilla algorithm across different evaluation metrics:
Devg
Σe. A BES
base
S
Snax
8base
S [AcCin ACCehitt; Accadv, ECE, AUROC) and < A, S and Sbase are the performances of a given method and vanilla, respectively. Seax denotes the best score of each metric: 100 for accu- racy metrics, 0 for ECE, and 1 for AUROC. Here. we note that we present the AUROC values as 100 x AUROC in our tables to maintain a consistent relative
Training details. All models are trained using AdamW (Loshchilov and Hutter, 2019) with its de- fault parameters (B1, 32, <0.9 0.98, 1e-6) and weight decay of 0.01. We use linear learning rate decay with warmup ratio 0.06 and learning rate 1y=ee-5 (Liu et al., 2019). The models are fine- tuned with batch size 16 for 10 epochs. All the experiments are conducted with RoBERTa-large (Liu et al., 2019) except for the experiments in Table 3. Both baselines and our method are opti- mized with their own hyper--arametee from set of candidates (described in Appendix A.2) based on the validation set. For RoAST, we use 5 0.1 with
scale with other metries. Also, average instead of direct average for measuring muste-perseective robustness, as it is more appro- priate for the problem at hand; to evaluate LMs" multi-perspective robustness, is crucial to equally incorporate multiple metrics from various perspec- tives. However, these metrics often have different scales, and the direct average is limited in prevent- ing single metric from dominating the aggregate results due its numerical magnitude. In addi- tion, we report the average rank, Rankavg. among multiple baseline algorithms averaged across five different measurements in S. More details about datasets are presented in Appendix A.I.
{0.01, 0.1, 0.5) for adversarial training. For the hyper- parameersss of gradient masking, we use a E [0.6, 0.95), 3 E {1,5, 10) along with a scaling term. More details are in Appendix A.3.

---

Table 1: Robustness measurements of RoBERTa- laare fine-tuned on SST-2 datasct for sentiment classification. All the values and error bars mean and standard deviation across random seeds. The best and the second best results are in bold and underline, respectively.
Method
Acein
AcCshift
AcCedv
ECE (4)
AUROC
Aavg
Rankavg
Vanilla
96.29
91.791013
66.30
7.1 2.1e
86.7213m
0.00
11.8
WoodDrop
+6.44
89.95
69.46=am
7.33
87.57
-1.14
1 11.2
R-Drop
+6.44+0.19
91.75+u21
69.00+1.52
6.05
89,19#120
9.02
9.0
HiddenCut
96.67 10.34
91,14.0020
70.32.a78
5.47 1o.4
89,91 1i.3s
12.26
5.8
AdvWeight
96.41 +0.29
91.92+u.15
65.47 =
7.36
87,80
1.33
10.6
AdvEmbed
96.48 10.0
91.75
69.901a90
5.5 Lo.16
90.79 19.35
13.70
6.0
FreeL.B
96.33
91.94
70.06
6.49 gast
89.82.19.
9.21
7.6
SMART
96.86
91,491033
70.091000
6.32 1034
90.89 _o..4
13.10
5.8
RIFT
96.41 2000
:000
89.55
70.67 1.00
2.85
89.93
3.31
8.6
ChildPTune
96.56+004
91.75+003
69.54
5.57+011
87.00=a11
7.98
8.2
R3F
96.56+0.0
91.79
69, 11.55
5,83
88,49
9.38
7.8
WCons LP-FT
96.601022
92.15
70.86_a12
1.0
89.61 11:03
15.47
3.6
96.33
91.85
72,48
4.05g000
89.46
16.75
5.6
RoAST (Ours)
96.87+0.0
92.38+0112
72.57=a53
5.45+a40
90.37+aas
18.39
1.8
4.2
Main results
while the previous best method achieves 4.4. These results demonstrate that RoAST could serve as a simple yet strong method for robustifying LMs. In- terestingly, advanced fine-tuning methods are more effective in the sentiment classification task than in the entailment task. One possible reason is the difference in the intrinsic difficulty of the task and training dataset size; while the size of the train- ing data is much smaller for SST-2 (67k vs MNLI: 393k), the accuracy is much higher in the senti- ment classification task. This indicates that LMs could be more vulnerable to overfitting, and hence regularization of training could be more effective.
To evaluate the effectiveness of ROAST for im- proving the robustness of LMs, compare it with various baselines by fine-tuning RoBERTa-large model (Liu al., 2019). Tables and summarize the experimental results on sentiment classification and entailment task, respectively. First, it is worth noting that the common evaluation method using the accuracy on the validation set is not enough to capture the robustness of given model. For example, all the baselines successfully improve the vanilla fine-tuning the validation accuracy (Accin), but their robustness sometimes degrades severely as listed Table 2. Such results sup- port the necessity of considering multi-perspective model robustness naively focusing on validation accuracy. Also, we note that there is no single best method when con- sidering multiple perspectives of model robustness; this result indicates the value of unified measure- ment to facilitate robustness evaluation.
To further demonstrate the effectiveness of ROAST, we verify its compatibility across differ- ent types of pre-trained LMs. Specifically, in ad- dition to RoBERTa-large, we conduct additional experiments with five recent state-of-the--ee LMs which have the similar number of model parame- ters: BERT-large (Kenton and Toutanova, 2019), ALBERT-xxlarge (Lan et al., 2020), XLNet-large (Yang et al., 2019), ELECTRA- (Clark et al., 2020), and DeBERTa-large (He et al., 2021). We note that the best hyper-parameters found in Ta- bles and 2 are inherited without additional cost from separate search. In Table we present the experimental results by comparing the average rel- ative improvement of ROAST compared to two representative baselines, AdvEmbed and WConsol,
unified manner, rather than
In this sense, one can observe that ROAST con- sistently outperforms the baseline fine-tuning meth- ods. To be specific, across 4 different robustness metrics along with validation accuracy, ROAST exhibits 18.39 % and 7.63% average relative im- provement compared vanilla fine-tuning on senti- ment classification and entailment, respectively. As a result, ROAST achieves an average ranking of 2.7

---

Table 2: Robustness measurements of RoBERTa-large fine-tuned on MNLI dataset for entailment task. All the values and emrrr bars mean and standard deviation across random seeds. ROAST again achieves the best overall performance across different evaluation aspects.
Method
Accin
ACCshirt
ACCadv
ECE (L)
AUROC
Dasg
Rankavg
Vanilla
89.97
64.3 40.s
48,60=131
12.64
92.09+23s
0.00
9.2
WordDrop
90.35
63.20+044
52.45 -0.7
12.17=004
87.97+1.40
-8.15
8.2
R-Drop
90.64 Loco
62.74
11.73
91.89
2.45
6.4
HiddenCut
90,48
63.84+026
50.43+au1
11.83+006
91.64++039
1.60
6.8
AdyWeight
90.19 10.1
62.87
48.00
12.381.071
91.01 1uM0
-2.97
11.4
AdvEmbed
90.24
64.23
50.20
12,48
93,83
5.72
6.8
FreeLB
90.27 1:030
64.9410014
49.48 2044
12.26
93.24
4.74
6.4
SMART
90.74 1oo0
63.27 10:1
52.27 10
11.41 Loas
91.42.10
2.56
5.8
RIFT
89.73
62.33
53.26
13.46
92.88 1.554
0.85
9,4
ChildPTune
90.08=0.7
64.09
46.48
13,63+3.29
86.60+500
-16.14
12.0
R3F
90.4 +007
+3.54
50.29
11.39
91.81
2.31
6.8
WCons
20.54
25.04
49,00=a00
10.84
91.49
2.99
5.2
LP-FT
90.42+0.44
64.70+0.4
48, +0.0
12.64
93.63+uue
5.11
6.6
RoAST (Ours)
90.64
63.95 10::1
51.33
11.02 +.44
93.25
7.63
3.6
Table 3: Robustness with different language models. Average relative improvements (Aaug) compared to vanilla fine-tuned BERT-large are reported for each model. All the values are mean across random seeds. More detailed experimental results, such the performances on individual robustness metries, are presented in Appendix D.
Entailment
Sentiment Classification AdvEmbed
Model Vanilla
AdvEmbed
WCons
RoAST
Vanilla
WCons
ROAST
BERT-large
00.00
22.85
20.23
25.34
00.00
18.24
15.85
22.76
RoBERTa-large
37.98
39.35
40.75
41.31
38.40
44.94
47.35
48.33
ALBERT-xXlarge
42.86
42.74
41.43
44.29
24.75
45.34
42.36
51.13
Xl.Net-large
32.79
34.48
33.19
36.04
37.24
37.03
34.38
41.46
ELECTRA-laue 39.47
41.36
38.49
42.96
47.85
48.24
46.21
49.76
DeBERTa-larg 36.18
39.73
37.74
44.48
40.21
44.84
42.33
52.23
which show competitive performance in Tables and 2. Here, to facilitate the comparison between different LMs, we calculate the average relative im- provement using the vanilla fine-tuned BERT-large the common baseline for Stase in Eq. 5. We observe that RoAST significantly improves the ro- bustness of fine-tuned models regardless of the type of LMs. More interestingly. RoAST could be use- ful to reveal the true potential of LMs; DeBERTa- large becomes the most robust LM with ROAST while it was originally far from the best. Detailed results on each dataset and LM are presented in Appendix D and E,, respectively.
specifically, we fine-tune RoBERTa-large on SST-2 by varying each component of ROAST: (a) ad- versarial perturbation (Advp) in Eq. and selective update of the model parameters with (b) threshold- ing (Thre) using relative importance s()) in Eq.2. (c) scaling (Scal) with smooth approximation us- ing p(0) in Eq.3, and (d) sampling (Samp) from Bernoulli distribution with mn() in Eq.4. Then, we evaluate the robustness of each method using the constructed robustness benehmark as same as Table 1. We summarize the results in Table As shown in Table 4, the incorporation of the proposed impor- tance score via re-scaling, denoted by (d), performs slightly worse than naive adversarial training, de- noted by (a). However, the incorporation of the importance score via selective update with hard thresholding, denoted by (c), outperforms both.
4.3 More analysis with ROAST
Ablation study. To validate the proposed compo- nents of ROAST, we conduct an ablation study;

---

Table Ablation study with each component of RoAST on the sentiment classification with RoBERTa-large All the values and error bars mean and standard deviation across random seeds.
Method
AdvP
Thre
Scal
Samp
AcCin
ACCghift
AcCadv
ECE (1)
AUROC
A
Vanilla
26.29
91.79 a117
66.30
7.1 2asz
86.72
0.00
(a)
96.48
91.75+032
69.90
5,51 #0.16
90.79+aas
13.70
(b)
96.67
91.99 tcuas
70.25 20.12
5.65
89.92
13.80
(c)
96.7 +024
91.88
72.01 +o.73
5.14
89.66+a1
15.83
(d)
/
96.44_nzs
91.65
70.48
6. 6.17
91.23
12.28
ROAST (Ours)
96.87=0.1
92.38+au1z
72.57 +0.53
5.45+040
90.37
18.39
Table Robustness of fine-tuned RoBERTa-large with different ways enhance the robustness by training model under perturbation while preserving pre- rreitrined knowledge. Rand and Min are RoAST with different masking strategies. All the values are mean across random seeds and results with variance are presented in Appendix E.
Entailment
Sentiment Classification
Method
AcCan
ACCehist
AcCade
ECE (L)
AUROC
Awg
AcC
AcCehitt
AcCadr
ECE (4)
AUROC
A
Vanilla
89.97
64.31
48.60
12.64
92.09
0.00
96.29
91.79
66.30
7.11
86.72
0.00
WCons"
90.62
65,20
50.96
12.01
93.23
6.52
96.71
91.89
71.46
4.99
89.13
15.16
LP-FT"
90.55
64.88
48.95
11.46
92.83
5.29
96.10
92.14
70,69
5.18
90.97
14.24
RoAST
90.64
63.95
51.33
11.02
93.25
7.63
96.87
92.38
72.57
5.45
90.37
18.39
Rand
90.65
64.13
50.98
11.39
91.13
1.67
96.22
92.25
72.68
5.19
90.03
14.88
Min
90.68
64.37
50.87
11.89
90.89
0.44
96.22
91.97
71.45
5.61
86.70
7.26
This result indicates that the better robustness of s0) in Table is not solely due to incorporating the importance score into the model update, but rather to the careful design of using them via selective training with sparse gradients. Moreover, RoAST explicitly constrains the model update with m(f), significantly enhancing the robustness from its two distinct advantages for training the model. Firstly, it preserves the parameters by selectively training them using masked (i.e., sparse) gradients. Sec- ondly, it reduces distortion from the original gradi- ents by sampling the mask instead of deterministic selection. Here, the gain from the first advantgge is not only obtainable from m(0). Hard threshold- ing via (0) also shares the same advantages of selective training and it can be verified with the results in Table 4, i.e., (c) (a), (b), (d). However, is important to note that its effectiveness can be limited since thresholding can distort the gradients from the original direction through deterministic updates of important parameters. Therefore, the second advantage of (0) is crucial, as it improves selective training by reducing the risk of distortion. By sampling the mask for updates from the distribu- tion p(0), which prioritizes important parameters, m0) continuously benefits from selective training while also covering overlooked parameters through
stochastic sampling. This essential advantage of integrating m0) is demonstrated by improvements over (c.d) Table 4.
Comparison to methods with similar intuition. The key intuition of RoAST is effectively lcarning given task under challenging perturbation while preserving the generalizable knowledge within pre- trained L.M. ROAST achieves this goal via selec- tive model training, but one can consider different ways for the same purpose. For example, SMART (Jiang et al., 2020) and RIFT (Dong et al., 2021) introduce additional regularization to the preser- vation of pre-trained knowledge upon the adver- sarial training. To demonstrate the superiority of RoAST, we additionally consider the extra base- lines with similar intuition, WCons* and LP-FT". by incorporating adversarial perturbation to the methods for the preservation. model robust un- der perturbations as well. As shown in Table and 2, ROAST significantly outperforms SMART and RIFT; since both approaches additionally introduce the regularization loss, it can induce the learning difficulty to find Pareto optimal between learning and preservation. Also, in Table 5, one can further verify that ROAST significantly outperforms both WCons* and LP-FT" This result clearly demon- strates the effectiveness of the proposed selective

---

0.05
Centered Kernel Alignment
SST-2 (Ini)
SST-2 (Last)
0.5
(MNL))
Aag (%)
Similarity
0.8
10-
5
0.2
AdvEmbed Weak
AcvEmbed
AdVEmeed
AdvEmbed
(Oun)
(Ouns)
(Oun)
0.0
0.85
0
Strong
Weak
Strang
Epochs
Distance pre-trained LM
Effect on robustness
(c) Similarity of sampled masks
Figure 2: Qualitative analysis of ROAST. (a) Similarity between the initial pre-trained model and fine-tuned one. (b) Robustmess under the different strengths of adversarial perturbation and improvement from RoAST. (c) Dynamics of the sampled gradient masks for selective training.
training scheme compared to naive combination with existing works.
the intersection union between (1) masks the first epoch and other epochs (SST-2 (Init)), (2) masks at the last epoch and other epochs (SS7-2 (Last)), and (3) masks from SST-2 and MNLI at each epoch (SST-2 (MNLJ)). As the model is trained by focusing on the few task-relevant parameters under ROAST, the sampled masks become task- specific and far from the initial one as training goes (Figure 2(c)). The adaptability of RoAST for each task is further observed from a low similarity between the masks of different tasks.
Additionally, conduct experiments to verify the effect of different strategies of selective train- ing. Instead of focusing on the relatively impor- tant parameters in RoAST (Eq.3), Min gives more weights on the unimportant ones by considering the reversed order to obtain the normalized score, i.e., 3(0max) 0 < 5(0) < = 3(Om1n). Rand randomly assigns 5(0) regardless of the relative importance score s0)) From Table 5, we find that the effectiveness of Min and Rand is largely de- graded compared RoAST. Remarkably, Min shows worse results than Rand, which reveals the importance of focusing on the task-relevant, impor- tant model parameters for selective training.
Conclusion
In this paper, we propose to consider multiple as- pects of model robustness for the reliable deploy- ment of LMs in real-world settings. To improve the robustness of fine-tuned LMs, we present a new fine-tuning method (RoAST) by leveraging adversarial perturbation while preventing its poten- tial risk with efficient selective training. Through extensive experiments under constructed bench- mark, we demonstrate the effectiveness of RoAST and its generalizability to multiple state-of-the-an LMs. As investing multiple aspects of model ro- bustness in a unified viewpoint is under-explored in the literature, we expect our work to contribute to this research direction to enable us to use the well well-perfooming pre-trained language models with more reliability. Furthermore, since our proposed method of robust fine-tuning is task- and domain- agnostic, we believe that ROAST can benefit other NLP tasks (e.g.. question answering) and domains (e.g., vision and graph) well, interesting future work directions.
Qualitative analysis. We further present quali- tative analysis on ROAST. To this end, we con- sider RoBERTa-large on SST-2 with two different strengths of adversarial perturbation to facilitate the analysis: Weak (A = 0.01) and Strong (A 0.5). Then, we measure the similarity between the ini- tial pre-trained model and fine-tuned centered kernel alignment (CKA) (Kornblith et al., 2019). Specifically, we measure the average CKA between each layer's hidden features across train- ing epochs. In Figure 2(a), we observe that stronger adversarial training incurs larger deviation from the pre-trained model, and it could be effectively prevented by ROAST. This result indicates that RoAST helps fine-tuned model to preserve the gen- eralizable knowledge of the pre-trained LM while learning new task under adversarial perturbation, and hence can effectively improve the model ro- bustness (Figure 2(b)). In addition, we investigate the dynamics of gradient masking by measuring
using

---

Limitations
2016 conference on machine translation. In Proceed- ings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 13 131-198.
Although we have conducted comprehensive exper- iments on two representative NIP tasks with a wide range of LMs and multiple representative robust- ness aspects, results and analysis on more datasets, tasks, and domains (c.g., computer vision) would likely draw a more decisive conclusion. In addition to empirical evidence, theoretical analysis of why gradient masking improves language model robust- ness in aspects like model calibration and anomaly detection is also worth further exploration.
Shikha Bordia and Samuel R Bowman. 2019. Identify- ing and reducing gender bias in word-level language models. arXiv preprint arXiv: 7 1904.03035.
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models few-shot learners. In Advances in Neural Information Process- ing Systems (NeurIPS).
Jiaao Chen, Dinghan Shen, Weizhu Chen, and Diyi Yang. 202 Hiddencut: Simple data augmentation for natural language understanding with better gener- alizability. In Annual Meeting of the Association for Computational Linguistics (ACL).
Ethics Statement
The robustmess issue of language models has been extensively investigated and a simple yet effective fine-tuning algorithm, ROAST, has been proposed to address this issue. Similar to other fine-tuning methods, the behavior of trained language models with the proposed method might highly rely on the training dataset. Therefore, they can suffer from the inherent problems of the given dataset, e.g,, gen- der bias (Bordia and Bowman, 2019) or annotation artifacts (Gururangan et al., 2018). However, as the proposed idea of ROAST is general and can be easily extended to other training objectives, we believe that the concerns on such problems could be alleviated by incorporating recently proposed solutions for those problems (Sagawa et al., 2020; Moon et al., 2021); for example, one could add new training objective in E to address each prob- lem while preserving the idea of gradient masking of the overall objective (Eq.2).
Jifan Chen, Eunsol Choi, and Greg Durrett. 202 Can nli models verify qa systems' predictions? In Find- ings of the Association for Computational Linguistics: EMNLP 2021, pages 3841-3854.
Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu, and Xiangzhan Yu. 2020. Recall and learn: Fine-tuning deep pretrained language models with less forgetting. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
Kevin Clark, Minh-Thang Luong, Quoe V Lee. and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representa- tions (ICLR).
Brian Davies, 2002. Integral transforms and their ap- plications, volume 41. Springer Science & Business Media.
Shrey Desai and Greg Durreut 2020. Calibration of pre- trained transformers. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
References
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta.
Xinshuai Dong, Anh Tuan Luu, Min Lin, Shuicheng Yan, and Hanwang Zhang. 2021. How should pre- trained language models be fine-tuned towards adver- sarial robustness? In Advances in Neural Informa- tion Processing Systems (NeurIPS).
2021. Better fine-tuning by reducing representational collapse. In International Conference on Learning Representations (ICLR). Dara Bahri, Hossein Mobahi, and Yi Tay. 2022. Sharpness-awwer minimization improves language model generalization. In Annual Meeting of the As- sociation for Computational Linguisties (ACL).
Desmond Elliott, Stella Frank, Khalil Sima' an, and Lu- cia Specia. 2016. Multi30k: Multilingual english- german image descriptions. In Proceedings of the 5th Workshop on Vision and Language, pages 70-74.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raf- fel. 2019. Mixmatch: A holistic approach to semi- supervised learning. In Advances in Neural Informa- tion Processing Systems (NeurIPS).
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. 2021. Sharpness-aware mini- mization for efficiently improving generalization. In International Conference on Learning Representa- tions (ICLR).
Ondicj Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, An- tonio Jimeno Yepes, Philipp Koehn, Varvara Lo- gacheva, Christof Monz, et al. 2016. Findings of the
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and hamnessing adversar- ial examples. In International Conference on Learn- ing Representations (ICLR).

---

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Wein- berger. 2017. On calibration of modern neural net- works. In International Conference on Machine Learning (ICML).
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre- traiiriiing of deep bidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT.
Demi Guo, Yoon Kim, and Alexander M Rush. 2020. Sequence-level mixed sample data augmentation. In Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP).
Jachyung Kim, Dongyeop Kang, Sungsoo Ahn, and Jin- woo Shin. 2022. What makes better augmentation strategies? augment difficult but not too different. In International Conference on Learning Representa- rions (ICLR).
Abhishek Gupta, Alagan Anpalagan. Ling Guan, and Ahmed Shaharyar Khwaja. 2021. Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues. Array, 10:100057.
Klim Kireev, Maksym Andriushchenko, and Nicolas Flammarion. 2022. On the effectiveness of adversar- ial training against common corruptions. In Uncer- tainty in Artificial Intelligence (UAT).
Suchin Gururangan, Swabha Swayamdipla, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A Smith. 2018. Annotation artifacts in natural language infer- ence data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume 2 (Short Papers).
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Barnkkka, et al. 2017. Over- coming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences. I1 (14(13):521-3352
Pengeheng He. Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhancee enhanced bert with disentangled attention. In International Conference Learning Representations (ICLR).
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Similarity of neural network representations revisited. In International Conference on Machine Learning (ICML).
Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dxiedzic, Rishabh Krishnan, and Dawn Song. 2020. Pretrained transformers improve out-of-distribution robustness. In Annual Meeting of the Association for Computational Linguistics (ACL).
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang. 2022. Fine-tuning can distort pretrained features and underperform out-of- distribution. In International Conference on Learn- ing Representations (ICLR).
Dan Hendrycks, Andy Zou, Mantas Mazeika, Leonard Tang, Bo Li, Dawn Song, and Jacob Steinhardt. 2022, Pixmix: Dreamlike pictures comprehensively im- prove safety measures. In Conference on Computer Vision and Pattern Recognition (CVPR).
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Fppttein, Maa Polosukhin, Jacob Devlin, Ken- ton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics. 7:453- 466.
Haoming Jiang, Pengcheng He, Weizhu Chen, Xi- aodong Liu, Jianfeng Gao, and Tuo Zhao. 2020. Smart: Robust and efficient fine-tuning for pre- trained natural language models through principled regularized optimization. In Anmual Meeting of the Association for Computational Linguistics (ACI.).
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. Albert: A lite bert for self-supervised learning of language representations. In International Confer- ence on Learning Representations (ICLR).
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? strong base- line for natural language attack on text classification and entailment. In Proceedings of the AAAI confer- ence on artificial intelligence.
Ken Lang. 1995. Newsweeder: Learning to filter netnews. In International Conference on Machine Learning (ICML).
Longlong Jing and Yingli Tian. 2020. Self-supervised visual feature learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In Thir- teenth international conference the principles of knowledge representation and reasoning.
Divyansh Kaushik, Eduard Hovy, and Zachary C Lipton. 2020. Learning the difference that makes differ- ence with countefiactually--ngmeen ternational Conference on Learning Representations (ICLR).
Linyang Li. Ruotian Ma. Qipeng Guo, Xiangyang aannyynng& Xue. and Xipeng Qiu. 2020. Bert-attack: Adversarial attack against bert using bert. In Conference Empirical Methods in Natural Language Processing (EMNLP).
data. In In-

---

Zongyi Li, Jiamhan Xu, Jichang Zeng, Linyang Li, Xiao- qing Zheng. Qi Zhang. Kai-Wei Chang, and Cho-Jui Hsieh. 2021. Searching for an effective defender: Benchmarking defense against adversarial word sub- stitution. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3137- 3147.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial nli: A new benchmark for natural language under- standing. In Anmual Meeting of the Association for Computational Linguistics (ACL).
Timothy Niven and Hung- Yu Kao. 2019. Probing neu- ral network comprehension of natural language ar- guments. In Annual Meeting of the Association for Computational Linguistics (ACL).
Alisa Liu, Swabha Swayamdipta, Noah A Smith, and Yejin Choi. 2022. Wanli: Worker and ai collaboration for natural language inference dataset creation. arXiv preprint ar arXiv:201.059555.
Jungsoo Park, Gyuwan Kim, and Jaewoo Kang. 2022. Consistency training with virtual adversarial discrete perturbation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computationall Linguisties: Human Language Technologies.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907.11997.11999
Christopher Potts, Zhengxuan Wu, Atticus Geiger, and Douwe Kiela. 2021. Dynasent: A dynamic bench- mark for sentiment analysis. In Annual Meeting of the Association for Computational Linguistics (ACL).
Nyy Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Confer- ence on Learning Representations (ICLR).
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 188)99.
Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng. and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Annual Meeting of the Association for Computational Linguistics (ACL).
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2020. Distributionally robust neu- networks for group shifts: On the importance of regularization for worst-case generalization. In In- lernational Conference on Learning Representations (ICLR).
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant adversarial attacks. In International Conference on Learning Representations (ICLR).
I Lakshay Sharma, Laura Graesser, Nikita Nangia, and Utku Evci. 2019. Natural language understanding with the quora question pairs dataset. arXiv preprint arXiv:1907.0m 90.00141...
Julian McAuley and Jure Leskovec. 2013. Hidden fac- tors and hidden topics: understanding rating dimen- sions with review text. In Proceedings of the 7th ACM conference on Recommender systems, pages 165-172.
Dinggang Shen, Guorong Wu, and Heung- Suk. 2017. Deep learning in medical image analysis. Annual review of biomedical engineering. 19:221.
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuris- ties in natural language inference. In Annual Meet- ing of the Association for Computasional Linguistics (ACL).
Emily Sheng and David C Uthus. 2020. Investigating societal biases in a poetry composition system. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 93-106.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over sentiment treebank. In Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP).
Scung Jun Moon, Sangwoo Mo, Kimin Lec, Jacho Lee, and Jinwoo Shin. 2021. Masker: Masked keyword regularization for reliable text classification. In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence.
Junhyun Nam, Jachyung Kim, Jaeho Lee, and Jinwoo Shin. 2022. Spread spurious attribute: Improving worst-group accuracy with spurious attribute esti- mation. In International Conference on Learning Representations (ICLR).
Jihoon Tack, Sangwoo Mo. Jongheon Jeong, and Jinwoo Shin. 2020. Csi: Novelty detection via contrastive learning on distributionally shifted instances. In Ad- vances Neural Information Processing Systems (NeurIPS).
Nathan Ng. Kyunghyun Cho, and Marzyeh Ghassemi. 2020. Ssmba: Self-supervised manifold based data augmentation for improving out-of-domain robust- ness. Conference on Empirical Methods Natural Language Processing (EMNLP).
Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning re- sults. Advances in Neural Information Processing Systems (NeurIPS).

---

James Thome. Andreas Vlachos. Christos Christodoulopoulos, and Arpit Mittal. 2018. Fever: large-scale dataset for fact extraction and verification. In NAACL-HLT.
Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, and Aaron Courville. 2021. Can subnetwork stric- ture be the key to out-of-distribution generalization? In International Conference on Machine Learning (ICML).
Rheeya Uppaal, Junjie Hu, and Yixuan Li. 2023. Is fine- tuning needed? pre-trained language models are near perfect for out-of-domain detection, Annual Meet- ing of the Association for Computational Linguisties (ACL).
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing. Laurent El Ghaoui, and Michael Jordan. 2019. Theo- retically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICML).
Alex Wang. Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Glue: A multi-task benchmark and analysis platform for natural language understanding. In International Conference Learning Representations (ICLR).
Xiang Zhang, Jumbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems (NeurIPS).
Boxin Wang, Shuohang Wang, Yu Cheng. Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. 2021a. Infobert: Improving robustness language models from an information theoretic perspective. In International Conference Learning Representations (ICLR).
Wenxuan Zhou, Fangyu Liu, and Muhao Chen. 2021. Contrastive out-of-distributtin detection for pre- trained tramsformers. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Gold- stein, and Jingjing Liu. 2020. Freelb: Enhanced ad- versarial training for natural language understanding. In International Conference Learning Representa- tions (ICLR).
Boxin Wang, Chejian Xu, Shuohang Wang, 7he Gan, Yu Cheng, Jianfeng Gao. Ahmed Hassan Awadal- lah, and Bo Li. 2021b. Adversarial glue: multi- task benchmark for robustness evaluation of language models. In Thirty-fifth Conference Neural In- formation Processing Systems Datasets and Bench- marks Track (Round?) 2).
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume: 1 (Long Papers), pages 1112-1122 1122.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. 2020. Adversarial weight perturbation helps robust general- ization. In Advances in Neural Information Process- ing Systems (NeurIPS).
Lijun Wu, Juntao Li, Yue Wang, Qi Meng, Tao Qin. Wei Chen, Min Zhang, Tie- Yan Liu, et al. 2021. R- drop: Regularized dropout for neural networks. In Advances in Neural Information Processing Systems (NeurIPS).
Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang. 2021. Raise child in large language model: To- wards effective and generalizable fine-tuning. In Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP).
LI Xuhong, Yves Grandvalet, and Franck Davoine. 2018. Explicit inductive bias for transfer learning with convolutional networks. International Con- ference on Machine Learning (ICML).
Zhilin Yang. Zihang Dai, Yiming Yang, Jaime Car- bonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for lan- guage understanding. Advances Neural Infor- mation Processing Systems (NeurIPS).

---

A Details on Experimental Setups
5. Hence, similar to Yelp, take review score and negative, and 4 and as positive. This binarized dataset is available at https://huggingaaatt co/ aetastts//mazzn_polaiiitt
A.1 Datasets for robustness benchmarks
As described in Section 1,1, we consider two popu- lar NLP tasks, sentiment classification and entail- ment tasks, to verify the multiple aspects of model robustness with fine-tuned LMs. To fine-tune LMs for both tasks, we use SST-2 (Socher et al., 2013) and MNLI (Williams al., 2018) datasets.
IMDB (Maas et al., 2011): IMDB is dataset for binary sentiment classification movie reviews. It is composed of 25,000 labeled training samples and 25,000 test samples.
SST-2: a binary single sentence classification task about movic reviews with human labels of their sentiment (positive or negative). It is composed of 67k training and 872 validation samples.
We use IMDB dataset provided at https: / //huggingface wot.eecctttttttiinnaaaa
cIMDB (Kaushik et al., 2020): Given docu- ments and their initial labels, (Kaushik et al., 2020) asked people to change each one that it matched counterfactual target la- bel, as long as they avoided making any unnecessary changes to facts that were se- mantically unrelated to the label's applicabil- ity and produced revisions that resulted in internally consistent documents. The con- structed cIMDB dataset is publicly avail- able at http:////tthhuu..c//c.c/aaaaa/ counterfactuallyy--ggeeeeent
MNLI: a ternary entailment task which is composed of 393k training and 20k devel- opment samples. Given a pair of sentences (premise and hypothesis), the given task is to predict whether the hypothesis is entail- ment, contradiction, or neutral with respect to the premise.
Both datasets are available at https: // /ggluebenchmark. com/tasks. After that we measure the following aspects using the described datasets.
Poem (Sheng and Uthus, 2020): Poem is a binary single classification task about the sen- timent of poem verses from Project Guten- berg. There are 892 training, 105 valida- tion, 104 test samples, respectively. The dataset is available at https://huugiigg:: co/da tasets ppoesesss////pmseeeeeeee http:/ooommmststeeeeeeee sentiment
1. In-distribution accuracy (Accin): To measure the performance with respect to training distribu- tion (i.e., in-distribution), we measure the accuracy on the validation sets provided from both datasets, following usual practice (Wang et al., 2019).
2. Distribution-shie generalization (Accenirt): To evaluate the model's capability on distribution- shift (i.e., out-of-domain) generalization, mea- sure the accuracy on multiple distribution shifted datasets. For sentiment classification, we use the following five different sentiment datasets based on the setups (Potts et al., 2021).
For entailment task, we follow the setups in the recent work (Liu et al., 2022); 7 different entail- ment datasets used to evaluate the distribution- shift generalization of entailment classifier; here, some of the datasets are binary classification rather than ternary (denoted by *). Hence, following (Liu ct al., 2022), the MNLI classifier treated as binary classifier by merging the predicts as neu- tral or contradiction into not entailment. All the datasets are available at https: htt::://ggtttu.... thwb.ccubooooo... com/ all i sawuffles/wanli.
Yelp (Zhang et al., 2015); The Yelp re- views dataset consists of reviews from Yelp. It is extracted from the Yelp Dataset Chal- lenge 2015 data. As Yelp has five star-rating categories, we bin these ratings by taking the lowest two ratings to be negative and the highest two ratings to be positive, fol- lowing (Potts et al., 2021). This dataset is available at https://huut http://ggggggn.e...//// co/ dataastss/yelp_reviie_.fo
Diag (Wang et al., 2019): NLI Diagnostics uses naturally-occurrigg sentences from sev- eral domains to evaluate the variety of linguis- tic phenomena.
HANS (McCoy al., 2019): Based on the lex- ical overlap between the premise and the hy- pothesis, HANS seeks faulty syntactic heuris- tics.
Amazon (McAuley and Leskovec, 2013):
The Amazon reviews dataset consists of re- views from amazon with a rating from

---

QNLI (Wang et al., 2019): QNLI binary classification task that decides whether the given (question, sentence) pair contains the correct answer (entailment) or not.
TextFooler (Jin et al., 2020): We denote the dataset with the adversarial texts from vanilla fine-tuned BERT as (1) TF-B, Sim- ilarly, the dataset with the adversarial text from vanilla fine-tuned RoBERTa is denoted (2) 7F-R. The official code of TextFooler is available at https://gitinn com/ jindiiin TextFooler.
WNLI (Levesque et al., 2012): Winograd NIL (WNLI) is from the Winograd Schema Challenge (Levesque et al., 2012), which checks the correct coreference through com- mon sense. By replacing the proper referent, an entailed hypothesis is created, and by re- placing the incorrect referent, a non-entailed hypothesis is created.
DynaSent (Potts et al., 2021): DynaSent is dynamically constructed through multiple it- erations of training a classifier model and find- ing its adversarial samples by involving hu- man annotator in the loop. In our experiments, we the dataset from the first round, (3) DynaSent-Ro RI, and the dataset from the second round, (4) DynaSent-R2. As DynaSent is ternary sentiment classification (Positive, Neu- Iral, and Negative), we remove the samples with Neutral. Also, we use both validation and test sets for the evaluation. The datasets are publicly released at ht https: /ttps:///hggigggg nggace.. co/ co/datassess/ddmaeeee///// dynasent.
NQ-NLI (Chen al., 2021b): Using Natu- ral Questions QA dataset (Kwiatkowski et al., 2019), NQ-NLI creates decontextualized sentence from the original context for the premise and a hypothesis from a question-and- answer candidate converted into a declarative form.
FEVER-NLI (Thorne et al., 2018): It is adapted from the FEVER dataset (Thorne et al., 2018). In each case, the hypothesis is statement that either supported (implied), re- futed (contradicted), or neither (neutral), and the premise is brief context from Wikipedia.
AdvGLUE (Wang et al., 2021b): To con- struct principled and comprehensive bench- mark for adversarial robustness NLP tasks, (Wang et al., 2021b) systematically apply 14 textual adversarial attack methods to GLUE tasks to construct AdvGLUE, which is further validated by humans for reliable annotations. Hence, we measure the robustness of senti- ment classifier using the dataset for SST-2 in AdvGLUE and denote it as (5) AdvGLUE (SST-2). AdvGLUE dataset available at https: / http://vvvvvraaaaaaalllaaaaa
WANLI (Liu et al., 2022): Starting with an existing dataset such MNLI, the new sam- ples are automatically generated with GPT-3 focusing on the ambiguous samples. To fur- ther improve the quality constructed dataset, automatic filtering is applied and then each sample is annotated by human labelers.
github.... io.
Similarly, for entailment task, we use the following nine different adversarially constructed datasets to evaluate the adversarial robustness of entailment classifier. Here, -m indicates that the dataset is constructed from MNLI's matched validation set and mmm indicates from mismatched validation set.
3. Adversarial robustness (Acczav): To mea- sure the adversarial robustness of model, we first construet the text-level adversarial examples using TextFooler (Jin al., 2020) on vanilla fine-tuned BERT and RoBERT models, following (Wang et al., 2021a). We also consider the datasets con- structed via dynamic adversarial data collection with human-in-the-loop (Nie et al., 2020; Potts et al., 2021). In addition, we use the datasets from a recent benchmark for adversarial robustness, Ad- vGLUE (Wang et al., 2021b), which incorporate the various types of adversarial noises. Overall, for the sentiment classification, the following five dif- ferent adversarially constructed datasets are used to measure the adversarial robustness of fine-tuned LMs.
TextFooler (Wang et al., 2019): TF-B and TF-R are defined in the same way with the case of sentiment classification. Hence, we consider (1) TF-B-m, (2) TF-B-mm, (3) TF-R-m, and (4) TF-R-mm. We remark that the corresponding datasets constructed from other researchers are available https://drrve..ooglee. com/fi//eee.//////////ooo 1 wWw.ABFFeeGGEERR1xxx--vveveeee view.

---

ANLI (Nie al., 2020): Similar the case of DynaSent, ANLI dynamically constructed through multiple iterations of training a clas- sifier model and finding its adversarial sam- ples by involving human annotator in the loop. As there are three rounds in overall, we utilize all of these datasets: (5) ANLI- RI, (6) ANLI-R2, and (7) ANLI-R3. ANLI dataset is available https://huggingface. co/datasets/anli.
publicly available at http://qwone.com "jason/22Newsgroupss/
QQP (Sharma et al., 2019): QQP is bi- nary classification datasets for entailment task, where the goal is to determine if two ques- tions in given pair semantically equiv- alent not. As part of GLUE benchmark, it is available at https: http:::/hhgginggaaee co/ datasets/glue.
AdvGLUE (Wang ct al., 2021b): We use the datasets for MNLI in AdvGLUE and denote it as (8) AdvGLUE-m and (9) AdvGLUE-mm.
In addition, we consider that the one dataset be- comes anomaly samples to the other, i.., SST-2 become anomaly dataset with respect to MNLL. Hence,
total six anomaly datasets in case of sentiment classification and five datasets in case of entailment, respectively.
4. Model calibration (ECE): To measure the model's calibration performance, we report the av- erage Expected Calibration Error (Guo et al., 2017), denoted ECE, calculated during the all evalua- tions on different datasets including in-distribution, distribution-shifted, and adversarial datasets intro- duced in above. Namely, we report the average ECE across 11 datasets for sentiment classification and 18 datasets for entailment.
A.2 Baselines
We consider various baseline fine-tuning algo- rithms in NLP tasks. Specifically, we first consider a wide range of perturbation-bast fine-tuning al- gorithms and their training loss Lurain can be de- scribed as follow:
5. Anomaly detection (AUROC): To measure the performance about the anomaly detection, we use the following four external datasets as anomaly samples based on the setups in the recent related works (Hendrycks et al., 2020; Zhou et al., 2021).
Ltrain = Lrask (fo(x). y) + (taak(FF(F
+ ACcons (f(()), (6)
where fe(x) indicates the perturbed prediction of model fe for input x. Also, Leone is a bidirectional KL divergence introduced in Eq. Here, for better explanation, we slightly abuse the notations of in- puts for Cesck and Lcons, compared to Eq. 1. With Eq.6, the baselines in this categories only have a difference in how they impose the perturbation for the prediction:
WMT16 (Bojar et al., 2016): WMT is translation dataset based on the data from statmt.org and versions exist for different years using a combination data sources. We use the English source side of English source side of English-German WMT 16, follow- ing the previous works. WMT dataset could be downloaded from https://huggingfaet co toddaassets/wmtt
WordDrop (Guo et al., 2020) impose the per- turbation by randomly dropping the input to- kens with probability Pwa similar to Dropout. We select PxD E (0.05, 0.10, 0.15).
Multi30K (Elliott al., 2016): Multi30K is a translation datasets which extends the Flickr30K dataset with German translations created by professional translators over a subset of the English descriptions, and in- dependently crowd-sourced descriptions of the original English descriptions. The dataset is available at https::/e //gi thub. mult multi30k/dataset
HiddenCut (Chen et al., 2021a) drops the contiguous spans within the hidden features of Transformer model during fine-tuning. As the attention- based strategy for sampling the spans shows the best results in (Chen et al., 2021a), we adopt it and tune the Hidden- Cut ratio Pac E {0.1, 0.2, 0.3) with the fixed selection ratio of 0.4. The official code is available at https: //gi thub. com/ SALT-NLP/HiddenCUu
20 NG (Lang, 1995): 20 Newsgroup is a dataset for topic classification con- sists of 20 classes. 20 NG dataset is

---

AdyWeight (Bahri et al., 2022) adds adver- sarial noise on the model parameters rather noteworthy that this method is also called as Sharpness-Aware Minimiza- tion (SAM) (Foret et al., 2021). We tune the step size p for gradient ascent step among {0.01, 0.03, 0.05). We adopt the codes from https://githhue com/ davda54/sam.
too much from the initial pre-trained model to pre- serve the generalizable knowledge of pre-trained language models during fine-tuning:
than input.
R3F (Aghajanyan et al., 2021) introduces a noise-based consistency regularization to pre- vent representation collapse. Hence, we use the same candidate for À E {0.01, 0.1, 0.5). In addition, we consider the fixed variance of noise 0=le-5 with the two noise distribu- tions additional payper-parmmnn lowing the original paper (Aghajanyan et al., 2021). Also, the official code is available https://gi com wwt.c/otttt//tttcctetoooo cairges
AdvEmbed (Madry al., 2018) imposes ad- versarial perturbation the word embeddings of imput tokens. We set the same values be- tween the magnitude perturbation and step size for the gradient ascent step; a step size is tuned among ( (le-5, le-3, le-1) under foo norm.
[V fol-
Weight Consolidation (Chen et al., 2020) in- corporates &2 distance between trained and pre-trained models regularization during fine-tuning. To gradually control the strength of such regularization, the authors consid- ers the sigmoid annealing function X(4) 1/(1 + exp(-k: (t ta))) where t E [0, 1]. Here, we tune the hyper-parameters & and to among {0.1, 0.5, 1.0} and {0.1, 0.5}. re- spectively. We denote it as WConsol. We use the official code from https: h/tt::/b.oohhooomommm Sanyuan-CCennReccdaa
FreeLB (Zhu et al., 2020) proposes an ef- ficient way to construct multi-step adver- sarial perturbation via gradient accumula- tion. We follow the best hyper-parameters provided by the authors after careful tun- ing. The official code available at
SMART (Jiang et al., 2020) additionally in- corporates the regularization based on Breg proximal point method. Specifically, they use EMA model (Tarvainen and Valpola, 2017) and consistency loss between fine-tuned model. We set the same hyper-parameters in the paper, except the coefficient between each loss; for such coefficient, we tune among (0.01, 0.1, 1.0). The official code available at
Child-tuning (Xu et al., 2021) selectively up- date the subset of model parameters (called child network) with fixed child model. As the task-driven approach shows the better per- formance compared to task-free variant in (Xu et al., 2021), adopt the task-driven one as baseline. We tune the child net- work's 's sparsity pp among {0.1, 0.2,0.3.333 fol- lowing the original paper (Xu et al., 2021). We denote this method as ChildTune in our paper. Official code by the authors is publicly released at https://g/t http://gttttu....e thub. com/ PKUnlp-icler/Childtue
RIFT (Dong et al., 2021) introduce reg- ularization loss which is derived from snffrmmtiion-theorete perspective. RIFT encourages objective model to retain the features learned from the pre-trained model throughout the entire fine-tuning process. We tune hyper-parameters a among 10.1, 0.3, 0.7). following the offi- cial code by authors: https: /gi thub. com/ dongxinshuai/RRe SwtpssuaaaRRRRT-NNNARA NNur-NNN
LP-FT (Kumar et al., 2022) uses a two-step strategy of linear probing and then full fine- tuning. For a linear probing, we train the lin- ear classifier on the frozen backbone using Adam optimizer with fixed learning rate n le-3 and 5 epochs. Then, tune the learn- ing rate 7/t during the full fine-tuning among [1c-6, 3c-5, lc-5).
Also, we commonly tune the among {0.01, 0.1, 0.5) in addition to the specific parerttareeeter of each method.
A
A.3 ROAST
On the other hand, we also consider the recent methods that prevents the model from deviating
As described in Section 4.1, we use fixed step size = 0.1 for the gradient ascent step to construct

---

Algorithm 1 ROAST: Robustifying LMs via Ad- versarial Perturbation with Selective Training
B Proof of Corollary
In this section, we present a formal proof of the Corollary. To this end, we first present the theoreti- cal results by (Xu et al., 2021):
Input:
Pre-trained LM fe. training data D, learning rate 7/, update frequency T. masking ratio a, smoothness factor B, adversarial noise magnitude 6, coefficient of regularization À
Theorem. (Xu al., 2021) Suppose L denotes the loss function on the parameter for multiple data instances in the training set x ~ D, the gra- dients obey a Gaussian distribution N PC 70 og 1x). For a randomly sampled batch B ~ S, when the learning algorithm is SGD with learning rate n) the probability of the gradient mask from Bernoulli distribution is p. then the mean and covariance of the update AO := -n( 30 @ m(0)) are
/* Obtaining ini Anitial gradient */ ( < InitGrad D) for each iteration do
% T then
/* Get relative importance */
{s(0)]0 E }} + g()), g(+) t 0
/x Sample gradient mask */
{m(0)] E }} + Mask( (()) a, 5)
ac
end if
|| 11 88
/* Sampling training data */
(x, y) D
17 n
(1 - p)v) diag
OC 12
Σ[A0]
** Add adversarially perturbation */
p|B
p
x x+x x+
/0x|
where E is the covariance matrix and diag( X) is the diagonal matrix of the vector X.
x* Get gradient during backward */
g(0) & @Ltrain/00, Ltrain
Here, the key difference between the above prob- lem setup (Xu et al., 2021) and our case is the probability for masking: (Xu et al., 2021) assumes the identical Bernoulli distribution, while we as- sume the element- Bernoulli distribution with the different probability for each model parame- However, in below Corollay, show that our problem can be also proved in the almost same way.
(x.eu(xx))) + Lraex(X,, (x, + (x,
/ Update with masked gradients */
0 + 0 - n- ((m(()/p((0) @ g(0))
/* Accumulate training gradients */
(0)) + ((0)
end for
the adversarial perturbation. Also, similar to the case of perturbation- based regularization methods, we tune the coefficient of consistency regulariza- tion Lcous with À E {0.01, 0.5) (Eq. 1). For the hyper-parameters. of gradient masking, we use a [0.6, 0.95) and 8 E {1, 5, 10} along with the application of a scaling term. We remark that a relatively higher masking ratio a has been effec- tive for sentiment classification, while the smaller a has been effective for entailment during our ex-
Corollary. We consider the same assumption in Theorem by (Xu et al., 2021), expect the probability of the gradient mask follows different Bernoulli distribution for each parameter p(f) and m(0) Ber(p(0)). Then, the mean of the update 20 is,
E[20] = 1] 00
and its covariance is bounded as,
|Σ[A0]|E
periments. Based on such observation, we tune a among (0.95, 0.9, 0.8} for sentiment classifica- tion, {0.6, 0.65, 0.7} for entailment task along with 3 E (1, 5, 10). We accumulate the gradient infor- mation through each training epoch, then sample the gradient mask from them for the next epoch. the case of InitGrad in Algorithm 1, similar to the setups in (Xu et al., 2021), we gather the sum of the square of gradients with respect to vanilla cross-entropy loss, since the classifier is not trained at that time. We use NVIDIA A100 GPUs in our experiments.
17 20
(1 p)y diagt
ac 12 30
d
B
D
where p = minp(())
Proof. Let (()) the gradient of sample x;, 1
< [B], then ()))) ( 22 by the assump-
tion. Let g - Σ [
g B
then we have
AO -m
O m0) -ng 3 m0)
B

---

When we consider 9, the followings are obtained:
in clean accuracy (Dong al., 2021) due the relatively large perturbation on discrete space. In contrast, the embedding-level perturbation can be constructed by adding small noise to continuous space, making it more feasible to train the model without the loss of accuracy (Zhu et al., 2020; Jiang et al.. 2020)
ac 0A
0g 1l
Elg]
[وgl
[5
Suppose g = (m(0)/p(0)) @ 9. then we have:
p(0) Oc p(0)
Ac
E 9
x
= Elgl
To further validate the generalization of RoAST beyond the embedding-level perturbation, we con- duct additional experiments by adapting RoAST with discrete token-level adversarial perturbations by fine-tuning RoBERTa-large using VAT-D (Park et al., 2022). We used the same hyper-parameters for discrete token-level perturbation as in Park al. (2022), and for RoAST, we used the same values previously found with embedding-leve. perturba- tion. The results are shown in the table below.
a
00
Let Dis 9s. Pt are the i-th dimension of g. 9. Then,
Var
[od
B (E(
- Di )2] (E
pi
Elg²' (EL
(I[ + Var [gr] (E(
Here, observe that the discrete token-level adversarial perturbation can improve the multi- perspective robustness, especially for adversarial robustness (Accaav). However, it comes at the cost of degradation in clean accuracy (Accin). With RoAST, such degradation can be mitigated and the overall robustness of the model could be improved.
Pi
Varlgi] (1 (E)
Pt
Pi
Since is a decreasing function regarding pt. one can derive following bound with P == min; Pi+
C.2 Absolute average improvement of ROAST
og
8C p) diag 30
12
H <
B
P
During the experiments, we used the average of relative improvement, instead of absolute improve- ment, as it more appropriate measure the multi- perspective robustness than the average of absolute improvement, not to scale up the values. However, we also recognize its weak points, such as the risk of amplification, which is the reason why we addi- tionally report Rankavg. which does not have simi- lar issues. We emphasize that RoAST exhibits the lowest rank among the state-of-the-ar fine-tuning methods both sentiment classification and en- tailment tasks. Nevertheless, to address the con- cemss about this, we additionally calculate the ab- solute average improvement (Absaug) of (Acein. Accesift, Accaav, 100 ECE, 100 AUROC), on the sentiment classification task. The results are presented in Table 7. Here, one can observe that our method still outperforms the state- saae--offie of-the-art fine- tuning method with large gap; RoAST exhibits 46.72% relative improvement on AbSaug compared to SMART (1.16% 0.79%). This result further demonstrates the effectiveness of our method, and we do believe that our empirical results clearly show the merit the proposed framework.
C More Quantitative Results with ROAST
C.1 Generalization beyond embedding-level perturbation
For ROAST, we chose embedding-lewe perturba- tion (Zhu et al., 2020; Jiang et al.. 2020) over token- level perturbation (Jin et al., 2020; Li et al., 2020), it is more computationally efficient and better suited for enhancing multi-perse pealpiiperspettvee robustness. Specifically, the construction of token-level adver- sarial perturbations requires more computation to solve the discrete optimization problem, and ad- ditional regularization is often introduced to pre- vent degenerate cases such as significant changes in semantic or lexical violation (Jin et al., 2020; Li et al., 2020; Park et al., 2022). For example, relatively simple construction of token-level pertur- bation with Park al. (2022) requires 15% more times per iteration, compared to embedding-level perturbation. addition, while the token-level ad- versarial perturbation effective for adversarial robustness, often comes at the cost of decrease

---

Table Generalization of ROAST with discrete token-level adversarial perturbation. Here, for such perturbation. use VAT-D (Park al., 2022). All the values mean across 3 random seeds.
Method
Accin
AcCshift
AcCadv
ECE (L)
AUROC
A Avvg
Vanilla
96.29
91.79
66.30
7.11
86.72
0.00
VAT-D
95.83
90.86
70.37
6.33
90.39
5.37
VAT-D ROAST
96.27
90.45
72.12
6.94
92.16
8.73
Table 7: Average absolute improvements with different baselines. All the values are mean actoss 3 random seeds.
Mathol
Vanilla
WeedDoop
R-Drop
HidlenCht
AdiWhigh
Adviimbed
Freet.8
SMART
RIFT
ChildPTine
R3F
WC
LP-FT
RoAST (Ours)
Abssez
76.47
76.91
76.34
77.20
77.13
77.29
74.72
77.05
C.3
Additional comparison with FreeLB++
10 and 11, the results on sentiment classification and entailment are presented, respectively.
Here, present additional experimental results with FreeLB++ (Li et al., 2021) the sentiment classification task, which runs adversarial pertur- bation for 10 steps without constraints of norm- bounded projection. We also tuned the adver- sarial step size appropriately as the number of steps varies. The results are summarized in Ta- ble 8. Here, one can observe that FreeLB++ out- performs FreeLB, especially in ACCadv (70.07 + 72.45). Consequently, FreeLB++ is better than FreeLB for multi-perspective robustness as well (Aavg 99.1 + 11.02). However, RoAST still out- performs FreeLB++ with a large gap, which further demonstrates the effectiveness of our method. On the other hand, these results indicate a potential room for further improvement in our method at the additional cost of the increased number of steps, since RoAST also uses single step for construct- ing adversarial perturbation.
Next, we provide the additional results with dif- ferent vanilla algorithm to calculate Aavgi Table 3, we use BERT-large as a universal vanilla al- gorithm to calculate relative improvement across different LMs to facilitate comparison in terms of multi-perspective robustness. This choice was made to provide additional insight into the ques- tion of "which LM is the most robust". While answering this question is important, we also ac- knowledge that using the corresponding LM's score as the baseline could provide additional insights into how RoAST performs with each specific LM. Hence, we recalculate Table 3 with the correspond- ing LM's score and present it in Table 12. One can observe that RoAST mostly outperforms the baselines except in only case, and achieves large improvements compared to baselines both tasks. This result demonstrates the robustness and effec- tiveness of RoAST with respect to different LMs.
C.4 Relationship between overfitting and robustness
Lastly, we conducted additional experiments on the sentiment classification task with GPT2-large (Radford et al.. 2019). a popular decoder-only LM. to validate the applicability of our approach. Fol- lowing Radford al. (2019), we added linear classifier head on the last token's 's embedding output for fine-tuning. The results are presented Table 13. Here, we first observe that the improvements with baseline methods are largely limited. We spec- ulate that this ineffectiveness may occur due to the different nature of decoder-only LMs compared to encoder-only ones, such as BERT, as it could result in different effectiveness of the baseline algorithms and tuned saper-parameters, which were originally developed and demonstrated only using encoder- only models; for instance, most of the baselines (Jiang et al., 2020: Zhu et al., 2020; Chen et al.,
To investigate the relationship between overfitting and robustness of LMs, we performed additional experiments by training RoBERTa-large on SST-2, using the Vanilla method with a constant learning rate for 100 epochs. As shown in Table 9, the model's robustness significantly decreases as the training progresses. This outcome confirms that preserving the parameters is critical for maintaining model robustness, which is fundamental principle of selective training with m(f) in ROAST.
Additional Results with Various LMs
First, we present the detailed results on the individ- ual robustness metrics like Tables and In Table

---

Table 8: Comparison with additional baseline, FreeLB++ (Li et al., 2021) on the sentiment classification task. All the values mean across random seeds.
Method
Accin
AcCehift
AcCadv
ECE (4)
AUROC
Davg
Vanilla
96.29
91.79
66.30
7.11
86.72
0.00
FreeLB
96.33
91.94
70.07
6.49
89.82
9.21
FreeL.B++
96. 14
91.79
72.45
5.44
90.79
11.02
ROAST (Ours)
96.87
92.38
72.57
5.45
90.37
18.39
Table 9: Tradcoff between robustness and training epochs. All the values mean across random seeds.
Method
AcCin
ACCahift
ACCadv
ECE ())
AUROC
Davg
Vanilla (Orig)
96.29
91.79
66.30
7.11
86.72
0.00
Epoch 20
94.88
90.66
64.82
7.75
83.83
-17.45
Epoch 40
93.92
88.52
60.54
9.81
77.04
-46.38
Epoch 60
93.42
88.11
59.63
10.16
72.28
-58.74
Epoch 80
93.39
87.43
59.03
10.68
73.83
-60.08
Epoch 100
93.31
87.52
58.28
10.62
67.61
-69.94
2020; Xu et al., 2021) have been demonstrated un- der encoder-only models and not shown the results of decoder-only ones. Nevertheless, our RoAST ap- proach continues enhance the multi-perspective robustness of GPT2-large, with an average relative improvement of 5.07% compared to the Vanilla method. These results indicate that the effective- ness of our approach is not limited to BERT-based based LMs with Transformer-encoue architecture.
E Individual Experimental Results
Next, we present the results on each dataset for each robustness metric. First, we present the re- sults from sentiment classification. Specifically, we report the accuracy and ECE on distribution shifted datasets in Table 14 and 15, respectively. Also, report the accuracy and ECE adver- sarially constructed datasets in Table 16 and 17, respectively. The results of anomaly detection are shown in Table 18. Next, we present the results from entailment task; we report the accuracy and ECE on distribution shifted datasets in Table 19 and 20, respectively. Then, we report the accuracy and ECE on adversarially constructed datasets in Table 21 and 22, respectively. Finally, report the results of anomaly detection in Table 23.

---

Table 10: Robustness measurements six different LMs fine-tuned on SST-2 dataset for sentiment classification. All the values and bars are and standard deviation across 3 random seeds.
Type LM
Method
Accin
AfCehift
ACCadv
ECE (J)
AUROC
Vanilla
94.042mm
89.30gas4
57.10
83.682mse 9.21
AdvEmbed
94.422n14
88. 28775
61.16
9.46
83.40
BERT
WConsol
93.81
89.4 39.46
16.34
9.35
85.09
ROAST (Ours)
94.07
88.87
60.31
7.00
85,89
Vanilla
96.29
91.79
66,30s214
7. +
86.72+360
AdvEmbed
96.48
91.75
69. 69.90=s0)
5.5
90.79
RoBERTa
WConsol
96.60
92.15
70.86
89.61 5,01
ROAST (Ours)
96.871sm
92.38
72.57.0m1
5.45
90.37
Vanilla
95.24
88.63
64. 13mm
8.71
88.15
AdvEmbed
96.52+a.
91.78
73.63
5.88
87.34
ALBERT
WConsol
96.25 90.92
71.95
5.75
87.44
RoAST (Ours)
96.62+6m
92.33
78.20
5.00
89.32
Vanilla
95.87=0m4
90.80
68.33
6.57
86.76.247
AdvEmbed
96.251mmM
91. 91.22
67.39
7.56
88.261a42
XL.Net
WConsol
95.26.am
g006
67.77
6.82
88.581391
ROAST (Ours)
96.02asm
90.61 j,0e
69.61
5.48
92,25
Vanilla
96.96+s0s
91.62+0u0
74.15
4.92+am
82,43+1a:
AdvEmbed
90.97 #:446
74.81
1.20
=
ELECTRA
WConsol
97.08
91.51 11.1ashe 20/2
73.
1620m
5,55
12.40
ROAST (Ours)
97.022a19
91.75
74.74
4.97.am
88.872067
Vanilla
96.481am
90.95
69.96.1.2
6.631am
86.722447
AdvEmbed
96.64100s
91.83gow
71.18
90.21 guse
DeBERTa
WConsol
96.602a14
91.34
19.74.mm
6.231am:
88.01aam
ROAST (Ours)
97.13
92.31 10004
74.25
4.46.om
89.74gum

---

Table I1: Robustness measurements of six different LMs fine-tuned on MNLI dataset for entailment task, All the values and error bars are mean and standard deviation across 3 random seeds.
Type of LM
Method
Accin
AcCahift
AcCudv
ECE (4)
AUROC
Vanilla
86.25gen
60.20
39.25
23.03
70.88
AdvEmbed
86.99
60.77
42.90
18.50gam
81.98asm
BERT
WConsol
86.27 60.52:004
39.34
17.38
75.7
ROAST (Ours)
26.89
20.38gn41
42.58
14.97
82.41 2n0
Vanilla
89.97 1cmm
64.31 gnss
48. 21.1
12.64
92.09.2.z
AdvEmbed
90.24
64.23 =0as
50.20
12,48
93.83
RoBERTa
WConsol
90.54
65. 04:0as
a0.0
10.84
91.49=140
RoAST (Ours)
90.64
63.95gmm
51.332074
11.02am.s
93.25
Vanilla
90.62 +o.1
64.94
48279
10.25
83.14
AdvEmbed
10.60
64. 19
58.
38.82
10.92
86.63
ALBERT
WConsol
90.43 aocan
64.37 2::3
9.67
80.65
RoAST (Ours)
90.72
66.57 1003
59.24
11.83
91.47
Vanilla
89. 19.299.mmm
63.74_mm
49.332mm
14.04
77.53
AdvEmbed
89. 29.46..m
63.56gn91
50.52240
12.69
21
XLNet
WConsol
89.33
63,55 :nos
49.9 +
14.64
81.35
RoAST (Ours)
a0.03
63.69
50.11
10.28
28.53
Vanilla
90.68
66.20aneo
56,06g1.1
13.87
82.75.2.0
AdvEmbed
90.6 60.64
65.66 65.66=05)
56.95
12.96
88.43
ELECTRA
WConsol
90.5 se.1
67.21 =04:
55.43a0s4
15.36
84,05
ROAST (Ours)
91.07
64.80
58.38
10.17
81.01
Vanilla
90.09
65.01
52.61
15,87
87.89
AdvEmbed
a0.43
65. 2m
54,91 20:0
13.29
86.37
DeBERTa
WConsol
90.02.t.a
64.83 1.:3
53.67
14.51
89.03
ROAST (Ours)
90.97...
66.07 2m:
16.53
9.94gum
88.15
Table 12: Robustness with different language models. Average relative improvements (Azug) compared to each vanilla fine-tuned LM are reported for each LM. All the values are mean across random seeds.
Entailment
Sentiment Classification AdyEmbed
Model
Vanilla
AdvEmbed
WCons
RoAST
Vanilla
WCons
ROAST
Barge
0.00
22.85
20.23
25.34
0.00
18.24
15.85
22.76
RoBERTa-large
0.00
5.72
2.99
7.63
0.00
13.70
15.47
18.39
AL.BRT-xxlarg 0.00
2.60
-3.83
8.19
0.00
21.34
18.25
30.62
-erre-la
0.00
2.44
3.30
0.00
1.44
-1.76
12.74
ELECTRA Allagg
0.00
7.89
-0.70
4.39
0.00
6.44
-2.83
8.29
DeBERTa-large
0.00
2.81
3.79
11.93
0.00
11.95
8.25
18.90
Table 13: Robustness with GPT, LM with Transformer-decoder architecture. Average relative improvements (Anvg) compared to each vanilla fine-tuned GPT. All the values are mean across random seeds.
Method
Accin
ACCshift
AcCadv
ECE (4)
AUROC
Aavg
Vanilla
94.99
84.33
62.38
8.01
95.30
0.00
AdvEmbed
94.92
84.19
63.58
7.90
95.81
2.60
WCons
94.88
80.72
61.80
9.06
93.57
-15.37
ROAST (Ours)
95.03
81.91
64.96
8.52
97.16
5.07

---

Table 14: Accuracy of RoBERTa-large fine-tuned using SST-2 dataset for sentiment classifcation task. One in- distribution validation set and five distribution shifted datasets are evaluated. All the values with larger font are across 3 random seeds. The values with smaller font and plus-minus sign (=) corresponding variance. Numbers in bracket means the number of samples.
Distribution Shifted Datasets
SST-2
Yelp (20K)
IMDB c-IMDB
Poem
Amazon
Method
(872)
(25K)
(2440)
(359)
(100K)
Vanilla
96.29
96.43
88.82
91.79
88.02
93.91
20.14
#0022
10.10
40.45
1:060
20.11
WordDrop
96.44
96.31
88.50
89.06
82.37
93.52
#0,0
10.33
+0.70
10.14
R-Drop
96.44
96.18
88.78
91.80
88.21
93.78
+0.19
#0.11
+0.40
+0021
#:125
=0.07
HiddenCut
96.67
96.17
88.89
91.39
85.79
93.47
2:.34
=:0.88
20.17
2:045
#0.2
20.00
Adv AdvWeight
96.41
96.34
88.33
91.78
89.88
93.25
AdvEmbed
96.48
96.34
89.21
91.42
87.93
93.87
+00.0
#:0.14
+0010
+60.34
#:1.25
=0.05
FreeLB
96.33
96.34
89.20
91.27
88.95
93.96
2:0.34
::0.49
+b0.33
2:064
#:2788
20.5
SMART
96.86
96.23
88.49
90.50
89,04
93.18
+:005
160.66
RIFT
96.41
95.63
88. 14
89.25
81,80
92.93
10.05
1:0000
#0021
1:089
1:047
20.44
ChildPTune
96.56
96.69
89.53
91.93
86.26
94.33
+0.04
#0.11
R3F
96.56
96.24
88.83
90.97
89,23
93.67
2:0000
=:0.22
#0.16
de0.5
66
20.12
WConsol
96.60
97.02
89.75
91.93
87.56
94.49
=:0.22
=10007
10.30
1:023
1:199
20.11
LP-FT
96.33
97.54
89.84
90.04
87,28
94.57
ROAST (Ours)
96.87
96.90
89.52
92.10
89.04
94.32
-:0.30
+0.07
$0.50
20.35

---

Table 15: Expected Calibration Error (ECE) of RoBERTa-large fine-tuned using SST-2 dataset for sentiment classifcation task. One in-distribution validation set and five distribution shifted datasets are evaluated. All the values with larger font are mean across random seeds. The values with smaller fout and plus-minus sign (#) are corresponding variance. Numbers in bracket means the number of samples. Lower ECE value indicates the better calibration.
Distribution Shifted Datasets
SST-2 (872)
Yelp (20K)
IMDB
c-IMDB
Poem
Amazon (100K)
Method
(25K)
(2440)
(359)
Vanilla
3.80
5.30
3.70
5.13
5.87
5.03
1:.77
2:133
WordDrop
5.93
6.97
9.77
10.13
8.57
7.80
#1.54
#:1.6
R-Drop
4.90
5.27
3.80
6.17
5.53
5.30
1145
20017
11.56
12.36
22.69
HiddenCut
3.10
4,00
6.70
6.50
7.23
6.37
+0.62
:10.85
11.31
10.16
10.57
1.3
AdvWeight
5.30
7.20
9.60
7.87
7.27
6.30
AdvEmbed
3.27
2.77
4.07
5.43
5.73
5.33
FreeLB
3.63
5,47
6.17
5.97
4.37
4.50
114
22.13
+2.88
20661
$0.59
21.11
SMART
4.30
7.73
7.50
6.60
5.93
5.73
10.19
11.556
11.55
10.45
1:066
20.3
RIFT
4.40
4.97
7.97
7.20
8.63
5.57
ChildPTune
3.33
4.43
2.40
3.87
5.33
4.20
20.34
20067
1:.16
11.2
R3F
4.60
4.30
6.90
6.60
5.70
6.13
10.96
11:24
1.1.0
1..50
1:086
1121
WConsol
2.50
3.00
4.20
5.00
6.13
5.57
210.44
11.366
20.73
1:100
11.48
2141
LP-FT
3.83
2.27
1.83
2.60
6.43
4.00
+:0.17
=:0.12
40.24
1:0.44
1:001
-022
ROAST (Ours)
3.50
4.03
7.17
7.07
6.90
4.80
=:049
200900
+1.32
#0026
#1.61
2163

---

Table 16: Accuracy RoBERTa-large fine-tuned using SST-2 dataset for sentiment classification task. Five adversarially constructed datasets are evaluated. All the values with larger font are mean across 3 random seeds. The values with smaller font and plus-minus sign (#) are corresponding variance. Numbers in bracket means the number of samples.
Adversarially Constructed Datasets
TF-B
TF-R
Dynasent-RI
Dynasent-R2
AdvGLUE
Method
(694)
(686)
(4800)
(960)
(148)
Vanilla
73.05
45.87
78.19
77.64
56.76
1862
1132
10.38
10:30
11
WordDrop
77.38
60.50
76.34
76.35
56.76
+0.37
R-Drop
75.50
58.16
77.85
77.43
56.08
11.71
$2.08
1005
10005
4:452
HiddenCut
78.15
60.88
76.33
77.01
59.23
1095
1.007
+0.57
10.6
+:177
AdyWeight
73.01
53.26
75.30
76.25
49.55
#0.36
#0.54
+0023
10.47
#2000
AdvEmbed
75.36
61.03
77.23
77.57
58.33
10.6
1.099
10.36
1891
12.3
Freet BB
74.54
57.92
77.94
77.53
62.39
+0.59
#2.48
+0.26
+0.79
#285
SMART
77.95
65.31
75.82
76.22
55.18
1862
L1377
10.33
10.30
11.59
RIFT
77.14
67.35
74.35
77.33
57.21
1041
12.95
10662
1101
11.77
ChildPTune
75.55
54.86
79.57
78.92
58.78
#111
#1.92
+003
10.18
#253
R3F
76.03
58.94
77.04
77.12
56.53
1101
12:00
1:044
10.1
1:4
WConsol
76.70
55.64
80.75
79.96
61.26
+0.58
+0.56
+0.31
+0.50
#2.49
LP-FT
76.32
57.82
81.92
81.04
65.31
10.7
1.0.2
1.037
10.4
1864
ROAST (Ours)
77.09
61.18
79.79
79.27
65.54
+:0.4
#0.56
+041
+0.47
+240

---

Table 17: Expected Calibration Error (ECE) of RoBERTa-large fine-tuned using SST-2 dataset for sentiment classification task. Five adversarially constructed datasets are evaluated. All the values with larger font are mean across 3 random seeds. The values with smaller font and plus-minus sign (#) are corresponding variance. Numbers in bracket means the number of samples. Lower ECE value indicates the better calibration.
Adversarially Constructed Datasets
TF-B
TF-R
Dynasent-RI
Dynasent-R2
AdvGLUE
Method
(694)
(686)
(4800)
(960)
(148)
Vanilla
5.57
19.73
5.53
4.90
13.60
1193
1443
11.41
10.48
15506
WordDrop
8.53
6.97
7.57
4.97
+2.38
R-Drop
5.30
5.40
5.80
10.63
1159
1300
4281
10.50
13.00
HiddenCut
5,90
4.43
2.70
2.87
10.40
2113
10.74
20.96
10.9
16
AdyWeight
5,43
8.30
4.17
3.90
15.63
#0.2
11.28
2062
#127
#4.15
AdvEmbed
3,97
9.10
3.23
4.50
13.17
10.78
11:77
10.12
1107
12.10
Freet BB
6.43
13.70
3.93
5.47
11.77
#0.17
#6.25
#2007
#098
+5.00
SMART
7.53
6.53
3.70
4.37
9.57
1033
1.008
10.16
1090
1380
RIFT
11.43
11.13
2.73
3.17
8.10
10:46
12.11
10034
1024
11.14
ChildPTune
4.13
12.93
2.77
3.73
14.17
#037
#0.57
30019
+0.70
$3.27
R3F
5,07
6.77
2.43
3.53
12.10
11
12.16
1061
10:17
12.14 14
WConsol
6.50
8.10
3.50
3.27
7.33
+1.00
+0.96
+0.36
#0.66
+0.53
LP-FT
5.13
6.00
3.33
3.27
5.80
11.17
1.441
10.12
10034
1843
ROAST (Ours)
6.00
5.13
4.37
4.70
6.30
+:094
#0.00
+081
+0.44
+0.59

---

Table 18: Anomaly detection performance (AUROC) of RoBERTa-large fine-tuned using SST-2 dataset for sentiment classification task. Six anomaly datasets are evaluated. All the values with larger font are mean across random seeds. The values with smaller font and plus-minus sign (+) corresponding variance. Numbers in bracket means the number of samples.
Anomaly Datascts
WMT16
Multi30K (3071)
20News
QQP
MNLI-m
MNI.I-m
Method
(2999)
(592)
(40K)
(9815)
(9832)
Vanilla
83.47
84.87
96.03
87.33
84.17
84.47
#441
#3.65
WordDrop
85.40
88.03
93.90
86.47
86.20
85,40
#1.30
R-Drop
86.07
90.87
96.33
89.97
86.40
85.53
HiddenCut
87.90
92.53
95.40
89.23
87.47
86.90
1:005
10:99
1:120
AdvWeight
84.60
90.77
92.93
88.67
85.07
84.77
+1.31
:1087
2:000
20.94
1:1.30
:: 1.65
AdvEmbed
88.30
91.90
97.13
89.47
89.30
88.63
11080
11.18
1.0.1 12
1:1.88
1.066
1:0.8
Freet B
86.97
89.63
96.37
91.03
87.43
87.50
+:017
+1.55
=2.00
+0.82
#0.44
=00.78
SMART
88.63
94.33
95.80
89.70
88.53
88.33
10.70
110.1
10.22
1.0.2
1.066
RIFT
85.97
91.00
96.83
88.93
88.80
88.03
ChildPTune
84.64
82.03
96.17
88.40
85.23
85.53
+192
#3.84
=0.75
#:1.1
#1.58
#:1.76
R3F
84,90
91.33
95.57
87.80
86.13
85.23
WConsol
87,50
90.27
96.30
90.30
86.90
86.37
10091
LP-FT
86.23
91.57
95,20
90.60
87.10
1100
.10.73
2:0455
2000
11083
ROAST (Ours)
88.60
91.30
95.43
90.87
88.1 17
87.83
+0.47

---

Table 19: Accuracy of RoBeRTa-large fine-tuned using MNLI dataset entailment task. Two in-distribution validation sets (MNLI-m and MNLI-mm) and seven distribution shifted datasets evaluated. All the values with larger font are mean across 3 random seeds. The values with smaller font and plus-minus sign (+) are corresponding variance. Numbers in bracket means the number of samples.
Distribution Shifted Datasets
MNL
MNI.I-mm
Diag (1104)
HANS
QNLI
WNLI
NQ-NLII
FEVER-NLI (20K)
WANLI
Method
(9815)
(9832)
(30K)
(5266)
(635)
(4855)
(5000)
Vanilla
90.15
89.80
66.09
75.70
60.35
53.91
61.94
69.62
62.55
#0.21
#077
WordDrop
90.44
90.26
64.98
76.70
54.84
52.86
61.28
68.82
62.93
+1.63
#044
+0.79
R-Drop
90.61
90.67
65,46
74.82
54.88
51.34
60.56
68.92
63,20
HiddenCut
90.62
90.35
66.76
76.75
54.55
53.75
62.11
69.14
63.79
AdvWeight
90.39
90.00
65.94
74.83
54.91
51.71
61.29
69.19
62.21
+1.56
$0.30
AdvEmbed
90.51
89.97
67.75
77.77
55.77
53.23
61.81
69.29
63.99
+0.03
#0.56
20.8
FreeLB
90.38
90.16
67.39
79.47
55.22
56.30
62.62
69.44
64.12
10.00
#0.13
1:0.00
4022
#0.08
+0005
20112
1:014
SMART
90.78
65.53
77.73
53.95
$1.42
60.87
69.82
63.57
10:05
=0.4
d:0.25
10.5
#0.24
+0012
20.11
4:011
RIFT
89.75
65,90
67.22
$7.85
52.17
60.79
69.01
63.00
10.35
13
10.51
1.083
1:180
21133
1084
1042
1:029
ChildPTune
90.14
90.02
65.94
75.19
$7.83
53.86
62.38
69.46
63.96
+2.45
#0.58
R3F
90.57
90,25
65.55
76.75
56.54
52.55
61.53
69.10
62.75
+1.00
+0.80
WConsol
90.71
90.37
67.09
76.52
61.64
55,91
61.28
70.15
62,72
LP-FT
90.57
90.28
67.60
77.12
56.85
55.59
62.28
69.86
63.63
ROAST (Ours)
90.78
90.50
67.18
75.82
58.26
53.75
60.24
69.03
63.34
:10.08
10211
10.96
4:2.66
::3.44
:0:45
10.9
1061
1988

---

Table 20: Expected Calibration Error (ECE) of RoBERTa-large fine-tuned using MNLI dataset for entailment task. Two in-distribution validation sets (MNLI-m and MNLI-mm and seven distribution shifted datasets are evaluated. All the values with larger font are mean across 3 random seeds. The values with smaller font and plus-minus sign (#) are corresponding variance. Numbers in bracket means the number samples. Lower ECE value indicates the better calibration.
Distribution Shifted Datasets
MNLI-m (9815)
MNLI-mm (9832)
Diag (1104)
HANS (30K)
QNLI
WNLI
NQ-NLI'
FEVER-NLI
WANLI
Method
(5266)
(635)
(4855)
(20K)
(5000)
Vanilla
10.10
11.03
4.73
8.67
27.07
9.17
34,23
4.70
3.03
WordDrop
12.80
12.33
6.97
13.17
24.43
24.70
8.73
5.80
+0.44
+1.8
#0.36
R-Drop
8.73
6.97
24.83
26.30
5.13
4.97
#041
HiddenCut
11.70
6.70
9.17
26.40
30.20
5.67
1.70
11:28
1033
11.32
1UA8
1002
10.7
1100
10299
1008
AdvWeight
11.40
11.33
4.83
8.97
28.03
29.73
6.13
4.40
1192
11.36
11.58
L100
.327
11.10
11.19
AdvEmbed
8.87
4.37
9.13
31.00
10.33
34.13
3.37
4.30
+2.00
#2.61
+:0.0
+:031
2441
2323
#4.71
#1.52
11.4
FreeLB
9.60
8.70
4.40
10.35
27.80
7.75
32.75
5.45
3.45
+1.550
$0.50
1:000
1:465
27.40
11:45
$3.05
2065
#025
SMART
10.30
10.20
7.05
5.05
25.55
7.00
26.65
5.00
6.30
#022
::.15
#:0.0
#0.25
#5035
#322
4HL15
#1125
49.55
RIFT
12.50
12.48
4.20
12.20
29.83
7.48
30.93
5.53
2.00
10.66
#::99
#:025
$1.59
#3.11
#1.72
#239
$0.00
+:104
ChildPTune
8.03
7.87
4.83
8.70
29,40
10.37
35.13
5.67
4,30
1207
R3F
9.93
10.73
4.90
8.43
23.33
7.80
29.00
5.03
3.33
+1.35
+:10.62
#1.35
+0.5
WConsol
8.77
8.57
7.23
28.27
5,00
3.20
+2.19
+1.28
LP-FT
11.03
5.40
11.13
26.57
6.90
32.67
4.03
2.57
10.70
10.71
11.63
10077
13.35
1083
1166
11.20
1061
ROAST (Ours)
7.40
7.43
6.47
10.30
22.00
7.27
26.83
5.73
3.97
10.39
10.56
10.49
42.13
::..84
10074
1:.70
1060

---

Table 21: Accuracy of RoBERTa-large fine-tuned using MNLI dataset for entailment task. Nine adversarially constructed datasets evaluated. All the values with larger font across random seeds. The values with smaller font and plus-minus sign (+) are corresponding variance. Numbers in bracket means the number of samples.
Adversarially Constructed Datasets
TF-B-m (772)
TF-B-mm (746)
TF-R-m
TF-R-mm
ANLI-RI
ANII-R2
ANLI-R3
AdvGLUE-m
AdvGI-UF-ai
Method
(775)
(775)
(1000)
(1000)
(1200)
(121)
(162)
Vanilla
71.11
68.36
52.52
42
28.70
27.56
55.37
40.95
1042
21.76
1370
21:00
13.74
20:1
20:11
13.99
22.3
WordDrop
74.61
73.28
58.06
61.63
42.77
30.33
26.61
62.81
41.98
13144
20177
10.0
10.74
21
20036
1:124
2:30
R- R-Drop
73.36
74.04
59.57
63.48
42.47
29.37
27.39
56.75
34.77
HiddenCut
73.06
70.46
55.74
57.98
43.07
29.67
26.86
58.13
38.89
10,85
Weight
53.59
42.30
52.62
#1.95
$1.06
$2.48
#2.26
+0.71
20.2
1:119
40.39
$1.54
AdvEmbed
71.76
71.13
54.02
57.03
44.43
29.60
28.14
59.23
36.42
19.56
20.66
63.25
$0.66
19.50
2:039
2:241
1:431
FreeLB
70.27
69.37
52.06
50,19
45.45
27.70
28.00
61.57
40.74
+1.10
1122
#2.00
#3.10
+:315
10.40
2:085
+:441
2:123
SMART
73.32
74.80
58.97
64.71
43.85
31,20
28.46
57.44
37.65
+0.75
RIFT
77.08
75.74
65.13
66.13
43.45
28.15
26.83
59.50
39.97
ChildPTune
67.75
66.67
47.01
48.17
44,10
25.87
26.22
54.27
38.27
+1.70
$1.97
#2.11
$4.30
+:102
20.0
2:099
1:.56
24.50
R3F
72.93
71.49
56.64
$7.98
41.80
27.97
26.67
56.20
40.95
2011
1::99
1:1.5
21.55
2:1055
1:294
210.8
WConsol
72.06
71.36
51.87
50,97
46.90
26.20
24.86
56.47
40.33
I.P-TT
70.90
70.24
49.76
51.23
45.90
27.07
23.69
$7.58
43.00
+3..0
2:079
+1.40
20.77
ROAST (Ours)
75.60
72,74
56,30
56.60
45.23
28.0
25.92
58.95
42,59
+1.44
2084
+1.30
#226
+1.00
20.14
#042
#077
#2002

---

Table 22: Expected Calibration Eiror (EECE) of RoBERTa-large fine-tuned using MNLI dataset for entailment task. Nine adversarially constructed datasets are evaluated. the values with larger font are mean across random seeds. The values with smaller font and plus-minus sign (+) are corresponding variance. Numbers in bracket means the number of samples. Lower ECE value indicates the better calibration.
Adversarially Constructed Datascts
TF-B-m (772)
TF-B-mm (746)
TE-R-m (775)
TF-R-mm
ANLI-R1
ANLI-R2
ANLI-R3
AdvGLUE-m (121)
AdvGLUUE-m
Method
(775)
(1000)
(1000)
(1200)
(162)
Vanilla
26.17
16.00
+100
$1.58
40.00
20.17
+4.20
#231
$2.90
#2.77
#1.8
WordDrop
13.43
12.83
8.17
6.13
17.47
20.77
12.57
7.70
+0.70
20.56
$0.39
20.8
#0.1
#1.55
1:44
11.50
#100
R-Drop
11.40
12.50
6.73
7.67
8,90
19.40
20.37
9.73
13.30
1335
21186
11.15
11.16
21
2:024
#2.11
2000
HiddenCut
7.93
6.23
3.17
4.83
10.53
23.17
24.90
8.20
12.43
Adv Weight
8.97
9.30
6.80
5.67
10.63
23.03
22.03
8.83
14.90
+2.43
#245
AdvEmbed
5.20
25.50
26.47
7.37
18.40
43.44
22.32
$1.55
11.28
13.63
23.9
4:408
11.22
$1.28
FreeLB
6.70
6.55
8.15
26.30
26.65
9.55
13.30
1::000
2135
11.00
25.55
30095
$2.95
21.00
SMART
12.15
13.35
8.25
10.40
5,35
16.50
17.55
10.35
8.45
40.5
#1.66
+0.5
60.70
+0.15
$0.50
20.15
#0.5
#:115
RIFT
11.40
9.95
8,68
8.05
11.15
26.68
26.43
9.53
14.13
41886
40.78
ChildPTune
28.00
41.87
#1.9
+6.32
26.37
+7.38
2881
#9.67
+1.03
$7.53
R3F
8.27
8.20
4.77
5.27
9.47
22.80
23.47
10.03
10.17
4092
#2.30
+1.35
11.13
+3.27
#2.1
14.00
+1001
2:300
WConsol
10.63
11.10
6,27
4.83
5,43
22.27
22.73
10.63
8.43
#2.255
+3.77
6:.26
LP-FT
7.17
6.53
6.57
6.60
9.00
27.30
29.43
11.10
12.63
RoAST (Ours)
11.47
7.17
6.10
20.43
21.63
10.23
7.47
11.26
2132
$1.11
11.35
10.2
21.00
1:18
1119
200

---

Table 23: Anomaly detection performance (AUROC) of RoBERTa-large fine-tuned using MNLI dataset for entailment task. Five anomaly datasets are evaluated. All the values with larger font are mean across 3 random seeds. The values with smaller font and plus-minus sign (#) are corresponding variance. Numbers in bracket means the number of samples.
Anomaly Datasets
WMT16
Multi3KK
SST-2
20News
QQP
Method
(2999)
(3071)
(872)
(592)
(40K)
Vanilla
94.93
98.23
97.50
85.67
84.10
#1.06
#3.41
=411
WordDrop
92.00
99.47
97.97
74.97
75.43
11.21
1.0.1
10.62
14.39
0.88
R-Drop
96.03
98.50
98.30
84.67
81.97
#1.06
#1.2
#0.57
#1.55
=1.99
HiddenCut
95.57
97.87
97.50
85.23
82.03
#1.28
#1.68
#1.04
#2.17
=1.17
AdvWeight
96.70
99.43
98.70
78.37
81.87
1.092
1:0.38
10.45
::5000
0.82
AdvEmbed
96.93
99.00
98.23
88.60
86.40
#0.53
+0.49
#0.09
#2.50
=1.04
FreeI.B
98.15
99.35
98.50
84.45
85.75
4:2.35
22.6
SMART
94.55
99.30
96.10
87.20
79.95
RIFT
93.18
98.35
97.28
91.80
82.00
#3.08
1:135
+0.99
1:55
1.40
ChildPTune
82.87
95.90
87.43
83.57
83.27
1449
1:4.1
+7.23
#:151
=381
R3F
93.63
99.07
95.73
87.20
83.40
#2.10
1:038
+2.88
1:1.37
=2.38
WCCnsool
90.67
98.80
94.40
90.17
84.43
+:029
+0.47
1.88
LP-FT
93.33
98.63
94.40
91.03
90.73
20.08
#1.65
22.08
RoAST (Ours)
94.90
98.37
96.67
90.83
85.47
#1.14
1:061
11.17
#1.88
-0.62


Reviews:
[{"review_id": "ZI1VwRlzcf", "reviewer": "Reviewer_o3bN", "summary": "", "questions": "- Could the authors discuss the choice of embedding-level perturbation? Would the method also work for discrete token-level adversarial perturbations?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your efforts and insightful comments to improve the manuscript. We respond to each of your comments one-by-one in what follows. In tables, all the values are mean across 3 random seeds.\n\n---\n\n#### **[R]. The main weakness of the paper is the limited experimentation in the tasks/datasets side. Even though the baselines and models used are thorough, admittedly the authors could have used more datasets from each tasks and more challenging benchmarks.**\n**[A].** We appreciate your recognition of the thoroughness of our baselines and models and thank you for the great suggestion! We agree that incorporating more datasets and challenging benchmarks would further strengthen our experiments. \n\nHowever, we would like to note that a significant number of diverse datasets were already included in our experiments to evaluate multi-perspective robustness on each task. For example, we used 16 and 22 datasets for sentiment classification and entailment tasks, respectively. Moreover, constructing a benchmark to evaluate multi-perspective robustness for a new task is not a trivial task, as it involves identifying and integrating separately explored datasets for each perspective. Nevertheless, we believe the additional results on more challenging tasks would greatly strengthen our paper. We will continue to work and supplement more results in the final draft.\n\nOverall, we believe that our experimentation serves as a useful contribution and a foundation for pursuing a unified approach to evaluate and enhance the multi-perspective robustness of LMs. We also hope that our benchmarks will continue to be extended with new datasets and tasks by the community, and serve as a meaningful step towards the more reliable deployment of LMs.\n\n---\n\n#### **[Q]. Could the authors discuss the choice of embedding-level perturbation? Would the method also work for discrete token-level adversarial perturbations?**\n\n**[A].** We chose embedding-level perturbation over token-level one, as it is more computationally efficient and better suited for enhancing multi-perspective robustness. \n\n- Construction of token-level adversarial perturbations requires more computation to solve the discrete optimization problem, and additional regularization is often introduced to prevent degenerate cases such as significant changes in semantic or lexical violation [1,2,3]. For example, a relatively simple construction of token-level perturbation with [3] requires 15% more times per iteration, compared to embedding-level perturbation.  \n- While the token-level adversarial perturbation is effective for adversarial robustness, it often comes at the cost of a decrease in clean accuracy [4,5] due to the relatively large perturbation on a discrete space. In contrast, the embedding-level perturbation can be constructed by adding small noise to continuous space, making it more feasible to train the model without the loss of accuracy [6,7]\n\nMoreover, to address your comments, we conduct additional experiments to validate the effectiveness of $\\text{RoAST}$ with discrete token-level adversarial perturbations by fine-tuning RoBERTa-large using $\\text{VAT-D}$ [3]. We used the same hyper-parameters for discrete token-level perturbation as in [3], and for $\\text{RoAST}$, we used the same values previously found with embedding-level perturbation. The results are shown in the table below. \n\n| Method  | Acc$_{\\tt in}$ | Acc$_{\\tt shift}$ | Acc$_{\\tt adv}$ | ECE ($\\downarrow$) | AUROC | $\\Delta_{\\tt avg}$ | \n|:--------|:------------:|:----:|:------------:|:------------:|:----:|:------------:|\n| $\\text{Vanilla}$ | $\\textbf{96.29}$ | $\\textbf{91.79}$ | $66.30$ | $7.11$ | $86.72$ | $0.00$ | \n| $\\text{VAT-D}$ | $95.83$ | $\\underline{90.86}$ | $\\underline{70.37}$ | $\\textbf{6.33}$ | $\\underline{90.39}$ | $\\underline{5.37}$ |  \n| $\\text{VAT-D + RoAST}$ | $\\underline{96.27}$ | $90.45$ | $\\textbf{72.12}$ | $\\underline{6.94}$ | $\\textbf{92.16}$ | $\\textbf{8.73}$ |  \n\nHere, we observe that the discrete token-level adversarial perturbation can improve the multi-perspective robustness, especially for adversarial robustness ($\\text{Acc}\\_{\\tt adv}$). However, it comes at the cost of a degradation in clean accuracy ($\\text{Acc}\\_{\\tt in}$). With $\\text{RoAST}$, such degradation can be mitigated and the overall robustness of the model could be improved. We will include the respective discussions and results in the final draft.\n\n---\n\nPlease let us know if there are any more comments; we would be more than happy to continue the discussion! \n\nSincerely,  \nAuthors.\n\n[1] Jin et al., Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment., AAAI 2020  \n[2] Li et al., BERT-ATTACK: Adversarial Attack Against BERT Using BERT., EMNLP 2020  \n[3] Park et al., Consistency Training with Virtual Adversarial Discrete Perturbation., NAACL 2022  \n[4] Jia et al., Certified Robustness to Adversarial Word Substitutions., EMNLP 2019  \n[5] Dong et al., Towards Robustness Against Natural Language Word Substitutions., ICLR 2021  \n[6] Zhu et al., FreeLB: Enhanced Adversarial Training for Natural Language Understanding., ICLR 2020  \n[7] Jiang et al., SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization., ACL 2020", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well written, well motivated and easy to follow.\n- The proposed approach, RoAST, is novel given that is adds a multi-perspective robustness of LMs in a unified way.\n- The experimental setup is solid, with multiple baselines and thorough analysis (and ablation), making the results and conclusions strong.", "reasons_to_reject": "- The main weakness of the paper is the limited experimentation in the tasks/datasets side. Even though the baselines and models used are thorough, admittedly the authors could have used more datasets from each tasks and more challenging benchmarks.", "questions_to_authors": "- Could the authors discuss the choice of embedding-level perturbation? Would the method also work for discrete token-level adversarial perturbations?\n", "paper_topic_and_main_contributions": "The paper is addressing the problem of lack of robustness in language models. Even though it is a well studied problem, the authors present limitations of previous work and motivate their work as an exploration of a single effective way for fine-tuning LMs to improve its robustness in multiple perspectives. They propose a fine-tuning method to address this problem, namely Robustifying LMs with Adversarial perturbation with Selective Training (ROAST). The method is based on the idea of combining robustness on the perturbed input and generalizable knowledge learned during pre-training of LMs, during the fine-tuning stage. The authors show the superiority of their approach compared to various baselines in sentiment analysis and entailment tasks.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "15eQAP94wL", "reviewer": "Reviewer_C6Wu", "summary": "", "questions": "What does the ‘coined’ mean in line 70? \n\nWhy is the matric of the experiment use a relative average form? Except for ECE, what is the advantage of it compared to a directly average form? \n\nWhy does the score of accuracy and AUROC in the same form in every table but in line 368-369 it says the best score of each metric: 100 for accuracy and 1 for AUROC?\n\nExperiments using different LMs aim to demonstrate the effectiveness of RoAST, rather than compare different LMs. Why using the vanilla fine-tuned BERT-large as the baseline to calculate the average relative improvement in the experiment of other LMs rather than use the corresponding LMs score? Does this change of the matric make no difference? \n", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your efforts and insightful comments to improve the manuscript. We respond to each of your comments one-by-one in what follows. In tables, all the values are mean across 3 random seeds.\n\n---\n\n#### **[R1-1]. This paper only incorporates one kind of perturbed inputs techniques and one kind of preserving generalizable knowledge techniques, and thus far from a general framework to incorporate two kinds of techniques.**  \n**[A].** Thank you for pointing it out. We first clarify that our goal is NOT to create a general framework to incorporate two kinds of techniques. Rather, our goal is to establish a highly effective single framework for enhancing the multi-perspective robustness of LMs simultaneously. We believe that this is an important direction in practice, as it can mitigate the complexities and inefficiencies associated with exploring numerous combinations.\n\nRegarding this, we remark that our approach consistently surpasses existing methods that attempt to merge two kinds of techniques, as highlighted in Section 4.3. Through comparisons with prior works [1,2] and other advanced technique combinations aiming at the same intuition of incorporating, our results demonstrate the effectiveness of our method as a single framework to incorporate two kinds of techniques.  \n\n---\n \n#### **[R1-2]. The paper formalizes and derives that under mild assumptions, the masked gradient serves as an unbiased estimate of the original gradient, and the norm of covariance is upper bounded. But the derivation seems to be not very clear.**   \n**[A].** Thank you for the insightful questions.   \n- ***Why $g^(i)$ is assumed to be in the form presented in line 1423?***: Indeed, this form is based on the Central Limit Theorem (CLT); CLT suggests that, under mild assumptions, the distribution of the sum (or average) of a large number of independent, identically distributed variables will be approximately normal, regardless of the underlying distribution. In the context of our work, this would imply that the gradients, which are calculated in the mini-batch of the samples, would approximate a normal distribution. We additionally remark that this is a common simplifying assumption in the literature [3,4,5]. \n\n- ***How to derive from line 1428 to line 1429?***: Line 1429 ($\\mathbb{E}[\\widetilde{g}] = \\mathbb{E}[g]$) is derived from line 1428 ($\\widetilde{g} := \\big(m(\\theta)/p(\\theta)\\big) \\odot g$) as follow. We will add these details in the final draft:  \n$\\mathbb{E}[\\widetilde{g}]=\\mathbb{E}[( m(\\theta) / p(\\theta) ) \\odot g]=\\mathbb{E}[ m(\\theta) \\odot g]/p(\\theta)=(\\mathbb{E}[m(\\theta)] \\odot \\mathbb{E}[g])/p(\\theta)$ $  = (p(\\theta) \\times \\frac{\\partial \\mathcal{L}}{\\partial \\theta}) / p(\\theta)  = \\frac{\\partial \\mathcal{L}}{\\partial \\theta}) = \\mathbb{E}[g]$  \n\n- ***Meanwhile, it should be explained what implications this corollary has, such as whether it indicates that gradient masking does not affect the convergence of the model, etc.***: The corollary establishes that, under certain conditions, the masked gradient $\\widetilde{g}(\\theta)$ is an unbiased estimator of the original gradient $g(\\theta)$. The implication of this result is that the masked gradient correctly identifies the true gradient on average; therefore, the variance introduced by masking doesn't introduce a systematic bias that could deviate the model from converging to its optimal parameters.  \n\n  In addition, the upper bound on the norm of covariance provides confidence in the stability of this estimator. In practical terms, it assures us that the masked gradient won't be too volatile or far off from the true gradient, thus safeguarding the convergence properties of the optimization process.  \n\n  To clarify the implications of the corollary, we will incorporate this discussion in the final manuscript.  \n \n---\n\n#### **[R2]. The selective training method this paper propose is simple and intuitive, and distinct methods for each perspective of robustness are available. The overall paper lacks novelty.**  \n**[A].** We would like to emphasize that this work is the first to explore the problem of multi-perspective robustness of LMs, with a novel approach for addressing this problem.\n- For reliable deployment to real-world applications, LMs must be robust across multiple dimensions simultaneously, rather than just a specific one. Despite this practical importance, the direction of multi-perspective robustness has yet to be explored. While each perspective of robustness for LMs has been separately explored with distinct methods (as we have also pointed out in the introduction), enhancing the multi-perspective robustness of LMs is still non-trivial; naively combining the methods would largely increase the computational cost and hyper-parameters and could even conflict with each other.\n- In addition to exploring a new problem, we propose a new training method, $\\text{RoAST}$, which is simple and intuitive, as highlighted by you. As shown in the experimental results, $\\text{RoAST}$ outperforms various approaches to enhance the robustness of LMs (including their combination) and is compatible with various types of LMs. Therefore, we expect that $\\text{RoAST}$ can serve as a unified way to effectively enhance the multi-perspective robustness of LMs.  \n\nOverall, we believe that both the introduction of a new problem of multi-perspective robustness of LMs and the design of a novel training method for it are valuable contributions to the community, as highlighted by Reviewer o3bN. We’ll make the contribution more clear during revision.\n\n---\n\n#### **[R3-1]. The experimental results show that on entailment task, this paper doesn’t achieve best score on any single matric. Moreover, considering the ‘relative’ part in the average relative improvement $\\Delta_{\\tt avg}$, this score is scaling up (for example, nearly ten times of $\\text{Acc}_{\\tt in}$) compared to the original score and thus 2 points improvements compared to the best score of state-of-the-art finetuning method is not satisfiable.**\n**[A.]** Firstly, we would like to clarify that the primary goal of this work is to improve the multi-perspective robustness of LMs, NOT to improve a single metric of a specific perspective. Our approach did not achieve the best score on any single metric in the case of the entailment task, but it is more important to recognize that our method demonstrates the best results when considering multi-perspective robustness. Additionally, we remark that our method even achieves the best scores for three different metrics in the case of the sentiment classification task.  \n\nMoreover, we use the average of relative improvement as it is more appropriate to measure the multi-perspective robustness than the average of absolute improvement, not to scale up the values (please see the detailed answer for this, in the response for Q2 below). However, we also recognize its weak points, such as the risk of amplification, which is the reason why we additionally report $\\text{Rank}_{\\tt avg}$ (as it does not have similar issues). We emphasize that $\\text{RoAST}$ exhibits the lowest rank among the state-of-the-art fine-tuning methods on both sentiment classification and entailment tasks.  \n\nNevertheless, to address your concerns about this, we additionally calculate the absolute average improvement ($\\text{Abs}\\_{\\tt avg}$) of {$\\text{Acc}\\_{\\tt in}, \\text{Acc}\\_{\\tt shift}, \\text{Acc}\\_{\\tt adv}$, $100 - \\text{ECE}$, $100 * \\text{AUROC}$}, on the sentiment classification task. The results are presented in the below table.\n\n| Method  | Vanilla | WordDrop | R-Drop | HiddenCut | AdvWeight | AdvEmbed | FreeLB | SMART | RIFT | ChildPTune | R3F | WCons | LP-FT | RoAST (Ours) |  \n|:--------|:------------:|:----:|:------------:|:------------:|:----:|:------------:|:------------:|:------------:|:----:|:------------:|:------------:|:----:|:------------:|:------------:|\n| $\\text{Abs}_{\\tt avg}$  | $76.47$ | $76.36$ | $76.96$ | $76.91$ | $76.94$ | $77.20$ | $77.13$ | $\\underline{77.26}$ | $76.95$ | $74.72$ | $76.93$ | $77.05$ | $76.99$ | $\\textbf{77.63}$ | \n\nHere, one can observe that our method still outperforms the state-of-the-art fine-tuning method with a large gap; $\\text{RoAST}$ exhibits 46.72% relative improvement on $\\text{Abs}_{\\tt avg}$, compared to $\\text{SMART}$ (1.16% vs 0.79%). This result further demonstrates the effectiveness of our method, and we do believe that our empirical results clearly show the merit of the proposed framework (as highlighted by reviewers Gq2X and o3bN). We will add this result and related discussion in the revised paper.\n\n---\n\n#### **[R3-2]. Although this paper includes numerous experiments, the analysis does not seem to match the workload. For instance, Table 2 does not show a significant improvement of ROAST compared to the baselines in the entailment task, but the authors have not analyzed the reason. Similarly, the analyses in the ablation study only describe the results in the tables, without providing sufficient reasoning or evidence for the effectiveness or shortcomings of ROAST.**\n\n**[A].** Thank you for the suggestion. We would like to first note that we presented various analyses of $\\text{RoAST}$ by focusing on designing appropriate experiments, such as measuring the distance to pre-trained LM or the importance of selection strategies, in Section 4.3. However, we agree that there is room for improvement by providing additional intuitions of the results. In the revision, we will incorporate more analysis of the results, especially for the following aspects as suggested.\n\n***Reasoning for different effectiveness between tasks.*** We clarify that the different effectiveness for each task is commonly observed, not only for $\\text{RoAST}$. Overall, advanced fine-tuning methods are more effective in the sentiment classification task than in the entailment task. One possible reason is the difference in the intrinsic difficulty of the task and training dataset size. While the size of the training data is much smaller for SST-2 (67k vs MNLI: 393k), the accuracy is much higher in the sentiment classification task. This indicates that the model could be more vulnerable to overfitting, and hence regularization of training could be more effective.\n   \n***More analyses in the ablation study.*** In addition to the presented ones (lines 496-514), we provide the following additional reasonings from the results of the ablation study in Table 4. \n- *Implications of performance degradation with $p(\\theta)$ (c -> d)*: (c) Hard thresholding for only updating the important parameters with $s(\\theta)$ can strictly preserve the original parameters of the pre-trained LM, which is less important to the given task, while (d) using $p(\\theta)$ updates all the model parameters with the re-scaled gradients. Therefore, the performance decrease with $p(\\theta)$ compared to $s(\\theta)$ indicates the advantages of imposing sparsity rather than re-scaling to update the model for a given task while preserving the generalizable knowledge of the pre-trained LM. \n- *Implications of performance improvement with $m(\\theta)$ (c,d -> $\\text{RoAST}$)*: While (c) the selective training with hard thresholding showed the improvement, its effectiveness can be limited as it distorts the model update from the original direction through deterministic updates of important parameters. Compared to this, $\\text{RoAST}$ can continuously take advantage of selective training while reducing such distortion by sampling and re-scaling the mask $m(\\theta)$ from $p(\\theta)$.  This advantage of $\\text{RoAST}$ is verified by the improvements over (c,d).\n\n---\n\n#### **[R3-3]. Only BERT-based models are considered in the paper. Decoder-only structure is more popular with the emergence of LLMs. It is promising to study the robustness on them.**\n**[A].** We agree that our approach has the potential to be applicable beyond BERT-based models, and demonstrating its effectiveness in broader types of models would strengthen our paper. In response to your suggestion, we conducted additional experiments on the sentiment classification task with GPT2-large, a popular decoder-only LM, to validate the applicability of our approach. Following [6], we added a linear classifier head on the last token’s embedding output for fine-tuning. The results are presented in the table below. \n\n| Method  | Acc$_{\\tt in}$ | Acc$_{\\tt shift}$ | Acc$_{\\tt adv}$ | ECE ($\\downarrow$) | AUROC | $\\Delta_{\\tt avg}$ | \n|:--------|:------------:|:----:|:------------:|:------------:|:----:|:------------:|\n| Vanilla | $\\underline{94.99}$ | $\\textbf{84.33}$ | $62.38$ | $\\underline{8.01}$ | $95.30$ | $0.00$ | \n| AdvEmbed | $94.92$ | $\\underline{84.19}$ | $\\underline{63.58}$ | $\\textbf{7.90}$ | $\\underline{95.81}$ | $\\underline{2.60}$ | \n| WCons | $94.88$ | $80.72$ | $61.80$ | $9.06$ | $93.57$ | $-15.37$ |\n| RoAST (Ours) | $\\textbf{95.03}$ | $81.91$ | $\\textbf{64.96}$ | $8.52$ | $\\textbf{97.16}$ | $\\textbf{5.07}$ |\n\nHere, we first observe that the improvements with baseline methods are largely limited. We speculate that this ineffectiveness may occur due to the different nature of decoder-only LMs compared to encoder-only ones, such as BERT, as it could result in different effectiveness of the baseline algorithms and tuned hyper-parameters, which were originally developed and demonstrated only using encoder-only models; for instance, most of the baselines [2,5,7,8] have been demonstrated under encoder-only models and not shown the results of decoder-only ones.\n\nNevertheless, our $\\text{RoAST}$ approach continues to enhance the multi-perspective robustness of GPT2-large, with an average relative improvement of 5.07% compared to the Vanilla method. These results indicate that the effectiveness of our approach is not limited to BERT-based LMs. Hence, we hope that our work could contribute to broader applications in the future. We will add more results and corresponding discussions to the final draft. Thank you for your suggestion to strengthen our paper.\n\n---\n\n#### **[Q1] What does the ‘coined’ mean in line 70?**  \n**[A].** The term ‘coined’ in line 70 refers to the act of introducing a new term or phrase for something, especially in an academic context. In our paper, the term \"Robustifying LMs with Adversarial perturbation with Selective Training\" is introduced as the full name of our proposed technique, which we then abbreviate as \"$\\text{RoAST}$\" for ease of reference throughout the paper.  \n\n---\n\n#### **[Q2] Why is the matric of the experiment use a relative average form? Except for ECE, what is the advantage of it compared to a directly average form?**\n**[A].** We used a relative average instead of a direct average for measuring multi-perspective robustness, as it is more appropriate for the problem at hand. To evaluate LMs' multi-perspective robustness, it is crucial to equally incorporate multiple metrics from various perspectives. However, these metrics often have different scales, and the direct average is limited in preventing a single metric from dominating the aggregate results due to its numerical magnitude. For instance, the average absolute improvement in $\\text{Acc}\\_{\\tt adv}$ is much larger than in $\\text{Acc}\\_{\\tt in}$ due to the former's relatively small values. In contrast, the relative average addresses this issue by considering normalized metrics. We'll make sure to explain this more clearly in our final manuscript.\n\n---\n\n#### **[Q3] Why does the score of accuracy and AUROC in the same form in every table but in line 368-369 it says the best score of each metric: 100 for accuracy and 1 for AUROC?** \n**[A].** Many thanks for the careful reading, particularly in highlighting the inconsistency between the tables and the text in lines 368-369. In our tables, we have presented the value as 100 × $\\text{AUROC}$ instead of $\\text{AUROC}$, for a better presentation by aligning its scale (including significant figures) with other metrics such as accuracy and $\\text{ECE}$. However, we remark that the metrics for comparison ($\\Delta_{\\tt avg}$ and $\\text{Rank}_{\\tt avg}$) remain unchanged and are unaffected by this re-scaling. To enhance clarity and prevent any confusion, we will integrate the following statement into the experimental setup in the final draft. Thank you again for pointing out this!:  \n“We present the AUROC values as 100 × AUROC in our tables to maintain a consistent scale with other metrics.”  \n\n---\n\n#### **[Q4] Experiments using different LMs aim to demonstrate the effectiveness of RoAST, rather than compare different LMs. Why using the vanilla fine-tuned BERT-large as the baseline to calculate the average relative improvement in the experiment of other LMs rather than use the corresponding LMs score? Does this change of the matric make no difference?** \n**[A].** Thank you for the insightful comments. We use BERT-large as a universal baseline to calculate relative improvement across different LMs to facilitate comparison in terms of multi-perspective robustness. This choice was made to provide additional insight into the question of *“Which LM is the most robust”*. While answering this question is important, we also acknowledge that using the corresponding LM's score as the baseline could provide additional insights into how $\\text{RoAST}$ performs with each specific LM.  \nHence, to address your concerns about the potential impact of this choice on the metrics, we recalculate Table 3 with the corresponding LM's score and present it below. One can observe that $\\text{RoAST}$ mostly outperforms the baselines except in only 1 case, and achieves large improvements compared to baselines in both tasks. This result clearly demonstrates the robustness and effectiveness of $\\text{RoAST}$ with respect to different LMs. We will add the corresponding discussions and results in the final draft.\n\n| Sentiment Classification  | Vanilla | AdvEmbed | WCons | RoAST | \n|:--------|:------------:|:----:|:------------:|:------------:|\n| BERT-large | $0.00$ | $\\underline{18.24}$ | $15.85$ | $\\textbf{22.76}$ | \n| RoBERTa-large | $0.00$ | $13.70$ | $\\underline{15.47}$ | $\\textbf{18.39}$ | \n| ALBERT-xxlarge | $0.00$ | $\\underline{21.34}$ | $18.25$ | $\\textbf{30.62}$ | \n| XLNet-large | $0.00$ | $\\underline{1.44}$ | $-1.76$ | $\\textbf{12.74}$ | \n| ELECTRA-large | $0.00$ | $\\underline{6.44}$ | $-2.83$ | $\\textbf{8.29}$ | \n| DeBERTa-large | $0.00$ | $\\underline{10.53}$ | $4.49$ | $\\textbf{20.62}$ | \n| Average | $0.00$ | $\\underline{11.95}$ | $8.25$ | $\\textbf{18.90}$ | \n\n| Entailment  | Vanilla | AdvEmbed | WCons | RoAST | \n|:--------|:------------:|:----:|:------------:|:------------:|\n| BERT-large | $0.00$ | $\\underline{22.85}$ | $20.23$ | $\\textbf{25.34}$ | \n| RoBERTa-large | $0.00$ | $\\underline{5.72}$ | $2.99$ | $\\textbf{7.63}$ | \n| ALBERT-xxlarge | $0.00$ | $\\underline{2.60}$ | $-3.83$ | $\\textbf{8.19}$ | \n| XLNet-large | $0.00$ | $2.44$ | $\\underline{2.75}$ | $\\textbf{3.30}$ | \n| ELECTRA-large | $0.00$ | $\\textbf{7.89}$ | $-0.70$ | $\\underline{4.39}$ | \n| DeBERTa-large | $0.00$ | $2.81$ | $\\underline{3.79}$ | $\\textbf{11.93}$ | \n| Average | $0.00$ | $\\underline{7.39}$ | $4.21$ | $\\textbf{10.13}$ | \n\n---\n\n#### **[Missing References] Some related works are missing.**\n**[A].** Thank you for letting us know about the additional related works! As the line of work for the adversarial robustness, we agree that their inclusion will significantly enhance the comprehensiveness and depth of our related works section. We will include them in the related work section of our final manuscript.  \n\n---  \nWe hope this response will clarify your concerns. Please let us know if there are any unclear points remaining. \n\nSincerely,  \nAuthors.\n\n[1] Dong et al., How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?., NeurIPS 2021  \n[2] Jiang et al., SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization., ACL 2020  \n[3] Smith and Le., A Bayesian Perspective on Generalization and Stochastic Gradient Descent., ICLR 2018  \n[4] Zhang et al., Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model., NeurIPS 2019  \n[5] Xu et al., Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning., EMNLP 2021  \n[6] Radford et al., Improving Language Understanding by Generative Pre-Training., OpenAI  \n[7] Zhu et al., FreeLB: Enhanced Adversarial Training for Natural Language Understanding., ICLR 2020  \n[8] Chen et al., Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting., EMNLP 2020", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The collaborative framework pursuing unified robustness has been less explored and this paper incorporates two important sources: robustness on the perturbed inputs and generalizable knowledge in pre-trained LMs and thus improve the robustness of LMs in multiple perspective.\n\n2. This paper conducts extensive experiments and have best weighted average performance across the four robustness aspects along with a standard validation accuracy compared to state-of-the-art finetuning methods.\n", "reasons_to_reject": "General weakness:\n\n1. This paper only incorporates one kind of perturbed inputs techniques and one kind of preserving generalizable knowledge techniques, and thus far from a general framework to incorporate two kinds of techniques. The paper formalizes and derives that under mild assumptions, the masked gradient serves as an unbiased estimate of the original gradient, and the norm of covariance is upper bounded. But the derivation seems to be not very clear. Why is $g^{(i)}$ assumed to be in the form presented in line 1423? How to derive from line 1428 to line 1429? Meanwhile, it should be explained what implications this corollary has, such as whether it indicates that gradient masking does not affect the convergence of the model, etc.\n\n2. The selective training method this paper propose is simple and intuitive, and distinct methods for each perspective of robustness are available. The overall paper lacks novelty.\n\n3. The experimental results show that on entailment task, this paper doesn’t achieve best score on any single matric. Moreover, considering the ‘relative’ part in the average relative improvement $\\Delta_{avg}$, this score is scaling up (for example, nearly ten times of $Acc_{in}$) compared to the original score and thus 2 points improvements compared to the best score of state-of-the-art finetuning method is not satisfiable. Although this paper includes numerous experiments, the analysis does not seem to match the workload. For instance, Table 2 does not show a significant improvement of ROAST compared to the baselines in the entailment task, but the authors have not analyzed the reason. Similarly, the analyses in the ablation study only describe the results in the tables, without providing sufficient reasoning or evidence for the effectiveness or shortcomings of ROAST.\nOnly BERT-based models are considered in the paper.\nDecoder-only structure is more popular with the emergence of LLMs. It is promising to study the robustness on them.\n\n", "questions_to_authors": "What does the ‘coined’ mean in line 70? \n\nWhy is the matric of the experiment use a relative average form? Except for ECE, what is the advantage of it compared to a directly average form? \n\nWhy does the score of accuracy and AUROC in the same form in every table but in line 368-369 it says the best score of each metric: 100 for accuracy and 1 for AUROC?\n\nExperiments using different LMs aim to demonstrate the effectiveness of RoAST, rather than compare different LMs. Why using the vanilla fine-tuned BERT-large as the baseline to calculate the average relative improvement in the experiment of other LMs rather than use the corresponding LMs score? Does this change of the matric make no difference? \n", "paper_topic_and_main_contributions": "This paper proposes a simple fine-tuning technique Robustifying LMs via Adversarial perturbation with Selective Training (RoAST) that aims to improve the multi-perspective robustness of LMs. To be specific, RoAST introduce adversarial embedding-level continuous perturbation during finetuning while the model parameters are selectively updated upon the square of their gradients. They conduct experiments on six different types of LMs on sentiment classification and entailment tasks. ", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Some related works are missing. List a few:\n\nCreAT\nWu et al. Toward adversarial training on contextualized language representation\n\nTRADES\nZhang et al. Theoretically principled trade-off between robustness and accuracy\n\nAWP\nWu et al. Adversarial weight perturbation helps robust generalization\n"}}, {"review_id": "cEZYvpQ9f4", "reviewer": "Reviewer_Gq2X", "summary": "", "questions": "- Can you show some results of FreeLB++, which runs adversarial perturbation for 10 steps instead of one? Li et al, EMNLP 2021, Searching for an effective defender: Benchmarking defense against adversarial word substitution.\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your efforts and insightful comments to improve the manuscript. We respond to each of your comments one-by-one in what follows. In tables, all the values are mean across 3 random seeds.\n\n---\n\n#### **[R1]. Why does adversarial training with Adam perform worse than $s(\\theta)$? A more detailed study may make this clear.**\n**[A].** Thank you for your insightful question. We'd like to provide some clarification here. As you pointed out, the introduced importance score $s(\\theta)$ is closely related to the second-order momentum in Adam, as both utilize the square of gradients of parameters. \n\nHowever, the ways in which this gradient information is used are totally different; $s(\\theta)$ in Table 4 (denoted in (c)) uses it to select the parameters to update via hard thresholding, i.e., the model is updated with sparsed gradients. On the other hand, Adam uses it to re-scale the gradients to update the parameters showing large fluctuations with smaller steps. In the context of incorporating the gradient information as the re-scaling term, $p(\\theta)$ in Table 4 (denoted in (d)) is more closely related to Adam, as it also re-scales the gradients to update the model using $p(\\theta)$, which is derived from $s(\\theta)$ in Equation 3 (described in lines 504-511).  \n \nAs shown in Table 4, the incorporation of the proposed importance score via re-scaling, denoted by (d), performs slightly worse than adversarial training with Adam, denoted by (a). However, the incorporation of the importance score via a selective update, denoted by (c), outperforms both. This result indicates that the better robustness of $s(\\theta)$ in Table 4 is not solely due to incorporating the importance score into the model update, but rather to the careful design of using them via selective training with sparse updates.\n\nTo avoid potential confusion, we will clarify the naming of columns in Table 4 $\\big(s(\\theta), p(\\theta), m(\\theta)\\big)$ to be more explicit in the revised draft. We will also incorporate the above discussion to make it clearer.\n\n---\n\n#### **[R2]. Why does using $p(\\theta)$ even hurt performance when compared to $s(\\theta)$ only?**\n**[A].** First, we clarify that there is a key difference between the two approaches in Table 4, (c) and (d), regarding how they incorporate the importance score into the model update. (c) (i.e., $s(\\theta)$ only) uses thresholding (i.e., sparsity) by only keeping the gradient of a relatively important parameter, which has a high value of $s(\\theta)$, via hard thresholding. In contrast, (d) (i.e., using $p(\\theta)$ from $s(\\theta)$) updates the model by re-scaling gradients according to their relative importance in terms of the values of $p(\\theta)$. \n\nAs a result, the two approaches exhibit significantly different behavior. $s(\\theta)$ can strictly preserve the original parameters of the pre-trained LM, which is less important to the given task, while $p(\\theta)$ updates all the model parameters; therefore, the observed performance decrease with $p(\\theta)$ compared to $s(\\theta)$ could be due to this difference. Consequently, this result indicates the advantages of imposing sparsity rather than re-scaling to update the model for a given task while preserving the generalizable knowledge of the pre-trained LM.\n\n---\n\n#### **[R3]. How does the integration of $m(\\theta)$ increase so much? If this is because of preserving the parameters, does it indicate that training a model for 100 epochs will be less robust than 10 epochs only? Does it indicate that a sparse update paradigm without $p(\\theta)$ as the denomination will also achieve some robustness?**\n**[A].** Thank you for the insightful questions and suggestions!  \n- ***How does the integration of $m(\\theta)$ increase so much?***: We first clarify that integrating $m(\\theta)$ has two distinct advantages for training the model. Firstly, it preserves the parameters by selectively training them using masked (i.e., sparse) gradients. Secondly, it reduces distortion from the original gradients by sampling the mask instead of deterministic selection.  \nHere, the gain from the first advantage is not only obtainable from $m(\\theta)$. Hard thresholding via $s(\\theta)$ also shares the same advantages of selective training and it can be verified with the results in Table 4, i.e., (c) > (a), (b), (d). However, it is important to note that its effectiveness can be limited since thresholding can distort the gradients from the original direction through deterministic updates of important parameters.   \nTherefore, the second advantage of $m(\\theta)$ is crucial, as it improves selective training by reducing the risk of distortion. By sampling the mask for updates from the distribution $p(\\theta)$, which prioritizes important parameters, $m(\\theta)$ continuously benefits from selective training while also covering overlooked parameters through stochastic sampling. This essential advantage of integrating $m(\\theta)$ is demonstrated by improvements over (c,d) in Table 4.\n- ***Model for 100 epochs will be less robust than 10 epochs only?***: As per your suggestion, we performed additional experiments by training RoBERTa-large on SST-2, using the $\\text{Vanilla}$ method with a constant learning rate for 100 epochs. As shown in the table below, the model's robustness significantly decreases as the training progresses. This outcome confirms that preserving the parameters is critical for maintaining model robustness, which is a fundamental principle of selective training with $m(\\theta)$ in $\\text{RoAST}$.  \n| Method  | Acc$_{\\tt in}$ | Acc$_{\\tt shift}$ | Acc$_{\\tt adv}$ | ECE ($\\downarrow$) | AUROC | $\\Delta_{\\tt avg}$ | \n|:--------|:------------:|:----:|:------------:|:------------:|:----:|:------------:|\n| Vanilla (Orig)  | $96.29$ | $91.79$ | $66.30$ | $7.11$ | $86.72$ | $0.00$ | \n| Epoch 20  | $94.88$ | $90.66$ | $64.82$ | $7.75$ | $83.83$ | $-17.45$ |\n| Epoch 40  | $93.92$ | $88.52$ | $60.54$ | $9.81$ | $77.04$ | $-46.38$ |\n| Epoch 60 | $93.42$ | $88.11$ | $59.63$ | $10.16$ | $72.28$ | $-58.74$ |\n| Epoch 80 | $93.39$ | $87.43$ | $59.03$ | $10.68$ | $73.83$ | $-60.08$ |\n| Epoch 100 | $93.31$ | $87.52$ | $58.28$ | $10.62$ | $67.61$ | $-69.94$ | \n\n- ***A sparse update paradigm without $p(\\theta)$ as the denomination will also achieve some robustness?***: To answer your question, we additionally train RoBERTa-large on SST-2, using $\\text{RoAST}$ without the denomination term by $p(\\theta)$ (in Equation 4). The results are presented in the table below. \n| Method  | Acc$_{\\tt in}$ | Acc$_{\\tt shift}$ | Acc$_{\\tt adv}$ | ECE ($\\downarrow$) | AUROC | $\\Delta_{\\tt avg}$ | \n|:--------|:------------:|:----:|:------------:|:------------:|:----:|:------------:|\n| $\\text{Vanilla}$ | $96.29$ | $91.79$ | $66.30$ | $7.11$ | $86.72$ | $0.00$ | \n| $\\text{RoAST}$ without $p(\\theta)$ | $\\underline{96.62}$ | $\\underline{92.08}$ | $\\underline{72.21}$ | $\\bf 5.27$ | $\\bf 90.99$ | $17.57$ |  \n| $\\text{RoAST}$ | $\\bf 96.87$ | $\\bf 92.38$ | $\\bf 72.57$ | $\\underline{5.45}$ | $\\underline{90.37}$ | $\\bf 18.39$ |  \n\n  From the table, we can observe that a sparse update paradigm from $m(\\theta)$ without $p(\\theta)$ also achieves the strong multi-perspective robustness; this result implies that the aforementioned two advantages by introducing $m(\\theta)$ is key of the improvement from RoAST, and the scaling via denomination provides additional gains upon that.   \n\nWe believe that these questions and responses provide many insights to understand our work. We will incorporate these in the revised manuscript accordingly.\n\n---\n\n#### **[R4]. In Table 5, a random/min s(theta) still outperforms the baseline, especially for sentiment classification tasks. Does this mean that the importance score is not that important?**\n**[A].** Thank you for the question. Firstly, it is important to note that both the $\\text{Rand}$ and $\\text{Min}$ methods in Table 5 use not only selective training with varying masking strategies, but also adversarial perturbation during training. Therefore, a more appropriate comparison for these methods would be with $\\text{AdvEmbed}$ from Tables 1 and 2, rather than the $\\text{Vanilla}$ baseline presented in Table 5.\n\nFor your convenience, we have provided $\\Delta_{\\tt avg}$ scores for the methods from the paper in the table below.\n\n| Methods \\ Tasks | Sentiment Classification | Entailment |\n|:--------|:------------:|:----:|\n| $\\text{AdvEmbed}$ | $13.70$ | $\\underline{5.72}$ |\n| $\\text{Min}$  | $7.26$ | $0.44$ |\n| $\\text{Rand}$ | $\\underline{14.88}$ | $1.67$ |\n| $\\text{RoAST}$  | $\\bf 18.39$ | $\\bf 7.63$ |\n\nFrom the table, one can observe that:\n- $\\text{Min}$ significantly lags behind the $\\text{AdvEmbed}$ baseline, indicating the ineffectiveness of updating only the less important parameters.\n- $\\text{Rand}$ shows a slight improvement in sentiment classification, possibly due to the additional benefit of sparse updates in a task that is more susceptible to overfitting. This is suggested by the higher validation performance overall, despite fewer training samples for sentiment classification.\n- There is a clear pattern of improvement with selective updates as the importance of the selected parameters increases, as demonstrated by the order: $\\text{Min}$ < $\\text{Rand}$ < $\\text{RoAST (Max)}$. \n\nOverall, these results clearly reveal the importance of selecting parameters based on their importance scores. We will make this clearer in our revised manuscript.\n\n---\n\n#### **[Q]. Can you show some results of FreeLB++, which runs adversarial perturbation for 10 steps instead of one?** \n**[A].** Following your suggestion, we conducted additional experiments with FreeLB++ on the sentiment classification task, which runs adversarial perturbation for 10 steps without constraints of norm-bounded projection [1]. We also tuned the adversarial step size appropriately as the number of steps changed. The results are summarized in the below table. \n\n| Method  | Acc$_{\\tt in}$ | Acc$_{\\tt shift}$ | Acc$_{\\tt adv}$ | ECE ($\\downarrow$) | AUROC | $\\Delta_{\\tt avg}$ | \n|:--------|:------------:|:----:|:------------:|:------------:|:----:|:------------:|\n| Vanilla | $96.29 $ | $91.79 $ | $ 66.30 $ | $7.11 $ | $86.72 $ | $0.00$ | \n| FreeLB | $\\underline{96.33}$ | $\\underline{91.94}$ | $ 70.07 $ | $6.49 $ | $89.82 $ | $9.21$ | \n| FreeLB++ | $96.14 $ | $91.79$ | $\\underline{72.45}$ | $\\bf 5.44$ | $\\bf 90.79$ | $\\underline{11.02} $ | \n| RoAST (Ours) | $\\bf 96.87$ | $\\bf 92.38$ | $\\bf 72.57 $ | $\\underline{5.45}$ | $\\underline{90.37}$ | $\\bf 18.39$ | \n\nHere, one can observe that $\\text{FreeLB++}$ outperforms $\\text{FreeLB}$ with a large gap, especially in $\\text{ACC}\\_{\\tt adv}$ (70.07 $\\rightarrow$ 72.45). Consequently, $\\text{FreeLB++}$ is better than $\\text{FreeLB}$ for multi-perspective robustness as well ($\\Delta\\_{\\tt avg}: 9.21 \\rightarrow 11.02$). Since $\\text{RoAST}$ also uses a single step for constructing adversarial perturbation, these results indicate a potential room for further improvement in our method at the additional cost of the increased number of steps. We will include the respective discussions and results in the final draft.\n\n---\n\nWe hope this response will clarify your concerns. Please let us know if there are any unclear points remaining. \n\nSincerely,  \nAuthors.\n\n[1] Li et al, Searching for an Effective Defender: Benchmarking Defense Against Adversarial Word Substitution, EMNLP 2021", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The experiments are comprehensive and the results are great.\n- The approach has the potential to be a stronger and more generalised technique, rather than just a few NLP tasks.", "reasons_to_reject": "- Further analysis is required to analyze why the importance score may benefit the model, as it is close to the update formula in Adam optimizer. In Table 4, the s(theta) row has better adversarial robustness, even better than the AT baseline. Why? The approach to computing s(theta) is in spirit to the second order momentum in Adam, which measures the variance of gradients. For parameters with large fluctuations, Adam will update with smaller steps. My question is, why does adversarial training with Adam perform worse than s(theta)? A more detailed study may make this clear. In addition, some explanations are not convincing enough and maybe more experiments would make them stronger. \n\n- Why does using p(theta) even hurt performance when compared to s(theta) only? \n\n- How does the integration of m(theta) increase so much? If this is because of preserving the parameters, does it indicate that training a model for 100 epochs will be less robust than 10 epochs only? Does it indicate that a sparse update paradigm without p(theta) as the denomination will also achieve some robustness? \n\n- Some experimental results seem to be contrary to motivation. In Table 5, a random/min s(theta) still outperforms the baseline, especially for sentiment classification tasks. Does this mean that the importance score is not that important? ", "questions_to_authors": "- Can you show some results of FreeLB++, which runs adversarial perturbation for 10 steps instead of one? Li et al, EMNLP 2021, Searching for an effective defender: Benchmarking defense against adversarial word substitution.\n", "paper_topic_and_main_contributions": "The paper proposes an adaptive gradient update approach by reducing the update frequency of large variance parameters. The selective update strategy looks like \"gradient dropout\" by masking a gradient with a probability value p and rescaling it by 1/p if it is updated. Experimental results show that this effectively increases the robustness of the model.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an easy-to-follow approach called RoAST, which aims to improve the robustness of language models (LMs) from multiple perspectives. 
This approach is unique as it considers both robustness on perturbed inputs and generalizable knowledge in pre-trained LMs. 
The experimental setup is robust, including multiple baselines and thorough analysis, strengthening the results and conclusions. The paper's extensive experiments show the best weighted average performance in four robustness aspects, alongside a standard validation accuracy compared to other finetuning methods. Overall, RoAST has the potential to become a more robust and more generalized technique for various NLP tasks.

From the reviewers, this paper only showed experiments using limited tasks and datasets, thus recommending the use of challenging benchmarks and non-BERT-based models like decoder-only structures.
Overall, the paper is deemed to require further analysis to explain the difference in approaches, which could be more exciting to the community.
Decision: Accept-Findings