AD-DROP: Attribution-Driven Dropout for Robust Language Model Fine-Tuning
Authors: Tao Yang
Jinghao Deng
Xiaojun Quan
Qifan Wang
Shaoliang Nie
Venue: NeurIPS 2022 Accept (2022)
Keywords: dropout
self-attention
attribution
fine-tune
language model

Abstract: Fine-tuning large pre-trained language models on downstream tasks is apt to suffer from overfitting when limited training data is available. While dropout proves to be an effective antidote by randomly dropping a proportion of units, existing research has not examined its effect on the self-attention mechanism. In this paper, we investigate this problem through self-attention attribution and find that dropping attention positions with low attribution scores can accelerate training and increase the risk of overfitting. Motivated by this observation, we propose Attribution-Driven Dropout (AD-DROP), which randomly discards some high-attribution positions to encourage the model to make predictions by relying more on low-attribution positions to reduce overfitting. We also develop a cross-tuning strategy to alternate fine-tuning and AD-DROP to avoid dropping high-attribution positions excessively. Extensive experiments on various benchmarks show that AD-DROP yields consistent improvements over baselines. Analysis further confirms that AD-DROP serves as a strategic regularizer to prevent overfitting during fine-tuning.

AD-DROP: Attribution-Driven Dropout for Robust Language Model Fine-Tuning
Tao Yang', Jinghao Deng', Xiaojun Quan'; Qifan Wang", Shaoliang Nie² 'School of Computer Science and Engineering, Sun Yat-sen University Meta Al [yangt225, dengjh27)@mail2 sysu. edu. cn, quanxj3mmail sysu. edu, cn "[qffrr snie)
Abstract
Fine-tuning large pre-trained language models on downstream tasks is apt to suffer from overfitting when limited training data is available. While dropout proves to be an effective antidote by randomly dropping proportion of units, existing research has examined its effect on the self-attention mechanism. In this paper, we investigate this problem through self-attention attribution and find that dropping attention positions with low attribution scores can accelerate training and increase the risk of overfitting. Motivated by this observation, we propose Attribution- Driven Dropout (AD-DROP), which randomly discards some high-atriibution positions to encourage the model to make predictions by relying more on low- attribution positions to reduce overfitting. We also develop a cross-tuning strategy to alternate fine-tuning and AD-DROP to avoid dropping high-attribution positions excessively. Extensive experiments on various benchmarks show that AD-DROP yields consistent improvements over baselines. Analysis further confirms that AD-DROP serves as a strategic regularizer to prevent overfitting during fine-tuning.
Introduction
Pre-training large language models (PrLMs) on massive unlabeled corpora and fine-tuning them on downstream tasks has become a new paradigm [13. Their success can be partly attributed
to the self-attention mechanism 4 yet these self-attention networks are often redundant [5] and tend to cause overfitting when fine-tuned on downstream tasks due to the mismatch between their overparameterization and the limited annotated data [7H13]. To address this issue, various regularization techniques such data augmentation [14 15). adversarial training [16] I7),, and dropout-based based methods [II 13 [8] have been developed. Among them, dropout- methods widely adopted for their simplicity and effectiveness. Dropout [19] which randomly discards proportion of units, is at the core of dropout-base based methods. Recently, several variants of dropout have been proposed, such Concrete Dropout [20]. DropBlock (21), and Artooropout 22 However, these variants generally follow the vanilla dropout to randomly drop units during training and pay little attention to the effect of dropout self-attention. In this paper, seek fill this gap from the perspective of self-attention attribution (33 and aim to reduce overfitting when fine-tuning PrLMs.
Attribution 24 is an interpretability method that attributes model predictions to input features via saliency measures such as gradient (25 26 is also used to explain the influence patterns of self-attention in recent literature 23 27 28 Our prior experiment of self-attention attribution (Section [.2] reveals that attention positions are not equally important in preventing overfitting, and dropping low-atrribution positions is more likely cause overfitting than discarding high-attribution positions. This observation suggests that attention positions should not be treated the same in dropout.
"Corresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Motivated by the above, we propose Srabutioo-Driven Dropout (AD-DROP) to better fine-tune PrI.Ms based on self- attention attribution. The general idea of AD- DROP is to drop set of self-attention posi- tions with high attribution scores. We illus- trate the difference between vanilla dropout and AD-DROP by their attention maps in Fig- ure I When fine-tuning PrLM batch of training samples, AD-DROP involves four steps. First, predictions are made through forward computation without dropping any attention position. Second, compute the attribution score of each attention position by gradient [25] or integrated gradient [26) attribution methods. Third, we sample a set of positions with high attribution scores and generate a mask matrix for each attention map. Finally. the mask matrices are applied to the next forward computation to make predictions for backpropagation. as a strategic dropout regularizer that forces the model to make predictions by relying more on low-attribution positions to reduce overfitting. Nevertheless, excessive neglect of high-attribution positions would leave insufficient information for training. Hence, we further propose a cross-tuning strategy that performs fine-tuning and AD-DROP alternately to improve the training stability.
Vanilla dropout
AD-DROP
Figure I: Attention maps of vanilla dropout and our AD-DROP. Darker attention positions indicate higher attribution scores, and crossed circles mean dropped attention positions. Red-dotted boxes refer to can- didate discard regions with high attribution scores. Unlike vanilla dropout which randomly discards atten- tion positions, AD-DRO focuses on dropping high- attribution positions in candidate discard regions.
can be regarded
To verify the effectiveness of AD-DROP, we conduct extensive experiments with different PrLMs (i.e., BERT D. RoBERTa (2). ELECTRA 29) and OPUS-MT (30) on various datasets (i.e., GLUE
B1. CoNLL-2003 32) WMT 2016 EN-RO and TR-EN KK HANS 34,, and PAWS-X 35 ).
Experimental results show that the models tuned with AD-DROP obtain remarkable improvements over that tuned with the original fine-tuning approach. For example, on the GLUE benchmark. BERT achieves an average improvement of 1.98/0.87 points on the dev/test sets while RoBERTa achieves an average improvement of 1.29/0.62 points. Moreover, ablation studies and analysis demonstrate that gradient- based attribution 25 26 is a more suitable saliency measure for implementing AD-DROP than directly using attention weights or simple random sampling. Moreover, they also demonstrate that the cross-tuning strategy plays crucial role in improving training stability.
To sum up, this work reveals that self-attention positions are not equally important for dropout when fine-tuning PrLMs. Arguably, low-attribution positions are more difficult to optimize than nigh-attributton positions, and dropping these positions tends not to relieve but accelerate overfitting. This leads to novel dropout regularizer, AD-DROP driven by self-attention attribution. Although proposed for self-attention units, AD-DROP can be potentially extended other units as dropout.
2 Methodology
2.1 Preliminaries
Since Transformers [4] are the backbone of PrLMs, we first review the details of self-attention in Transformers and self-attention attribution (23). Let X gaxd be the input of Transformer block. where n is the sequence length and d is the embedding size. Self-attention in this block first maps X into three matrices Q. K, and Vn via linear projections query, key, and value respectively for the h-th head. Then, the attention output of this head is calculated as:
QrK,T T + +MA Vn.
Attention (Qr, Kr. Vn. = A VA softmax
(I)
dah
where Vde a scaling factor. M, is the mask matrix to apply dropout in self-attention, and elements in MA will be -oo if the corresponding positions in attention maps are masked and 0 otherwise.

---

Based on the attention maps A [A1, ... ".....AA "A] for H attention heads, gradient attribution 25 36 directly produces an attribution matrix B,, by computing the following partial derivative:
(A)
B dA
where Fe(.) denotes the logit output of the Transformer for class c.
To provide theoretically more sound attribution method, Sundararajan et al. (26) propose integrated gradient, which is employed by Hao et al. [23] as a saliency measure for self-attention attribution. Specifically, Hao et al. [23] compute the attribution matrix B,, as:
A m
OF ( A)
B
AA
where is the number of steps for approximating the integration in integrated gradient, and @ is the element-wise multiplication operator. Despite its theoretical advantage over gradient attribution, integrated gradient requires times more computational effort, which is especially expensive when it is applied to all the attention heads Transformers. Moreover, our experiments in Section 3.4 show that gradient attribution achieves comparable performance with integrated gradient but requires much less computational cost, suggesting that gradient attribution is more desirable for AD-DROP.
2.2 A Prior Attribution Experiment
To better motivate our work, we first conduct a prior experiment on MRPC
pookon
wwg.hhhhheeeee.ooo. pedkon -a- by rexce seepig
dappie
37 to investigate how different po- sitions self-attention maps affect
0.5
fine-tuning performance based on at- tribution results. RoBERTa_base [2] is used as the base model. To begin with, we first perform a forward com- putation of the model on each batch of training samples to obtain the logit output of each sample corresponding to the gold label. Then, we obtain an attribution matrix B, for the self- attention positions in the first layer" by gradient attribution with Eq. 2 and sort each row of the matrix. Finally, we sample a set of self-attention positions with high or low attribution scores in each row to generate mask matrix M,. which is fed into Eq. (1) make the final predictions. After each epoch of training, we evaluate the model on the development set. Two baseline dropping strategies (i.e., dropping by random sampling and without dropping any position) are employed for comparison. We plot the loss curves of the model with these dropping strategies on both training and development sets in Figure 2 The observations are threefold. First, dropping low- attribution positions makes the model fit the training data rapidly, whereas it performs poorly on the development set, indicating that the model is not properly trained. Second, compared with the other dropping strategies, dropping high-attribution positions reduces the fitting speed significantly. Third, random dropping only slightly reduces overfitting, compared to the training without dropping. These observations suggest that attention positions are of different importance in preventing overfitting. We conjecture that low-attribution positions are more difficult to optimize than high-attribution positions. While dropping low-attribution positions tends to accelerate overfitting, discarding high-attribution positions helps reduce overfitting.
16
66
0.3
66-
02
04-
02
Epoch
(8) Training loss
(b) Validation
Figure 2: Results of training and validation losses when fine- tuning RoBERTa with different dropping strategies on MRPC. The dropping rate is set to 0.3 if it applies.
2.3
Attribution-Dri Driven Dropout
we propose novel regularizer, AD-DROP, to better prevent overfitting when adapting PrLMs downstream tasks. The motivation of AD-DROP is to minimize the over-reliance of these models on particular features which may affect their generalization.
Inspired by the observations in Section 2.2
We provide results and discussions in Appendix B.
3

---

Formally, given a training set D = { Bi of N samples, where 2; is the i-th sample and ys is its label, the goal of AD-DROP to fine-tune PrLM F(-) of L layers on D. Same the vanilla dropout 19 AD-DROP is only applied in the training phase.
Softmax
Pseudo Label
Loss
Softmax
FFN
Logit
FFN
Sort& Sample
Add
Nx
Atribution
Nx
Mask Matrices
Self-A Ulentoon Maps
A Atribution Matrices
Self A tention Maps
Step1
Step2
Step3 3
Step
X
Figure 3: Illustration of AD-DROP in four steps. (1) Conduct the first forward computation to obtain pseudo label č. (2) Generate attribution matrices B via computing the gradient of logit output Fa (A) with respect to each attention head. (3) Sort B and strategically drop positions to produce mask matrices M. (4) Feed M into the next forward computation to compute the final loss.
the idea of AD-DROP can be described in four steps. First, we conduct forward computation of the model to obtain the label with the highest probability as pseudo label. The reason we adopt pseudo labels rather than gold labels for attribution will be explained shortly. Specifically, for the input 24 with n tokens, we apply F(-) to encode it and obtain its pseudo label
As shown in Figure 3
= arg max (PF
(4)
where Pr(c)x;) is the probability of class for n;. After the forward computation, also obtain set of attention maps A = A2. A for each layer according to Eq. ①
Second, we compute the attribution matrices B [B1, B2. ... Bh] for H heads according to Eq. (2). Specifically, the attribution matrix B,, for the hithh head computed as:
B 0F (A)
(5)
dA
where F& (A) is the logit output of pseudo label before softmax
Third, we generate a mask matrix M, based on B,, To this end, we first sort each row of B, in ascending order and obtain sorted attribution matrix B₁, Then, we define candidate discard region Sn, in which each element st is defined as:
Si. ={
{
1, bij < <biint(((Ip)) otherwise
where bij and by.5 are elements of B, and B,, respectively, int(.) is an integer function, and p E (0, 1) is used to control the size of the candidate discard region. Next, apply dropout in the region to produce the mask matrix M, as:
00,
(sng+ =0
mijj
O. otherwise
where Wi,j BBrnnouli(1 - q) is an element of matrix Un B and y is the dropout rate.
Finally, MA is fed into self-attention of Eq. [ for the second forward computation, and the final output is used to calculate the loss for backpropagation.
The negative loss will be used for both regression and token-level tasks, introduced in Appendix A.

---

Discussion The reasons that AD-DROP uses pseudo labels for attribution are twofold. First, adopting gold labels will divulge label information and lead inconsistency between training and inference. Second, for misclassified samples in the first forward computation, AD-DROP with gold labels tends to continue make incorrect predictions because high-attribution attention positions derived from gold labels may be located in low-atrribution regions derived from pseudo labels. Therefore, dropping these positions does not help the model correct wrong predictions, while AD- DROP with pseudo labels urges the model to rely on important features in the current pass and may correct the wrong predictions. The attribution with gold labels will be investigated Section 3.4
2.4 Cross- Tuning Algorithm
We further design a cross-tuning algorithm to avoid dropping high-attribution positions excessively when applying AD-DROP. The idea of cross-tuning is to execute the original fine-tuning and AD- DROP alternatively. Specifically, it performs the original fine-tuning at odd epochs and AD-DROP even epochs. The overall process of cross cross-tuning is described in Algorithm I where Lines 3-5 are the original fine-tuning operations and Lines 7-9 describe the process of AD-DROP.
Algorithm Cross-tuning
Input: shuffled training samples D - (o) PrLM F with parameters W
Output: updated parameters W
1: Initialize F with W, epoch
2: while not converged do
3: Calculate the prediction (x)()()) and loss via forward computation.
4: if' epoch%2 = then
5:
Backpropagate the loss update model parameters W.
6: else
7:
Perform AD-DROP by Eq. 44-77
7 obtain mask matrices M [M, M2. .......M. MM].
8:
Calculate the new prediction (r( (x:) and new loss by feeding M into Eq. (I) Backpropagate the new loss update model parameters W.
9:
10:
epoch- = epoch +
11:
return W W
3
Experiments
3.1
Datasets
We conduct our main experiments on eight tasks of the GLUE benchmark [31,
including SST-2
38 MNLI 39. QNLI [40], QQP [41] CoLA 422, STS-B [43, MRPC 37 and RTE 44] The
evaluation metrics are Matthew's Corrcoef (Mee) 45 for CoL.A, Pearson Corrcoef (Pcc) 46 for
STS-B, and Accuracy (Acc) for the others. To demonstrate that AD-DROP applies to token-level tasks as well, we conduct experiments on Named Entity Recognition (CoNLL-2003 (32)) and Machine Translation (WMT 2016 (33] datasets, the results of which are shown in Appendix A.2. Besides, we also evaluate AD-DROP on two out-of-distribution (OOD) datasets, including HANS 34 and
PAWS-X 35| The details of these datasets are introduced in Appendix C.I.
3.2 Implementation Details
We implement our AD-DROP in Pytorch with the Transformers package [7]. We train the selected PrLMs on GeForce RTX 3090 GPUs. We tune the learning rate (lc-5, 2c-5, 3e-5) and the batch size in (16, 32, 64). Following Miao et al. [17,, we perform early stopping to choose the number of training epochs on GLUE. The two critical hyperparameters p and q are searched within [0.1. 0.9) with step size 0.1. For integrated gradient in Eq. (3). we follow Hao et al. (23] and set m to 20. We apply AD-DROP only in the first layer for the datasets SST-2, MNLI, QNLI, QQP, and STS-B since the fine-tuning datasets, we apply AD-DROP all layers. We provide the detailed hyperparameter settings on each dataset in Appendix C.2. Our code is available https: / gi thub. com/Taaoangg25///D A2-
these datasets is stable and less likely to cause overfitting. For the rest
5

---

Table 1: Overall results of fine-tuned models the GLUE benchmark. The symbol i denotes results directly taken from the original papers. The best average results are shown bold.
Methods
SST-2
MNLI
QNLI
QQP
CoLA
STS-B
MRPC
RTE
Average
Development
BERT buse
92.3
84.6
91.5
91.3
60.3
89.9
85.1
70.8
83.23
+SCAL
17
92.8
84.1
90.9
91.4
61.7
69.7
+SuperTi
48
93.4
84.5
91.3
91.3
58.8
89.8
87.5
72.5
83.64
+R-Drop
18
93.0
85.5
92.0
91.4
62.6
89.6
87.3
71.1
84.06
+AD-DROP
93.9
85.1
92.3
91.8
90.4
88.5
75.1
85.21
RoBERTa
95.3
87.6
92.9
91.9
64.8
90.9
90.7
79.4
86.69
+R-Drop 18
95.2
87.8
93.2
91.7
64.7
91.2
90.5
80.5
86.85
HidddenCul
15
95.8
88.2
93.7
92.0
66.2
91.3
92.0
83.4
87.83
+AD-DROP
95.8
88.0
93.5
92.0
91.4
92.2
84.1
87.98
Test
BERT base
93.6
84.7
90.4
89.3
52.8
85.6
81.4
68.4
80.78
+AD-DROP
94.3
85.2
91.6
89.4
53.3
86.6
84.1
68.7
81.65
RoBERTabas
94.8
87.5
92.8
89.6
58.3
88.7
86.3
75.1
84.
+AD-DROP
95.9
87.6
93.4
89.5
58.5
89.3
87.9
76.0
84.76
3.3
Overall Results
We report the overall results of the fine-tuned models in Table I We first compare AD-DROP with existing regularization methods on the development sets, including the original fine-tuning, SCAL [17. SuperT [8], R- R-Drop [18]. and HiddenCut (15). We observe that AD-DROP surpasses the baselines on most of the datasets. Specifically, AD-DROP yields an average improvement 1.98 and 1.29 points BERT base and RoBERTapase. Tabases respectively. We then discuss the performance of AD-DROP on the test sets. Results Table show that AD-DROP achieves consistent improvement, boosting the average scores of BERT hase and RoBERTa by 0.87 and 0.62, respectively. Besides, compared with large datasets, AD-DROP achieves more gains on small datasets, which are more likely overfitting, illustrating that AD-DROP is more suitable for small data scenarios.
3.4 Ablation Study
We conduct ablation experiments on four small datasets investigate the impact of different components. Due to the limited number of submissions imposed the GLUE server for evaluation, the results here reported the development sets.
Table 2: Results of ablation studies, in which r/w means "replace with" and w/o means "without".
Methods
CoLA
STS-B
MRPC
RTE
BERT base
60.3
89.9
85.1
70.8
+AD-DROP (GA)
64.6
90.4
88.5
75.1
r/w IGA
63.8
90.7
88.5
74.4
Attribution methods AD-DROP can be implemented with different attribution methods to generate the mask matrix in Eq. ( such integrated gradient attri- bution (IGA) introduced Eq. (3), atten- tion weights for attribution (AA), and domly generating the discard region (RD) in Eq. (6). We replace the gradient attribu- tion (GA) in Eq. (5) with these methods. From Table 2 we can make three obser- vations. First. AD-DROP with gradient- based attribution methods (GA and IGA) surpasses that with the other methods (AA or RD) most of the datasets, illustrating that gradient- based methods are better at finding features that are likely to cause overfitting. Second, IGA outperforms GA in some cases. Although IGA provides better theoretical justification than GA for attribution, requires prohibitively more computational cost than GA (see Section 4.7 for efficiency analysis), making GA more desirable choice for AD-DROP Third, AD-DROP improves the original
rAw AA
63.6
90.0
88.0
74.7
r/w RD
62.1
90.2
87.8
74.7
r/w gold labels
63.2
88.0
74.4
w/o cross-tuning
62.1
90.4
87.3
71.5
RoBERTabase
64.8
90.9
90.7
79.4
+AD-DRO (GA)
66.8
91.4
92.2
84.1
r/w IGA r/w AA
68.1
91.6
91.4
82.7
66.3
91.5
91.2
82.3
r/v RD
66.5
91.5
92.2
82.0
rAv gold labels
66.4
91.2
82.0
w/o cross-tuning
67.3
91.3
90.4
80.5

---

BERT base and RoBERTabese with any of the masking strategies, demonstrating the robustness of AD-DROP overfitting when fine-tuning these models.
Pseudo labels vs gold labels In Section 2.3 we discuss the motivation of using pseudo labels for attribution in AD-DROP. To verify the reasonability, for attribution. As the results show in Table 2 performance, illustrating that AD-DROP with pseudo labels for attribution is preferable.
conduct an experiment with gold labels using gold labels for attribution deteriorates the
Cross-tuning To verify the effectiveness of the cross-tuning strategy, we ablate it and ap- ply only AD-DROP in all training epochs. As shown Table 2 removing moos-tuning causes noticeable performance degradation on most of the datasets. This can be explained by the intu- ition that AD-DROP without cross-fune tross-tuning tends discard high-attribution positions excessively and make the model difficult to converge nor- mally. To vividly demonstrate the effect of AD) DROP with without cross-tuning, we visual- ize the distributions the performance on the RTT development set when enumerating the parameters p and in the range of [0.1, 0.9]. The results plotted in Figure 4 where each blue/orange point denotes the accuracy with a pair of and values. We observe from the fig- ure that A AD-DROP without cross-tuning cannot be trained properly under tings. However, it works well for most parameter settings when cross-tuning is applied, demonstrating that cross-tuning is vital for improving training stability.
80
Accuracy
70
65
60
55
w/o cross-tuming
with cross-tuning
Figure 4: Results of AD-DROP with and without cross-tuning when enumerating p and q in [0.1, 0.9]. RoBERTa is chosen as the base model. Re- sults show that "with cross-tuning" leads to much lower variance and higher performance.
parameter set-
4 Analysis
In this section, we further conduct several experiments for more thorough analysis.
4.1 Repeated Experiments
To reduce the influence of random- ness, we conduct repeated experi- ments four small datasets (i.e., CoLA, STS-B, MRPC and RTE). We repeat the training of each model with five random seeds and report the average
Table 3: Results of repeated experiments. Each score is the average of five runs with a standard deviation.
Methods
CoLA
STS-B
MRPC
RTE
BERT
61.8
89.440.6
85.2+1.3
71.2+1.2
+AD-DROP
63.4
90.1
87.420.9
73.9
and standard deviation on the development sets. From Table 3 observe that AD-DROP outper- forms the original fine-tuning on all the datasets. In addition, AD-DROP results in lower standard deviations on most of the datasets, showing that AD- AD-DROP is more robust in fine-tuning PrLMs than the original approach.
RoBERTabuee
64.3
91.0.1.2
89.8
79.1
+AD-DROP
66.4
91.2
91.3..0.7
82.5
4.2 Effect of Data Size
To study the impact of data size, we compare AD-DROP with the original fine-tuning (FT) approach on QNLI and QQP two relatively large datasets, and report their performance when the number of training samples changes. RoBERTa chosen as the base model. Figure shows that AD-DROP
Results on other datasets are shown in Appendix D.1.
Ressuls QQP shown in Appendix D.2.
7

---

outperforms FT consistently on QNLI. Moreover, AD- DROP improves the efficiency of data use as training AD- DROP with 60% training samples produces comparable performance to FT with full data.
Accuracy 92
4.3 Hyperparameter Sensitivity
AD-DROP involves two hyperparameters p and y to con- trol the number of discarded attention positions. To inves- tigate the sensitivity of AD-DROP to them, we show the results of different p and g combinations on CoLA and RTE in Figure in which we apply MaxAbsScaler project the difference between the results of AD-DROP and FT into interval of [-1.0, 1.0]. We observe that
vanilla FT
AD-DROP
20
30
40
50
70
100
Size
Figure Results of AD-DROP and FT as the number of training samples changes.
BERT with AD-DROP is not apperparameeee-sensttiii as it outperforms the baseline under most settings. contrast, RoBERTa with AD-DROP is more sensitive and requires careful search for optimal hyperparameter settings. The possible reason that RoBERTa pre-trained with more data and more effective tasks than BERT, making it less prone overfitting than BERT.
62
6A
65
06
66
(a) BERT on CoLA
(b) RoBERTa CoLA
(c) BERT on RTE
(d) RoBERTa
Figure 6: Results of sensitivity study CoLA and RTE. Rows correspond to and columns refer to q. Blue blocks indicate the results of AD-DROP below the baseline (FT), and red blocks mean results of AD-DROP above the baseline. Darker colors mean greater gaps with the baseline.
4.4 Larger Model Size
To verify the scalability of AD-DROP for a larger model size, we evaluate AD-DROP with RoBERTaaage on the RTE and MRPC datasets. Ta- ble shows the average scores and standard devi- ations of five random seeds. There are two main observations. First, AD-DROP achieves consistent improvements over the larger RoBERTa model, il- lustrating that AD-DROP is scalable to large models. Second, compared with RoBERTahes on RTE in Table 3, the larger model significantly reduces the deviation (from 1.7 to 0.86), suggesting that larger model size indeed helps to improve the stability. AD-DROP further improves the performance and reduces the deviation.
Table 4: Testing AD-DROP on a larger model.
Methods
MRPC
RTE
RoBERTa
90.83
85.99 1..96
+AD-DROP
91.62 10.53
88.01 10.48
4.5 Few-shot Scenario
In this subsection, test the performance of AD-DROP under few-shot scenarios. Specifically, carry out 16-, 64-, and 256-shot experiments on SST-2 and CoLA with RoBERTabase as the base model and the baseline. We report the average scores and standard deviations of five random seeds in Table We observe that RoBERTa with AD-DROP consistently outperforms the original fine-tuning approach. Besides, AD-DROP tends to bring more benefits when fewer samples are available.
http:///ssttttttoooooooooooooeee haw.

---

Table 5: Testing AD-DROP in few-shot settings. RoBERTa with AD-DROP achieves higher perfor- mance and lower deviations than that with the original fine-tuning approach.
SST-2
CoLA 64-shot
Methods
16-shot
64-shot
256-shot
16-shot
256-shot
RoBERTabass aasas
74.50 +3.03
89.06 00.83
91.44+t0.17
23.18 23.18 66.38 26.70+4.96
39. 49.70
51.11
+AD-DROP
80.16
91.61_0.52
92.61.0.18
46.41
52.47.1.16
4.6
Out-of-Distribution Generalization
To further demonstrate AD-DROP is beneficial to reducing overfitting, we test AD-DROP with RoBERTahase IWO out-of-distribution (OOD) datasets, i.e., HANS and PAWS-X. For HANS, we the checkpoints trained MNLI and test their performance on the validation set (the test set is not supplied). For PAWS-X, we use the checkpoints trained on QQP and examine its performance on the test set. The evaluation metric is accuracy. From T we can see that RoBERTa with AD-DROP achieves better generalization, where AD-DROP boosts the performance by 0.66 on HANS and 3.35 on PAWS-X, illustrating that the model trained with AD-DROP generalizes better OOD data.
Table 6: Testing AD-DROP on OOD datasets.
Methods
HANS
PAWS-X
RoBERTa base 69.83
47.90
+AD-DROP
70.49
51.25
4.7 Computational Efficiency
To analyze the computational efficiency, we quantitatively study the computational cost of AD-DROP with different dropping strategies (GA, IGA, AA, and RD) relative to the original fine-tuning on CoLA, STS-B, MRPC, and RTE. BERT is chosen as the base model for this experiment. As shown in Table although IGA achieves more favorable performance on one of the datasets, it requires higher computational costs than its counterparts, especially when applied in all the layers. In contrast, AD-DROP with GA is more competitive in both performance and computational cost.
Table 7: Results of performance and computational cost of AD-DROP with different masking strategies (GA, IGA, AA, and RD) relative the original fine-tuning, The symbol means AD- DROP is only applied in the first layer. BERT is chosen as the base model.
CoLA
STS-B*
MRPC
RTE Time
Methods
Mee
Time
Pec
Time
Acc
Time
Acc
RD
+1.8
x1.42
+0.3
1.38
+2.7
1.31
+3.9
x1.42
AA
+3.3
x1.42
+0.1
x x.44
+2.9
1.94
+3.9
x x1.58
GA
+4.3
x3.58
+0.5
1.95
+3.4
x4.13
+4.3
x4.50
IGA
+3.5
x99.61
+0.8
x 15500
+3.4
X x110.12
+3.6
X 125.667
5
Related Work
Dropout Dropout is a widely used regularizer to mitigate overfitting when training deep neural networks. Vanilla dropout [19] randomly selects neurons with a predefined probability and sets their values zeros during training. By doing so, the neurons cannot co-adapt and the trained networks can lead to better generalization. In recent years, many variants of dropout have emerged. One body of research aims adopt different strategies drop units in neural networks. For example, DropConnect [49] randomly selects connections between neurons to discard. DropBlock [2]] defines a structured dropout that randomly drops units specific contiguous region of a feature map. AutoDropout 22) aims to improve the dropout pattern of DropBlock by introducing an automatic method to design dropout structures. HiddenCut [1.5] drops contiguous spans within the hidden space, in which the attention weights are utilized select the dropped spans strategically.

---

Another body of research devotes to addressing the inconsistency between training and inference when dropout is applied. For instance, mixout [T] randomly replaces selected parameters with original pre-trained weights rather than setting them to zeros. CHILD-TUNING 13 selects child network and masks out the gradients of the non-child network during the backward step, only updating weights in the child network. R-Drop [88] performs dropout twice in the forward steps to produce two sub-models and then applies KL-divergence for their output distributions, forcing the two sub-models to be consistent with each other. However, most of these methods follow the random sampling strategy of dropout and pay little attention to the different importance of self-attention positions in PrLMs.
Attribution Numerous studies have been undertaken to interpret the behaviors of deep neural networks (DNNs). As a theory for interpretability, attribution aims to evaluate the impact of input features on predictions [2 Generally, attribution methods can be divided into perturbation- baaab
[50 gradient-related [25 53 26], and attention viention-bbsed [56] methods. We focus on reviewing the gradhent-related methods as they are more relevant to our work. Specifically, earlier works try to explain model decisions via gradients since gradients indicate the direction and rate that changes the loss the fastest. However, Sundararajan et al. 26 point out that gradient attribution violates the sensitivity axiom in some cases that the gradients will be zero for the function in saturated areas, and propose integrated gradient as a theoretically more sound attribution method.
25 57 58
Other efforts have been devoted to revealing the behavior patterns of PrLMs. Kovaleva et al. 8 and Clark et al. [ use attention weights for attribution to investigate what specific knowledge BERT i learns. Jain and Wallace [27] and Brunner et al. [59] investigate the identifiability of attention weights and conclude that attention weights are not a faithful explanation for model predictions. Hao et al. [23] apply integrated gradient (26] as saliency measure for self- aiffaattntion attribution in BERT, and use the attribution result interpret information interactions inside Transformers. Similarly, Lu et al. [28 develop influence patterns based on integrated gradient to explain information flow in BERT. Unlike these works, we aim to examine the effect of dropout on self-attention through self-attention attribution and to reduce overfitting when fine-tuning PrLMS.
6 Conclusion
We propose a novel dropout regularizer, AD-DROP, to mitigate overfitting when fine-tuning PrLMs on downstream tasks. Unlike previous dropout-based methods that generally adopt the random sampling strategy to discard units, AD-DROP draws inspiration from self-attention attribution which reveals that attention positions are not equally important in reducing overfitting and that dropping inappropriate positions may exacerbate the problem. Therefore, AD-DROP focuses on discarding high-attribution attention positions to prevent the model from relying heavily on these positions to make predictions. Besides, we propose a cross gross-tuning strategy that performs the original fine-tuning and our AD-DROP alternately to stabilize the fine-tuning process. Extensive experiments and analysis on the GLUE benchmark demonstrate the effectiveness of AD-DROP. Although originally proposed and evaluated based on self-attention attribution, AD-DROP can be potentially extended other neural network units as vanilla dropout, which deserves further research efforts.
Acknowledgments
We appreciate the anonymous reviewers for their valuable comments. We thank Feifan Yang, Zihong Liang, and Hong Ding for their early discussions. This work was supported by the National Natural Science Foundation of China (No. 62176270), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (No. 20172T07X355), and the Foundation of Key Laboratory of Machine Intelligence and Advanced Computing of the Ministry of Education.
References
[1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 18101 10.04805 2018.
10

---

[2] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach. arXiv preprint arXiv: 1907. 1692, 2019.
[3] Tom Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877-1901, 2020.
[4] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems, pages 5998-6008, 2017.
[5] Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontañón, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, viriual, 2020.
[6] Sebastian Jaszezur, Aakanksha Chowdhery, Afroz Mohiuddin, Lukasz Kaiser, Wojciech Gajew- ski, Henryk Michalewski, and Jonni Kanerva. Sparse is enough in scaling transformers. In Advances in Newral Information Processing Systems 34: Annual Conference on Neural Informa- fion Processing Systems 1021, NeurlPS 2021, December 6-14, 2021, virtual, pages 2021.
[7] Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? Advances in Neural Information Processing Systems, 32, 2019.
[8] Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. Revealing the dark secrets of bert. In Proceedings of the 2019 Conference Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) pages 4365 4374, 2019.
[9] Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. What does bert look at? an analysis of bert's attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276-2866 286, 2019.
[10] Yihe Dong, Jean- Man--Bapistt Cordonnier, and Andreas Loukas. Attention not all you need: Pure attention loses rank doubly exponentially with depth. In International Conference on Machine Learning, pages 2793-2803. PMLR, 2021.
[11] Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang. Mixout: Effective regularization to finetune large-scale pretrained language models. In International Conference on Learning Representations, 2019.
[12] Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow, On the stability of fine- tuning bert: Misconceptions, explanations, and strong baselines. International Conference on Learning Representations, 2020.
[13] Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang. Raise child in large language model: Towards effective and generalizable fine-tuning. In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pages 9528, 2021.
[14] Jacob Andreas. Good-enough compositional data augmentation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7556-7566, 2020.
[15] Jiaao Chen, Dinghan Shen, Weizhu Chen, and Diyi Yang. Hiddencut: Simple data augmentation for natural language understanding with better generalizability. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Pupers), pages 4380-4390. 2021.
11

---

[16] Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. arXiv preprint arXiv:2004. 08994,, 2020.
[17] Deshui Miao, Jiaqi Zhang, Wenbo Xie, Jian Song, Xin Li, Lijuan Jia, and Ning Guo. Simple contrastive representation adversarial learning for nip tasks. arXiv preprint arXiv:2111.3330 2021.
[18] Lijun Wu, Juntao Li, Yue Wang. Qi Meng, Tao Qin, Wei Chen, Min Zhang, Tie- Yan Liu, et al. R-drop: regularized dropout for neural networks. In Advances in Neural Information Processing Systems, volume 34, 2021.
[19] Nitish Srivastava, Geoffre Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929-199 1958, 2014.
[20] Yarin Gal, Jiri Hron, and Alex Kendall. Concrete dropout. In Proceedings of the 31st Interna- lional Conference on Neural Information Processing Systems, pages 3584-3593, 2017.
[21] Golnaz Ghiasi, Tsung-Yi Yi Lin, and Quoe V Le. Dropblock: A regularization method for convolutional networks. Advances in Neural Information Processing Systems, volume 31, 2018.
[22] Hieu Pham and Quoc Le. Autodropout: Learning dropout patterns to regularize deep networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1359,9359, 2021.
[23] Yaru Hao, Li Dong. Furu Wei, and Ke Xu. Self-attention attribution: Interpreting information interactions inside transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages
2021.
[24] Yu Zhang, Peter Tiño, Ales Leonardis, and Ke Tang. A survey neural network interpretability. IEEE Transactions on Emerging Topics in Computational Intelligence, 2021.
[25] David Bachrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller. How to explain individual classification decisions. The Journal of Machine Learning Research, 11:1803-1831, 2010.
[26] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319-3328, PMLR, 2017
[27] Sarthak Jain and Byron C Wallace. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 3543-3556 2019.
(28) Kaiji Lu, Zifan Wang, Piotr Mardziel, and Anupam Datta. Influence patterns for explaining information flow in bert. In Advances in Neural Information Processing Systems, volume 34, 2021.
[29] Kevin Clark, Minh- Shah-Thang Luong, Quoc V Le, and Christopher D Manning. Electra: Pre-training text encoders discriminators rather than generators. arXiv preprint 200.:20000..0555, 2020.
[30] Jörg Tiedemann and Santhosh Thottingal. OPUS-MT Building open translation services for the World. In Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT), Lisbon, Portugal, 2020.
[31] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv: 1804.07461, 2018.
[32] Erik Tjong Kim Sang and Fien De Meulder. Introduction to the con1l-2003 shared task: manguage-independenn named entity recognition. Proceedings of the Seventh Conference on Natural Language Learning HLT-NAACL 2003, pages 142-147, 2003.
12

---

[33] Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz. al. Findings of the 2016 conference on machine translation. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 131-198, 2016.
[34] R Thomas McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. arXiv preprint arXiv: 192.000000200077 2019.
[35] Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. Paws-x: A lingull adversarial dataset for paraphrase identification. arXiv preprini arXiv: 1908. 1828, 2019.
[36] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In In Workshop at International Conference on Learning Representations. Citeseer, 2014.
[37] William B Dolan and Chris Brockett. Automatically constructing corpus of sentential para- phrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005.
[38] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, 2013.
[39] Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv: 2017.
[40] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606 052500 2016.
[41] Zihan Chen, Hongbo Zhang, Xiaoji Zhang, and Leqi Zhao. Quora question pairs. URI https://www. kaggle.
catpouoorr-uusttooo-uoee 2018.
[42] Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. Cola: The corpus of linguistic acceptability (with added annotations). 2019.
[43] Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio. and Lucia Specia. Semeval-2017 task I: Semantic textual preprint arXiv: 00055, 2017.
and cross-lingual focused evaluation. arXiv
[44] Luisa Bentivogli. Peter Clark, Ido Dagan, and Danilo Giampiccolo. The fifth pascal recognizing textual entailment challenge. In TAC, 2009.
[45] Brian W Matthews. Comparison of the predicted and observed secondary structure of 14 phage lysozyme. Biochimica et Biophysica Acta
Structure, 405(2):442-451, 1975.
[46] Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. In Noise reduction in speech processing, pages 1-4. Springer, 2009.
[47] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: Sffte-of-the- art natural language processing. In Proceedings of the 2020 Conference
Empirical Methods in Natural Language Processing: System Demonstrations, pages 38 45, Online, October 2020. Association for Computational Linguistics. URL https: /ww. aclweb. org/ anthology/
2020. emn emnlp-demos 6
[48]
Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, and Weizhu Chen. Super tickets in pre-trained language models: From model compression to improving generalization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1hth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages
2021.
13

---

[49] Li Wan, Matthew Zeiler, Sixin Zhang, Yann LeCun, and Rob Fergus. Regularization of neural networks using dropeonnect. In Proceedings of the 30th International Conference on International Conference on Machine Learning-Volle Volume 28, pages 111-1058, 2013.
[50] Jiwei Li, Will Monroe, and Dan Jurafsky. Understanding neural networks through representation erasure. arxiv preprint 1rI1v:1612.082200 2016.
[51] Shi Feng, Eric Wallace, Alvin Grissom II, Mohit lyyer, Pedro Rodriguez, and Jordan Boyd- Graber. Pathologies of neural models make interpretations difficult. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3719-3728, 2018.
[52] Karl Schulz, Leon Sixt, Federico Tombari, and Tim Landgraf. Restricting the flow: Information bottlenecks for attribution. In International Conference on Learning Representations, 2020. URL https::s///peerreiieeeee/ htt.://w.....tooommmmaoaaaaa111.
[53] Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Woj- ciech Samek. Layer-wise relevance propagation for neural networks with local renormalization layers. In International Conference Artificial Neural Networks, pages 63-71. Springer, 2016.
[54] Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, and Phil Blun- som. Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.000 2015.
[55] Sofia Serrano and Noah A Smith. Is attention interpretable? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2931-2951,200 2019.
[56] Sarah Wiegreffe and Yuval Pinter. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Lunguage P Processing (EMNLP-ICCNLP), pages 11 11-20, 2019.
[57] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv: 1 131.60334 2013.
[58] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6804 2014.
[59] Gino Brunner, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, and Roger Wattenhofer. On identifiability transformers. In 8th International Conference on Learning Representations (ICLR virtaa), International Conference on Learning Representations, 2020.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] Our contributions and scope are accurately reflected in the abstract and introduction.
(b) Did you describe the limitations of your work? [Yes] We discuss the potential limita- tions of our work in Appendix E.
(c) Did you discuss any potential negative societal impacts of your work? [No] Our work a general training method and will not bring any negative societal impact.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] We have read the guidelines and ensured our paper conforms to them.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material as URL)? [Yes] We have submitted our code and data in the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We describe our training details in Section 3.2 and Appendix C.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] We conduct repeated experiments in Section 4.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We describe the type of GPUs used in this work in Section 3.2
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We provide appropri- ate citations for the used data and tools.
(b) Did you mention the license of the assets? [No] All used datasets are publicly available.
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
15


Reviews:
[{"review_id": "pfJXd9jP-GU", "reviewer": "Reviewer_XQcd", "summary": "This paper introduces AdDrop, a novel dropout approach which chooses only high-attribution components of attention to apply dropout to. This paper presents experiments on the GLUE benchmark using BERT base and RoBERTa base, and does an excellent job of presenting experiments, ablations, and hyperparameter tuning to showcase the positives and negatives of the proposed approach.", "questions": "I think where you say \"drop by random sampling\" you mean \"drop by uniform sampling\"?", "limitations": "The authors appropriately listed limitations, though it could be made more explicit that the experimental evidence only covers BERT base and RoBERTa base, so the intended use here likely just covers similar pretrained language models during fine-tuning. Further experiments are necessary to evaluation if this works with other models or other types of data.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThis paper has many strengths, including the clarity of the writing, the ablations motivation the different experimental choices, and well-chosen baselines (though there could be more baselines, there are so many variants of dropout now it's not possible to cover all baselines here).\nThe gains presented in Table 1 are surprisingly consistent. My initial thought was that we should have some presentation of the variance (e.g., from random seed), then I found Table 3 which shows exactly that. Table 3 also shows that the variance in the AD-Drop experiments is lower than in the BERT and RoBERTa experiments, which is a benefit of the approach that the authors could emphasize further.\nI really like the hyperparameter sensitivity results in Figure 6, as I was reading I was hoping to see something like this. It's clear that RoBERTa has less benefit, but that there is still a range of q for which performance improves.\nThe appendix is also great, I love that the authors reported hyperparameters, the size of the GLUE datasets, and more experiments. \nThe transparency here is great, and I believe this paper has a lot of necessary reproducibility information.\n\nWeaknesses:\nIt's clear to me that the authors appropriately allocated their limited budget to provide experimental evidence supporting their claims that this approach improves performance, so I don't count this as a negative in my review, but to increase adoption of AD-Drop in the community the authors may consider running experiments with larger models (e.g., BERT large and RoBERTa large are more standard). The results may look different; sometimes increased scale means more regularization is necessary, sometimes it means the models are more stable.\nI would recommend having the caption of each figure be descriptive enough that it can be considered stand-alone (without also reading the main body of the paper). For example, the caption to Figure 2 should say the results are for MRPC, and Figure 4 should state what p and q are. It's better to have the main takeaway for each figure in the caption as well (\"with cross-tuning\" leads to much lower variance and higher performance).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Z6bUS5ZU5LC", "reviewer": "Reviewer_VyWF", "summary": "This work proposes AD-DROP to alleviate overfitting. AD-DROP involves four steps. First, predictions are made through a  forward computation without dropping any attention position. Second, compute the attribution score of each attention position by attribution methods. Third,  randomly sample a set of positions with high attribution scores and generate a masking matrix for each attention map. Finally, the masking matrices are applied to the next forward computation to make predictions for backpropagation.\n\nIn experiments, it is shown that AD-DROP is stable, and gives visible improvements over strong regularization baselines.", "questions": "Question:\nIn equation6, what is n?\nI don't quite get the intuition behind cross-tuning, why not just use a smaller dropout probability?\nIn Table 1, why is the baselines in bert and roberta different?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\nThe method is novel in that it involves attribution methods. Instead of pure random.\nIn experiments it outperform some strong baselines. \n\nWeakness:\nExperiments maybe not complete.(refer to my questions)\nSince AD-Drop could prevent overfitting, I'd like to see how it performs in few-shot learning. (Sec4.2 is helpful, but I think it's better to see more datasets)\nThe attribution method would cost some computation.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "J4mKzzAs1M", "reviewer": "Reviewer_PC5L", "summary": "This paper presents a new fine-tuning strategy called AD-DROP to prevent overfitting for PLM on language understanding tasks based on the attribution scores on attention positions. The idea is intuitive that dropping attention positions with low attribution scores increases the chance of overfitting. Based on this observation, this paper proposes to randomly discard some of the attention distribution with high attribution scores to make the model exploit more information from low attribution positions thus it is less likely to overfit. The empirical results on NLU tasks demonstrate the success of the proposed strategy, especially on small datasets.", "questions": "See Weakness.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- This paper is well-written and easy to understand.\n- The proposed method is simple and easy to apply.\n- The empirical results on NLU tasks look promising compared to full fine-tuning approaches. \n\n\n**Weakness**\n- Limited tasks and analysis. Experiments are only conducted on GLUE, which leaves questions on the applicability of the proposed approach. Other than the aspect indicated by the authors that the AD-DROP could potentially be applied to different units in the model, there are still multiple dimensions that could have been considered to make the evaluation of the approach more thorough, convincing, and insightful. For example, 1) only Masked PLMs are considered; what happens if the base model is an autoregressive LM? 2) Other than NLU tasks, can it be applied to generation tasks? 3) How the approach scales with model sizes? In terms of overfitting, both limited data and large model size could be the source and this should also be investigated. \n- The motivation for including the hyper-parameter $q$ is unclear. I understood it is used to control the strength of enforcing the masking to the softmax, but isn't this sort of covered by the $p$? It seems to me that a random sampling based on uniform distribution to decide the positions (in addition to Eq 6) could do the job. Including $q$ seems to require much more hyper-parameter tuning. Besides, no description of $q$ is given around Line 145, I had to guess what $q$ is.\n- Some confusion about the experiments/analysis\n> 1. In Table 1, the improvement of R-Drop with RoBERTa Base seems quite limited compared to the improvement with RoBERTa Large on the original paper. I assume the authors implemented and ran the R-Drop with RoBERTa. However, there is no description of how they chose the hyper-parameter for R-Drop and the search space for that.\n>2. Comparing Table 3 to Table 1, it looks like the variance of the improvement is rather notable. For example, on RTE with BERT, the improvement reported in Table 1 is **4.3** while when averaging over multiple rounds, the improvement in Table 3 is **2.7**.  It looks to me that it is more reasonable to report averaging numbers in Table 1 rather than the best possible numbers that are ever achieved?  \n>3. In Table 4, I may misunderstand something here. It looks to me that it is almost impossible that the computation cost of AD-DROP on STS-B is the same as FT. AD-DROP needs **TWO** forward passes of the model. It requires running a **forward pass** of the model first to obtain the pseudo label as well as the **attribution scores computation**. Lastly, the model will run the forward pass again for back-propagation. On the other hand, FT only requires **ONE** forward pass and back-propagation. Would you clarify how the computational cost is defined and calculated here such that they are the same?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4EEOma_0WoM", "reviewer": "Reviewer_XaQX", "summary": "* The paper presents a novel variant of dropout for self-attention layers in Transformers, by dropping self attention units with high attribution scores. The authors motivate this by showing that dropping low attribution scores leads to faster overfitting, and by dropping high attribution units, the model tends to overfit less. \n* The authors also present cross-tuning (alternatively training with and without AD-Drop across different epochs) as a method to counter excessively dropping high attribution scores\n* Results presented on GLUE benchmark shows that the AD Drop helps improve performance over vanilla dropout methods\n* The authors also present additional ablation experiments, improvements across varying dataset sizes, sensitivity to choices of hyper-parameters and compute efficiency analysis of the proposed method.\n\n[Update]\nUpdated scores based on author response.", "questions": "Questions:\n* Why is cross-tuning done at an epoch level ? How do the results change if it were done at a say minibatch level ? \n* Given that size of STS-B is smaller than CoLA, why was AD-Drop only applied to the first layer for STS-B (Line 175) ? ", "limitations": "I think including the limitations section as a part of the main paper, as opposed to the last section of the Appendix, would be more informative in terms of informing the readers of the limitations of the work. Additionally, in it's current form, one of the big limitations that is not addressed is the lack of diversity of fine-tuning tasks: if the proposal of the paper is a new general purpose regularization method, I think not having additional tasks is a big limitation.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper presents a novel regularization method. The idea of leveraging saliency maps to inform which units to drop is quite interesting\n* The results for base model training show decent improvements over baseline methods. \n\nWeaknesses\n* Given that the paper presents AD Drop as a new regularization method, the results would be more convincing if carried out on a diverse set of tasks. Right now, the results presented are only for classification tasks. Even if this was for encoder only models, showing results on a token level task (eg: SQuAD QnA, NER etc) would help show the generality of the proposed method. Eg: [1] also propose a novel regularization methods, and demonstrate it's applicability across Machine Translation, Summarization, GLUE, Language Modeling and Image Classification. \n* While the presented method is quite interesting, the authors don't address _why_ it works. If the hypothesis is that dropping high attribution self-attention units makes the model rely less on spurious features, it would be good to show that the proposed method does better on OOD datasets (eg: HANS for MNLI, PAWS-X for QQP etc). \n* It would be good to report mean of 4 / 5 seeds for all the tasks (not just the small tasks). From Table 1, if we account for the mean of 4 seeds for small tasks, then the overall gain from AD-Drop reduces to 84.7 (taking results from Table 3). Thus the gains might be a bit over-stated, and it would be good to account for the variance due to random seeds in the results.\n* It is very hard to understand the impact of different hyper-parameter values from Figure 4. Instead, it might be more informative to plot the accuracy as a function of the parameter values for with and without cross-tuning.\n* Since that the compute cost can go up by as much as 4.5x, I think it would be informative to compare against other (non-dropout related) strategies for regularization (eg: Sharpness Aware Minimization; see [2]). For [2], the authors show consistent gains at an 2x performance hit (not considering the efficient SAM results). Thus, it would be useful to see how the proposed method compares against that. \n* [Minor] The description of candidate discard region is a bit misleading (Line 140). If my understanding is correct, then candidate discard region S_{i,j} = 1implies the self-attention logit is kept, and not discarded.\n\nReferences\n\n[1] Wu, Lijun, et al. \"R-drop: Regularized dropout for neural networks.\" Advances in Neural Information Processing Systems 34 (2021): 10890-10905.\n\n[2] Bahri, Dara, Hossein Mobahi, and Yi Tay. \"Sharpness-aware minimization improves language model generalization.\" arXiv preprint arXiv:2110.08529 (2021).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept