Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces AdDrop, a novel dropout approach which chooses only high-attribution components of attention to apply dropout to. This paper presents experiments on the GLUE benchmark using BERT base and RoBERTa base, and does an excellent job of presenting experiments, ablations, and hyperparameter tuning to showcase the positives and negatives of the proposed approach.
strengths_and_weaknesses: Strengths:
This paper has many strengths, including the clarity of the writing, the ablations motivation the different experimental choices, and well-chosen baselines (though there could be more baselines, there are so many variants of dropout now it's not possible to cover all baselines here).
The gains presented in Table 1 are surprisingly consistent. My initial thought was that we should have some presentation of the variance (e.g., from random seed), then I found Table 3 which shows exactly that. Table 3 also shows that the variance in the AD-Drop experiments is lower than in the BERT and RoBERTa experiments, which is a benefit of the approach that the authors could emphasize further.
I really like the hyperparameter sensitivity results in Figure 6, as I was reading I was hoping to see something like this. It's clear that RoBERTa has less benefit, but that there is still a range of q for which performance improves.
The appendix is also great, I love that the authors reported hyperparameters, the size of the GLUE datasets, and more experiments. 
The transparency here is great, and I believe this paper has a lot of necessary reproducibility information.

Weaknesses:
It's clear to me that the authors appropriately allocated their limited budget to provide experimental evidence supporting their claims that this approach improves performance, so I don't count this as a negative in my review, but to increase adoption of AD-Drop in the community the authors may consider running experiments with larger models (e.g., BERT large and RoBERTa large are more standard). The results may look different; sometimes increased scale means more regularization is necessary, sometimes it means the models are more stable.
I would recommend having the caption of each figure be descriptive enough that it can be considered stand-alone (without also reading the main body of the paper). For example, the caption to Figure 2 should say the results are for MRPC, and Figure 4 should state what p and q are. It's better to have the main takeaway for each figure in the caption as well ("with cross-tuning" leads to much lower variance and higher performance).

questions: I think where you say "drop by random sampling" you mean "drop by uniform sampling"?
limitations: The authors appropriately listed limitations, though it could be made more explicit that the experimental evidence only covers BERT base and RoBERTa base, so the intended use here likely just covers similar pretrained language models during fine-tuning. Further experiments are necessary to evaluation if this works with other models or other types of data.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: * The paper presents a novel variant of dropout for self-attention layers in Transformers, by dropping self attention units with high attribution scores. The authors motivate this by showing that dropping low attribution scores leads to faster overfitting, and by dropping high attribution units, the model tends to overfit less. 
* The authors also present cross-tuning (alternatively training with and without AD-Drop across different epochs) as a method to counter excessively dropping high attribution scores
* Results presented on GLUE benchmark shows that the AD Drop helps improve performance over vanilla dropout methods
* The authors also present additional ablation experiments, improvements across varying dataset sizes, sensitivity to choices of hyper-parameters and compute efficiency analysis of the proposed method.

[Update]
Updated scores based on author response.
strengths_and_weaknesses: Strengths:
* The paper presents a novel regularization method. The idea of leveraging saliency maps to inform which units to drop is quite interesting
* The results for base model training show decent improvements over baseline methods. 

Weaknesses
* Given that the paper presents AD Drop as a new regularization method, the results would be more convincing if carried out on a diverse set of tasks. Right now, the results presented are only for classification tasks. Even if this was for encoder only models, showing results on a token level task (eg: SQuAD QnA, NER etc) would help show the generality of the proposed method. Eg: [1] also propose a novel regularization methods, and demonstrate it's applicability across Machine Translation, Summarization, GLUE, Language Modeling and Image Classification. 
* While the presented method is quite interesting, the authors don't address _why_ it works. If the hypothesis is that dropping high attribution self-attention units makes the model rely less on spurious features, it would be good to show that the proposed method does better on OOD datasets (eg: HANS for MNLI, PAWS-X for QQP etc). 
* It would be good to report mean of 4 / 5 seeds for all the tasks (not just the small tasks). From Table 1, if we account for the mean of 4 seeds for small tasks, then the overall gain from AD-Drop reduces to 84.7 (taking results from Table 3). Thus the gains might be a bit over-stated, and it would be good to account for the variance due to random seeds in the results.
* It is very hard to understand the impact of different hyper-parameter values from Figure 4. Instead, it might be more informative to plot the accuracy as a function of the parameter values for with and without cross-tuning.
* Since that the compute cost can go up by as much as 4.5x, I think it would be informative to compare against other (non-dropout related) strategies for regularization (eg: Sharpness Aware Minimization; see [2]). For [2], the authors show consistent gains at an 2x performance hit (not considering the efficient SAM results). Thus, it would be useful to see how the proposed method compares against that. 
* [Minor] The description of candidate discard region is a bit misleading (Line 140). If my understanding is correct, then candidate discard region S_{i,j} = 1implies the self-attention logit is kept, and not discarded.

References

[1] Wu, Lijun, et al. "R-drop: Regularized dropout for neural networks." Advances in Neural Information Processing Systems 34 (2021): 10890-10905.

[2] Bahri, Dara, Hossein Mobahi, and Yi Tay. "Sharpness-aware minimization improves language model generalization." arXiv preprint arXiv:2110.08529 (2021).
questions: Questions:
* Why is cross-tuning done at an epoch level ? How do the results change if it were done at a say minibatch level ? 
* Given that size of STS-B is smaller than CoLA, why was AD-Drop only applied to the first layer for STS-B (Line 175) ? 
limitations: I think including the limitations section as a part of the main paper, as opposed to the last section of the Appendix, would be more informative in terms of informing the readers of the limitations of the work. Additionally, in it's current form, one of the big limitations that is not addressed is the lack of diversity of fine-tuning tasks: if the proposal of the paper is a new general purpose regularization method, I think not having additional tasks is a big limitation.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper presents a new fine-tuning strategy called AD-DROP to prevent overfitting for PLM on language understanding tasks based on the attribution scores on attention positions. The idea is intuitive that dropping attention positions with low attribution scores increases the chance of overfitting. Based on this observation, this paper proposes to randomly discard some of the attention distribution with high attribution scores to make the model exploit more information from low attribution positions thus it is less likely to overfit. The empirical results on NLU tasks demonstrate the success of the proposed strategy, especially on small datasets.
strengths_and_weaknesses: **Strengths**
- This paper is well-written and easy to understand.
- The proposed method is simple and easy to apply.
- The empirical results on NLU tasks look promising compared to full fine-tuning approaches. 


**Weakness**
- Limited tasks and analysis. Experiments are only conducted on GLUE, which leaves questions on the applicability of the proposed approach. Other than the aspect indicated by the authors that the AD-DROP could potentially be applied to different units in the model, there are still multiple dimensions that could have been considered to make the evaluation of the approach more thorough, convincing, and insightful. For example, 1) only Masked PLMs are considered; what happens if the base model is an autoregressive LM? 2) Other than NLU tasks, can it be applied to generation tasks? 3) How the approach scales with model sizes? In terms of overfitting, both limited data and large model size could be the source and this should also be investigated. 
- The motivation for including the hyper-parameter $q$ is unclear. I understood it is used to control the strength of enforcing the masking to the softmax, but isn't this sort of covered by the $p$? It seems to me that a random sampling based on uniform distribution to decide the positions (in addition to Eq 6) could do the job. Including $q$ seems to require much more hyper-parameter tuning. Besides, no description of $q$ is given around Line 145, I had to guess what $q$ is.
- Some confusion about the experiments/analysis
> 1. In Table 1, the improvement of R-Drop with RoBERTa Base seems quite limited compared to the improvement with RoBERTa Large on the original paper. I assume the authors implemented and ran the R-Drop with RoBERTa. However, there is no description of how they chose the hyper-parameter for R-Drop and the search space for that.
>2. Comparing Table 3 to Table 1, it looks like the variance of the improvement is rather notable. For example, on RTE with BERT, the improvement reported in Table 1 is **4.3** while when averaging over multiple rounds, the improvement in Table 3 is **2.7**.  It looks to me that it is more reasonable to report averaging numbers in Table 1 rather than the best possible numbers that are ever achieved?  
>3. In Table 4, I may misunderstand something here. It looks to me that it is almost impossible that the computation cost of AD-DROP on STS-B is the same as FT. AD-DROP needs **TWO** forward passes of the model. It requires running a **forward pass** of the model first to obtain the pseudo label as well as the **attribution scores computation**. Lastly, the model will run the forward pass again for back-propagation. On the other hand, FT only requires **ONE** forward pass and back-propagation. Would you clarify how the computational cost is defined and calculated here such that they are the same?
questions: See Weakness.
limitations: Yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This work proposes AD-DROP to alleviate overfitting. AD-DROP involves four steps. First, predictions are made through a  forward computation without dropping any attention position. Second, compute the attribution score of each attention position by attribution methods. Third,  randomly sample a set of positions with high attribution scores and generate a masking matrix for each attention map. Finally, the masking matrices are applied to the next forward computation to make predictions for backpropagation.

In experiments, it is shown that AD-DROP is stable, and gives visible improvements over strong regularization baselines.
strengths_and_weaknesses: Strength:
The method is novel in that it involves attribution methods. Instead of pure random.
In experiments it outperform some strong baselines. 

Weakness:
Experiments maybe not complete.(refer to my questions)
Since AD-Drop could prevent overfitting, I'd like to see how it performs in few-shot learning. (Sec4.2 is helpful, but I think it's better to see more datasets)
The attribution method would cost some computation.

questions: Question:
In equation6, what is n?
I don't quite get the intuition behind cross-tuning, why not just use a smaller dropout probability?
In Table 1, why is the baselines in bert and roberta different?
limitations: Yes.
review_title: null

Meta-review: metareview: The paper proposes a method AD-DROP to drop attention weights in a network to alleviate overfitting.  
It randomly sample a set of token positions with respect to attribution score calculated in first pass. 
The authors provide a variety of experiments on multiple tasks (SNLI, NER, MT, etc.) showing effectiveness comparing to other methods. The method is slower since it needs a separate pass to calculate attention attribution. 


confidence: Less certain
Decision: Accept