RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Preliminaries
Since Transformers [ 4 ] are the backbone of PrLMs, we first review the details of self-attention in Transformers and self-attention attribution [ 23 ] . Let {\bf{X}}\in{\mathbb{R}^{n\times d}} be the input of a Transformer block, where n is the sequence length and d is the embedding size. Self-attention in this block first maps \bf{X} into three matrices {\bf{Q}}_{h} , {\bf{K}}_{h} and {\bf{V}}_{h} via linear projections as query, key, and value respectively for the h -th head. Then, the attention output of this head is calculated as:

{\rm{Attention}}\left({{\bf{Q}}_{h},{\bf{K}}_{h},{\bf{V}}_{h}}\right)={{\bf{A}}_{h}}{{\bf{V}}_{h}}={\rm{softmax}}\left({\frac{{{\bf{Q}}_{h}{{{\bf{K}}_{h}}^{\rm{T}}}}}{{\sqrt{d_{k}}}}}+{\bf{M}}_{h}\right){\bf{V}}_{h}, (1)

where \sqrt{d_{k}} is a scaling factor. {\bf{M}}_{h} is the mask matrix to apply dropout in self-attention, and elements in {\bf{M}}_{h} will be -\infty if the corresponding positions in attention maps are masked and 0 otherwise.

Based on the attention maps {\bf{A}}=\left[{{\bf{A}}_{1},{\bf{A}}_{2},\cdots,{\bf{A}}_{H}}\right] for H attention heads, gradient attribution [ 25 , 36 ] directly produces an attribution matrix \bf{B}_{h} by computing the following partial derivative:

{\bf{B}}_{h}={\frac{\partial{F}_{c}\left({\bf{A}}\right)}{{\partial{\bf{A}}_{h}}}}, (2)

where F_{{c}}{\left({\cdot}\right)} denotes the logit output of the Transformer for class c .

To provide a theoretically more sound attribution method, Sundararajan et al. [ 26 ] propose integrated gradient, which is employed by Hao et al. [ 23 ] as a saliency measure for self-attention attribution. Specifically, Hao et al. [ 23 ] compute the attribution matrix {\bf{B}}_{h} as:

{{{{\bf{B}}}}_{h}}=\frac{{{\bf{A}}_{h}}}{m}\odot\sum\limits_{k=1}^{m}{\frac{{\partial{F}_{c}\left({\frac{k}{m}{\bf{A}}}\right)}}{{\partial{\bf{A}}_{h}}}}, (3)

where m is the number of steps for approximating the integration in integrated gradient, and \odot is the element-wise multiplication operator. Despite its theoretical advantage over gradient attribution, integrated gradient requires m times more computational effort, which is especially expensive when it is applied to all the attention heads in Transformers. Moreover, our experiments in Section 3.4 show that gradient attribution achieves comparable performance with integrated gradient but requires much less computational cost, suggesting that gradient attribution is more desirable for AD-Drop .
A Prior Attribution Experiment
To better motivate our work, we first conduct a prior experiment on MRPC [ 37 ] to investigate how different positions in self-attention maps affect fine-tuning performance based on attribution results. RoBERTa_base [ 2 ] is used as the base model. To begin with, we first perform a forward computation of the model on each batch of training samples to obtain the logit output of each sample corresponding to the gold label. Then, we obtain an attribution matrix {\bf{B}}_{h} for the self-attention positions in the first layer 1 1 1 We provide more results and discussions in Appendix B . by gradient attribution with Eq. ( 2 ) and sort each row of the matrix. Finally, we sample a set of self-attention positions with high or low attribution scores in each row to generate a mask matrix {\bf{M}}_{h} , which is fed into Eq. ( 1 ) to make the final predictions. After each epoch of training, we evaluate the model on the development set. Two baseline dropping strategies (i.e., dropping by random sampling and without dropping any position) are employed for comparison. We plot the loss curves of the model with these dropping strategies on both training and development sets in Figure 2 . The observations are threefold. First, dropping low-attribution positions makes the model fit the training data rapidly, whereas it performs poorly on the development set, indicating that the model is not properly trained. Second, compared with the other dropping strategies, dropping high-attribution positions reduces the fitting speed significantly. Third, random dropping only slightly reduces overfitting, compared to the training without dropping. These observations suggest that attention positions are of different importance in preventing overfitting. We conjecture that low-attribution positions are more difficult to optimize than high-attribution positions. While dropping low-attribution positions tends to accelerate overfitting, discarding high-attribution positions helps reduce overfitting.
Discussion
The reasons that AD-Drop uses pseudo labels for attribution are twofold. First, adopting gold labels will divulge label information and lead to inconsistency between training and inference. Second, for misclassified samples in the first forward computation, AD-Drop with gold labels tends to continue to make incorrect predictions because high-attribution attention positions derived from gold labels may be located in low-attribution regions derived from pseudo labels. Therefore, dropping these positions does not help the model correct wrong predictions, while AD-Drop with pseudo labels urges the model to rely on important features in the current pass and may correct the wrong predictions. The attribution with gold labels will be investigated in Section 3.4 .
Cross-Tuning Algorithm
We further design a cross-tuning algorithm to avoid dropping high-attribution positions excessively when applying AD-Drop . The idea of cross-tuning is to execute the original fine-tuning and AD-Drop alternatively. Specifically, it performs the original fine-tuning at odd epochs and AD-Drop at even epochs. The overall process of cross-tuning is described in Algorithm 1 , where Lines 3-5 are the original fine-tuning operations and Lines 7-9 describe the process of AD-Drop .

Input: shuffled training samples \mathcal{D}=\left\{{\left({{x_{i}},{y_{i}}}\right)}\right\}_{i=1}^{N} , PrLM F with parameters \bf W Output: updated parameters {\bf{\widetilde{W}}}
Datasets
We conduct our main experiments on eight tasks of the GLUE benchmark [ 31 ] , including SST-2 [ 38 ] , MNLI [ 39 ] , QNLI [ 40 ] , QQP [ 41 ] , CoLA [ 42 ] , STS-B [ 43 ] , MRPC [ 37 ] , and RTE [ 44 ] . The evaluation metrics are Matthew’s Corrcoef (Mcc) [ 45 ] for CoLA, Pearson Corrcoef (Pcc) [ 46 ] for STS-B, and Accuracy (Acc) for the others. To demonstrate that AD-Drop applies to token-level tasks as well, we conduct experiments on Named Entity Recognition (CoNLL-2003 [ 32 ] ) and Machine Translation (WMT 2016 [ 33 ] ) datasets, the results of which are shown in Appendix A.2 . Besides, we also evaluate AD-Drop on two out-of-distribution (OOD) datasets, including HANS [ 34 ] and PAWS-X [ 35 ] . The details of these datasets are introduced in Appendix C.1 .
Implementation Details
We implement our AD-Drop in Pytorch with the Transformers package [ 47 ] . We train the selected PrLMs on GeForce RTX 3090 GPUs. We tune the learning rate in {1e-5, 2e-5, 3e-5} and the batch size in {16, 32, 64}. Following Miao et al. [ 17 ] , we perform early stopping to choose the number of training epochs on GLUE. The two critical hyperparameters p and q are searched within [0.1,0.9] with step size 0.1. For integrated gradient in Eq. ( 3 ), we follow Hao et al. [ 23 ] and set m to 20. We apply AD-Drop only in the first layer for the datasets of SST-2, MNLI, QNLI, QQP, and STS-B since the fine-tuning on these datasets is stable and less likely to cause overfitting. For the rest datasets, we apply AD-Drop in all layers. We provide the detailed hyperparameter settings on each dataset in Appendix C.2 . Our code is available at https://github.com/TaoYang225/AD-DROP .
Overall Results
We report the overall results of the fine-tuned models in Table 1 . We first compare AD-Drop with existing regularization methods on the development sets, including the original fine-tuning, SCAL [ 17 ] , SuperT [ 48 ] , R-Drop [ 18 ] , and HiddenCut [ 15 ] . We observe that AD-Drop surpasses the baselines on most of the datasets. Specifically, AD-Drop yields an average improvement of 1.98 and 1.29 points on BERT base and RoBERTa base , respectively. We then discuss the performance of AD-Drop on the test sets. Results in Table 1 show that AD-Drop achieves consistent improvement, boosting the average scores of BERT base and RoBERTa base by 0.87 and 0.62, respectively. Besides, compared with large datasets, AD-Drop achieves more gains on small datasets, which are more likely to cause overfitting, illustrating that AD-Drop is more suitable for small data scenarios.
Ablation Study
We conduct ablation experiments on four small datasets to investigate the impact of different components. Due to the limited number of submissions imposed by the GLUE server for evaluation, the results here are reported on the development sets.
Attribution methods
AD-Drop can be implemented with different attribution methods to generate the mask matrix in Eq. ( 1 ), such as integrated gradient attribution (IGA) introduced Eq. ( 3 ), attention weights for attribution (AA), and randomly generating the discard region (RD) in Eq. ( 6 ). We replace the gradient attribution (GA) in Eq. ( 5 )-( 6 ) with these methods. From Table 2 , we can make three observations. First, AD-Drop with gradient-based attribution methods (GA and IGA) surpasses that with the other methods (AA or RD) on most of the datasets, illustrating that gradient-based methods are better at finding features that are likely to cause overfitting. Second, IGA outperforms GA in some cases. Although IGA provides better theoretical justification than GA for attribution, it requires prohibitively more computational cost than GA (see Section 4.7 for efficiency analysis), making GA a more desirable choice for AD-Drop . Third, AD-Drop improves the original BERT base and RoBERTa base with any of the masking strategies, demonstrating the robustness of AD-Drop to overfitting when fine-tuning these models.
Pseudo labels vs gold labels
In Section 2.3 , we discuss the motivation of using pseudo labels for attribution in AD-Drop . To verify the reasonability, we conduct an experiment with gold labels for attribution. As the results show in Table 2 , using gold labels for attribution deteriorates the performance, illustrating that AD-Drop with pseudo labels for attribution is preferable.
Cross-tuning
To verify the effectiveness of the cross-tuning strategy, we ablate it and apply only AD-Drop in all training epochs. As shown in Table 2 , removing cross-tuning causes noticeable performance degradation on most of the datasets. This can be explained by the intuition that AD-Drop without cross-tuning tends to discard high-attribution positions excessively and make the model difficult to converge normally. To vividly demonstrate the effect of AD-Drop with or without cross-tuning, we visualize the distributions of the performance on the RTE 3 3 3 Results on the other datasets are shown in Appendix D.1 . development set when enumerating the parameters p and q in the range of [0.1, 0.9]. The results are plotted in Figure 4 , where each blue/orange point denotes the accuracy with a pair of p and q values. We observe from the figure that AD-Drop without cross-tuning cannot be trained properly under some parameter settings. However, it works well for most parameter settings when cross-tuning is applied, demonstrating that cross-tuning is vital for improving training stability.
Analysis
In this section, we further conduct several experiments for more thorough analysis.
Repeated Experiments
To reduce the influence of randomness, we conduct repeated experiments on four small datasets (i.e., CoLA, STS-B, MRPC and RTE). We repeat the training of each model with five random seeds and report the average score and standard deviation on the development sets. From Table 3 , we observe that AD-Drop outperforms the original fine-tuning on all the datasets. In addition, AD-Drop results in lower standard deviations on most of the datasets, showing that AD-Drop is more robust in fine-tuning PrLMs than the original approach.
Effect of Data Size
To study the impact of data size, we compare AD-Drop with the original fine-tuning (FT) approach on QNLI and QQP, 4 4 4 Results on QQP are shown in Appendix D.2 . two relatively large datasets, and report their performance when the number of training samples changes. RoBERTa is chosen as the base model. Figure 5 shows that AD-Drop

outperforms FT consistently on QNLI. Moreover, AD-Drop improves the efficiency of data use as training AD-Drop with 60% training samples produces comparable performance to FT with full data.
Hyperparameter Sensitivity
AD-Drop involves two hyperparameters p and q to control the number of discarded attention positions. To investigate the sensitivity of AD-Drop to them, we show the results of different p and q combinations on CoLA and RTE in Figure 6 , in which we apply MaxAbsScaler 5 5 5 https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MaxAbsScaler.html to project the difference between the results of AD-Drop and FT into the interval of [-1.0,1.0] . We observe that BERT with AD-Drop is not hyperparameter-sensitive as it outperforms the baseline under most settings. In contrast, RoBERTa with AD-Drop is more sensitive and requires a careful search for optimal hyperparameter settings. The possible reason is that RoBERTa is pre-trained with more data and more effective tasks than BERT, making it less prone to overfitting than BERT.
Larger Model Size
To verify the scalability of AD-Drop for a larger model size, we evaluate AD-Drop with RoBERTa large on the RTE and MRPC datasets. Table 4 shows the average scores and standard deviations of five random seeds. There are two main observations. First, AD-Drop achieves consistent improvements over the larger RoBERTa model, illustrating that AD-Drop is scalable to large models. Second, compared with RoBERTa base on RTE in Table 3 , the larger model significantly reduces the deviation (from 1.7 to 0.86), suggesting that a larger model size indeed helps to improve the stability. AD-Drop further improves the performance and reduces the deviation.
Few-shot Scenario
In this subsection, we test the performance of AD-Drop under few-shot scenarios. Specifically, we carry out 16-, 64-, and 256-shot experiments on SST-2 and CoLA with RoBERTa base as the base model and the baseline. We report the average scores and standard deviations of five random seeds in Table 5 . We observe that RoBERTa with AD-Drop consistently outperforms the original fine-tuning approach. Besides, AD-Drop tends to bring more benefits when fewer samples are available.
Out-of-Distribution Generalization
To further demonstrate AD-Drop is beneficial to reducing overfitting, we test AD-Drop with RoBERTa base on two out-of-distribution (OOD) datasets, i.e., HANS and PAWS-X. For HANS, we use the checkpoints trained on MNLI and test their performance on the validation set (the test set is not supplied). For PAWS-X, we use the checkpoints trained on QQP and examine its performance on the test set. The evaluation metric is accuracy. From Table 6 , we can see that RoBERTa with AD-Drop achieves better generalization, where AD-Drop boosts the performance by 0.66 on HANS and 3.35 on PAWS-X, illustrating that the model trained with AD-Drop generalizes better to OOD data.
Computational Efficiency
To analyze the computational efficiency, we quantitatively study the computational cost of AD-Drop with different dropping strategies (GA, IGA, AA, and RD) relative to the original fine-tuning on CoLA, STS-B, MRPC, and RTE. BERT is chosen as the base model for this experiment. As shown in Table 7 , although IGA achieves more favorable performance on one of the datasets, it requires higher computational costs than its counterparts, especially when applied in all the layers. In contrast, AD-Drop with GA is more competitive in both performance and computational cost.
Dropout
Dropout is a widely used regularizer to mitigate overfitting when training deep neural networks. Vanilla dropout [ 19 ] randomly selects neurons with a predefined probability and sets their values to zeros during training. By doing so, the neurons cannot co-adapt and the trained networks can lead to better generalization. In recent years, many variants of dropout have emerged. One body of research aims to adopt different strategies to drop units in neural networks. For example, DropConnect [ 49 ] randomly selects connections between neurons to discard. DropBlock [ 21 ] defines a structured dropout that randomly drops the units in a specific contiguous region of a feature map. AutoDropout [ 22 ] aims to improve the dropout pattern of DropBlock by introducing an automatic method to design dropout structures. HiddenCut [ 15 ] drops contiguous spans within the hidden space, in which the attention weights are utilized to select the dropped spans strategically.

Another body of research devotes to addressing the inconsistency between training and inference when dropout is applied. For instance, mixout [ 11 ] randomly replaces selected parameters with original pre-trained weights rather than setting them to zeros. CHILD-TUNING [ 13 ] selects a child network and masks out the gradients of the non-child network during the backward step, only updating weights in the child network. R-Drop [ 18 ] performs dropout twice in the forward steps to produce two sub-models and then applies KL-divergence for their output distributions, forcing the two sub-models to be consistent with each other. However, most of these methods follow the random sampling strategy of dropout and pay little attention to the different importance of self-attention positions in PrLMs.
Attribution
Numerous studies have been undertaken to interpret the behaviors of deep neural networks (DNNs). As a theory for interpretability, attribution aims to evaluate the impact of input features on predictions [ 24 ] . Generally, attribution methods can be divided into perturbation-based [ 50 , 51 , 52 ] , gradient-related [ 25 , 53 , 26 ] , and attention-based [ 54 , 55 , 56 ] methods. We focus on reviewing the gradient-related methods as they are more relevant to our work. Specifically, earlier works [ 25 , 57 , 58 ] try to explain model decisions via gradients since gradients indicate the direction and rate that changes the loss the fastest. However, Sundararajan et al. [ 26 ] point out that gradient attribution violates the sensitivity axiom in some cases that the gradients will be zero for the function in saturated areas, and propose integrated gradient as a theoretically more sound attribution method.

Other efforts have been devoted to revealing the behavior patterns of PrLMs. Kovaleva et al. [ 8 ] and Clark et al. [ 9 ] use attention weights for attribution to investigate what specific knowledge BERT [ 1 ] learns. Jain and Wallace [ 27 ] and Brunner et al. [ 59 ] investigate the identifiability of attention weights and conclude that attention weights are not a faithful explanation for model predictions. Hao et al. [ 23 ] apply integrated gradient [ 26 ] as a saliency measure for self-attention attribution in BERT, and use the attribution result to interpret information interactions inside Transformers. Similarly, Lu et al. [ 28 ] develop influence patterns based on integrated gradient to explain information flow in BERT. Unlike these works, we aim to examine the effect of dropout on self-attention through self-attention attribution and to reduce overfitting when fine-tuning PrLMs.
Conclusion
We propose a novel dropout regularizer, AD-Drop , to mitigate overfitting when fine-tuning PrLMs on downstream tasks. Unlike previous dropout-based methods that generally adopt the random sampling strategy to discard units, AD-Drop draws inspiration from self-attention attribution which reveals that attention positions are not equally important in reducing overfitting and that dropping inappropriate positions may exacerbate the problem. Therefore, AD-Drop focuses on discarding high-attribution attention positions to prevent the model from relying heavily on these positions to make predictions. Besides, we propose a cross-tuning strategy that performs the original fine-tuning and our AD-Drop alternately to stabilize the fine-tuning process. Extensive experiments and analysis on the GLUE benchmark demonstrate the effectiveness of AD-Drop . Although originally proposed and evaluated based on self-attention attribution, AD-Drop can be potentially extended to other neural network units as vanilla dropout, which deserves further research efforts.
Acknowledgments
We appreciate the anonymous reviewers for their valuable comments. We thank Feifan Yang, Zihong Liang, and Hong Ding for their early discussions. This work was supported by the National Natural Science Foundation of China (No. 62176270), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (No. 2017ZT07X355), and the Foundation of Key Laboratory of Machine Intelligence and Advanced Computing of the Ministry of Education.
Checklist
For all authors…

Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? [Yes] Our contributions and scope are accurately reflected in the abstract and introduction.

Did you describe the limitations of your work? [Yes] We discuss the potential limitations of our work in Appendix E .

Did you discuss any potential negative societal impacts of your work? [No] Our work is a general training method and will not bring any negative societal impact.

Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] We have read the guidelines and ensured our paper conforms to them.

If you are including theoretical results…

Did you state the full set of assumptions of all theoretical results? [N/A]

Did you include complete proofs of all theoretical results? [N/A]

If you ran experiments…

Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] We have submitted our code and data in the supplemental material.

Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We describe our training details in Section 3.2 and Appendix C .

Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] We conduct repeated experiments in Section 4.1 .

Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We describe the type of GPUs used in this work in Section 3.2 .

If you are using existing assets (e.g., code, data, models) or curating/releasing new assets…

If your work uses existing assets, did you cite the creators? [Yes] We provide appropriate citations for the used data and tools.

Did you mention the license of the assets? [No] All used datasets are publicly available.

Did you include any new assets either in the supplemental material or as a URL? [N/A]

Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [N/A]

Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]

If you used crowdsourcing or conducted research with human subjects…

Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]

Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]

Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]