Introduction
Pre-training large language models (PrLMs) on massive unlabeled corpora and fine-tuning them on downstream tasks has become a new paradigm [ 1 , 2 , 3 ] . Their success can be partly attributed to the self-attention mechanism [ 4 ] , yet these self-attention networks are often redundant [ 5 , 6 ] and tend to cause overfitting when fine-tuned on downstream tasks due to the mismatch between their overparameterization and the limited annotated data [ 7 , 8 , 9 , 10 , 11 , 12 , 13 ] . To address this issue, various regularization techniques such as data augmentation [ 14 , 15 ] , adversarial training [ 16 , 17 ] ), and dropout-based methods [ 11 , 13 , 18 ] have been developed. Among them, dropout-based methods are widely adopted for their simplicity and effectiveness. Dropout [ 19 ] , which randomly discards a proportion of units, is at the core of dropout-based methods. Recently, several variants of dropout have been proposed, such as Concrete Dropout [ 20 ] , DropBlock [ 21 ] , and AutoDropout [ 22 ] . However, these variants generally follow the vanilla dropout to randomly drop units during training and pay little attention to the effect of dropout on self-attention. In this paper, we seek to fill this gap from the perspective of self-attention attribution [ 23 ] and aim to reduce overfitting when fine-tuning PrLMs.

Attribution [ 24 ] is an interpretability method that attributes model predictions to input features via saliency measures such as gradient [ 25 , 26 ] . It is also used to explain the influence patterns of self-attention in recent literature [ 23 , 27 , 28 ] . Our prior experiment of self-attention attribution (Section 2.2 ) reveals that attention positions are not equally important in preventing overfitting, and dropping low-attribution positions is more likely to cause overfitting than discarding high-attribution positions. This observation suggests that attention positions should not be treated the same in dropout.

Motivated by the above, we propose A ttribution- D riven Drop out ( AD-Drop ) to better fine-tune PrLMs based on self-attention attribution. The general idea of AD-Drop is to drop a set of self-attention positions with high attribution scores. We illustrate the difference between vanilla dropout and AD-Drop by their attention maps in Figure 1 . When fine-tuning a PrLM on a batch of training samples, AD-Drop involves four steps. First, predictions are made through a forward computation without dropping any attention position. Second, we compute the attribution score of each attention position by gradient [ 25 ] or integrated gradient [ 26 ] attribution methods. Third, we sample a set of positions with high attribution scores and generate a mask matrix for each attention map. Finally, the mask matrices are applied to the next forward computation to make predictions for backpropagation. AD-Drop can be regarded as a strategic dropout regularizer that forces the model to make predictions by relying more on low-attribution positions to reduce overfitting. Nevertheless, excessive neglect of high-attribution positions would leave insufficient information for training. Hence, we further propose a cross-tuning strategy that performs fine-tuning and AD-Drop alternately to improve the training stability.

To verify the effectiveness of AD-Drop , we conduct extensive experiments with different PrLMs (i.e., BERT [ 1 ] , RoBERTa [ 2 ] , ELECTRA [ 29 ] , and OPUS-MT [ 30 ] ) on various datasets (i.e., GLUE [ 31 ] , CoNLL-2003 [ 32 ] , WMT 2016 EN-RO and TR-EN [ 33 ] , HANS [ 34 ] , and PAWS-X [ 35 ] ). Experimental results show that the models tuned with AD-Drop obtain remarkable improvements over that tuned with the original fine-tuning approach. For example, on the GLUE benchmark, BERT achieves an average improvement of 1.98/0.87 points on the dev/test sets while RoBERTa achieves an average improvement of 1.29/0.62 points. Moreover, ablation studies and analysis demonstrate that gradient-based attribution [ 25 , 26 ] is a more suitable saliency measure for implementing AD-Drop than directly using attention weights or simple random sampling. Moreover, they also demonstrate that the cross-tuning strategy plays a crucial role in improving training stability.

To sum up, this work reveals that self-attention positions are not equally important for dropout when fine-tuning PrLMs. Arguably, low-attribution positions are more difficult to optimize than high-attribution positions, and dropping these positions tends not to relieve but accelerate overfitting. This leads to a novel dropout regularizer, AD-Drop , driven by self-attention attribution. Although proposed for self-attention units, AD-Drop can be potentially extended to other units as dropout.