PATS : Sensitivity-aware Noisy Learning for Pretrained Language Models (2022)

Abstract: A wide range of NLP tasks benefit from the fine-tuning of pretrained language models (PLMs). However, a number of redundant parameters which contribute less to the downstream task are observed in a directly fine-tuned model. We consider the gap between pretraining and downstream tasks hinders the training of these redundant parameters, and results in a suboptimal performance of the overall model. In this paper, we present PATS (Perturbation According To Sensitivity), a noisy training mechanism which considers each parameter’s importance in the downstream task to help fine-tune PLMs. The main idea of PATS is to add bigger noise to parameters with lower sensitivity and vice versa, in order to activate more parameters’ contributions to downstream tasks without affecting the sensitive ones much. Extensive experiments conducted on different tasks of the GLUE benchmark show PATS can consistently empower the fine-tuning of different sizes of PLMs, and the parameters in the well-performing models always have more concentrated distributions of sensitivities, which experimentally proves the effectiveness of our method.

## introduction with a huge number of model parameters and well designed training objectives, pretrained language models (plms) have brought a new era to nlp @xcite @xcite . fine-tuning plms such as bert @xcite has become a basic and effective way in many downstream tasks @xcite @xcite . however, recent study has shown that aggressive fine-tuning can induce an unstable and suboptimal performance of the models especially with insufficient data @xcite , which attracts some researchers to figure there could be some connection between the gaps caused by overfitting of pretraining tasks and the redundancy of parameters. we consider it could be the gaps between pretraining and downstream tasks that hinder the training of these redundant parameters. sage enlarges the learning rates of insensitive parameters to help their training. however, with the sensitivity measurement considered, the insensitive parameters usually have smaller gradients, so enlarged learning rates may help them little to escape the sub-optimal areas compared to involving additional noise. one noisy training method to alleviate the gaps is noisytune, in which parameters of a matrix in a plm are added with noise according to the standard deviation of the matrix before fine-tuning. nevertheless, there are few explanations about why or whether the parameters in the same matrix should be perturbed with the same intensity. considering different parameters have different contributions to the model, noise from a unified distribution may disturb knowledge of some sensitive parameters, resulting in a loss of performance. besides, since each task needs to capture an appropriate textual pattern and the data of it usually comes from a special domain, different downstream tasks could have different kinds of gaps with those of the pretraining. so the noise added to overcome the gaps should also be related to the downstream task data. in this paper, we propose a novel parameterwise noisy fine-tuning method called pats (perturbation according to sensitivity) to make full use of perturbation on parameters to handle the problems above. we focus on balancing the contributions of all parameters in the model by activating the insensitive ones to play better roles in downstream tasks. so the main idea of our method is adding different intensities of noise to parameters according to their sensitivity when fine-tuning plms, different from noisytune (fig. 1 (b) ) in which noise added to a matrix of parameters is from a unified distribution and unrelated to downstream task data. specifically, during fine-tuning in pats (fig. 1 (c )), larger noise will be added to the parameters with lower sensitivity (such as the parameter shown in red), while sensitive parameters (such as the parameter shown in purple) will be barely perturbed. our contributions can be summarized as follows: 1) we propose a simple but effective method to help all parameters be trained sufficiently when fine-tuning plms in downstream tasks. 2) among all the training methods with noise, pats is the first sensitivity-aware one which perturbs models with noise of different distributions according to parameters' sensitivity, to the best of our knowledge. 3) extensive experiments on the glue benchmark show pats makes a difference in boosting the performance of plms in downstream nlp tasks. ## approach in this section, we present our pats for plms fine-tuning. previous matrix-wise noisy methods perturb a plm by adding noise from a uniform distribution to a matrix of parameters. different in noisytune, a matrix of parameters in a plm are perturbed with the same intensity before fine-tuning; in pats, parameters with lower sensitivity (the "red" parameter) to downstream data are added with larger noise, and vice versa (like the "purple" parameter). from them, in pats, each parameter even from the same matrix will be paid to different attention according to its sensitivity. it is also worth noting that in pats, a plm is not perturbed in advance like noisytune, instead the perturbation happens during training as the task data comes in. in the following sections, we will introduce the calculation of parameter sensitivity first and then present the noisy learning mechanism in detail. ## conclusion we propose a novel noisy training method called pats to optimize fine-tuning of plms. since aggressive fine-tuning plms will leave a large number of insensitive parameters which contribute little to the overall model, pats activates them and balance the contributions of all parameters in downstream tasks by adding noise to each parameter according to its sensitivity in the process of training. pats is a simple mechanism without much computational and memory overhead compared to adversarial training which requires additional backwards passes. extensive experiments on eight tasks of the glue benchmark show that pats can consistently improve the performance of plms on downstream tasks with the sensitivity of the pa-rameters more concentrated, which is especially pronounced on small datasets.