Noise Stability Regularization for Improving BERT Fine-tuning (2021)

Abstract: Fine-tuning pre-trained language models suchas BERT has become a common practice dom-inating leaderboards across various NLP tasks. Despite its recent success and wide adoption,this process is unstable when there are onlya small number of training samples available. The brittleness of this process is often reflectedby the sensitivity to random seeds. In this pa-per, we propose to tackle this problem basedon the noise stability property of deep nets,which is investigated in recent literature (Aroraet al., 2018; Sanyal et al., 2020). Specifically,we introduce a novel and effective regulariza-tion method to improve fine-tuning on NLPtasks, referred to asLayer-wiseNoiseStabilityRegularization (LNSR). We extend the theo-ries about adding noise to the input and provethat our method gives a stabler regularizationeffect. We provide supportive evidence by ex-perimentally confirming that well-performingmodels show a low sensitivity to noise andfine-tuning with LNSR exhibits clearly bet-ter generalizability and stability. Furthermore,our method also demonstrates advantages overother state-of-the-art algorithms including L2-SP (Li et al., 2018), Mixout (Lee et al., 2020)and SMART (Jiang et al., 20)

## introduction large-scale pre-trained language models such as bert @xcite have been widely used in natural language processing tasks @xcite @xcite . a typical process of training a supervised downstream dataset is to fine-tune a pre-trained model for a few epochs. in this process, most of the model's parameters are reused, while a random initialized task-specific layer is added to adapt the model to the new task. fine-tuning bert has significantly boosted the state of the art performance on natural lan- †contribution during internship at baidu research. * equal contributions. correspondence. guage understanding (nlu) benchmarks such as glue @xcite and superglue @xcite . however, despite the impressive empirical results, this process remains unstable due to the randomness involved by data shuffling and the initialization of the task-specific layer. the observed instability in fine-tuning bert was first discovered by @xcite ; @xcite , and several approaches have been proposed to solve this problem @xcite @xcite . in this study, we consider the fine-tuning stability of bert from the perspective of the sensitivity to input perturbation. this is motivated by @xcite and @xcite who show that noise injected at the lower layers has very little effect on the higher layers for neural networks with good generalizability. however, for a well pre-trained bert, we find that the higher layers are still very sensitive to the lower layer's perturbation (as shown in figure 1 ), implying that the high level representations of the pre-trained bert may not generalize well on downstreaming tasks and consequently lead to instability. this phenomenon coincides with the observation that transferring the top pre-trained layers of bert slows down learning and hurts performance @xcite . in addition, @xcite also point out that in transfer learning models for object recognition, the lower pre-trained layers learn more general features while the higher layers closer to the output specialize more to the pre-training tasks. we argue that this result also applies to bert. intuitively, if a trained model is insensitive to the perturbation of the lower layers' output, then the model is confident about the output, and vice versa. based on the above theoretical and empirical results, we propose a simple and effective regularization method to reduce the noise sensitivity of bert and thus improve the stability and performance of fine-tuned bert. figure 1 : attenuation of injected noise on the bert-large-uncased model on the mrpc task (x-axis: the layer index. y-axis: the l 2 norm between the original output and noise perturbed output). a curve starts at the layer where a scaled gaussian noise is injected to its output whose l 2 norm is set to 5% of the norm of its original output. as it propagates up, the injected noise has a rapidly decreasing effect on the lower layers but becomes volatile on the higher layers, which indicates the poor generalizability and brittleness of the bert top layers. moreover, models with higher accuracies (marked in the upper right) usually have lower error ratios or higher noise stability in top layers. to verify our approach, we conduct extensive experiments on different few-sample (fewer than 10k training samples) nlp tasks, including cola @xcite , mrpc @xcite , rte @xcite @xcite , and @xcite . with the layerwise noise stability regularization, we obtain strong empirical performance. compared with other stateof-the-art models, our approach not only improves the fine-tuning stability (with a smaller standard deviation) but also consistently improve the overall performance (with a larger mean, median and maximum). in summary, our main contributions are: ## using noise stability as a regularizer one of the central issues in neural network training is to determine the optimal degree of complexity for the model. a model which is too limited will not sufficiently capture the structure in the data, while one which is too complex will model the noise on the data (the phenomenon of over-fitting). in either case, the performance on new data, that is the ability of the network to generalize, will be poor. the problem can be regarded as one of finding the optimal trade-off between the high bias of a model which is too inflexible and the high variance of a model with too much freedom @xcite @xcite . to control the trade-off of bias against variance of bert models, we impose an explicit noise regularization method. ## analysis 5.1 ablation study to verify the effectiveness of our proposed lnsr model, we conduct several ablation experiments including fine-tuning with more training epochs and 2 https://huggingface.co/transformers/index.html noise perturbation without regularization (we inject noise directly to the output of a specific layer, and then use the perturbed representation to conduct propagation and then calculate loss, this process is similar to a vector-space represent augmentation). the results are shown in ## conclusion in this paper, we propose the layer-wise noise stability regularization (lnsr) as a lightweight and effective method to improve the generalizability and stability when fine-tuning bert on few training samples. our proposed lnsr method is a general technique that improves model output stability while maintaining or improving the original performance. furthermore, we provide a theoretically analysis of the relationship of our model to the lipschitz continuity and tikhonov regularizer. extensive empirical results show that our proposed method can effectively improve the generalizability and stability of the bert model.