Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning (2020)

Abstract: Recently, leveraging pre-trained Transformer based language models in down stream, task specific models has advanced state of the art results in natural language understanding tasks. However, only a little research has explored the suitability of this approach in low resource settings with less than 1,000 training data points. In this work, we explore fine-tuning methods of BERT - a pre-trained Transformer based language model - by utilizing pool-based active learning to speed up training while keeping the cost of labeling new data constant. Our experimental results on the GLUE data set show an advantage in model performance by maximizing the approximate knowledge gain of the model when querying from the pool of unlabeled data. Finally, we demonstrate and analyze the benefits of freezing layers of the language model during fine-tuning to reduce the number of trainable parameters, making it more suitable for low-resource settings.

## introduction pre-trained language models have received great interest in the natural language processing (nlp) community in the last recent years @xcite @xcite @xcite . these models are trained in a semi-supervised fashion to learn a general language model, for example, by predicting the next word of a sentence @xcite . then, transfer learning @xcite @xcite can be used to leverage the learned knowledge for a down-stream task, such as text-classification @xcite @xcite . @xcite introduced the "bidirectional encoder representations from transformers" (bert), a pre-trained language model based on the transformer architecture @xcite . bert is a deeply bidirectional model that was pre-trained using a huge amount of text with a masked language model objective where the goal is to predict randomly masked words from their context @xcite . the fact is, bert has achieved state of the art results on the "general language understanding evaluation" (glue) benchmark @xcite by only training a single, task-specific layer at the output and fine-tuning the base model for each task. furthermore, bert demonstrated its applicability to many other natural language tasks since then including but not limited to sentiment analysis @xcite @xcite , relation extraction @xcite and word sense disambiguation @xcite @xcite , as well as its adaptability to languages other than english @xcite @xcite . however, the fine-tuning data set often contains thousands of labeled data points. this plethora of training data is often not available in real world scenarios @xcite @xcite . in this paper, we focus on the low-resource setting with less than 1,000 training data points. our research attempts to answer the question if pool-based active learning can be used to increase the performance of a text classifier based on a transformer architecture such as bert. that leads to the next question: how can layer freezing techniques @xcite @xcite , i.e. reducing the parameter space, impact model training convergence with fewer data points? to answer these questions, we explore the use of recently introduced bayesian approximations of model uncertainty @xcite for data selection that potentially leads to faster convergence during fine-tuning by only introducing new data points that maximize the knowledge gain of the model. to the best of our knowledge, the work presented in this paper is the first demonstration of combining modern transfer learning using pre-trained transformer-based language model such as the bert model with active learning to improve performance in low-resource scenarios. furthermore, we explore the effect of trainable parameters reduction on model performance and training stability by analyzing the layer-wise change of model parameters to reason about the selection of layers excluded from training. the main findings of our work are summarized as follows: a) we found that the model's classification uncertainty on unseen data can be approximated by using bayesian approximations and therefore, used to efficiently select data for manual labeling in an active learning setting; b) by analyzing layer-wise change of model parameters, we found that the active learning strategy specifically selects data points that train the first and thus more general natural language understanding layers of the bert model rather than the later and thus more task-specific layers. ## qualitative analysis overall observation figure 6 visualizes the number of training elements in t grouped by their label c after each acquisition iteration of picking 100 elements from u . as the labels in the data sets and thus the subset u are equally distributed, we expect the same equal distribution when sampling randomly from the pool elements, which is apparent in the lower row of figure 6 where the training data was actually randomly chosen from the pool of data. in contrast, we observed that when using the bald acquisition with non-deterministic forward passes, the distribution shows a stronger bias to a particular class. this bias increases when sampling more pool data, whereas the difference between the biggest and smallest class in t stays constant during random acquisition (see which class? as the goal of active learning is to maximize the knowledge gain of the model with minimum cost, the choice of data the acquisition strategy selects from u may give further insight into the models understanding of the input data. one characteristic when applying active learning on the mnli data set, that is apparent in figure 6 , is that samples from the neutral class are queried the most. it might indicate that the model is most confused about the samples from this class. this observation may be justified by the fact that the hypothesis stated in that datum may be unrelated to the associated premise, confusing the model. another observation is that the model queries data points from the contradiction class more often than data points from the entailment class at the beginning of the training, while at the end number of queries from the different classes is almost equal. this indicates that, at the start of training, the model is more confused about contradictions, which may be explained by the fact that a contradiction can differ from an entailment only by a single negation at the correct place in the input, making it harder to differentiate from an entailment. an example of negated statements in a contradiction can be seen in the last row of layer freezing in deep transformers experiments by @xcite indicated that the first layers of a lm capture a more general language understanding, while later layers capture more taskspecific knowledge. with this motivation, @xcite introduced gradual unfreezing of the transformer layers during each epoch, beginning with the last layer. @xcite analyzed the loss surfaces in dependency of the model parameters before and after training and came to the same conclusion that lower layers contain more transferable features. however, none of the work has considered the training set size as a dependent parameter as our experiments presented in this paper. active learning in nlp there is some prior work regarding active learning for nlp tasks using deep neural networks. @xcite explored pool-based active learning for text classification using a model similar to our setting. however, they used word-level embeddings @xcite and focus on representation learning, querying pool points that are expected to maximize the gradient changes in the embedding layer. @xcite used active learning for named entity recognition tasks. they proposed a acquisition strategy named maximum normalized log-probability which is a normalized form of the constrained forward-backward confidence estimation @xcite . using this strategy, they achieved on-par performance in comparison to a model using the bald acquisition function and mc dropout without needing multiple forward passes. however, this approach is not suitable for any arbitrary model architecture but requires conditional random fields (crfs) for the approximation of model uncertainty. ## conclusion in this paper, we evaluated the performance of a pre-trained transformer model -bert -in an active learning scenario for text classification in low-resource settings. we showed that using monte-carlo dropout in the classification architecture is an effective way to approximate model uncertainty on unlabeled training elements. this technique enables us to select data for annotation that maximize the knowledge gain for the model fine-tuning process. experimental results on glue data set show that it improves both model performance and training stability. finally, in order to improve the efficiency of the fine-tuning process with a small amount of data, we explored the reduction of trainable model parameters by freezing layers of the bert model up to a certain level of depth. comparing the exclusion of layers in the front or the back of the bert model from training, we found it to be advantageous for training stability when freezing the layers closest to the output.