Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning (2022)

Abstract: While transferring a pretrained language model, common approaches conventionally attach their task-specific classifiers to the top layer and adapt all the pretrained layers. We investigate whether one could make a task-specific selection on which subset of the layers to adapt and where to place the classifier. The goal is to reduce the computation cost of transfer learning methods (e.g. fine-tuning or adapter-tuning) without sacrificing its performance.We propose to select layers based on the variability of their hidden states given a task-specific corpus. We say a layer is already “well-specialized” in a task if the within-class variability of its hidden states is low relative to the between-class variability. Our variability metric is cheap to compute and doesn’t need any training or hyperparameter tuning. It is robust to data imbalance and data scarcity. Extensive experiments on the GLUE benchmark demonstrate that selecting layers based on our metric can yield significantly stronger performance than using the same number of top layers and often match the performance of fine-tuning or adapter-tuning the entire language model.

## introduction transfer learning from a pretrained language model (plm) is now the de-facto paradigm in natural language processing (nlp). the conventional approaches of leveraging plms include fine-tuning all the parameters in the language model (lm) and some lightweight alternatives that can decrease the number of tuning parameters such as adaptertuning @xcite @xcite @xcite . these methods have one thing in common: they all involve the entire plm and attach a classifier to its top layer. however, plms were optimized via the language modeling objective and thus their top * work done during internship at tti-chicago. layers have been specialized in producing representations which facilitate optimizing that objective. such mismatch between the pretraining and finetuning objectives poses the following questions: ① given a pretrained language model and a downstream task, can we measure how "well-specialized" each layer has already been in that task, without any task-specific tuning? in this paper, we take a technically principled approach to investigate the research questions ① and ②. first, we define a metric in section 3.1 to measure the "task-specialty" of each layer in a given plm. our task-speciality score is inspired by the neural collapse (nc) phenomenon which has been widely observed in the computer vision community @xcite : as training converges, the top-layer representations of the images with the same label form an extremely tight cluster. in our setting, we examine the variability of the representations of the linguistic sequences given by each layer of the plm, and define our layerwise task-specialty to be the within-class variability normalized by the between-class variability. computing our metric does not require any training or hyperparameter tuning. experiments on the glue benchmark demonstrate that it is highly correlated with layer-wise probing performance, thus giving a clear "yes" to the question ① above. we propose several layer-selecting strategies in section 3.2 based on our proposed task-specialty metric. our strategies are complementary to all the major paradigms of transfer learning (such as fine-tuning and adapter-tuning) and thus can take advantages of the state-of-the-art at the time: only the selected layers will be tuned (e.g., via finetuning or using adapters) such that the computation cost of the tuning methods can be further reduced. experiments on the glue benchmark demonstrate that our proposed strategies are highly effective: under comparable computation budget, fine-tuning or adapter-tuning the layers selected by our strategies can achieve significantly higher performance than using the layers selected by the widely adopted baseline strategies; it can even often match the performance of fine-tuning or adapter-tuning the entire plm which takes 500% more computation cost. through extensive ablation studies, we demonstrate the comparable advantages of our proposed task-specialty metric over potential alternatives (such as cca and mutual information) as well as its robustness to data scarcity and data imbalance. ## technical background in this paper, we focus on classification tasks. technically, each classification task has a corpus of training data {(x n , y n )} n @xmath0,1 , . . . , x n,t ) is a sequence of linguistic tokens and each y @xmath1. such tasks include • sentiment analysis. each x is a single sequence of words such as "this movie is fantastic" and y is a sentiment label from {positive, negative}. thus, the sentiment analysis can be cast as a binary-class classification problem. figure 1 : an illustration of our variability-based taskspecialty metric with hypothetical data. each dot denotes a two-dimensional hidden state vector and its color denotes its target label. each colored star denotes the mean vector of its class. the net f is also called "classification head". transfer learning is to maximize the log probability of the ground-truth label y n -i.e., log p(y n | x n )-by learning the parameters of the classification head f as well as certain methodspecific parameters: • fine-tuning updates all the plm parameters @xcite . ## the method our goal is to answer the research questions ① and ② introduced in section 1. that involves finding a layer-specific metric ν (1) , . . . , ν (ℓ) , . . . , ν (l) where each ν (ℓ) measures the task-specialty of layer ℓ. suppose that we use s (ℓ) to denote the task score that we can achieve by letting the classification head read the layer ℓ hidden state h (ℓ) n,0 of the cls token. if ν (ℓ) is highly (positively or negatively) correlated with s (ℓ) , then the answer to question ① is yes. to answer question ② involves designing ν-based strategies that select a subset of layers to use in transfer learning approaches. in this section, we introduce our task-specialty metric ν (ℓ) (section 3.1) along with a few strategies for selecting layers (section 3.2). in section 4, we will empirically demonstrate the effectiveness of our proposed metric and strategies. ## experiments we evaluated the effectiveness of our taskspecialty metric along with our layer-selecting strategies through extensive experiments on the six classification tasks of the glue benchmark @xcite . the tasks are: cola, mnli, mrpc, qnli, qqp, and sst-2.all of them are sequencelevel classification tasks related to natural language understanding, thus being very different from how language models are pretrained. we chose the widely accepted roberta model @xcite to be our plm and used the pretrained roberta-large instance (355m parameters) downloaded from huggingface @xcite . our experiments are mainly conducted with this model. we also experimented with deberta @xcite to investigate whether our methods generalize across models: foot_0 those results are in appendix c.2 and are similar to the roberta results. prior work @xcite found that fine-tuning roberta on glue could be unstable, so we ran each of our experiments with five random seeds and reported the means and standard errors. experiment details (e.g., hyperparameters) can be found in appendix b. our code is implemented in pytorch @xcite and heavily relies on huggingface. it will be released after the paper is published. implementation details can be found in appendix b.2. ## conclusion in this paper, we present a comprehensive study on how to measure the task-specialty of each layer of a pretrained language model as well as how to leverage that knowledge to improve transfer learning. our proposed layer-wise task-specialty metric is based on the variability of the hidden states of each layer given a task-specific corpus. our metric is highly correlated with the layer-wise probing performance, though it is cheap to compute and does not require any training or hyperparameter tuning. we propose a couple of strategies based on the metric for selecting a subset of the layers to use in the plm-based transfer learning methods. extensive experiments demonstrate that our strategies can help fine-tuning and adapter-tuning achieve strong performance under a greatly reduced computation budget. our strategies are complementary to all the major paradigms of plm-based transfer learning and thus they will also benefit other methods.