O n the I nterplay B etween F ine-tuning and S entence-level P robing for L inguistic K nowledge in P re-trained T ransformers (2020)

Abstract: Fine-tuning pre-trained contextualized embedding models has become an integral part of the NLP pipeline. At the same time, probing has emerged as a way to investigate the linguistic knowledge captured by pre-trained models. Very little is, however, understood about how fine-tuning affects the representations of pre-trained models and thereby the linguistic knowledge they encode. This paper contributes towards closing this gap. We study three different pre-trained models: BERT, RoBERTa, and ALBERT, and investigate through sentence-level probing how fine-tuning affects their representations. We find that for some probing tasks fine-tuning leads to substantial changes in accuracy, possibly suggesting that fine-tuning introduces or even removes linguistic knowledge from a pre-trained model. These changes, however, vary greatly across different models, fine-tuning and probing tasks. Our analysis reveals that while fine-tuning indeed changes the representations of a pre-trained model and these changes are typically larger for higher layers, only in very few cases, fine-tuning has a positive effect on probing accuracy that is larger than just using the pre-trained model with a strong pooling method. Based on our findings, we argue that both positive and negative effects of fine-tuning on probing require a careful interpretation.

## introduction transformer-based contextual embeddings like bert @xcite , roberta @xcite and albert @xcite ) recently became the state-of-the-art on a variety of nlp downstream tasks. these models are pre-trained on large amounts of text and subsequently fine-tuned on task-specific, supervised downstream tasks. their strong empirical performance triggered questions concerning the linguistic knowledge they encode in their representations and how it is affected by the training objective and model architecture @xcite . one prominent technique to gain insights about the linguistic knowledge encoded in pre-trained models is probing @xcite . however, works on probing have so far focused mostly on pre-trained models. it is still unclear how the representations of a pre-trained model change when fine-tuning on a downstream task. further, little is known about whether and to what extent this process adds or removes linguistic knowledge from a pre-trained model. addressing these issues, we are investigating the following questions: to answer these questions, we investigate three different pre-trained encoder models, bert, roberta, and albert. we fine-tune them on sentence-level classification tasks from the glue benchmark @xcite and evaluate the linguistic knowledge they encode leveraging three sentence-level probing tasks from the senteval probing suite @xcite . we focus on sentence-level probing tasks to measure linguistic knowledge encoded by a model for two reasons: 1) during fine-tuning we explicitly train a model to represent sentence-level context in its representations and 2) we are interested in the extent to which this affects existing sentence-level linguistic knowledge already present in a pre-trained model. we find that while, indeed, fine-tuning affects a model's sentence-level probing accuracy and these effects are typically larger for higher layers, changes in probing accuracy vary depend-ing on the encoder model, fine-tuning and probing task combination. our results also show that sentence-level probing accuracy is highly dependent on the pooling method being used. only in very few cases, fine-tuning has a positive effect on probing accuracy that is larger than just using the pre-trained model with a strong pooling method. our findings suggest that changes in probing performance can not exclusively be attributed to an improved or deteriorated encoding of linguistic knowledge and should be carefully interpreted. we present further evidence for this interpretation by investigating changes in the attention distribution and language modeling capabilities of fine-tuned models which constitute alternative explanations for changes in probing accuracy. ## related work probing a large body of previous work focuses on analyses of the internal representations of neural models and the linguistic knowledge they encode @xcite @xcite @xcite . in a similar spirit to these first works on probing, @xcite were the first to compare different sentence embedding methods for the linguistic knowledge they encode. @xcite extended this approach to study sentence-level probing tasks on english and polish sentences. alongside sentence-level probing, many recent works @xcite @xcite @xcite have focused on token-level probing tasks investigating more recent contextualized embedding models such as elmo @xcite , gpt @xcite , and bert @xcite . two of the most prominent works following this methodology are @xcite and @xcite . while liu et al. (2019a) however, in contrast to our work, most studies that investigate pre-trained contextualized embedding models focus on pre-trained models and not fine-tuned ones. moreover, we aim to assess how probing performance changes with fine-tuning and how these changes differ based on the model architecture, as well as probing and fine-tuning task combination. fine-tuning while fine-tuning pre-trained language models leads to a strong empirical performance across various supervised nlp downstream tasks @xcite , fine-tuning itself @xcite and its effects on the representations learned by a pre-trained model are poorly understood. as an example, @xcite show that downstream accuracy can benefit from an intermediate fine-tuning task, but leave the investigation of why certain tasks benefit from intermediate task training to future work. recently, @xcite extended this approach using eleven diverse intermediate fine-tuning tasks. they view probing task performance after finetuning as an indicator of the acquisition of a particular language skill during intermediate task finetuning. this is similar to our work in the sense that probing accuracy is used to understand how finetuning affects a pre-trained model. @xcite try to understand whether the performance on downstream tasks should be attributed to the pre-trained representations or rather the fine-tuning process itself. they fine-tune bert and roberta on a large set of symbolic reasoning tasks and find that while roberta generally outperforms bert in its reasoning abilities, the performance of both models is highly context dependent. most similar to our work is the contemporaneous work by @xcite . they investigate how fine-tuning leads to changes in the representations of a pre-trained model. in contrast to our work, their focus, however, lies on edgeprobing @xcite and structural probing tasks @xcite and they study only a single pre-trained encoder: bert. we consider our work complementary to them since we study sentence-level probing tasks, use different analysis methods and investigate the impact of fine-tuning on three different pre-trained encoders: bert, roberta, and albert. ## methodology and setup the focus of our work is on studying how finetuning affects the representations learned by a pretrained model. we assess this change through sentence-level probing tasks. we focus on sentencelevel probing tasks since during fine-tuning we explicitly train a model to represent sentence-level context in the cls token. model task cola sst-2 rte squad devlin et al. (2019) 52.1 93.5 66.4 80.8/88.5 bert 59.5 92.4 64.6 78.6/86.5 roberta 60.3 93.6 73.6 81.7/89.3 albert 45.8 88.5 69.6 79.9/87.6 to focus more on syntactic, semantic or discourse information. the extent to which knowledge of a particular linguistic level is needed to perform well differs from task to task. for instance, to judge if the syntactic structure of a sentence is intact, no deep discourse understanding is needed. our hypothesis is that if a pre-trained model encodes certain linguistic knowledge, this acquired knowledge should lead to a good performance on a probing task testing for the same linguistic phenomenon. extending this hypothesis to fine-tuning, one might argue that if fine-tuning introduces new or removes existing linguistic knowledge into/from a model, this should be reflected by an increase or decrease in probing performance. 1 however, we argue that encoding or forgetting linguistic knowledge is not necessarily the only explanation for observed changes in probing accuracy. hence, the goal of our work is to test the abovestated hypotheses assessing the interaction between fine-tuning and probing tasks across three different encoder models. ## conclusion we investigated the interplay between fine-tuning and layer-wise sentence-level probing accuracy and found that fine-tuning can lead to substantial changes in probing accuracy. however, these changes vary greatly depending on the encoder model and fine-tuning and probing task combination. our analysis of attention distributions after fine-tuning showed, that changes in probing accuracy can not be attributed to the encoding of linguistic knowledge alone but might as well be caused by changes in the attention distribution. at the same time, our perplexity analysis showed that finetuning has profound effects on the representations of a pre-trained model but our probing analysis can not sufficiently detail whether it leads to forgetting of the probed linguistic information. hence we argue that the effects of fine-tuning on pre-trained representations should be carefully interpreted.