On the Nature of BERT : Correlating Fine-Tuning and Linguistic Competence (2022)

Abstract: Several studies in the literature on the interpretation of Neural Language Models (NLM) focus on the linguistic generalization abilities of pre-trained models. However, little attention is paid to how the linguistic knowledge of the models changes during the fine-tuning steps. In this paper, we contribute to this line of research by showing to what extent a wide range of linguistic phenomena are forgotten across 50 epochs of fine-tuning, and how the preserved linguistic knowledge is correlated with the resolution of the fine-tuning task. To this end, we considered a quite understudied task where linguistic information plays the main role, i.e. the prediction of the evolution of written language competence of native language learners. In addition, we investigate whether it is possible to predict the fine-tuned NLM accuracy across the 50 epochs solely relying on the assessed linguistic competence. Our results are encouraging and show a high relationship between the model’s linguistic competence and its ability to solve a linguistically-based downstream task.

## introduction in the last few years, interest in assessing the linguistic generalization abilities of neural language models (nlms) has given rise to numerous studies aimed at investigating how the models are able to encode different types of linguistic phenomena. to this end, the most widespread methodology is the probing classification approach @xcite @xcite , which showed that nlms are able to encode a variety of language properties @xcite . even if most of the work is focused on the linguistic competence of pre-trained models, a complementary line of research is devoted to understanding whether and to what extent the existing competence is modified by fine-tuning the pretrained model on a supervised downstream dataset @xcite . in this paper, we continue this line of research and carry out a study aimed at adopting a probing task approach to investigate how the linguistic competence of one of the most prominent nlm model, bert @xcite , is altered after a fine-tuning stage with the main focus on which specific phenomena are preserved or forgotten and which fine-tuning epochs are mainly involved. as a testbed, we chose a downstream task where linguistic information plays the main role. it consists in predicting the evolution of written language competence relying on the linguistic style of chronologically ordered productions written by language learners @xcite . to the best of our knowledge, it has been never considered in the nlm interpretability literature, and differently from previous studies, the task was carried out at sentence rather than at document level. the present study addresses two further related issues: the first one concerns the debated relationship between the linguistic competence of an nlm and its ability to resolve a task @xcite . secondly, we aim to investigate whether it is possible to predict the accuracy of a fine-tuned model across multiple training epochs solely relying on the linguistic probing task results reflecting bert's linguistic competence. contributions. in this paper, we i) compared the ability of bert's and a support vector machine classifier, which uses a set of explicit linguistic features, in the prediction of the chronological order of two sentences written by the same student across two school years, ii) showed the impact of the fine-tuning stage on bert's linguistic competences, focusing in particular on how they change across 50 epochs of fine-tuning, iii) assessed which types of linguistic competence are mainly related to bert's ability to solve the downstream task, iv) predicted the fine-tuned bert's performance across 50 epochs relying on the linguistic competence of the model. ## related work several methods have been proposed in the literature to obtain meaningful explanations of how nlms are able to capture syntax-and semanticsensitive phenomena @xcite , also taking inspiration from human language experiments @xcite . despite highly debated @xcite , one of the mostly explored methods is the definition of probing tasks which a model can solve only if it has encoded a precise linguistic phenomenon within its representations. particularly relevant to our study are the multiple papers focused on the impact of the fine-tuning stage on how and the extent to which the linguistic knowledge encoded in the pre-trained model is modified, such as @xcite ; merchant et al. (2020); durrani et al. (2021); sarti et al. (2021); miaschi et al. (2020) ; @xcite to mention the most recent ones. even with some main differences concerning the nlms, and the fine-tuning and probing tasks considered, they agree that the main changes are typically larger for the output layers of a fine-tuned model than for the first ones. however, a shared consensus about whether the linguistic information are preserved, reinforced or forgotten is still missing. for example, @xcite ; @xcite ; @xcite report both a general not catastrophic forgetting and in some cases a substantial improvement of the linguistic competence directly involved in the resolution of the downstream task. on the contrary, @xcite found that finetuning on phrase meaning composition sets does not exhibit noteworthy benefits, and @xcite show that bert tends to lose its precision in encoding a wide set of linguistic features after the fine-tuning process. however, to the best of our knowledge, none of the prior works provide a comprehensive analysis of the impact of the fine-tuning stage across the training epochs. an orthogonal debated issue we are going to address in this study regards the question raised by @xcite and concerning the extent information encoded in nlms is indicative of information needed to perform downstream tasks. differently from the authors who demonstrated that nlms are able to encode linguistic properties even if they are not needed for a given downstream task, our results show that there is a strong correlation between the ability to encode a linguistic property and the accuracy to solve the task. an additional novelty of this study is the language we focus on, i.e. italian. while the vast majority of these studies are focused on english, relatively little work has been done to interpret non-english models. exceptions are represented by @xcite focused on dutch and by @xcite ; @xcite dealing with italian. ## our approach our study includes multiple experiments. firstly, we test bert's ability to solve the downstream task, by comparing the performance of the finetuned model against a support vector machine classifier which uses a set of explicit features resulted particularly relevant in the resolution of the task, as shown by @xcite ; @xcite . then, we use a suite of probing tasks to test the linguistic abilities of pre-trained and fine-tuned bert, with a specific focus on how they change across multiple training epochs of fine-tuning and on which linguistic properties are mostly correlated with the resolution of the downstream task. as linguistic probing tasks, we use the same set of linguistic features used by the svm. the last experiment is devoted to assess whether it is possible to predict the fine-tuning performance using the accuracy of the linguistic probing tasks. ## predicting the evolution of written language competence our first experiment is aimed at assessing bert's ability to predict the evolution of written language competence of l1 learners. following @xcite and @xcite , we modelled the task as a binary classification one. we started from the assumption that, given a set of chronologically ordered essays written by the same student, a document d j should have a higher quality level with respect to the ones written previously (d i ). however, differently from previous studies, our analysis was carried out at the sentence level. thus, given two sentences s i and s j , belonging to document d i and d j respectively, the task consists in predicting whether t(s j ) > t(s i ), where t(s i ) identifies the time in which the sentence s i was written. to this end, we built a dataset composed of all possible pairs (s i , s j ) of sentences contained in the cita corpus and written by the same student in the two school years. we considered only the sentences contained in the first essay written during the first year and the penultimate of the second year of four different schools, for a total of 3,562 sentences and 33,566 pairs. this is motivated by the fact that this represents the widest temporal span. for each pair, we assigned two labels: 1 if t(s j ) > t(s i ) and 0 otherwise. to balance the dataset, we included only one of the two labels, i.e. 50% of the pairs with label 0 (randomly selected) and the remaining of the pairs with label 1. cita statistics. as a preliminary analysis, we investigated which linguistic phenomena are mostly involved in the writing transformations across the two years. thus, we compared the distribution of the linguistic features automatically extracted from the set of paired sentences and described in ## how does linguistic competence change across the fine-tuning epochs? bert linguistic competence was tested by adopting a supervised probing classifier approach, which accounts for 68 probing tasks. as detailed in section 3.3, we trained a linearsvr probing model which uses the sentence-level representations extracted from the pre-trained and fine-tuned model. since many of our probing features are strongly related to sentence length, we compared the results with the ones obtained using a baseline computed by using a probing model trained using only sentence length as an input feature. pre-training linguistic competence. figure 2 reports the probing task results obtained using the pre-trained bert's representations. as a general remark, we can notice that bert's scores always outperform the baseline ones, with the obvious exception of the sentence length (n_tokens). however, as expected, the results are more similar for the probing tasks whose resolution is highly influenced by the length of the sentence. this is the case of global and local properties of the sentence such as the depth of the syntactic tree (avg_max_depth) and the length of the longest dependency links (max_links_len). they correspond to linguistic phenomena that the nlm masters very well. nevertheless, differently from the baseline, bert shows to encode quite accurately also the only lexical property we considered, i.e. the type/token ratio, and a raw text feature highly related to the use of lexicon, i.e. the length of tokens (char_per_tok). pretrained representations are also very good at predicting the distribution of functional information both in terms of parts-of-speech such as prepositions (xpos_dist_e), coordinating conjunctions (*_cc), determinative articles (*_rd), and of dependency relations linking a functional pos to its head (i.e. dep_dist_case, *_cc, *_det respectively), and of punctuation, in particular of commas (*_ff) and full stops (*_fs). lastly, concerning bert's knowledge about verbs, we can note that the model encodes quite accurately the tense, person and mood of auxiliary verbs (aux_*) and the distribution of verbal heads (verbal_head_per_sent). when we move to the analysis of how the competence changes across layers, we can observe that it generally tends to decrease when the output layer is approaching. this is in line with previous findings @xcite and it could be due to the fact that transformer layers trade off between task-oriented (e.g. masked language modeling) information and general linguistic competence. a such decreasing trend can be specifically appreciated by observing the evolution of the black dotted line reported in figure 3 . it shows how bert's competence in the 9 groups of linguistic phenomena introduced in fine-tuning impact. coloured lines in figure 3 report the layer-wise ρ scores we obtained using the representations extracted by each of the 50 epochs of fine-tuning on the cita corpus. differently from scholars who found that some linguistic abilities are reinforced after the fine-tuning process, such as @xcite and @xcite , but more similarly to @xcite , our analysis shows that bert tends to lose the linguistic competence acquired during the pretraining phase. however, such loss has a different impact across the 50 epochs and the 12 layers. for all epochs, all linguistic competencies are mostly altered starting from the 8th layer, thus resulting mostly modified in the 12th one. this might be possibly due to the fact that during these layers the model is storing task-specific information at the expense of its ability to encode general knowledge about the language @xcite . in addition, note that the impact of the finetuning stage is less evident in the first epochs (blue lines in the figure) than in the last ones (green lines). since the main changes are observed in the 12th to further investigate which group of features is forgotten most rapidly across the epochs, we calculated a slope of a linear regression line between the 50 epochs and the decay of competencies for each linguistic group. 4 results are reported in ## task resolution and linguistic competence correlating task accuracy with linguistic competence. for each single feature and each group of linguistic features, figures 5 and 6 report the results of the spearman correlations between the layer-wise probing scores and the accuracy of bert in the resolution of the downstream task for all epochs. the results were compared with a baseline computed as the spearman correlation between bert's accuracy on the downstream task and a list of numbers from 50 to 1 (ρ=0.55) 5 . as a general remark, in the first layers (from 1st to 7th), we noticed a non-significant or low correlation, most of the time lower than the baseline scores. on the contrary, the scores are higher, always above the baseline, in the central layers (8-10) and they tend to decrease in the 11th and 12th layers, where task-specific information is stored. the trend suggests that the higher bert's linguistic competence, the more accurate the resolution of the downstream task is. this is a global trend as showed by all features, even if there are some differences among the studied linguistic phenomena. namely, information about verbs (both in terms of inflectional morphology and predicate structure) and the order of nuclear elements of sentence results to be the linguistic competence mostly correlated with the resolution of the task. in particular, the distribution of auxiliary verbs (both in terms of pos and dependency relations), clitic pronouns, tense, person and mood of both auxiliary and lexical verbs, of subjects in their canonical order (i.e. in the pre-verbal position), show a very high correlation (ρ ≥ 0.70) in either the 9th or 10th layer. interestingly, these linguistic phenomena are the mostly involved in the writing transformations across the two school years (see predicting task accuracy with linguistic competence. the last experiment is aimed at testing whether it is possible to predict bert's accuracy in the resolution of the downstream task on the basis of its linguistic competence assessed by the linguistic probing tasks. for this purpose, we trained a linear support vector regressor using a leaveone-out strategy. namely, each time, we used as training the average accuracy of the probing tasks (in terms of ρ scores) achieved at the 12th layer of 49 epochs, and the excluded epoch as the test. as an evaluation metric, we used the spearman correlation and the mean squared error (mse) between the predicted values and the actual ones. we devised two baselines: the first one consists of a list of 50 numbers corresponding to the average bert's accuracy in the resolution of the downstream task considering all the epochs. in this case, since the value is always the same for all the epochs, the prediction was evaluated only in terms of mse. the second baseline is a linear svr using a decreasing list of numbers from 50 to 1, simulating the decreasing trend of the probing accuracy across the epochs. here, we were able to compute also the correlation scores. the results show that the svr achieves very high scores (ρ=0.76, @xmath0.011) which outperform the two baselines (ρ=na, @xmath1.017; ρ=0.54, @xmath2.017). this demonstrates that the bert accuracy can be reliably predicted by studying its linguistic abilities. the reliability of this result is corroborated by figure 7 where we compare the trends of the predicted (predicted) and bert actual accuracy. it is worth noting that the two lines are almost overlapping, showing a very similar trend. ## conclusion in this paper, we carried out an extensive study aimed at investigating in detail the relation between the fine-tuning stage and the linguistic knowledge encoded by bert. for our study, we chose the prediction of the evolution of written language competence of native language learners as a downstream task, since the morpho-syntactic and syntactic information plays an important role in the resolution. we showed in particular how the knowledge assessed during the pre-training stage is progressively forgotten across 50 epochs of fine-tuning. we observed that the main changes concern the 12th layer of all epochs and they are more evident in the last than in the first epochs. an in-depth analysis showed that the types of linguistic phenomena forgotten most rapidly across epochs are not related to the linguistic knowledge of the pre-trained nlm but possibly to some characteristics useful to solve the downstream task. we also found a strong cor-relation between the linguistic knowledge encoded by bert and its ability to resolve the task. specifically, we showed that the verbal inflectional morphology, the predicate structure, and the canonical order of subject and object are the most correlated aspects. interestingly, these linguistic characteristics, in particular those referring to verbs, resulted to be the most predictive types of features exploited by the svm classifier and the most involved in the writing transformations. a such strong relationship between the ability to the resolution of the task and linguistic competence is particularly evident in our last experiment, where we obtained very strong accuracy in predicting bert's performance using the encoded linguistic competence. besides investigating the nature of bert this outcome could open the way to define a new linguistically motivated stop-function of the fine-tuning process.