How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy (2021)

Abstract: It is widely accepted that fine-tuning pre-trained language models usually brings about performance improvements in downstream tasks. However, there are limited studies on the reasons behind this effectiveness, particularly from the viewpoint of structural changes in the embedding space. Trying to fill this gap, in this paper, we analyze the extent to which the isotropy of the embedding space changes after fine-tuning. We demonstrate that, even though isotropy is a desirable geometrical property, fine-tuning does not necessarily result in isotropy enhancements. Moreover, local structures in pre-trained contextual word representations (CWRs), such as those encoding token types or frequency, undergo a massive change during fine-tuning. Our experiments show dramatic growth in the number of elongated directions in the embedding space, which, in contrast to pre-trained CWRs, carry the essential linguistic knowledge in the fine-tuned embedding space, making existing isotropy enhancement methods ineffective.

## introduction recently, several studies have focused on the remarkable potential of pre-trained language models, such as bert @xcite , in capturing linguistic knowledge. they have shown that pretrained representations are able to encode various linguistic properties @xcite @xcite @xcite @xcite , among others, syntactic, such as part of speech @xcite and dependency tree @xcite , and semantic, such as word senses @xcite and semantic dependency @xcite . despite their significant potential, pre-trained representations suffer from important weaknesses. frequency and gender bias are two well-known problems in cwrs. while the former hurts the semantic expressiveness of embedding space, the latter reflects the unwanted social bias in training data @xcite @xcite . the representation degeneration problem is another issue that limits their linguistic capacity. @xcite showed that the weight tying trick @xcite in the pre-training procedure is mainly responsible for the degeneration problem in the embedding space. in such a case, the embeddings occupy a narrow cone in the space @xcite . several approaches have been proposed to improve the isotropy of pre-trained models, which in turn boosts the representation power and downstream performance of cwrs @xcite . however, previous studies have mainly focused on the anisotropy of pre-trained language models. here, we investigate the impact of fine-tuning on isotropy. specifically, we try to answer the following questions: to answer these questions, we consider the semantic textual similarity (sts) as the target task and leverage the metric proposed by @xcite for measuring isotropy. the pretrained bert and roberta @xcite underperform static embeddings on sts, while fine-tuning significantly boosts their performance, suggesting the considerable change that cwrs un-dergo during fine-tuning @xcite . our analysis on the fine-tuned embedding space of bert and roberta demonstrates that word representations are highly anisotropic across all layers. an evaluation specifically carried out on the [cls] tokens approves a similar pattern but to a greater extent. moreover, experimental results show fading of local clustered areas in pre-trained cwrs during fine-tuning, which could be a possible reason for the improved performance. interestingly, the fine-tuning procedure can change the linguistic knowledge encoded in dominant directions of embedding space from unnecessary information to the essential knowledge for the target task such that eliminating them toward making isotropic space hurts the performance of contextual representations. ## related work following the research line in understanding the reasons behind the outstanding performance of pre-trained language models and their capabilities, most recent investigations on fine-tuning have been done through probing tasks and by evaluating the encoded linguistic knowledge @xcite @xcite . these studies demonstrate that most changes in fine-tuning are applied to the upper layers, such that those layers encode taskspecific knowledge, while lower layers are responsible for the core linguistic phenomenon @xcite . moreover, the results show that some linguistic information is surprisingly eliminated by this procedure @xcite . studies on the multi-head attention structure suggest a similar trend in their patterns during fine-tuning; in higher layers, attention weights experience more significant changes @xcite . more detailed analysis on self-attention modules indicates that dense and value projection matrices have heavily been affected by fine-tuning @xcite . however, geometric analysis on the embedding space and changes applied to the structure of embeddings during fine-tuning are aspects that have not been properly understood. furthermore, evaluating the fine-tuning effect on frequency bias in cwrs is another aspect that distinguishes our work from previous studies. ## conclusions in this paper, we explored the effect of fine-tuning on the structure of the embedding space of bert and roberta. our analysis demonstrates that the remarkable performance usually gained as a result of fine-tuning is not due to its enhancement of isotropy in the embedding space. similarly to their pre-trained counterparts, fine-tuned cwrs have elongated directions towards different dimensions across all layers, and the number of these directions tends to increase by fine-tuning. we have also found that fine-tuning changes the nature of the linguistic knowledge encoded in dominant directions such that removing them hurts the performance (unlike pre-trained models for which removing such directions often result in performance improvements). moreover, the clustered structure of pre-trained models is entirely modified upon fine-tuning, producing unbiased embedding space from the viewpoint of word frequency. as future work, we plan to experiment with more target tasks and different fine-tuning strategies to expand our knowledge about the fine-tuning procedure. furthermore, we aim at exploring the type of linguistic knowledge encoded in specific dimensions or subspaces in the semantic space.