Investigating Semantic Subspaces of Transformer Sentence Embeddings through Linear Structural Probing (2023)

Abstract: The question of what kinds of linguistic information are encoded in different layers of Transformer-based language models is of considerable interest for the NLP community. Existing work, however, has overwhelmingly focused on word-level representations and encoder-only language models with the masked-token training objective. In this paper, we present experiments with semantic structural probing, a method for studying sentence-level representations via finding a subspace of the embedding space that provides suitable task-specific pairwise distances between data-points. We apply our method to language models from different families (encoder-only, decoder-only, encoder-decoder) and of different sizes in the context of two tasks, semantic textual similarity and natural-language inference. We find that model families differ substantially in their performance and layer dynamics, but that the results are largely model-size invariant.

## introduction it is more or less generally assumed that the success of bidirectional masked language models (mlms), such as bert @xcite , on downstream tasks is largely due to the fact that in pre-training they learn to compute rich and well-structured representations of their inputs. more precisely, it is often argued that the task of masked language modelling encourages models to successively aggregate lexical/collocational, syntactic, and semantic information from the input text as the activations progress through encoder layers @xcite . the extent to which bert-like models follow the stages of the classical nlp pipeline @xcite or accumulate contextual information @xcite has been questioned. however, the association of middle layers of mlms with syntax and higher levels with semantic information is not widely disputed as a general principle and is taken for granted in many papers on model analysis and downstream applications @xcite @xcite . despite the high volume of literature on transformer representations, these studies are mostly constrained in two ways: (i) they deal almost exclusively with word/token-level and not sentence-level embeddings, and (ii) the lion's share of attention is given to encoder-only mlms, notably variants of bert (cf. @xcite @xcite . as a result, the representations computed by text-to-text models, such as t5, and causal language models, such as members of the gpt family, remain understudied. this can be largely attributed to the fact that the standard way of analyzing pretrained language models, namely probing, proceeds by applying linear classifiers to token representations at different layers @xcite . this approach is not as easily applicable to mlmderived sentence representations or to representations computed by other types of models. 1 conversely, while it is possible to provide nuanced analyses of causal transformer-based models @xcite , such analyses are not easily transferable to mlms @xcite . in this study, we propose a unified methodology for studying layer-specific sentence-level representations extracted from masked, text-to-text, and causal language models. we analyze these representations via structural semantic probing, largely inspired by @xcite . instead of directly predicting features of interest from representations, structural probing projects them onto lower-dimensional subspaces where distances are interpretable in terms of task properties, or where different classes of data points are directly linearly separable. by varying the dimensionality of the projection space, we can gauge the amount of information contained in the embeddings. while @xcite identify well-structured syntactic subspaces, i.e. those encoding the topology and labels of universal dependency trees, we target sentence-level semantic subspaces and carry out experiments on two semantic tasks, viz. sentence similarity and natural language inference (nli). our contributions are as follows: the structure of the paper is as follows: § 2 introduces structural probing and its application; § 3 lays out our experimental setup; § 4 presents and discusses our findings, and § 5 concludes. ## semantic structural probing in all our experiments, we assume that we are given a set of sentences s @xmath0,m,@xmath1, where each element is indexed with a sentence, a model from which it was derived, and the model layer. to study the semantic organisation of sentence representations, we aim to find a projection matrix m to a lower-dimensional space, such that we can directly 'read off' the answer from the application of the matrix to elements of r. for the sts task, we choose an m that minimises the differences between the gold-label similarities and the euclidean distance between embeddings (averaged over the mini-batch): this corresponds to learning an approximation to the mahalanobis matrix m t m , that is, to learning a distance metric in the embedding space. (this interpretation carries over to our other experiments.) this distance metric is optimised to correlate well with the manually provided similarity judgements. correspondingly, we evaluate the performance of the probing approach by computing the spearman correlation between ||m r s i -m r s j || 2 and l i,j . the number of columns of m is equal to the dimensionality of the embedding space, but we can control the number of rows and thus vary the dimensionality of the projection subspace. in all experiments, we use the sequence of the powers of two from 2 1 to 2 9 , augmented with the embedding dimension of the model (e.g., 768 for bert base). we apply a similar approach for a subset of data from the te task: for sentence triplets where we have both an entailment e i and a contradiction c i for a given premise p i , we define q @xmath2 it must be stressed that by 'evaluation' we mean a proxy measure of the informativeness of vanilla embeddings and not a measure of how well the models can solve the original task. the labels of the tasks themselves constitute an 'abuse of notation' as in all cases we are dealing with reformulations of the original tasks, which in the case of nli involve a considerable simplification. thus, numbers should not be compared to results on the original tasks. nevertheless, we believe that our proxy tasks can provide interesting insights into the models. ## results in this section, we first describe our presentation of the results of the experiments ( § 4.1) and then go over individual tasks and model types ( § 4.2).