Grammatical information in BERT sentence embeddings as two-dimensional arrays (2023)

Abstract: Sentence embeddings induced with various transformer architectures encode much semantic and syntactic information in a distributed manner in a one-dimensional array. We investigate whether specific grammatical information can be accessed in these distributed representations. Using data from a task developed to test rule-like generalizations, our experiments on detecting subject-verb agreement yield several promising results. First, we show that while the usual sentence representations encoded as one-dimensional arrays do not easily support extraction of rule-like regularities, a two-dimensional reshaping of these vectors allows various learning architectures to access such information. Next, we show that various architectures can detect patterns in these two-dimensional reshaped sentence embeddings and successfully learn a model based on smaller amounts of simpler training data, which performs well on more complex test data. This indicates that current sentence embeddings contain information that is regularly distributed, and which can be captured when the embeddings are reshaped into higher dimensional arrays. Our results cast light on representations produced by language models and help move towards developing few-shot learning approaches.

## introduction transformer-based models have taken the nlp world, and not only, by storm in recent years. they have even reached super-human performance on standard benchmarks such as superglue @xcite and squad @xcite , and the output of gpt-* and chatgpt are often worryingly difficult to distinguish from humanproduced data @xcite . considering such performance, the expectations are high that the word and sentence representations produced by transformer-based architectures can also be useful for finer-grained tasks, such as those that target specific grammatical phenomena. long-distance agreement, a specific and simple grammatical phenomenon, is often used to test the syntactic abilities of deep neural networks @xcite @xcite . long-distance agreement tests are usually framed as a prediction task: whether the model gives higher probability to the correct form of the verb, despite intervening attractors -nouns appearing between the subject and the verb. this decision is somehow implicit: the language model gives a prediction based on the data it has been built on. in this paper, we investigate what happens when we target subject-verb agreement explicitly in bert sentence embeddings for detecting subjectverb agreement in french sentences. we work with raw bert sentence embeddings -not fine-tuned for specific tasks -and investigate how this specific information is encoded in the distributed representations. more specifically, we ask: can we detect subject-verb agreement in sentence embeddings? and can we manipulate the raw sentence embeddings to make this phenomenon more easy to detect? to address these questions, we adopt the framework and task definition for rule-like learning described in @xcite . in this framework, learning a given linguistic property or rule is formulated as a task of implicit rule detection in a multiple-choice setup, based on input sequences that share the target property. for the problem under investigation here, each sequence consists of sentences that share subject-verb agreement, but have different distances between the subject and the verb, different clause structures, and different agreement patterns. we show that bert sentence embeddings encoded as a one-dimensional array are only successful at detecting subject-verb agreement when provided with large amounts of data. reshaping the embeddings to two-dimensional arrays, and combining these with vae-based architectures, allows a system to detect better the shared patterns in the input sequences, while relying on much smaller amounts of simpler data. these results open up new avenues of exploration for few-shot learning @xcite @xcite . they also support further analyses of more disentangled representations, those representations that encode underlying rule-like generalisations, typical of human knowledge representation, but not of neural networks @xcite . the contributions of this paper are: the code and the data are available here: terminology sentence embeddings can be read from the output of bert systems as a 1 × n vector (n usually 768 or 1024). this can be viewed as the projection of the sentence into an n-dimensional space. in this paper, we use the word dimensions to refer to the shape of the data structure used to represent the sentence embeddings. in particular, we use one-dimensional array to refer to the 1 × n vector sentence representation obtained directly from bert, and 2d representations to refer to the 2d reshaped (rows × columns) array. ## related work producing sentence representations is a non-trivial issue, mainly because of the structural grammatical and semantic relations they express and their varying complexity and length @xcite . the deep learning framework has allowed for a variety of elegant solutions to explicitly learn sentence representations or to induce them as a side-effect or modeling of a more complex problem @xcite @xcite . transformer architectures, such as bert @xcite , have provided one such solution, where the representation of an input text as a one-dimensional array (usually 1x768 or 1x1024 for the large versions of the model) can be readily obtained from the output of the model. depending how the system is trained, the sentence embedding can be obtained from the encoding of the [cls] token, or as a combination of the embeddings of the tokens in the sentence. sentence transformers (reimers and gurevych, 2019) implement architecture and training regime changes on bert to optimize sentence embeddings for downstream tasks. @xcite analyze the relation between specific sentence properties (e.g. the contribution of different pos) and the geometry of the embedding space of sentence transformers. whether obtained from sentence transformers or directly from the output of a bert-based system, sentence embeddings have been shown to capture information about syntactic and semantic properties. for example, @xcite show that attention heads capture information about dependency relations in transformer models, and thrush et al. (2020) show the bert representations contain important information about argument structure and the meaning of verbs. subject-verb agreement is one of the phenomena used to probe a deep-learning system's syntactic abilities. while it is a simple word-level phenomenon, it encodes long-distance relations between words and requires knowledge of structural sentence properties to be correctly learned. @xcite shows that sentence embeddings capture this property, by testing the language model learned by bert in predicting the contextually appropriate form of the verb. @xcite include an overview of work that analyzes deep-learning models on this task. while the models tested show high performance in predicting the contextually correct form of a verb, they are guided -and misled -by biases within the corpus on which they were trained, e.g. they pay undue attention to the first noun in the sentence. linzen and baroni also include a survey of work that probe deep learning models to understand how grammatical information is encoded. @xcite al. (2018); conneau et al. (2018); mccoy et al. (2018) show that specific grammatical information -such as the plurality of the subject, the maximal depth of the parse tree of the sentence, the verb auxiliaries -can be decoded from the sentence encodings (or the hidden state) of the respective systems. @xcite analyze the actual architecture of an lstm language model @xcite , and track the impact of each unit on the long-distance agreement performance. they uncover a combination of a sparse mechanism -two units -and a larger distributed circuit that together keep track of number and syntactic structure. @xcite focus on how bert encodes grammatical number in english and how this information is used for performing number agreement. the focus is on word embeddings and quantifying how much number information they encode at various layers of the bert architecture. using a combination of probing approaches, they discover that subjects and predicates embeddings do encode number information, but at different layers. further investigations into where and how the number information is shared reveals that number information is not directly shared, but rather passed through intermediate tokens. we also target bert embeddings to investigate the subject-verb agreement property. rather than looking at properties of word/token embeddings, we analyze sentence embeddings as the embeddings of the special @xcite token. we investigate how accessible the number agreement is in raw bert sentence embeddings in several steps: • test whether the subject-verb agreement rule can be recovered through the sentence representation • test whether different shapes of the sentence embedding -1d and various 2d forms -make the targeted rule more easy to find • test these different shapes of sentence embeddings with several encode-decoder architectures, based on variational autoencoders (kingma and welling, 2013). example of contexts 1 the vase with the flower leaks. 2 the vases with the flower leak. 3 the vase with the flowers leaks. 4 the vases with the flowers leak. 5 the vase with the flower from the garden leaks. 6 the vases with the flower from the garden leak. 7 the vase with the flowers from the garden leaks. 8 ??? example of answers the vase with the flower and the garden leaks. coord the vases with the flowers from the garden leak. correct the vase with the flower leaks. wna the vase with the flower from the garden leak. ae the vases with the flower from the garden leak. wn1 the vases with the flowers from the gardens leak. wn2 figure 2: examples of actual sentences of type i data (original in french). @xcite ). we use blm-agrf @xcite . the structure of each problem in this task and dataset is inspired from rpm visual pattern tests -raven progressive matrices -where one problem consists of overlapping rules the solver must detect @xcite . a blackbird language matrix (blm) problem @xcite for subjectverb agreement consists of a context set of seven sentences that share the subject-verb agreement phenomenon, but differ in other aspects -e.g. number of intervening attractors between the subject and the verb, different grammatical numbers for these attractors, and different clause structures. an example template is illustrated in figure 1 , and an actual example in figure 2 . the dataset comprises three subsets, of increasing lexical complexity. type i data is generated based on manually provided seeds, and a template that captures the rules mentioned above. type ii data is generated based on type i data, by introducing lexical variation with the aid of a transformer, by generating alternatives for masked nouns. type iii data is generated by combining sentences from different instances from the type ii data. this will allow us to investigate the impact of lexical variation on the ability of a system to detect grammatical patterns. each subset contains an equal number of instances comprising three clause structures (we include complete instances -in french -in appendix a.1). these structural variations alter the distance and relative depth of the subject and verb to produce a variety of conditions, to allow us to investigate how the subject-verb agreement information is encoded in bert sentence embeddings. each problem is paired with a set of candidate answers. to allow for probing the learned model, apart from the correct answer, the answer sets contain negative examples built by corrupting some of the generating rules. this helps investigate the kind of information and structure learned, and the type of mistakes a system is prone to. ## experiments we report experiments on seven systems -the baseline ffnn, and two variations (the input as a stack of 1d or 2d sentence representations) for each of the other three architectures. our aim was to explore raw bert sentence embeddings, and not variations fine-tuned for specific tasks. the results reported here are based on sentence embeddings obtained using berttokenizer and bertmodel from the transformers python library, using the pretrained "bert-basemultilingual-cased" model foot_2 . 4 preliminary experiments have been used to determine the kernel size for processing the 2d and 3d tensors with the stack of 1d and 2d-ed sentence embeddings respectively. the optimal kernel for 2d tensors was 3x3, and for the 3d tensors was 3x15x15. this large kernel size imposes specific restrictions on the dimensions of the 2d-ed sentence embeddings. because a sentence embedding is a one-dimensional array of length 768, there are only 4 possible 2d values, where both dimensions are greater than 15 (16x48, 24x32, 32x24, 48x16). we have also explored the size of the latent layer (5:25, step 5), and chosen the latent size 5. all systems used a learning rate of 0.001 and adam optimizer, and batch size 100. for the baselines and experiments on the full training set for type ii and type iii data the training was done for 50 epochs, and for type i data and the sametrain set-up, the training was done for 120 epochs. the experiments were run on an hp pair workstation z4 g4 mt, with an intel xeon w-2255 processor, 64g ram, and a msi geforce rtx 3090 ventus 3x oc 24g gddr6x gpu. ## conclusion we have proposed and investigated variations of bert sentence representations for the task of iden-29 tifying the rule of subject-verb agreement and some of its properties in a dataset consisting of sequences of sentences that instantiate these underlying rules. bert sentence embeddings, as 1d arrays, have been successfully used for a variety of tasks, but the information they encode about specific phenomena is distributed over the vector. reshaping the sentence representation into two-dimensional inputs leads to improved results, and an additional step of abstracting these 2d-ed sentence embeddings leads to further improvements for the task. these more abstracted 2d-ed embeddings also learn a robust model based on a smaller amount of simpler training data, while showing good performance on the more lexically-complex data. we plan to explore whether bert sentence embedding may have even higher-dimension patterns, and whether such nd-ed bert sentence representation can be used to detect other grammatical or semantic phenomena. we also plan to directly distill 2d sentence representation in more compact and disentangled representation, to encode more explicitly some of the distributed information in these embeddings.