Assessing Phrasal Representation and Composition in Transformers (2020)

Abstract: Deep transformer models have pushed performance on NLP tasks to new limits, suggesting sophisticated treatment of complex linguistic inputs, such as phrases. However, we have limited understanding of how these models handle representation of phrases, and whether this reflects sophisticated composition of phrase meaning like that done by humans. In this paper, we present systematic analysis of phrasal representations in state-of-the-art pre-trained transformers. We use tests leveraging human judgments of phrase similarity and meaning shift, and compare results before and after control of word overlap, to tease apart lexical effects versus composition effects. We find that phrase representation in these models relies heavily on word content, with little evidence of nuanced composition. We also identify variations in phrase representation quality across models, layers, and representation types, and make corresponding recommendations for usage of representations from these models.

## introduction a fundamental component of language understanding is the capacity to combine meaning units into larger units-a phenomenon known as composition-and to do so in a way that reflects the nuances of meaning as understood by humans. transformers @xcite have shown impressive performance in nlp, particularly transformers using pre-training, like bert @xcite and gpt @xcite , suggesting that these models may be succeeding at composition of complex meanings. however, because transformers (like other contextual embedding models) typically maintain representations for every token, it is unclear how and at what points they might be combining word meanings into phrase meanings. this contrasts with models that incorporate explicit phrasal composition into their architecture, e.g. rnng @xcite , recursive models for semantic composition @xcite , or transformers with attention-based composition modules @xcite . in this paper we take steps to clarify the nature of phrasal representation in transformers. we focus on representation of two-word phrases, and we prioritize identifying and teasing apart two important but distinct notions: how faithfully the models are representing information about the words that make up the phrase, and how faithfully the models are representing the nuances of the composed phrase meaning itself, over and above a simple account of the component words. to do this, we begin with existing methods for testing how well representations align with human judgments of meaning similarity: similarity correlations and paraphrase classification. we then introduce controlled variants of these datasets, removing cues of word overlap, in order to distinguish effects of word content from effects of more sophisticated composition. we complement these phrase similarity analyses with classic sense selection tests of phrasal composition @xcite . we apply these tests for systematic analysis of several state-of-the-art transformers: bert @xcite , roberta @xcite , distilbert @xcite , xlnet @xcite and xlm-roberta @xcite . we run the tests in layerwise fashion, to establish the evolution of phrase information as layers progress, and we test various tokens and token combinations as phrase representations. we find that when word overlap is not controlled, models show strong correspondence with human judgments, with noteworthy patterns of variation across models, layers, and representation types. however, we find that correspondence drops substantially once word overlap is controlled, suggesting that although these transformers contain faithful repre-sentations of the lexical content of phrases, there is little evidence that these representations capture sophisticated details of meaning composition beyond word content. based on the observed representation patterns, we make recommendations for selection of representations from these models. all code and controlled datesets are made available for replication and application to additional models. 1 ## related work this paper contributes to a growing body of work on analysis of neural network models. much work has studied recurrent neural network language models @xcite @xcite @xcite and sentence encoders @xcite @xcite . our work builds in particular on analysis of information encoded in contextualized token representations @xcite @xcite @xcite and in different layers of transformers @xcite . the bert model has been a particular focus of analysis work since its introduction. previous work has focused on analyzing the attention mechanism @xcite , parameters @xcite @xcite and embeddings @xcite . we build on this work with a particular, controlled focus on the evolution of phrasal representation in a variety of state-of-the-art transformers. composition has been a topic of frequent interest when examining neural networks and their representations. one common practice relies on analysis of internal representations via downstream tasks @xcite @xcite @xcite . one line of work analyzes word interactions in neural networks' internal gates as the composition signal @xcite , extending the contextual decomposition algorithm proposed by @xcite . another notable branch of work constructs synthetic datasets of small size to investigate compositionality in neural networks @xcite @xcite . some work 1 datasets and code available at https://github.com/yulang/ phrasal-composition-in-transformers controls for word content, as we do, to study composition at the sentence level @xcite . we complement this work with a targeted and systematic study of phrase-level representations in transformers, with a focus on teasing apart lexical properties versus reflections of accurate compositional phrase meaning. our work relates closely to classic work on two-word phrases, which have used methods like landmark tests @xcite @xcite , or compared against distributionbased phrase representations @xcite . our work also draws on work using correlation with similarity judgments @xcite @xcite and paraphrase classification @xcite @xcite to assess quality of models and representations. we build on this work by combining these methods together, applying them to a systematic analysis of transformers and their components, and introducing controlled variants of existing tasks to isolate accurate composition of phrase meaning from capturing of lexical information. ## testing phrase meaning similarity our methods begin with familiar approaches for assessing representations via meaning similarity: correlation with human phrase similarity judgments, and ability to identify paraphrases. the goal is to gauge the extent to which models arrive at representations reflecting the nuances of composed phrase meaning understood by humans. we draw on existing datasets, and begin by testing models on the original versions of these datasets-then we tease apart effects of word content from effects of more sophisticated meaning composition by introducing controlled variants of the datasets. the reasoning is that strong correlations with human similarity judgments, or strong paraphrase classification performance, could be influenced by artifacts that are not reflective of accurate phrase meaning composition per se. in particular, we may see strong performance simply on the basis of the amount of overlap in word content between phrases. to address this possibility, we create controlled datasets in which word overlap is no longer a cue to similarity. as a starting point we focus on two-word phrases, as these are the smallest phrasal unit and the most conducive to these types of lexical controls, and because this allows us to leverage larger amounts of annotated phrase similarity data. ## representation types a variety of approaches have been taken for representing sentences and phrases when all tokens output contextualized representations, as in our tested transformers. to clarify the phrasal information present in different forms of phrase representation, we experiment with a number of different combinations of token embeddings as representation types. formally, let [t 0 , • • • , t k ] be an input sequence of length k + 1, with corresponding embeddings normal examples source phrase target phrase are crucial is absolutely vital (pos) was a matter of concern (neg) is an essential part (pos) are exacerbating (neg) controlled examples source phrase target phrase communication infrastructure telecommunications infrastructure (pos) data infrastructure (neg) we test the following forms of phrase representation, drawn from each model and layer separately: cls depending on specific models, this special token can be the first or last token of the input sequence (i.e. e i 0 or e i k ). in many applications, this token is used to represent the full input sequence. head-word in each phrase, the head word is the semantic center the phrase. for instance, in the phrase "public service", "service" is the head word, expressing the central meaning of the phrase, while "public" is a modifier. because phrase heads are not annotated in our datasets, we approximate the head by taking the embedding of the final word of the phrase. this representation is proposed as a potential representation of the whole phrase, if information is being composed into a central word: avg-phrase for this representation type we average the embeddings of the tokens in the target phrase (dashed box in figure 1 ). this type of averaging of token embeddings is a common means of aggregate representation @xcite . avg-all expanding beyond the tokens in "avg-phrase", this representation averages embeddings from the full input sequence. with some variation between models, the sep token is typically a separator for distinguishing input sentences, and is often the last token (e i k ) or second to last token (e i k-1 ) of a sequence. ## experimental setup embeddings of each token are obtained by feeding input sequences through pre-trained contextual encoders. we investigate the "base" version of five transformers: bert @xcite , roberta @xcite , distilbert @xcite , xlnet @xcite and xlm-roberta @xcite . for the models analyzed in this paper, we are using the implementation of @xcite , foot_0 which is based on pytorch @xcite . for correlation analysis, we first use the complete bird dataset, consisting of 3,345 phrase pairs. 3 we then test with our controlled subset of the data, consisting of 410 ab-ba pairs. for classification tasks, we first do preprocessing on ppdb 2.0, foot_2 filtering out pairs containing hyperlinks, nonalphabetical symbols, and trivial paraphrases based on abbreviation or tense change. for our initial classification test, we use 13,050 source-target phrase pairs (of varying word overlap) from this preprocessed dataset. we then test with our controlled dataset, consisting of 11,770 source-target phrase pairs (each with precisely 50% word overlap). for each paraphrase classification task, 25% of selected data is reserved for testing. we use a multi-layer perceptron classifier with a single hidden layer of size 256 with relu activation, and a softmax layer to generate binary labels. we use a relatively simple classifier following the reasoning of @xcite , that this allows examination of how easily extractable information is in these representations. for both correlation and classification tasks, we experiment with phrase-only inputs and contextavailable (full-sentence) inputs. to obtain sentence contexts, we search for instances of source phrases in a wikipedia dump, and extract sentences containing them. for a given phrase pair, target phrases are embedded in the same sentence context as the source phrase, to avoid effects of varying sentence position between phrases of a given pair. foot_3 6 results ## qualitative analysis: sense disambiguation the above analyses rely on testing models' sensitivity to meaning similarity between two phrases. in this section we complement these analyses with another test aimed at assessing phrasal composition: testing models' ability to select the correct senses of polysemous words in a composed phrase, as proposed by @xcite . each test item consists of a) a central verb, b) two subject-verb phrases that pick out different senses of the verb, and c) two landmark words, each associating with one of the target senses of the verb. for this example, models should produce phrase embeddings such that "horse ran" is closer to "gallop" and "color ran" is closer to "dissolve". we use the items introduced in @xcite , which consist of a total of 4 sets of landmark tests. we feed landmarks and phrases respectively through each transformer, without context, to generate corresponding representations p i rep for each layer l i and representation type rep. cosine similarity between each phrase-landmark pair is computed and compared against expected similarities. figure 6 shows the percentage of phrases that fall closer to the correct landmark word than to the incorrect one, averaged over 16 phrase-landmark word pairs. we see strong overall performance across models, suggesting that the information needed for this task is successfully captured by these models' representations. additionally, we see that the patterns largely mirror the results above for correlation and classification on uncontrolled datasets. particularly, avg-phrase and avg-all show comparatively strong performance across models. roberta and xlnet show stronger performance in early layers, dropping off in later layers, while bert and distilbert show more consistency across layers. xlm-roberta and xlnet show lower performance overall. for this verb sense disambiguation analysis, the head-word token is of note because it corresponds to the central verb of interest, so its sense can only be distinguished by its combination with the other word of the phrase. xlm-roberta has the weakest performance with head-word, while bert and distilbert demonstrate strong disambiguation with this token. as for the cls token, roberta produces the highest quality representation at layer 1, and bert outperforms other models starting from layer 6, with distilbert also showing strong performance across layers. notably, the observed parallels to our correlation and classification results are in alignment with the uncontrolled rather than the controlled versions of those tests. so while these parallels lend further credence to the general observations that we make about phrase representation patterns across models, layers, and representation types, it is worth noting that these landmark composition tests may be susceptible to lexical effects similar to those controlled for above. since these test items are too few to filter with the above methods, we leave in-depth investigation of this question to future work. ## discussion the analyses reported above yield two primary takeaways. first, they shed light on the nature of these models' phrase representations, and the extent to which they reflect word content versus phrasal composition. at many points in these models there is non-trivial alignment with human judgments of phrase similarity, paraphrase classification, and verb sense selection. however, when we control our correlation and classification tests to remove the cue of word overlap, we see little evidence that the representations reflect sophisticated phrase composition beyond what can be gleaned from word content. while we see strong performance on classic sense selection items designed to test phrase composition, the observed results largely parallel those from the uncontrolled versions of the correlation and classification analyses, suggesting that success on this landmark test may reflect lexical properties more than sophisticated composition. given the importance of systematic meaning composition for robust and flexible language understanding, based on these results we predict that we will see corresponding weaknesses as more tests emerge for these models' handling of subtle meaning differences in downstream tasks. our systematic examination of models, layers and representation types yields a second takeaway in the form of practical implications for selecting and extracting representations from these models. for faithful representations of word content, avg-phrase is generally the strongest candidate. if only the phrase is embedded, drawing from earlier layers is best in roberta, xlm-roberta, and xl-net, while middle layers are better in bert, and later layers in distilbert. if the phrase is input as part of a sentence, middle layers are generally best across models. though the cls token is often interpreted to represent a full input sequence, we find it to be a poor phrase representation even with phrase-only input, with the notable exception of the final layer of distilbert. as for representations that reflect true phrase meaning composition, we have established that such representations may not currently be available in these models. however, to the extent that we do see weak evidence of potential compositional meaning sensitivity, this appears to be strongest in distilbert's cls token in final layers, in roberta's avg-phrase representation in later layers, and in xlm-roberta's avg-phrase representation from later layers only when the phrase is contained within a sentence context. ## conclusions and future directions we have systematically investigated the nature of phrase representations in state-of-the-art transformers. teasing apart sensitivity to word content versus phrase meaning composition, we find strong sensitivity across models when it comes to word content encoding, but little evidence of sophisticated phrase composition. the observed sensitivity patterns across models, layers, and representation types shed light on practical considerations for extracting phrase representations from these models. future work can apply these tests to a broader range of models, and continue to develop controlled tests that target encoding of complex compositional meanings, both for two-word phrases and for larger meaning units. we hope that our findings will stimulate further work on leveraging the power of these generalized transformers while improving their capacity to capture compositional meaning.