Sentence Ambiguity, Grammaticality and Complexity Probes (2022)

Abstract: It is unclear whether, how and where large pre-trained language models capture subtle linguistic traits like ambiguity, grammaticality and sentence complexity. We present results of automatic classification of these traits and compare their viability and patterns across representation types. We demonstrate that template-based datasets with surface-level artifacts should not be used for probing, careful comparisons with baselines should be done and that t-SNE plots should not be used to determine the presence of a feature among dense vectors representations. We also show how features might be highly localized in the layers for these models and get lost in the upper layers.

## introduction pre-trained language models, such as bert, m-bert @xcite and gpt-2 @xcite , while being very efficient at solving nlp problems, are also notoriously difficult to interpret and their analysis and interpretation is an active area of research @xcite . one such technique of analysis is based on probing classifiers @xcite , which primarily consists of training and evaluating a shallow network multi-layer perceptron (mlp) as a classifier on top of the vector representations. probing classifiers are now fairly established in nlp @xcite @xcite . in this work, we build sentence representations from layer-wise contextual embeddings obtained from three different pre-trained language models and probe them for three linguistic traits: sentence ambiguity, grammaticality, and complexity using some well-established datasets. in the process, we show why having a reasonable baseline is a necessity for performance interpretation. we also demonstrate why simply visually checking the clustering of embeddings on datasets using t-sne, a popular dimension-reduction technique in probing, can lead to incorrect conclusions. motivation. the study of these traits is important for example in machine translation where disambiguation is necessary and grammaticality correction and simplification sometimes happen implicitly without any control. for the tasks of text simplification and grammar correction, it is crucial to be aware of whether and how general-purpose models encode these traits or whether they abstract the meaning from them. specifically, ambiguity detection has been investigated very little in contrast to other features. all of these three traits are orthogonal in their definitions, although their mutual relationships are unknown. for example, it may be that ambiguous sentences tend to be more complex and prone to lower grammaticality. we assimilate the definition of these traits from the respective datasets but nevertheless include examples in contribution. we carry out text classification tasks of ambiguity, grammaticality and complexity and demonstrate empirically that: ## data for each trait, we use a different dataset. their overall sizes are listed in ambiguity. we use sentences from the ms coco @xcite dataset, for our list of ambiguous and unambiguous sentences. the ms coco dataset comprises of a set of captions describing an image. captions containing ambiguous verbs corresponding to 461 images (ambiguous coco; elliott et al., 2016) constitute the ambiguous sentences for our experiment. 461 captions that were randomly sampled from ms coco con-dataset sentences ambiguity coco 0.9k complexity hcr english 1.2k paccss-it 1.1k grammaticality cola 5k blimp 67×2k complexity. corpus of sentences rated with human complexity judgments foot_0 @xcite and paccss-it @xcite contain 20 human ratings on the scale from 1 (not complex) to 7 (very complex) about sentences. we binarize these ratings and consider sentences below the average to be simple sentences and others to be complex sentences. the resulting dataset is class-balanced (complex/simple) in terms of examples (592 sentences of each class for english and 551 sentences for italian). the average sentence length for complex and simple examples is 24.84 and 13.95, respectively for english sentences. for italian sentences, the average sentence length for complex and simple examples is 21.61 and 12.26, respectively. the complexity could therefore be encoded solely in the sentence length. ## discussion the experiments with ambiguity reveal that the representations of the pre-trained models do not encode the ambiguity trait well. the description detailing how the ambiguous coco was created @xcite states that the dataset was created with the intention of testing the capabilities of multimodal translation systems. we posit that ambiguity as a trait is not encoded in an accessible way in the layer representations of pre-trained models. for blimp tasks related to morphology and syntax-semantics, the accuracy goes down in the upper layers, presumably because of increasing abstraction for both models (not shown in graphs). although we perform experiments without finetuning, the findings are in line with the experimental results of @xcite where finetuning on 3 tasks from the glue benchmark @xcite showed changes in probing performance mostly in the higher layers. fine-tuning however led to modest gains. the present setup which probes sentence representations from pretrained models shows that the middle layers fare far better in our probing tasks than the upper layers. this leads us to posit that the features of interest are highly localized and are lost in the upper layers (even with fine-tuning). although both bert and gpt-2 employ the transformer @xcite architecture, they have very different ways and locations for storing knowledge in their internal representations @xcite @xcite @xcite . the cls representations outperform the mean representations in only a few cases. this is expected since without fine-tuning the cls token in bert is trained to be used for the next sentence classification tasks. ## t-sne inadequacy given appropriate optimization and classifier, if two or more classes in a vector space form clusters, they are linearly separable and therefore the clas- ## conclusion in this work, we showed how large pre-trained language models represent sentence ambiguity in a much less extractable way than sentence complexity and stress the importance of using reasonable baselines. we document that template-based datasets, such as blimp used for sentence acceptability, are not suitable for probing because of surface-level artefacts and more datasets should be developed for probing more performant models. finally, we discuss why using t-sne visually for determining whether some representations contain a specific feature is not always a suitable approach.