Transformers in the loop: Polarity in neural models of language (2022)

Abstract: Representation of linguistic phenomena in computational language models is typically assessed against the predictions of existing linguistic theories of these phenomena. Using the notion of polarity as a case study, we show that this is not always the most adequate set-up. We probe polarity via so-called ‘negative polarity items’ (in particular, English ‘any’) in two pre-trained Transformer-based models (BERT and GPT-2). We show that – at least for polarity – metrics derived from language models are more consistent with data from psycholinguistic experiments than linguistic theory predictions. Establishing this allows us to more adequately evaluate the performance of language models and also to use language models to discover new insights into natural language grammar beyond existing linguistic theories. This work contributes to establishing closer ties between psycholinguistic experiments and experiments with language models.

## introduction recent transformer-based language representation models (lrms) -such as bert and @xcite -show impressive results on practical text analysis tasks. but do these models have access to complex linguistic notions? the results in this domain are less clearas well as ways to best approach this question. instead of asking whether lrms encode fragments of current linguistic theory, we will directly compare metrics derived from lrms to corresponding human judgments obtained in psycholinguistic experiments. the motivation for this is twofold. first, linguistic theories can be inaccurate -so, evaluating a model with respect to predictions of such theories is not informative about the model performance. second, robust abstract theoretical notions rarely correspond to robust judgments in * equal contribution. accepted to acl main conference. as a case study, we focus on polarity: a complex property of sentences at the intersection of grammar and semantics. we tackle polarity via the distribution of items that are sensitive to itnamely, so-called negative polarity items (npis) like english any. as a basic illustration of npi sensitivity to polarity, consider a pair of sentences in (1) (* = ungrammaticality): we find high levels of alignment between results of psycholinguistic experiments on monotonicity and npis, on the one hand -and our lrm-derived results, on the other hand. furthermore, show how lrms can be used to make new predictions about npis in contexts with different numerals and confirm these predictions in a psycholinguistic experiment. this case study contributes to the complement of the 'interpretability of neural lrms' research agenda: we can ask not only what linguistic tasks tell us about lrms, but also what these models can help us find out about natural language (see baroni 2021; linzen and baroni 2021 for a discussion along these lines). the paper is structured as follows. first, in section 2, we set up the context for our study: we describe the background in theoretical and experimental linguistics in the domains relevant for our discussion. section 3 discusses previous work on npis and polarity in computational linguistics. section 4 contains the description of our experimental method. first, we introduce our any dataset; then, we describe the tests and metrics we use with bert and with gpt-2 given our dataset. section 5 discusses our results. in section 6, we go beyond state-of-the-art knowledge in experimental semantics and pragmatics and study the effect of the numeral on npi acceptability -first, we do a bert study and then confirm the results on human participants. section 7 concludes: we propose directions for future work aligning experimental studies of language in humans and lrms. ## background npis are expressions with limited linguistic distribution. while their use is grammatical in some sentences, in other sentences their use results in ungrammaticality. the distribution of npis like any is governed by the notion of polarity that is much more intricate than the simple presence or absence of sentential negation, as in (1). for instance, in examples (2)-( 3 ), (2) are 'negative enough' to allow for (='license') any, while (3) are not -even though none of these sentences contain overt sentential negation. none of the boxes contain anything. b. nobody talked to anybody. c. at most five students did anything. d. few people had any thoughts (3) a. *some of the boxes contain anything. b. *somebody talked to anybody. c. *at least 5 students did anything. d. *many people had any thoughts the notion of polarity at play here relates to a semantic notion of monotonicity. 1 the notion of monotonicity builds on logical entailment. monotonicity of a linguistic environment defines its entailment patterns. in (4), the domain in square brackets is upward-entailing (ue), or upward-monotone, -as evidenced by the valid inference from sets (textbooks) to supersets (books): sentence (4-b) entails sentence (4-a). monotonicity is a highly abstract logical property interfacing with general reasoning. at the same time, it is deeply embedded into natural language grammar and it is relevant for understanding of inner workings of different linguistic expressions, such as npis. as shown by examples (1)-(3), de contexts give rise to negative polarity, as seen from npi acceptability; ue contexts are positive. there is conflicting evidence concerning non-monotone contexts @xcite . the connection between monotonicity and npi licensing is undeniable also beyond examples (1)-(3) (see @xcite @xcite @xcite @xcite . the order in pairs represents that the first element is judged as a better npi licenser than the second one or that it better supports de inferences (or both). that is, '@xmath0.'. the 'logical monotonicity' side of the table groups together all relations expected under the logical view of monotonicity; 'subjective monotonicity' contains additional asymmetries found experimentally that do not follow from the simple logical view. there is no single overarching psycholinguistic study testing the whole landscape of contexts. combined knowledge from an array of studies @xcite @xcite @xcite produces the picture summarized in ## previous work npis have been a topic of an investigation in the context of lrms, both as a subset of a more general test dataset @xcite , and as the main object of study @xcite @xcite . here we focus on @xcite as a representative case, as it shares with other previous studies its general set-up: assessment of lrms against predictions of linguistic theory. @xcite focus on npis in bert. using a variety of testing techniques, both zeroshot and with fine-tuning, they conclude that bert's ability to recognize npi licensing environments and, therefore, to tell licit uses of npis from illicit ones varies a lot depending on the type of context, scope configuration and the type of experimental setting. this might lead one to conclude that bert's ability to recognize polarity of a sentence is not so great across the board. indeed, reports from other tasks that involve polarity and/or monotonicity seem to support this. in particular, natural language inference has been reported to be hard for lrms @xcite @xcite . remarkably, geiger et al. (2020) report that fine-tuning bert on the snli dataset and then evaluating it on de sentences (their nmonli dataset) results in 2.2% accuracy -that is, the model practically ignores the monotonicity profile of the sentence. but is alleged poor polarity detection to blame here? importantly for our study, @xcite ) judge bert's recognition of npi acceptability against logical monotonicity rather than subjective monotonicity as uncovered by psycholinguistic experiments. so, we believe that these results deserve a second look. one of the measuring techniques in @xcite is very close to one of the two tech-niques we will adopt in this paper. it is a version of cloze test adapted for mlm, where probabilities of candidates for the masked position are compared. we discuss the set-up in section 4. finally, the idea of targeted lrm evaluations modeled after psycholinguistic experiments is being used in an increasing number of recent studies, albeit mainly in the domains of syntax and lexical semantics @xcite @xcite @xcite @xcite @xcite . we move on to describing our dataset, procedure and results. ## method we perform two types of tests using the dataset that we produce for this purpose. one experiment is done with bert, the other one with gpt-2. both experiments are performed in a zero-shot settingusing the pre-trained models without fine-tuning. the goal of these experiments is to test the contrasts between types of sentences described in first, we describe the dataset; then we explain the experiment procedure for bert and gpt-2; finally, we report and discuss the results. ## results of model evaluation we will discuss results from bert and gpt-2 together, because they mostly agree. one general result that allows us to limit our attention to one of the two bert metrics is that bert rank and bert probability produce the same order on all condition pairs of interest except for one (⟨at most, at least⟩) and we will only discuss bert probabilities in this section. the 20k synthetic data results are summarized in fig. 1 . the conditions in the 20k results are sorted for readability. 8k synthetic data results: ⟨no-, some-⟩: 99.76% (bert-prob); 99.56% (gpt-ppl-diff). in short, all predictions based on psycholinguistic evidence discussed in section 2 ( as a sanity check, we compare these results with the results of the same procedure on our natural dataset, and they are very similar: ⟨neg, aff⟩: 97.21% (bert-prob), 97.17% (gpt-ppl-diff); ⟨no-, some-⟩: 98.29% (bert-prob), 96.98% (gpt-ppl-diff). the take home message from these results is that lrms can tell between negative and positive polarity, as well as between different types of contexts by their monotonicity, as measured by establishing this, first of all, helps us make more sense of the metrics derived from such models and helps draw a more accurate line between noise and meaningful output. second, it encourages a closer tie between experiments with humans and with lrms: lrms encode a snapshot of numerous subjective linguistic intuitions, and maybe we can use lrms to get indirect access to speakers' shared intuitions as a source of new theoretically relevant linguistic generalisations. the next section is a pilot attempt in this direction. we establish a new generalization looking at lrm data -and then confirm it in a psycholinguistic experiment. ## next step: cardinality dependency for the conditions which involve numerals we left one parameter unexplored so far, namely, the numeral itself. in this section, we look at the dependency between npi acceptability and the numeral. there is no experimental data on this. theoretical literature tentatively suggests that the higher the numeral, the less acceptable an npi in its scope @xcite : @xcite exactly two of the boxes contain anything (11) ## discussion and outlook our experiments provide solid support for an approach under which lrm performance is compared directly to psycholinguistic data rather than to predictions of a linguistic theory. this opens up prospects for research that will result in a more empirically grounded picture of where the limits of lrm abilities lie. our results tell us something new about lrms but also suggest that lrms can be included in the experimental loop of theoretical semantics alongside with traditional experiments. to pilot this idea, we conducted an experiment on the effect of the numeral on npi acceptability. we confirmed our lrm findings in a parallel psycholinguistic study. in this paper, we only explore the connection between behavioral experiments and lrm-derived metrics. what about online measures in psycholinguistic studies? can we find a usable analogue to, for example, eye-tracking or reaction times in selfpaced reading studies -that is, studies that tell us which parts of input are important in processing? one obvious lrm-based candidate is attention. we took a preliminary look at bert attention distribution in sentences with any in an attempt to identify the attention head that contributes most to monotonicity-via-npis (see @xcite for a discussion of attention head specialization). to factor out linear position, we focused on the natural part of our dataset. we took the sentences that contained both a quantifier with a clear monotonicity profile (somebody, nobody, someone etc.) and any; calculated attention from any to the quantifier for every layer and every attention head and averaged it across sentences. then we sorted the results and went through the top of the resulting list. we found that the attention head (6,2) of bert-base-uncased model -6th layer, attention head 2 -seems to specialize in precisely what we are looking for. saliency maps below show that in a variety of contexts beyond the ones we checked for the purposes of this paper, monotonicity-affecting items are highlighted -buttressing the hypothesis that monotonicity is important for npi licensing (without, do-support in a question, if, lexical negation): [cls] we are unable to identify any others who knew of the scheme at the time it was being considered . additionally, this attention head reflects the role of the numeral in npi licensing that we established in section 6: in all contexts with numerals that we looked at, a lot of attention goes from any to both the quantifier (say, exactly) and the numeral that comes with it. moreover, the higher the numeral, the more attention goes to it, compared to the amount of attention that goes to the quantifier: more work is needed to verify and interpret these patterns systematically and compare them to other attribution measures and to online metrics in psycholinguistic studies.