Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers (2023)

Abstract: How well do language models deal with quantification? In this study, we focus on ‘few’-type quantifiers, as in ‘few children like toys’, which might pose a particular challenge for language models because the sentence components with out the quantifier are likely to co-occur, and ‘few’-type quantifiers are rare. We present 960 English sentence stimuli from two human neurolinguistic experiments to 22 autoregressive transformer models of differing sizes. Not only do all the models perform poorly on ‘few’-type quantifiers, but overall the larger the model, the worse its performance. This inverse scaling is consistent with previous work suggesting that larger models increasingly reflect online rather than offline human processing, and we argue that the decreasing performance of larger models may challenge uses of language models as the basis for natural language systems.

## introduction quantifiers can dramatically alter the meaning of an utterance. consider the sentences in (1). despite the fact that (a) and (c) have the same content words in the same syntactic arrangement, the statements have starkly different meanings. the same is true of (b) and (d). being able to successfully comprehend these differences is useful, and in an example such as this one, vitally important foot_0 . yet current work suggests that language models deal poorly with quantifiers-they struggle to predict which quantifier is used in a given context @xcite , and also perform poorly at generating appropriate continuations following logical quantifiers @xcite . this is especially concerning given the recent trend of using large language models (sometimes referred to as 'foundation models'; @xcite as general systems that can perform multiple tasks, including question answering, without specific training @xcite @xcite @xcite @xcite . it is thus crucial that such systems be able to distinguish among sentences like those in (1) in human-like ways both during training and when generating responses. the aim of the present study is to evaluate how well language models take into account the meaning of a quantifier when generating the text that follows it, and to investigate whether this scales with model size. we are particularly interested in the question of whether language models exhibit inverse scaling-that is, whether as model size increases, performance decreases rather than increases @xcite . inverse scaling is an issue of serious concern for developing and training new language models, since inverse scaling could indicate 'outer misalignment' @xcite -that the training approach is leading to models that produce undesirable outputs, which may get worse as performance at training objectives increases. inverse scaling is also a concern for models' ultimate use. as models increase in size and perform better at a wider range of benchmarks (for recent examples, see, e.g., @xcite , they may be increasingly assumed to be trustworthy and generalpurpose, and thus able to perform well tasks on which they have not been tested @xcite . this could lead to a range of possible harms, from misidentifying whether something is dangerous or not (as in the opening example), to amplifying biases @xcite . to test how well language models deal with quantifiers, we follow the approach of ettinger (2020) in using sentences from a study on human language comprehension to inform our evaluation. ettinger (2020) found that following a negation, the predictions of bert base and bert large in simple sentences expressing a proposition with or without negation @xcite do not appear sensitive to negation-for example, bert large predicts the final word of a robin is a bird to be more likely than a robin is a tree, but also predicts that a robin is not a bird is more likely than a robin is not a tree. in this way, the models' predictions more closely match those made by humans 'online'-that is, incrementally during the process of language comprehension-than our fully-formed 'offline' judgements: in their original study, @xcite found that the word bird elicited an n400 response of smaller amplitude than tree in both contexts, indicating that it was more strongly predicted. similar effects have been reported @xcite for other transformers such as transformer-xl @xcite , roberta @xcite , and albert @xcite , as well as elmo @xcite . worse, recent work suggests that as language models increase in size, their ability to deal with negation may degrade: an inverse scaling relationship has been reported for performance at a wide range of tasks when prompts include negation @xcite , though it is possible that this may reverse at extremely large scales @xcite . negation may be particularly challenging for statistical language models because its presence radically alters the meaning of a sentence, but negation occurs in only about 10% of sentences @xcite . quantifiers similarly impose radical modulations to meaning while also being relatively infrequent (see appendix b). in the present study, we focus on quantifiers indicating typicality such as most and few. to the best of our knowledge, only one study has evaluated model predictions following any quantifiers @xcite , and it focused on words corresponding to logical quantifiers such as all, every, and some. the few studies involving the quantifiers we address either focus on predicting the quantifier itself @xcite , or use rnns to investigate modeling significant ef-fects on the n400 without any form of evaluation @xcite . this study, therefore, represents the first attempt to explicitly evaluate the predictions of language models following most and few-type quantifiers. in the present study, we carry out two experiments. in the first, following ettinger (2020), we use the stimuli from a previously published n400 study @xcite . in it, @xcite found that while most and fewtype quantifiers do impact n400 amplitude, it is not enough to reverse predictions-few farmers grow crops elicits a smaller n400 response than few farmers grow worms, indicating that crops was more strongly predicted than worms, even though experimental participants judged it to be less plausible off-line. we test whether language models show the same pattern of insensitivity towards the quantifiers that humans do in online measures. in this way, we test how closely the predictions of language models correlate with those underlying the human n400 response. in our second experiment, we extend our study further. experiment 1 aims to replicate the original n400 results of @xcite ; however, one thing that it does not account for is that while a given complete sentence (e.g., few farmers grow crops.) can be highly unlikely and implausible, sentences beginning with the same words may not be (for example, in the plausible sentence few farmers grow crops in the winter). experiment 1 does not distinguish between these possibilities, and while it is important to test the sensitivity of language models to few-type quantifiers, if they fail to show a difference for complete sentences including the final period (e.g., few farmers grow crops.), this is more concerning. thus, in experiment 2, we run the same stimuli as experiment 1, but including a period following the final word (e.g., crops./worms.). in this experiment, we use all the stimuli from two experiments carried out by @xcite . these are made up of 120 sentence frames with 8 different sentence types falling into 4 experimental conditions, for a total of 960 sentences. the 4 conditions had a 2x2 design-each stimulus was either typical (t) or atypical (a), and had either a most-type or few-type quantifier. an example of the 8 sentence types comprising one sentence frame is shown in (2). (2) (a) most squirrels gather nuts... (t, most) (b) most squirrels gather nails... (a, most) (c) few squirrels gather nuts... (t, few) (d) few squirrels gather nails... (a, few) (e) squirrels often gather nuts... (t, most) (f) squirrels often gather nails... (a, most) (g) squirrels rarely gather nuts... (t, few) (h) squirrels rarely gather nails... (a, few) the quantifiers used in sentences (a)-(d) differed by sentence frame; see appendix b for a full list. ## general discussion in this study, we investigated whether language models show the same insensitivity towards fewtype and most-type quantifiers observed in the predictions made by humans during language comprehension, as indexed by the n400 response. we find that when tested on the same stimuli, they do, predicting the ostensibly implausible few squirrels gather nuts to be more likely than few squirrels gather nails. moreover, we find that as language models increase in size, they tend to show this effect to a greater extent, an example of inverse scaling. based on our analysis of sensitivity and accuracy with most-type quantifiers, we hypothesize that these results are due to a low degree of sensitivity to quantifiers and an increase in sensitivity to typicality. in other words, language models appear to be increasingly sensitive to the fact that squirrels gather nuts is more plausible than squirrels gather nails, but not to the effect on meaning that is caused by a preceding most or few. it is often assumed that as models increase in size and are trained on more data, their performance on natural language tasks generally improves-indeed, evidence supports this @xcite @xcite @xcite @xcite . however, the predictions of larger models and those trained on more data also increasingly correlate with human incremental online predictions, in particular those indexed by n400 amplitude @xcite @xcite @xcite . the two are often aligned-it is easier for humans to process well-formed sentences with plausible semantics @xcite . but in cases such as the present study, the two are not aligned, and we see instead that the predictions of larger models correlate better with human online predictions, even when these are contrary to offline judgements. thus, the increased performance we see at tasks corresponding to offline human judgements-and note that virtually all manually-annotated tasks are based on offline human judgements-may in fact be a by-product of the models' predictions resembling the online predictions. fortunately, the literature boasts a wealth of psycholinguistic studies where metrics of online prediction such as the n400 appear to conflict with offline judgements. future work could use these to identify phenomena where language models may struggle to make predictions in line with human judgements. such cases are important to detect as use of lms becomes more widespread. but by the same token, the present study shows that as language models increase in size, even when augmented by finetuning on desirable responses, they can make predictions that align less and less with explicit human judgements. this may be a clear indication of an inherent 'outer misalignment' present in language models: while humans might like language models to generate plausible sentences, by their nature they can only generate the most statistically probable ones. just as there is no guarantee of accuracy or coherence @xcite , there is no guarantee of plausibility. while it may be possible to tailor training to avoid specific known issues, this misalignment between probability and plausibility may pose a fundamental challenge with current approaches that aim to use language models as general-purpose natural language systems.