Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANN s (2024)

Abstract: Language models learn rare syntactic phenomena, but the extent to which this is attributable to generalization vs. memorization is a major open question. To that end, we iteratively trained transformer language models on systematically manipulated corpora which were human-scale in size, and then evaluated their learning of a rare grammatical phenomenon: the English Article+Adjective+Numeral+Noun (AANN) construction (“a beautiful five days”). We compared how well this construction was learned on the default corpus relative to a counterfactual corpus in which AANN sentences were removed. We found that AANNs were still learned better than systematically perturbed variants of the construction. Using additional counterfactual corpora, we suggest that this learning occurs through generalization from related constructions (e.g., “a few days”). An additional experiment showed that this learning is enhanced when there is more variability in the input. Taken together, our results provide an existence proof that LMs can learn rare grammatical phenomena by generalization from less rare phenomena. Data and code: https://github.com/kanishkamisra/aannalysis.

## introduction 1.motivation and prior work humans come to learn and use rare grammatical structures, even if they have encountered those structures only rarely or even not at all @xcite . for instance, humans accept the grammaticality of the pipp construction ("surprising though it may be...") even where the preposed element crosses a finite close boundary ("surprising though i know it may be that...") @xcite and even though they may plausibly have never encountered such a sentence in their linguistic experience (see @xcite the family spent a beautiful five days in... the family spent a beautiful five days in... the family spent five beautiful a days in... the family spent a beautiful five days in... a few weeks is all i need! figure 1 : we train lms on varied input corpora and measure learning of the aann ("a beautiful five days"), comparing across systematically manipulated corpora. e.g. we train on a control corpus, a corpus in which we remove all aanns, a corpus in which we replace all aanns with a corrupted version ("beautiful a five days"), and a corpus in which we remove aanns and remove related constructions like "a few weeks is". we measure learning of aanns and corrupted variants. corpus estimate). how people come to know an utterance is grammatical has occupied a central place in linguistics. specifically, mastery of never-beforeencountered grammatical structures has been taken to mean that people are endowed with innate linguistic knowledge @xcite @xcite . recent evidence, though, suggests that large language models (llms) can learn complex grammar @xcite @xcite even from human-scale amounts of input @xcite @xcite . this raises the possibility that input data, along with an appropriately sophisticated or weakly bi-ased statistical learning mechanism, is sufficient for learning rare constructions by allowing for models to emergently learn appropriate grammatical abstraction @xcite . but modern llms often have access to much more training input than people do and thus might memorize in a way that humans cannot @xcite @xcite . the possibility that llms are "stochastic parrots" @xcite , heavily reliant on memorization, is a common criticism of using llms to study human language (e.g., @xcite . there are different levels of memorization, though, requiring different levels of abstraction. consider the aann construction: "a beautiful five days in texas" @xcite @xcite , which is rarer than the default "five beautiful days in texas". a model that strictly memorizes this phrase might come to know that "a beautiful five days in texas" is grammatical but has no idea that "a beautiful four days in texas" is grammatical if it never appeared in its training. a model that generalizes just a bit more might know that "a beautiful five days in new york" is also grammatical by generalizing that any u.s. state can fill the slot. knowing that "an astonishing 200 pages" is acceptable requires generalization beyond mere lexical items. and knowing that "a blue five pencils" is not acceptable (because colors are "stubbornly distributive", @xcite ) requires yet more knowledge. even for an idealized learner, it is difficult to precisely formulate how these kinds of generalizations emerge. there is increasing evidence that llms can generate novel linguistic utterances @xcite , and also make subtle judgments on relatively rare constructions like these @xcite , including the aann @xcite ). if they do so by memorizing examples verbatim from an unrealistically large training corpus, that would not be particularly informative for human processing. but, if they do learn rare grammatical phenomena from smaller amounts of data and can generalize beyond just those verbatim instances, that would raise the question of how they do it and if it can inform theorizing about humans. for instance, in the context of the pipp construction, @xcite speculates that the comparative construction (e.g., "they are happier than we are.") "may be the key to all of this [i.e., learning the pipp]" because such constructions are "incredibly common" yet share abstract structure with the pipp. if llms learn rare grammatical structures in part by learning and generalizing structures from much more common constructions, that would be powerful evidence for abstraction in llms and raise the possibility that even quite general learners could learn very rare phenomena without strong innate priors, drawing in part on the long-posited linguistic hypothesis that apparently distinct grammatical phenomena often share underlying structure. to that end, our goal in this paper is to study a relatively rare grammatical phenomenon in lms trained on controlled input corpora that are (a) of human realistic scale, and (b) systematically manipulated with respect to the target constructions as well as key related constructions. our hypothesis is that generalization abilities of lms on such rare phenomena come from abstractions and structures learned from more frequent related phenomena-and that knowledge of more frequent phenomena is the "key to all of this." as a case study, we focus on the aforementioned aann construction, although we highlight how the methods used here could serve as a blueprint for work on other phenomena. our method is to train different instantiations of a transformer model on the 100m-word babylm corpus, which we systematically manipulate-via removal and replacement-to explore how frequent and related phenomena encountered during training facilitate generalization behavior in lms. to test for generalization, we subjected our lms to a series of acceptability tests on sentences which do not appear in the training corpus and which specifically target the special properties of the aann. this approach of training on a systematically manipulated corpus has been used to debias models @xcite , explore the effect of permuting words on pretrained models @xcite , and test whether lms can learn languages judged to be hard for humans @xcite . it is also becoming a fruitful method for giving causal answers to questions about syntactic learning in language models, including hypotheses about learning subject-verb agreement @xcite , the acquisition of negative polarity items @xcite , subject-auxiliary inversion @xcite , and the english passive alternation (leong and @xcite . using this "filtered pretraining" method, @xcite find evidence of syntactic generalization underlying models' success on syntactic benchmarks. while this related work has largely fo-cused on ubiquitous linguistic structures (e.g., passives, subject-verb agreement, etc.), we specifically zero in on a rare construction to explore learning in the linguistic "long tail", where there is relatively little evidence available in the input. ## experiment 2: keys to learning aanns experiment 1 reveals that, keeping everything else the same, lms learn the aann construction more accurately than they do its counterfactual variants. furthermore, we also see strong aann acceptability judgments in lms that have (almost) never encountered a single instance. this suggests that there could be instances in the training data that contribute to the learning of the construction. what might these be? below we enumerate four hypotheses, each of which tackles subtly different aspects of the aann construction, and then measure the effect of these phenomena by separately holding them out during training and computing the average slor of the well-formed instances of the aann tests. the effect of a particular phenomenon on the acceptability of aanns can therefore be measured by comparing slors before and after holding out instances of that phenomenon. methods for detecting the hypothesized phenomena can be found in app. c. as control, we additionally also hold out a random set of utterances, which do not conform to the target phenomena of interest. the presence of "the ann" phrases like "the beautiful five days" are common in corpora, which are not as unusual because "the" regularly takes plural nouns. we hypothesize that the acceptability of these structures affects the acceptability of aanns, since an lm might analogize from the general observation that 'a' or 'an' can substitute 'the' (e.g., a ball vs. the ball). therefore, we consider all cases where a determiner precedes the contiguous sequence of article, numeral, plural noun. a few/couple/dozen/etc. nns another related phenomenon that is more common relative to the aann construction involves phrases such as "a few days" or "a couple bottles". to an lm learner, they might provide evidence that measure noun phrases with plural nouns can be attached to an indefinite article (a/an; @xcite , as is the case in aanns. ## experiment 3: the role of variability results from experiment 2 highlight the importance of seen aanns in order for lms to generalize to unseen instances. what properties of these seen instances facilitate lms generalization behavior? this broadly relates to a longstanding question as to how the nature of the instances of a construction provided during learning affect its (partial) productivity @xcite . in the context of aanns, we consider the role of variability on the open slots of the construction as a factor that might play a role in lms' productivity on unseen instances. encountering a slot with a wide variety of lexical items could serve as a cue that the slot is flexible. the idea that instance-variability could affect learnability is motivated by theoretical claims in usage-based linguistics @xcite , as well as existing research on novel constructions @xcite , morphological productivity @xcite , and inductive inferences in cognitive psychology @xcite . we hypothesize that instances of aanns that provide natural evidence of greater open-slot variability-i.e. evidence that many different adjectives, numerals, and nouns can go in their respective positions in the aann construction-would lead lms to assign greater likelihood to unseen aanns. on the other hand, lms that encounter only a restricted set of instances might be more conservative in extending the coverage of possible aanns to novel combinations of the slot-fillers. to test this, we divided our set of 2,448 aanncontaining utterances in the babylm corpus into two roughly equal subsets-one that contained aanns which were restricted in which tokens occur in a particular slot (low variability), and the other where the aanns showed more variability in those slots. we obtain these subsets by performing a median split based on the number of unique occurrences in a target slot(s), which resulted in a set of 1224 low and high variability instances. we repeated this for all three open slots (adjective/numeral/noun) jointly as well as those slots individually-i.e., 4 different types of target slots and 2 conditions each (low vs. high variability). details about the slot fillers and examples from each set are provided in app. f. we then trained lms on the babylm corpus containing utterances involving either of these two cases. fig. 5 shows the average slors obtained from this experiment, along with those obtained by lms trained on unablated babylm and the no aann conditions. we see that the slor patterns of lms trained on corpora that differed in aann slot-variability lie between the slor values elicited by lms that never saw aanns and ones that saw every single aann in the original corpus. among these, lms that saw aanns that were highly variable in their open-slots elicited slors that were greater than those elicited by lms that saw aanns with low open-slot variability. this was true for all cases except when "numeral" was the target slot, where both variability conditions resulted in roughly similar slors. (we hypothesize that numerals may pattern differently since they may be inherently more fungible than other word classes.) overall, these results suggest that lms are sensitive to the nature of no aanns noun slot num slot adj slot all open slots unablated 1.6 1.7 1.8 1.9 2.0 2.1 2.2 avg. slor (95% ci; 3 lm runs) relative variability of observed aanns no aanns low high unablated figure 5: slors on aanns from mahowald (2023) for lms trained on babylm with low and high variability in the open slots of the observed aann instances. when models are presented with higher variability for a given slot, the construction is typically learned better. ## conclusion theoretically, there is, for good reason, considerable interest in how language models can handle what has been variously called the "long tail" of language @xcite , "extremely rare constructions" @xcite , "exceptions to syntactic rules" (leong and linzen, 2023), "rare linguistic phenomena" @xcite , inter alia. studies of such phenomena are important first because lms (and statistical models in general) are sensitive to frequency and often perform far better in data-rich environments and, second, because the human ability to generalize to rare phenomena is central to linguistics. empirically, we found that lms trained on modest amounts of data can learn a rare construction like the aann. we found that this learning occurs even without veridical instances of the construction in the training data, and that it is mediated by occurrences of other related constructions in training. as such, these results join a body of work showing the ability of llms to learn rare phenomena @xcite @xcite and to generalize from limited data in meaningful ways. methodologically, this work leave us optimistic that "controlled rearing" of lms is a fecund method for understanding models, as well as for gleaning insight into human language more generally.