Word Frequency Does Not Predict Grammatical Knowledge in Language Models (2020)

Abstract: Neural language models learn, to varying degrees of accuracy, the grammatical properties of natural languages. In this work, we investigate whether there are systematic sources of variation in the language models’ accuracy. Focusing on subject-verb agreement and reflexive anaphora, we find that certain nouns are systematically understood better than others, an effect which is robust across grammatical tasks and different language models. Surprisingly, we find that across four orders of magnitude, corpus frequency is unrelated to a noun’s performance on grammatical tasks. Finally, we find that a novel noun’s grammatical properties can be few-shot learned from various types of training data. The results present a paradox: there should be less variation in grammatical performance than is actually observed.

## introduction neural language models @xcite @xcite @xcite have achieved success in both text prediction and downstream tasks such as question-answering, text classification, and natural language inference. the strong performance of these models raises scientific questions about the knowledge they have acquired, in particular, about the abstractness and generality of their linguistic representations. previous work has investigated the linguistic representations of neural language models in several domains, and found varying evidence for how linguistically adequate these representations are @xcite @xcite . this work has employed psycholinguistic methodology in order to elicit grammatical judgments from these models, inferring the models' underlying representations from the patterns of judgments. in the current work, we focus on the variation in grammatical knowledge that potentially exists within a neural language model. just as in human psycholinguistic tasks, previous work on neural lms has observed variability in grammatical judgments between different sentences; not all violations of a grammatical constraint are judged to be equally bad. it is not clear, however, whether there are systematic sources of variation in these judgments, and if so, what the sources are. we will focus on variation among lexical items, using english subject-verb agreement and reflexive anaphora as a case study. we first ask whether language models learn the grammatical properties of some nouns more accurately than for others. we do this by measuring the accuracy of language models when making grammatical judgments involving different nouns. we find systematic variation among nouns: nouns that perform well on one task or language model are more likely to perform well on other tasks or other language models. we then consider possible sources of the observed variation between nouns, finding that the grammatical properties of nouns are paradoxically easy to learn; our results suggest that there should be much less variation than is actually observed. 1 ## approach we use the minimal pair methodology of @xcite in order to investigate the grammatical judgments of neural language models. given a minimal pair of sentences, i.e. a pair that differ from each other in their acceptability due to a difference in just one grammatical property. if the model understands the grammatical phenomenon being studied, it should assign higher probability to the grammatical sentence than to the ungrammatical sentence. ## methods in this section we describe the process of calculating a target noun's task performance score in more detail. ## few-shot learning for novel lexical items the results in the previous section provide evidence that nouns systematically vary in their performance on grammatical tasks; some nouns perform better than others across tasks and language models. however, this variation is not explained by frequency of occurrence in natural text. nouns that occur on the order of 100 times in a corpus do not have systematically worse performance than nouns that occur 10 6 times. the results raise a question: if frequency does not influence how well a noun is understood, what does? if low frequency nouns are understood as well as higher frequency nouns, then this suggests that language models few-shot learn the grammatical properties of nouns. we suggest that by study-ing what makes a noun learnable in a few-shot setting, it may be psosible to better understand the sources of the observed variation. we use a few-shot learning paradigm, introducing a new lexical item into the vocabulary of the language model, either "wug" (intended as a new singular noun), or "wuz" (intended as a plural). we then fine-tune the language model using several example sentences containing this word. note that this paradigm is distinct from nearly all of the fewshot learning experiments performed in @xcite ; brown et al. (2020), which operate on a known vocabulary. foot_4 ## discussion we have investigated the sources of variation in neural language models' grammatical judgments. we found that there are systematic differences between nouns: when a language model exhibits knowledge of a noun's grammatical properties in one task, it is more likely to do so in other tasks. moreover, when one language model exhibits this knowledge, other language models are more likely to as well. the study found two latent dimensions of variation between nouns: one corresponding to how well the models understood its behavior with reflexive pronouns, and the other corresponding to subject-verb agreement. subsequent analyses demonstrate a pair of empirical phenomena: these results suggest that nouns should vary less in their grammatical performance than is actually observed; the study finds excess variation in grammatical performance. if number agreement can be correctly learned from a few samples (fsl samples), then one would expect model performance to either a) improve with more data, as more fsl samples are observed, or b) improve with more data up to some threshold, and then asymptote after learning has saturated. in either case, for high frequency nouns, a sufficient number of fsl samples should be observed for these nouns to be learned very accurately. a potential explanation of the results is that they are caused by catastrophic forgetting @xcite : although a sufficient number of fsl samples are observed for a noun, these samples are forgotten during training, causing the performance of the noun to degrade. this explanation is implausible. if catastrophic forgetting is occurring, then the problem should be more severe for infrequent nouns than for frequent nouns, as the interval between training samples will be longer for infrequent nouns. this would predict better performance for frequent nouns. the three language models (tables 5 6 7 8 ); pairwise comparison between task performance for bert and gpt-2 (figures 7 and 8 ); and more fine-grained comparisons between word frequency and model performance (figures 9 and 10 ). figure 10: noun frequency vs. model performance on reflexive anaphora tasks