Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020)

Abstract: Humans can learn structural properties about a word from minimal experience, and deploy their learned syntactic representations uniformly in different grammatical contexts. We assess the ability of modern neural language models to reproduce this behavior in English and evaluate the effect of structural supervision on learning outcomes. First, we assess few-shot learning capabilities by developing controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. Second, we assess invariance properties of learned representation: the ability of a model to transfer syntactic generalizations from a base context (e.g., a simple declarative active-voice sentence) to a transformed context (e.g., an interrogative sentence). We test four models trained on the same dataset: an n-gram baseline, an LSTM, and two LSTM-variants trained with explicit structural supervision. We find that in most cases, the neural models are able to induce the proper syntactic generalizations after minimal exposure, often from just two examples during training, and that the two structurally supervised models generalize more accurately than the LSTM model. All neural models are able to leverage information learned in base contexts to drive expectations in transformed contexts, indicating that they have learned some invariance properties of syntax.

## introduction recurrent neural network language models @xcite have been shown to learn many aspects of natural language syntax including a number of long-miguel conducted this work while at ibm research scripts and data for this paper can be found online at https://github.com/wilcoxeg/fsl invar distance dependencies and representations of incremental syntactic state @xcite @xcite . however, previous studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. in this work, we assess neural models' ability to make robust syntactic generalizations about a token's nominal number or verbal argument structure based on minimal exposure with the token during training. because of the zipfian distribution of words in a corpus, the vast majority of word types will be seen only a handful of times during training @xcite . therefore, the few-shot learning capabilities of neural lms are critical to their robustness as an nlp system and as a cognitive model. however, human learning goes beyond simply learning syntactic properties in particular constructions. people apply the same properties across different constructions, meaning that their representations of the syntactic features of a word are in some sense invariant to the grammatical context of that word. for example, speakers and listeners are sensitive to a verb's argument structure relationships and can easily recognize that a verb which cannot take a direct object in active, declarative sentences cannot be passivized (as in the ungrammatical sentence "the ship was arrived.") the relationship between an active sentence and a passive sentence has been termed a transformation in the linguistic literature @xcite . many semantic-syntactic rules that govern word co-occurrence in one form, such as a verb's argument structure relationships, hold uniformly across transformations. it remains an open question whether models learn grammatical rules invariant to their surface realization, a property we call syntactic invariance. we combine assessment of few-shot learning and syntactic invariance for two grammatical features of english: whether a noun is singular or plural (nominal number) and whether a verb is transitive or intransitive (verbal argument structure). we assess whether a model is able to make different predictions based on number or argument structure in a simple active voice base context. we then assess whether models are able to make similar distinctions in a transformed context-passive voice for verbs and polar questions for nouns. in the transformed contexts, we test models with tokens that occur only in the base context during training. (as a control, we also test verbal passive voice with tokens that occur in the passive voice in the training data to establish that models have learned the proper syntactic rules for this context.) for models to succeed in the transformed contexts they must represent syntactic features in a way that is invariant to the specific realization of those features in terms of word co-occurrences in different constructions. for each grammatical feature, we introduce a suite of novel targeted test sentences, similar to those presented in @xcite . we find that all neural models tested are able to induce the proper syntactic generalizations in the base and transformed contexts after just two or three exposures, whereas a baseline n-gram model fails to learn the relevant generalizations. for all constructions tested our two neural models enhanced with explicit structural supervision outperform the purely sequence model. assessing invariance properties, we find that neural models demonstrate proper behavior in transformed contexts, even for tokens seen only in base contexts during training. this behavior indicates that models are able to deploy generalizations learned in one syntactic context into different syntactic environments, a key component of human linguistic capabilities that has been so far untested in the neural setting. ## nominal number in english, the matrix verb of a tensed clause must agree in number with the head of the subject noun phrase. neural lms are capable of learning this relationship based on a pure language modeling objective in multiple languages and across a variety of intervening material @xcite . previous work assessing neural learning of subject-verb number agreement has not directly compared the learning outcome to the amount of experience models receive during training. ## verbal argument structure in this section, we assess the ability of neural models to represent verbal argument structure, which we simplify to whether a verb is transitive or intransitive. if it is transitive, then it requires a theme, which must be realized as a direct object in the active voice. if it is intransitive then it requires an empty theme position and cannot have an object in the active voice. verbal argument structure is a hard task, insofar as both intransitivity and transitivity can only be inferred through indirect negative evidence. we assess neural models' sensitivity for indirect negative evidence by investigating how much experience models need with a particular verb before they make robust predictions about whether an object should follow that token at test time. ## discussion in this paper, we have tested the few-shot learning capabilities of neural language models, as well as whether these models can learn grammatical representations that are invariant to syntactic transformation. first, we addressed neural models' ability to learn nominal number, introducing a novel testing paradigm that leveraged polar questions to assess subject/verb number agreement learning in syntactically transformed settings. second, we turned to neural models' ability to represent verbal argument structure, developing two novel suites of tests that assessed preference for themes-either realized as direct objects or passive subjects-in both active contexts and passive contexts. in each experiment we assessed the effect of syntactic supervision on learning outcomes by comparing two supervised models to one purely sequence model. a summary of our results can be seen in because these experiments require careful and robust syntactic analysis of the training data, we evaluated models trained on a relatively small, human-annotated corpus. while the small training data poses some limitations when interpreting the results, it makes them more relevant to lowresource nlp applications and suggests that using structurally supervised models can lead to better generalization in a sparse data environment. while sub-word tokenization schemes such as byte-pair encoding @xcite have helped reduce the number of individual lexical items that need to learned, they do not completely eliminate the long tail of sub-word units. thus, robust fewshot generalization is still an important problem in these environments. it may be that larger amounts of training data support even better few-shot learning and syntactic invariance outcomes. scaling these carefully-controlled methods to the larger data setting will be an important next step. however, even with the relatively small models tested here, the results support a growing body of evidence that incremental statistical models of language are able to induce many key features of human linguistic competence. su wang, stephen roller, and katrin erk. 2017. distributional modeling on a diet: one-shot word learning from text only. in proceedings of the 8th international joint conference on natural language processing. ethan wilcox, roger levy, takashi morita, and richard futrell. 2018. what do rnn language models learn about filler-gap dependencies? in proceedings of the 2018 emnlp workshop black-boxnlp: analyzing and interpreting neural networks for nlp. ethan wilcox, peng qian, richard futrell, miguel ballesteros, and roger levy. 2019. structural supervision improves learning of non-local grammatical dependencies. in proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies. george kingsley zipf. 1949. human behavior and the principle of least effort. addison-wesley press.