Investigating Novel Verb Learning in BERT : Selectional Preference Classes and Alternation-Based Syntactic Generalization (2020)

Abstract: Previous studies investigating the syntactic abilities of deep learning models have not targeted the relationship between the strength of the grammatical generalization and the amount of evidence to which the model is exposed during training. We address this issue by deploying a novel word-learning paradigm to test BERT’s few-shot learning capabilities for two aspects of English verbs: alternations and classes of selectional preferences. For the former, we fine-tune BERT on a single frame in a verbal-alternation pair and ask whether the model expects the novel verb to occur in its sister frame. For the latter, we fine-tune BERT on an incomplete selectional network of verbal objects and ask whether it expects unattested but plausible verb/object pairs. We find that BERT makes robust grammatical generalizations after just one or two instances of a novel word in fine-tuning. For the verbal alternation tests, we find that the model displays behavior that is consistent with a transitivity bias: verbs seen few times are expected to take direct objects, but verbs seen with direct objects are not expected to occur intransitively.

## introduction contemporary deep learning models for language have been shown to learn many aspects of natural language syntax including a number of longdistance dependencies @xcite @xcite , selectional properties of verbs @xcite , representations of incremental syntactic state @xcite and information from which hierarchical structure can be linearly decoded @xcite @xcite . these and many other related studies demonstrate an impressive range of human-like linguistic knowledge that is automatically acquired by these models simply from exposure to large quantities of raw text. however, human-like grammatical abilities include not just rich and detailed linguistic knowledge but the ability to deploy this knowledge in using new words based on minimal exposure @xcite @xcite . it remains poorly understood what grammatical generalizations contemporary deep learning models are able to make regarding the behavior of words to which they have minimal exposure. in this work, we assess the syntactic generalization behavior of a contemporary neural network model (bert; @xcite ) on two novel phenomena in english and address the question of single-shot and few-shot learning, demonstrating that bert makes robust grammatical generalizations after fine-tuning on minimal examples of a novel token. we test bert's few-shot learning capabilities on two phenomena at the syntax-semantics interface: english verbal alternations, and verb/object selectional preferences. in english, verbs can appear in multiple syntactic frames; which frame a verb appears in is governed by its argument structure properties. often, frames are paired into alternation classes @xcite such that when english speakers hear a novel verb in one frame they can be confident that it can be used in its alternation-class pair. using the well-attested dative alternation as an example, if a listener hears the sentence "i daxed the tennis racket to my friend" they would expect that "i daxed my friend the tennis racket" is a grammatical english sentence, meaning approximately the same thing. they would not, however, have such an expectation for "i daxed my friend for the tennis racket." in addition, listeners may be attuned to semantic clustering of verbal arguments based on past experience. for instance, following the example above, english speakers may expect dax to take an animate indirect object, and would find examples such as "i daxed the court the tennis racket" to be surprising. we take inspiration for our testing regime from a class of psycholinguistic experiments known as 'novel word learning studies', which we adapt to the neural setting. in such experiments subjects are exposed to a novel word in context during a training phase, and assessed for what grammatical generalizations they have learned about the novel word during a later testing phase. novel word learning experiments have been used to assess human grammatical generalization since @xcite , and have been deployed to assess semantic, as well as syntactic, generalizations @xcite . in this work, we replicate the novel word learning paradigm in the neural setting by finetuning bert on tightly-controlled sentences that contain novel verbs and objects, and assessing the model on carefully constructed test sets that reveal what grammatical generalizations it has learned. we find that bert is able to make proper generalizations for both verbal alternations as well as semantic clustering for verbal arguments after just one or two exposures during training. ## methods for each test, we fine-tune bert with sentences that contain new tokens for novel words. we then assess the the model's learning outcomes in one of two testing settings, described below. foot_0 ## selectional preferences verbs can impose a variety of selectional restrictions on semantic properties of nouns to limit which clusters of nouns they accept. just to name a few, these restrictions can require an object to be animate or inanimate, a location, or a raw material @xcite . in this section, we ask what generalizations bert makes about a verb's selectional restrictions based on incomplete, limited exposure. for our experiments, we define selectional restrictions as a model's expectations for a verb and object to appear together in a simple active transitive sentence, and ask whether bert can make generalizations about selectional restrictions from indirect evidence, following the incomplete selectional network given in fig. 1 (a) . indirect evidence plays an important role in human language learning. the role of indirect negative evidence has been the focus of much debate in discussions of innate human learning biases @xcite , and indirect evidence has also been shown to play an important role in the learning of novel verbs in both adults and children @xcite @xcite to assess bert's ability to leverage indirect negative evidence for verbal selection classes, we finetune the model on 12 sentences with verb/object pairings that correspond to the solid lines in figure 1 (a ). the fine-tuning set (and the test set) consist of simple transitive sentences, following the form "the [mask] [verb1] the @xcite ." each novel verb and each novel noun occur twice in the fine-tuning set, meaning that this test assesses the model's few-shot generalization capabilities. the network of verb-noun relations in the 12 fine-tuning sentences implicitly creates two classes of verbs: verbs within a class can be connected with a path through the solid lines. if the model leverages this incomplete evidence to make class-based generalizations, we predict that novel in-class verb/object pairings (which we indicate with dashed lines in the figure) should be more expected than novel outclass verb-object pairings, despite neither having been directly attested in the fine-tuning data. in order to assess the learning outcome of the model, we follow our psycholinguistic generalization test methodology to derive the probabilities of the verbs in simple active transitive sentences across three testing contexts: in the attested inclass condition, we compute the average probability of the verbs in sentences where they are paired with their nouns seen during fine-tuning. this set consisted of 12 sentences, corresponding to the solid lines in figure 1 (a) . in unattested in-class we compute the probability of the verbs when paired with their unattested, but in-class nouns. this set consisted of 6 sentences, corresponding to the dashed lines in figure 1 (a) . in the unattested outclass we compute the probability of the verbs when paired with nouns from the other class. this set consisted of 18 sentences, corresponding to verbnoun combinations that are not connected by lines in figure 1 (a) . the results of this experiment can be seen in figure 1 (b) and (c). part (c) shows the average surprisal (or negative log probability) of the verbs in the three testing contexts. in (b) we see model 'accuracy', or the proportion of times the model assigns lower surprisal to the higher evidence verb/object pairs. for example, for the attested in-class vs. unattested in-class the y-axis is the proportion of the time the attested in-class verbs are given lower surprisal. results are averaged across all six novel verbs and the proportions are taken accross 200 random model seeds. our predictions are as follows: for the accuracy test, if the model is able to pick up patterns in the fine-tuning data, we expect the comparison between seen items and unseen items to be greater than the 50% random baseline. if the model is able to go beyond the patterns in the training data and make class-based generalizations, then we expect the unattested in-class vs. unattested out-class comparison, too, to be higher than the baseline. examining verb surprisal on the right, we see significant contrasts between each of the conditions (@xmath0.001); crucially, the unattested in-class pairings are less surprising (i.e. higher probability) than the unattested out-class pairings, despite the model having seen neither pairing during training. this pattern is confirmed with the accuracy scores, where all three contrasts are significantly higher than the 50% random baseline (@xmath1.001). these results provide strong evidence that bert is not only sensitive to the minimal amount of data on which it was fine-tuned, but also able to leverage indirect evidence during fine-tuning to make syntactic generalizations, which drive behavior at test time. ## verb alternation classes english is attested to have at least 83 distinct verbal alternation classes, which were analyzed and categorized in meticulous detail in @xcite . in these experiments we consider all verbal alternation classes for which there are two constant frames and for which levin provides a list of example verbs as well as a list of "distractor" verbs-verbs that fit in one frame but not the other-which we require for our embedding classification test paradigm. all of the alternation classes we test come from the first three sections of levin's 'english verb classes and alternations.' to give a brief flavor of the range of english verbal alternations, we give three examples below. understood reciprocal alternation a. the senator will meet the activist. b. the senator and the activist will meet. verbs like meet in example (2) undergo transitivity alternations, where the verb takes a direct object in one frame but not the other. verbs like spray in example (3) involve alternations for transitive verbs that take more than one non-subject argument, and allow for multiple ways of expressing the arguments. verbs like make in example (4) involve "oblique" subject alternations, where the verb takes one fewer argument in one verbal frame. it is important to note that levin makes a categorical distinction between these three types of verbal alternation classes and analyzes them each in their own section. the cup will break. the girls will floss their teeth. the girls will floss. the senator and the activist will meet. the senator will meet the activist. the boy will dress himself. the boy will dress. the girl will hit at the fence. the girl will hit the fence. the senator and the activist will meet. the senator will meet with the activist. the girl will admire the honestly in them. the girl will admire them for their honesty. the middle class will benefit from the tax cuts. the tax cuts will benefit the middle class. between the attested verbs and an out-class set of the 150 most frequent verbs from the corpus of contemporary american english (coca) @xcite , pruned of auxiliary and modal verbs and verbs that already appear in levin's lists. for each verbal alternation, we run two classification tests, one for each frame in the alternation. ## conclusion we used a novel word learning paradigm, inspired by classic studies from psycholinguistics, to assess bert's syntactic generalization behavior on two novel phenomena: english verb class alternations and verb/object selectional restrictions. in both cases we address the issue of single and few-shot learning by fine-tuning the model on just one or two positive examples, finding that bert makes some generalizations about a novel token based on minimal experience, and that these generalizations drive robust behavior during test time. this novel word learning paradigm can continue to be explored in later work through the use of large databases such as verbnet @xcite , which builds on levin's verb documentations by providing a larger database of verb alternations and sectional restrictions that can be turned into train and test sentences for bert without hand-crafting. for verbal/object selectional restrictions, we find that bert leverages indirect evidence to expect unattested but plausible verb/noun pairings more than unattested but implausible pairings. these results provide evidence for the view that the model is able to attend not just to patterns overtly realized in the data (direct evidence) but also implicit relationships between tokens (indirect evidence). the ability to use indirect evidence, specifically indirect negative evidence, is a hallmark of human language learning, and these results indicate that models are capable of similar behavior in a simple novel word learning paradigm. for verbal alternations, we find that when finetuned on a single frame, bert routinely expects the verb to occur in its sister frame with a higher likelihood than in unrelated verbal frames. interestingly, this behavior is consistently blocked when the model is asked to generalize from a frame that involves an object to a frame where the object is lacking. this behavior is consistent with a general bias towards transitivity in the model, and suggests an exciting direction for further study. whether such a general bias exists, whether it is restricted to settings with limited evidence, and whether it changes as verbs appear more frequently in the fine-tuning or training data is a question for future research. another question for future research is whether a multilingual bert would have the same success on alternation tests in other languages, and if if would exhibit the same biases that we see for english.