Filler-gaps that neural networks fail to generalize (2020)

Abstract: It can be difficult to separate abstract linguistic knowledge in recurrent neural networks (RNNs) from surface heuristics. In this work, we probe for highly abstract syntactic constraints that have been claimed to govern the behavior of filler-gap dependencies across different surface constructions. For models to generalize abstract patterns in expected ways to unseen data, they must share representational features in predictable ways. We use cumulative priming to test for representational overlap between disparate filler-gap constructions in English and find evidence that the models learn a general representation for the existence of filler-gap dependencies. However, we find no evidence that the models learn any of the shared underlying grammatical constraints we tested. Our work raises questions about the degree to which RNN language models learn abstract linguistic representations.

## introduction while sentences appear highly variable on the surface, many syntactic constructions share the same underlying constraints, which determine their acceptability or grammaticality, i.e. the extent to which they are considered "well formed" through adherence to the rules of grammar. one of the strongest pieces of evidence for the existence of these shared underlying constraints are filler-gap constructions such as: filler-gap constructions contain a dependency between an overt filler (what in (1)) and a gap site (underlined above). the filler is bound to a referent (e.g., robin's painting) that can fill the gap: there are well-known restrictions (islands; @xcite that prevent certain words from participating in a filler-gap dependency. for example, it isn't possible to form a filler-gap dependency with prenominal (left-branch) noun modifiers: foot_0 further, very different filler-gap constructions obey shared underlying principles (e.g., subjacency; @xcite . in this work, we probe recurrent neural network (rnn) language model understanding foot_1 of these underlying principles in english. recent work has claimed that recurrent neural network language models understand filler-gap dependencies @xcite @xcite . however, behavioral probing has suggested that this understanding is relatively superficial and doesn't reflect the underlying constraints that govern filler-gap acceptability @xcite ). an intermediate possibility, which we explore in this paper, is that rnns do acquire a basic understanding of the underlying constraints but that the learned representations of the constraints are too weak to correctly drive behavior in behavioral probing tasks. we use cumulative priming @xcite to test for representational overlap between disparate constructions. while we find some evidence that rnns learn a general representation for the existence of filler-gap dependencies (in keeping with @xcite , we find no evidence that rnns learn shared representation of the associated governing constraints (in keeping with @xcite . several recent papers have highlighted ways in which rnn behavior actually reflects shallow surface heuristics @xcite @xcite . note that the representational overlap we seek in the present study is actually a requirement for appropriate generalization of abstract knowledge to unseen data. this is what differentiates abstract knowledge from the surface heuristics that make rnn behavior fragile to adversarial methods. therefore, our finding that rnns fail to learn any shared abstract constraints across filler-gap constructions despite being sensitive to the existence of filler-gap dependencies raises questions about the ability of these models to learn abstract generalizable linguistic patterns. ## background there are a number of different kinds of filler-gap constructions whose behavior is governed by a variety of different underlying constraints (see cumulative priming has been introduced as a method for probing the linguistic representations encoded in rnns @xcite @xcite . 3 this approach involves fine-tuning pretrained models for a single epoch with a small amount of additional training data. the pretrained model acts as a filter on the linguistic features learned during finetuning. more salient features will be affected by the fine-tuning to a larger degree than less salient features. by measuring the responsiveness of the model to linguistic input before and after priming (fine-tuning), researchers can identify which linguistic features are salient for the pretrained model. importantly, if one construction primes a different construction, there is representational overlap between the two constructions within the model. therefore, this approach provides a direct method of separating abstract linguistic knowledge from surface heuristics. ## constructions we analyze eight types of syntactic constructions involving filler-gap dependencies in english. syntacticians often refer to these dependencies as extractions, which alludes to the linguistic theory that filler phrases originate at the gap site before being extracted to the filler site during language production. ## constraints we analyze four underlying syntactic principles that have been hypothesized to govern the behavior of the above constructions. ## models we focused in this work on recurrent neural language models with long short-term memory units (lstms; @xcite . we analyzed five of the highest performing models released by @xcite , 4 who showed that these models perform comparably to state-ofthe-art transformers gpt and bert on many simple syntactic agreement tasks. the models are 2layer lstms with 400 hidden units per layer, each with a unique random initialization, trained on 80 million training tokens of english wikipedia data. analyzing multiple similar models with different random seeds helps ensure that our results are more representative of a class of models rather than simply revealing how a single exceptional model behaves (e.g., bert; @xcite . this is very important given the speed with which new individual models supplant each other in the literature. our results are averaged across all five of our models. ## method we generated 40 sentences per construction type, partitioned into a prime set (15 sentences) and a test set (25 sentences). sentences are available in the supplementary materials. we measured model performance as the average surprisal (negative log-probability; @xcite ) experienced by a model m when processing each word w i of each sentence s j in a set s: priming is achieved by giving the model a single training epoch on the set of priming stimuli 4 https://zenodo.org/record/3559340 s p . this process produces a modified model m p whose performance on a test set s t differs from the original model in a way that gives insight into the representations of the original model. one way to think about this is that pretraining provides a certain kind of model initialization. priming the model moves the model representations along gradients which are characterized by an interaction of the priming stimuli and the initial pre-trained model state. if a set of priming stimuli has a consistent set of features, the initial state's sensitivity to that set of features can be probed with those stimuli. following @xcite , we denote this raw effect of priming (also known as adaptation) as: we are actually interested in the interaction between the original model and the priming set, but the above measure also includes the interaction between the original model and the test set. less expected (more surprising) test constructions can produce larger measures of priming simply because the original model has more room for improvement @xcite . therefore, we used linear regression to predict the size of the priming effect using the original model's performance on the test set: to obtain a more appropriate adaptation effect (ae) for analysis, we subtracted out the predicted linear relation between the original model's test performance and the size of the final priming effect: this measure of priming more directly reflects the interaction of the original model with the priming set, normalizing the adaptation effect by each model and prime construction, and producing a comparable measure to that studied by @xcite . across all analyses, greater values of the adaptation effect indicate greater similarity between adaptation and test structures. recently, @xcite have raised concerns about the efficacy of this technique in probing model representations. specifically, they showed that non-syntactic models can produce qualitative patterns that appear to mimic the effects of syntactic priming. however, their results demonstrate that while the qualitative patterns may be similar, syntactic priming produces much larger effects in syntactic models compared with non-syntactic models. therefore, their results should not be taken as an indictment of this methodology but simply that reasonable baselines must be employed when using this probing method. these shuffled null-prime stimuli do not contain any filler-gap dependencies, and so the adaptation effect from null-priming must represent phenomena in which we are not interested (e.g., lexical priming). our effect-of-interest (filler-gap knowledge) is therefore reflected in any adaptation effect in excess of the null-prime effect. ## probing for filler-gap constraints we next consider constraint-specific priming in order to determine whether the aforementioned abstraction of filler-gap dependency has shared substructure that would indicate understanding of the filler-gap constraints described in section 4. ## discussion our results support previous behavioral findings that recurrent neural language models acquire some abstract concept of "filler-gap dependency" (chowdhury and zamparelli, 2018; @xcite , but go farther than those findings by indicating that this concept is representationally shared across different constructions. however, our findings also indicate that rnns do not encode any of the syntactic filler-gap constraints we studied. both grammatical and ungrammatical sentences involving apparent filler-gap dependencies cause models to anticipate the existence of subsequent filler-gap dependencies. however, in two of four cases, the baseline adaptation effect from priming on unrelated filler-gap constructions was comparable to or greater than that from priming on other constructions governed by the same constraint. constructions that violated ecp and l-marking were represented similarly to constructions that adhered to those same constraints. and models assigned lower probability to sentences involving d-linking than sentences without d-linking, which is the opposite of human results. overall, our results provide robust evidence that chaves (2020) was correct that recurrent neural language models do not fully understand filler-gap constructions. while it is entirely possible that the constraints analyzed in this work do not actually govern filler-gap dependencies (i.e. these particular syntactic theories may be incorrect), we chose four constraints that have been very widely studied by syntacticians precisely because of their broad coverage. therefore, even if the underlying syntactic theories are incorrect, it is conceivable that rnns could induce these constraints as plausible abstractions to aid in filler-gap processing. it is therefore striking that rnns learn none of them. one might wonder whether our priming sets simply needed to be larger to observe the desired priming effects. our priming sets consisted of 15 items, which is comparable to the number of priming stim-uli used by @xcite and @xcite . since the constraints we study involve fewer surface cues, they could require more priming data to produce noticable effects. however, our non-baseline adaptation effects were around 1.5 bits, which is much larger than the 0.5-1 bit priming effects seen in those previous studies. since we are already seeing large priming effects with these constructions, it seems unlikely that increasing the amount of priming data would produce qualitatively different effects. we selected four common, well-studied fillergap influences from the syntax literature and tested whether rnns shared representational features across filler-gap constructions in a way that would suggest they had learned those constraints. while we did find evidence that rnns encode some general representation of the existence of filler-gap dependencies, we found no evidence for more abstract underlying shared constraints. that is, while we find that rnn language models can learn abstract representations that are shared across constructions, our work raises questions about the depth of such abstractions.