Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Task (2022)

Abstract: Although transformer-based Neural Language Models demonstrate impressive performance on a variety of tasks, their generalization abilities are not well understood. They have been shown to perform strongly on subject-verb number agreement in a wide array of settings, suggesting that they learned to track syntactic dependencies during their training even without explicit supervision. In this paper, we examine the extent to which BERT is able to perform lexically-independent subject-verb number agreement (NA) on targeted syntactic templates. To do so, we disrupt the lexical patterns found in naturally occurring stimuli for each targeted structure in a novel fine-grained analysis of BERT’s behavior. Our results on nonce sentences suggest that the model generalizes well for simple templates, but fails to perform lexically-independent syntactic generalization when as little as one attractor is present.

## introduction every english speaker would judge as grammatical the sentences in (1a)-(1b), but not those in (1c)-(1d), despite that they are all meaningless: colourless green ideas that cook the door sleep furiously. c. at least since @xcite , data like this has been taken as evidence that natural language grammars contain abstract syntactic rules that (i) are independent of the meaning of lexical items and (ii) obey hierarchical, rather than linear constraints. number agreement (henceforth na) between the subject (the cue) and the verb (the target) of the same clause in english is one of such rules @xcite . in fact, (1d) is ungrammatical, even though the closest noun door (typically referred to as attractor) has the same number as sleeps, because the noun belongs to an embedded relative clause. these na properties have made it one of the preferred test beds to investigate the ability of neural language models (nlms) to learn abstract, hierarchical syntactic structures @xcite @xcite @xcite . although recurrent and transformerbased nlms have been shown to possess syntactic abilities on the task, their nature is not fully understood @xcite . can nlms really perform lexically-independent number agreement, regardless of the syntactic structure? to answer this question, we test bert @xcite against the na task while controlling both the syntactic constructions and the meaningfulness of the stimuli presented to the model. our experiments provide two main findings. contrarily to previous observations that bert performs fairly well on @xcite syntactically well-formed but meaningless sentences @xcite , we show that its generalization abilities are not lexically-independent on syntactic constructions where an attractor is present foot_0 . though the model has been previously shown to ignore attractors belonging to an embedded clause independent of that containing the target (goldberg, 2019), we further provide insights on this lexical dependence that reveal the limitations of the model's abilities. our experiments rather show that the model is actually sensitive to the presence of attractors when semantic and lexical patterns are disrupted in its input sentence. goldberg (2019) further tested bert, a transformer-based model, against stimuli from @xcite , @xcite and @xcite . he found that bert substantially outperforms the previously tested lstm language models. @xcite have recently tested generalizations beyond @xcite data by extending the vocabulary at the target verb position. they show that though nlms' top predictions are generally correct verbforms, the models still struggle on the na task for infrequent verbs. in addition to testing the effect of meaningfulness by performing replacements at all positions of the sentence similarly to @xcite , we control for the syntactic constructions from @xcite : given a syntactic template, can bert generalize to any syntactically well-formed, but meaningless sentence? if not, when does lexical content matter? ## discussion previous na studies have led @xcite to claim that "the linguistic proficiency of neural networks extends beyond shallow pattern recognition". though it is undeniable that bert does generalize beyond its input and is able to carry out the na task on the simplest templates, our experiments also suggest that these generalizations can be lexically dependent. when naturally occurring lexical patterns are replaced with syntactically well-formed, but meaningless combinations, the model's syntactic ability seems to be heavily compromised, contrary to @xcite 's reported results on the @xcite stimuli. moreover, most disruption is caused by replacing the words closest to the target within the embedded phrase, that in principle should not affect the agreement relation. these two facts together indicate that some of bert's syntactic abilities are limited to specific word sequences that the model could have memorized during training, including words that are linearly close but belong to a different embedded phrase or clause. furthermore, the fact that the model improves its performance on data it has been trained on (i.e., the wiki dataset) over other meaningful, unseen sentences (i.e., the m&l dataset) is further evidence that at least part of its alleged generalization abilities might be just the effect of memorization. we can surmise that the model relies on a variety of heuristics acquired during training to approximate syntactic generalizations, in line with @xcite , who found two distinct mechanisms to accomplish agreement in transformerbased architectures. we find that those heuristics can therefore tend to be highly lexicalized, similarly to @xcite who showed that generalization is not systematic by testing a wide range of verbs. this is confirmed by bert's sensitivity to the main verb when there is no overt complementizer 5 , which prevents it from solving the na task. this suggests that the model has acquired semi-lexicalized syntactic information about verb subcategorization preferences. although bert's ability to approximate syntactic rules is probably more brittle than previously argued, this should not lead to rejecting its ability to learn natural language grammar. for instance, constructionist approaches @xcite have argued since long against a purely abstract grammar detached from lexical meaning, despite what the data in (1) have often been claimed to prove. the alternative view is a grammar consisting of constructions that differ for their level of abstractness and lexicalization. bert's lexicallydriven behavior could therefore be consistent with this less abstract conceptions of syntax. finally, given previous experiments @xcite , we can speculate that humans could also similarly manifest patterns of 5 cf. sentence type b no that errors driven by semantic, or lexical interferences from words linearly close to the target. though such patterns seem to differ between language models and humans @xcite , this in turn leads us to questioning our expectations regarding the syntactic abilities of neural language models. ## conclusion in this paper, we have shown that bert's ability to solve the na task on meaningless sentences strongly depends on the stimuli's syntactic template. while the model is able to perform lexicallyindependent generalization in simple settings, it fails when the agreement relation crosses an embedded phrase containing an attractor. we further provide insights on this lexical dependence, showing that the model relies mostly on the lexical content at the closest positions to the target of the agreement, though they belong to an independent embedded phrase. in the future, we want to get a better understanding of the mechanisms underlying the observed syntactic abilities of transformers, and in particular what makes some heuristics involved to solve a syntactic task lexically dependent. a more detailed analysis of the influence played by lexical combinations will help us understand the nature of the heuristics the model uses to solve complex na cases involving one or more attractors. moreover, we wish to compare bert's predictions with human judgments on our meaningless sentences.