Systematic Generalization by Finetuning? Analyzing Pretrained Language Models Using Constituency Tests (2023)

Abstract: Constituents are groups of words that behave as a syntactic unit. Many linguistic phenomena (e.g., question formation, diathesis alternations) require the manipulation and rearrangement of constituents in a sentence. In this paper, we investigate how different finetuning setups affect the ability of pretrained sequence-to-sequence language models such as BART and T5 to replicate constituency tests — transformations that involve manipulating constituents in a sentence. We design multiple evaluation settings by varying the combinations of constituency tests and sentence types that a model is exposed to during finetuning. We show that models can replicate a linguistic transformation on a specific type of sentence that they saw during finetuning, but performance degrades substantially in other settings, showing a lack of systematic generalization. These results suggest that models often learn to manipulate sentences at a surface level unrelated to the constituent-level syntactic structure, for example by copying the first word of a sentence. These results may partially explain the brittleness of pretrained language models in downstream tasks.

## introduction the study of syntax revolves around understanding how words and phrases are combined to form sentences. certain groups of words, known as constituents, behave as units in a sentence. in linguistics, groups of words that form constituents are often identified via constituency tests (see, e.g., @xcite . the tests involve transforming an input sentence using operations that substitute, displace, or otherwise modify constituents. for example, one well-known constituency test is proform substitution whereby a constituent is replaced by a corresponding pronominal form: john gave the book about syntax to the student-→john gave it to the student. the fact that the phrase the book about syntax can be replaced by it as a single unit, indicates that it is a constituent. other examples of constituency tests include clefting and wh-question formation. previous work has shown that pretrained transformer-based @xcite language models (lms), trained on large amounts of text achieve unparalleled performance on virtually every downstream task. nevertheless, these models suffer from robustness issues which call into question their reliability and ability to recover human-like linguistic generalizations. for example, in natural language inference, it has been shown that models rely on superficial cues such as lexical overlap between the premise and the hypothesis in order to make the correct predictions @xcite . a line of work has thus attempted to probe pretrained lms by training classifiers on top of frozen lm weights in order to extract some desired linguistic representation such as syntax trees @xcite . work by @xcite also shows that lstm based lms contain information about relative clauses in an interpretable manner. however, such analyses have several limitations. first, it is difficult to separate the contributions of the pre-trained lm weights from that of the probing classifier. in practice, it is necessary to place some constraints on the classifier (e.g., they must be linear) to ensure that extractive performance can be attributed to the pre-trained lms. more fundamentally, understanding what can be extracted from pre-trained representations is a different issue from the nlp system designer's ultimate concern, which is whether a fine-tuned system will perform well on new data which involves novel combinations of the units it has seen during pre-training and finetuning; that is, its ability to generalize systematically @xcite . in this paper, we propose analyzing the syntactic competence of pre-trained and fine-tuned lms. we focus on the phenomenon of constituency because of its core role in supporting semantic understanding and natural language inference, with applications to a wide variety of downstream tasks in nlu and nlg. we ask whether lms after fine-tuning are able to perform transformations on input sentences which correspond to well-known constituency tests from the literature. performing these transformations correctly requires that models represent the constituency structure of sentences. in our experiments, we systematically vary two dimensions of generalization: i) whether input sentences represent novel syntactic constructions and ii) whether the main verb of the sentence is novel. we also pose as a control a version of the input sentence where constituents of varying lengths are replaced by their head word. our results show that models are able to correctly transform input sentences only when tested on verbs and syntactic constructions that they were trained on. these results indicate that it is unlikely that these models have acquired a human-like representation of constituent structure, instead suggesting that they instead leverage surface-level cues. ## syntactic constructions our interest in this work is to probe the behaviour of a pre-trained language model m after it is finetuned to transform input sentences according to a number of constituency-sensitive transformations (cst) which are inspired by constituent tests from the literature. in this section, we describe the set of syntactic constructions which we use to construct our training and test datasets, as well as introduce a notation which allows us to compactly specify csts with respect to these constructions. we start with the set of simple declarative base sentences presented in @xcite (d base ). each of these sentences is constructed using a main verb which is either an english dative or locative alternator @xcite , which are well-understood syntactic alterations in the literature. each sentence s can appear as one of four types, which we denote using the feature a with the following notation s[a] ∈ {do, po, loc, in}. dative sentences using dative main verbs may either be in the double-object (s[a] = do) or the prepositional object construction (s[a] = po): our corpus also includes more complex sentences that result from modifying one of the arguments of each verb using a transformation inspired by a constituency test from the literature. pronominalization (p) proform substitution involves replacing one of the verbal arguments with an appropriate proform (he, it, them, etc.). we will use the feature p to indicate whether a sentence contains a noun phrase that is pronominalized, and if so, which syntactic position is pronominalized. putting this all together, the syntactic form of any sentence in our training or test sets can be described by specifying values for our four features a, c, p , and w . for example, the complete featural description of (2) would be: informally, a constituency-sensitive transformation t can be can be thought of as function s @xmath0, displaces, or otherwise modifies one or more constituents in a grammatical input sentence s in in such a way that grammaticality is maintained. in terms of the featural representation we have just introduced, a cst t can be described by (re)assigning a value val for feature f in the description of some sentence. we will denote this (re)assignment as t f ←val . for example, a cst which pronominalizes the direct object of a sentence would be written t p ←do (s) : s[p ] ← do. the sentence that results from applying this transformation to input s 2 that is, t p ←do (s 2 ), would then be (5). in our setting, a training or test item is a triple (t, s in , s out ) where t is a cst as described in the preceding section and s in and s out are sentences such that t (s in ) = s out . note that in our framework, all three parts of the triple are observed during training-the models observe which cst characterizes the relationship between s in and s out . test and training datasets are disjoint sets of such items. a model m will be finetuned on d train and tested according to its performance on d test . recall that all of the sentences in our dataset are built using main verbs which are either dative or locative alternators. thus, for all sentences, the value of the base argument structure feature a is one of do, po, loc, or in. the high-level idea of our experiments is to see if the models can correct apply a target cst to sentences which vary in terms of their similarity to training sentences. in each evaluation we choose a single cst called the target cst. in practice, we only evaluate csts which correspond to dative or locative alternations (i.e., we only test the following csts a ← po, a ← do, a ← ins, a ← loc). for the sake of concreteness, our description of training data construction below will use the target a ← po, but the other targets are handled analogously. these are simple declarative sentences in the double object construction without any further pronominalization, clefting, or wh-question formation applied. d train will consist of the union of following sets of triples (t, s in , s out ): i. non-base transformations applied to each @xmath1: ii. the target transformation applied to the base sentences: thus, training also includes the prepositional object alternation applied to double object base sentences. iii. the target verb alternation, applied in the opposite direction (i.e., a ← do), to a sentence which is in one of the more complex construction forms (pronominalized, clefted, wh-item). let s ′ be the base sentence s transformed to be in the po construction s ′ = t a←p o (s). test at test time, the model will be evaluated on its performance on the set d test , which consists of the target transformation applied to the output of i.: notably, the model is trained on each of the transformations involved in generating the test set (t a←p o , t c← * , t p ← * , t w ← * ), as well as on all construction types. our test sets vary (i) whether particular combinations of cst and input sentence type are held out (ii) whether particular verbs and arguments are held out and (iii) whether constituents vary in length between test and train. ## results we evaluate the models proposed in section 5 using the tasks proposed in section 3. we apply the four base transformations (t a←p o , t a←do , t a←loc , t a←in ) to the three separate non-base transformations (clefting, pronormalization and wh-question generation). we take an average of the results corresponding to the four base transformations. ## conclusion in this paper, we systematically vary the task setup and the training signals to do a behavioral analysis of pretrained sequence-to-sequence models. we design several linguistic tests including verb argument structure alternations, proform substitution, clefting and wh-question generation. we show that the models fail to generalize well when the target transformation is held-out. we attribute this to the failure of the pretrained language models in utilizing constituency information and relying on surface-level cues. we further show that simplifying the constituent boundaries improves the generalization capabilities of these models. furthermore, increasing the number of out-of-vocabulary tokens in the test corpus decreases the generalization performance of these models.