Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks

1 INTRODUCTION . Recent work has spotlighted significant shortcomings of neural network approaches to NLP in coping with compositional generalization ( CG ) ( Lake & Baroni , 2018 ; Finegan-Dollak et al. , 2018 ; Keysers et al. , 2020 ; Kim & Linzen , 2020 ; Shaw et al. , 2021 ) . In those studies , the test set combines in unfamiliar ways linguistic elements that may themselves be familiar from the training set ; for example , sentences in the test set may be longer than those observed in training , or may use familiar words in syntactic roles in which they did not occur in training . The performance of popular neural architectures in contemporary NLP models such as Transformers drops considerably when they are expected to generalize compositionally . Several approaches have been proposed to address this issue , including specialized architectures with compositional inductive biases ( Furrer et al . 2020 ) and compositional data augmentation ( Andreas , 2020 ) , but as of yet the problem is far from solved . In this paper , we study this challenge from the perspective of transferability of compositional generalization skills : is possible for a neural model to transfer compositional generalization skills acquired from one task to another task that also requires compositional generalization skills ? We ground our inquiries with the semantic parsing tasks on three very different datasets : GeoQuery ( Zelle , 1995 ) , COGS ( Kim & Linzen , 2020 ) and SCAN ( Lake & Baroni , 2018 ) . For each task , we use existing compositional training/test splits or create new ones . We propose a learning algorithm that can extract a compositional inductive bias from one task — a stage we refer to as pre-finetuning1 — and transfers that bias to a target task , improving the models ’ compositional generalization behavior on that task . To extract the inductive bias so as to transfer , we introduce a new training algorithm DUEL . In summary ( cf . Fig 1 ) , DUEL is designed to be compatible with pre-trained neural encoder-decoder models , such as T5 ( Raffel et al. , 2020 ) . We view the encoder as learning a representation for the inputs and the decoder as “ a task head ” that is specialized to different tasks . In pre-finetuing , framing the task ’ s two ( compositional ) splits as deriving representation from one split for zero-shot learning to the other split , DUEL trains the encoder and the decoder using different splits . In contrast to 1We borrow this terminology from the NLP research community . A large number of research papers have explored the idea of transfer learning , starting from a pre-trained language model , followed by training on a set of pre-finetuning tasks , and then fine-tuning on the target or downstream tasks ( Vu et al. , 2020 ; Gururangan et al. , 2020 ; Pruksachatkun et al. , 2020 ; Chen et al. , 2020a ; Aghajanyan et al. , 2021 ) . using standard supervised learning over both splits as a whole for pre-finetuning , DUEL encourages the encoder to learn to represent the input texts in a way that facilitates greater compositional generalization , and that this transfers across domains and persists through fine-tuning on the target task , as shown by our empirical studies . The rest of the paper is organized as follows . In §2 , we detail our setup . We describe our approach in §4 , followed by empirical studies §5 . We discuss related work in §6 and conclude in §7 . 2 PROBLEM SETUP . Our neural learner is given a pre-finetuning task and a target task . For the pre-finetuning task , we have a training data distribution s and a test/eval data distribution s̃ that may deviate from s. Likewise , for the target task , we have q and q̃ . In the case of compositional generalization , the difference between s and s̃ ( or q versus q̃ ) is defined in compositional terms : the ways in which elements combine differ systematically between the two parts of each datasets ( Keysers et al. , 2020 ; Kim & Linzen , 2020 ; Lake & Baroni , 2018 ; Hupkes et al. , 2020 ) . Our goal is to train a neural model on ( s , s̃ ) and then fine-tune it on q so that the final model performs well on q̃ , namely , attaining strong compositional generalization performance on the target task . Our assumption is that the difference between s and s̃ is similar to the difference between q and q̃ , such that an inductive bias acquired from ( s , s̃ ) can be transferred to the target task . A motivating example is that s̃ contains longer texts than s and so does q̃ than q : in this case , we would investigate whether learning to generalize from short to long texts is a transferrable skill . 3 TASK , DATA SETS AND THEIR COMPOSITIONAL SPLITS . Data Splits In this work , we focus on semantic parsing , the task of translating a natural language text into a formal representation of its semantics . Standard data splits for supervised learning partition a dataset randomly into a training set and testing set . As such , the examples in both portions are drawn from the same distribution . By contrast , compositional splits partition a dataset in a way that creates a distributional discrepancy between the training and the test set , such that strong performance on the test set requires compositional generalization from the training set . Compositional splits are straightforward to create for datasets that are generated from grammars or templates , such as SCAN ( Lake & Baroni , 2018 ) and COGS ( Kim & Linzen , 2020 ) , described in detail in the next section . For instance , the templates used for the test set can include specific words in syntactic roles they did not appear in in the training set , or they can nest syntactic constituents more deeply than in the training set , creating differences in lengths of the input texts . An alternative mechanism creates—based on an existing dataset for which the generative process is not available ( for example , real-world user queries ) —test sets that require compositional generalization by on splitting the dataset so as to obtain the maximum compound divergence ( MCD ) between the training and test examples ( Keysers et al. , 2020 ) . Here , compounds are complex expressions formed by composing atoms , and compound divergence is computed as DC ( V ‖W ) = 1 − C0.1 ( FC ( V ) ‖FC ( W ) ) where F is the distribution over compounds , and Cα ( P‖Q ) = ∑ k p α k q 1−α k . The intuition behind this method is that given an existing dataset it is difficult to limit the test examples to compounds that are completely absent from the training set , but by making the compound distributions as distinct as possible , we are primarily measuring the model ’ s performance on compounds are at least very infrequent in training . We use the MCD split generated for SCAN from Keysers et al . ( 2020 ) and the Target Maximum Compound Divergence ( TMCD ) splits generated for GeoQuery from Shaw et al . ( 2021 ) . TMCD splits extend the MCD methodology to create compositional splits of non-synthetic datasets and define atoms and compounds based on the known syntactic structure of the output logical forms . Datasets and Tasks Table 1 summarizes the 5 compositional splits we use in this paper . We also compared with standard splits ( std ) where the training and test come from the same distribution . As an example for our setting in §2 , taking COGS as our pre-finetuning task and GeoQuery as our target task , we learn from the CG General split of COGS and transfer to the CG General split of GeoQuery . In this example , COGS BASE is the distribution s and COGS CG is the distribution s̃ . GEO TMCD1 is our fine-tuning distribution q and GEO TMCD2 is our final test distribution q̃ . COGS ( Kim & Linzen , 2020 ) is a synthetic semantic parsing dataset generated from templates . The inputs are English sentences and the outputs are corresponding logical forms inspired by λ-calculus , e.g. , A dog ate the cake→ * cake ( x4 ) ; dog ( x1 ) AND eat.agent ( x2 , x1 ) AND eat.theme ( x2 , x4 ) . COGS has two components : a training dataset , which we refer to as COGS BASE , and a compositional generalization ( CG ) dataset COGS CG . Kim & Linzen ( 2020 ) show that the performance of neural models trained on COGS BASE degrades significantly when they are applied to COGS CG . COGS CG includes two kinds of CG challenges : lexical and structural . In lexical CG , familiar words need to be interpreted in new syntactic positions ; for example , in COGS BASE the word hedgehog may occur only in the subject position , in COGS CG it would need to be interpreted as as object . By contrast , structural CG involves new combinations of familiar syntactic structures . For example , in COGS BASE , prepositional phrases only modify objects ( Noah ate the cake on the plate ) , whereas in COGS CG they modify subjects ( The cake on the plate burned ) . Likewise , in COGS BASE prepositional phrases can only be nested once ( Ava saw the ball in the bottle on the table ) , but in COGS CG they are nested multiple times ( Ava saw the ball in the bottle on the table on the floor ) , creating longer sentences . COGS BASE has 24,155 examples and COGS CG has 21,000 examples , of which 18,000 are instances of lexical CG and 3,000 are instances of structural CG . The GeoQuery ( Zelle , 1995 ; Tang & Mooney , 2001 ) is an annotated semantic parsing dataset contains 880 natural language questions about US geography ( e.g. , What states border Texas ? ) . We use the same pre-processing as in Shaw et al . ( 2021 ) , replacing entity mentions with placeholders in the Functional Query Language ( FunQL ; Kate et al . 2005 ) output representations . The input what states border m0 , for example , is associated with the output answer ( intersection ( state , next_to_2 ( m0 ) ) ) . Atoms and compounds are defined over the FunQL expressions . We adopt the length and TMCD splits from Shaw et al . ( 2021 ) . For the length split GEO len , the training set GEO SHORT consists of examples with shorter inputs than the test set GEO LONG . We refer to the training and test set for the TMCD split as GEO TMCD1 and GEO TMCD2 . Both splits divide GeoQuery equally into two sets of 440 examples . The SCAN dataset ( Lake & Baroni , 2018 ) consists of over 20,000 natural language navigation commands and corresponding “ action sequences ” , e.g . jump twice→ JUMP JUMP . SCAN is not a semantic parsing dataset but is commonly used as a diagnostic dataset for evaluating compositional generalization . We adopt the length split ( SCAN len ) and MCD split ( SCAN cd ) introduced by Keysers et al . ( 2020 ) . 4 PROPOSED METHOD . Intuition and Main Idea As mentioned in §2 , our goal is to improve compositional generalization on our target task , where the training distribution is q and the evaluation distribution q̃ . The challenge is to leverage the information in s and s̃ during the pre-finetuning stage such that the model , when fine-tuned on q , generalizes better on q̃ . This contrasts with a standard supervised learning approach , which would collapse together the examples from s and s̃ during pre-finetuning , discarding the information about how examples are divided between s and s̃—information that , we hypothesize , can be useful for encouraging compositional generalization on the target task . We assume a neural encoder-decoder architecture ( Cho et al. , 2014 ) . Our proposed method , DUEL , iteratively updates the parameters of the encoder on examples from s̃ , while keeping the decoder fixed , and then updates the decoder parameters on examples from s , while keeping the encoder fixed . The encoder updates are then retained prior to fine-tuning on q , while the decoder updates are discarded ( as the pre-finetuning and the target tasks are different ) . In contrast to standard fine-tuning , DUEL encourages the encoder to learn to represent the input sequences in a way that facilitates compositional generalization , which we hypothesize will transfer across domains and persist through fine-tuning . Details For encoder-decoder architectures , the model ’ s parameters naturally partition into two sets , θ for the encoder and φ for the decoder , where θ parametrizes the representation f ( x ; θ ) of the input x and φ parametrizes the “ task head ” g ( · ) , which uses that representation to accomplish a task such as classification or semantic parsing . Taken together , the model ’ s output is p ( y|x ; θ , φ ) = g ( f ( x ; θ ) ; φ ) . Given a pre-finetuning task where both s and s̃ are given , our goal is to drive the learner to learn the optimal representation and task head parameters ( θ∗ , φ∗ ) such that g ( f ( x ; θ∗ ) ; φ∗ ) performs the best on both distributions . To achieve this , we design an iterative dueling game that the learner plays with the two distributions . Suppose the learner ’ s parameters are currently ( θ , φ ) . To perform optimally on s̃ , we would like to use the task head g ( · ; φ ) without further training it , i.e. , perform well in an “ zero-shot ” setting ; this setting directly anticipates how learning would proceed for the target task , where we will be training only on q , without exposure to q̃ . Since we are holding the task head g ( · ; φ ) fixed , we need to optimize the representation f ( · ; θ ) such that it models s̃ appropriately . To achieve this desiderata , we update the representation parameters iteratively for at most Tinner steps , according to θ ← θ + α 1 N ∇θ ∑ ( x , y ) ∈Bs̃ log p ( y|x ; θ , φ ) ( 1 ) where B s̃ is a batch of N samples from s̃ and α is the step size.2 Updates to φ follow the reverse logic . The representation function that results from updating θ defines a prior on how s should be represented . When updating the task head parameters φ , our goal is to have the task head perform optimally conditioned on this prior . Concretely , we hold θ fixed and apply iteratively at most Tinner steps to φ : φ← φ+ α 1 N ∇φ ∑ ( x , y ) ∈Bs log p ( y|x ; θ , φ ) ( 2 ) where Bs is a batch of samples from s. Crucially , we alternate between these two types of updates , such that s̃ informs the learner about the compositional inductive bias that should be incorporated into the representation f , and s teaches the learner how to use that representation to perform the task ( here semantic parsing ) . 2Eqns . ( 1-3 ) only aim at illustrating the computation of gradient . We use the Adam optimizer in the implementation . Algorithm 1 : DUEL Require : Data sets s , s̃ ; Learning rate α ; Batch size N ; Outer loop iterations Touter ; Inner loop iterations Tinner ; Inner loop early stopping criteria Tpatience ; Outer loop early stopping criteria Tmin ; Initialize model parameters θ , φ i← 0 while i < Touter do j ← 0 while j < Tinner do Sample a batch Bs from s Compute loss : L ( θ , φ , Bs ) = 1N ∑ ( x , y ) ∈Bs − log p ( y | x ; θ , φ ) Update parameters : φ← φ− α · ∇φL ( θ , φ , Bs ) j ← j + 1 if AccuracyDecreases ( θ , φ , s̃ , Tpatience ) then Early stop : break end end k ← 0 while k < Tinner do Sample a batch Bs̃ from s̃ Compute loss : L ( θ , φ , Bs̃ ) = 1N ∑ ( x , y ) ∈Bs̃ − log p ( y | x ; θ , φ ) Update parameters : θ ← θ − α · ∇θL ( θ , φ , Bs̃ ) k ← k + 1 if AccuracyDecreases ( θ , φ , s , Tpatience ) then Early stop : break end end if k < Tmin then Early stop outer loop : break end i← i+ 1 end Return : Model parameters θ , φ We contrast our approach with the standard supervised learning approach , where the two distributions are merged together as s ∪ s̃ , ignoring the compositional split : ( θ , φ ) ← ( θ , φ ) + η 1 N ∇ ( θ , φ ) ∑ ( x , y ) ∼Bs∪s̃ log p ( y|x ; θ , φ ) ( 3 ) DUEL is used for pre-finetuning . When fine-tuning the model on the target task q , we retain the representation component f ( · ; θ ) and re-initialize the task head g ( · ; φ ) . Both θ and φ are then updated , as is standard , to optimize the loss function on q. DUEL is illustrated schematically in the pre-finetuning panel of Fig . 1 . The pseudocode is listed in Algorithm 1 . The algorithm contains one outer loop for Touter rounds ( with possible early stopping ) , which alternates between inner loops updating θ and φ . The early-stopping monitoring function AccuracyDecreases takes as arguments the current model parameters θ and φ , a data distribution to use for evaluation ( s or s̃ ) , and the maximum patience Tpatience for early-stopping . This function returns true if accuracy on the dataset in question has not improved for Tpatience consecutive steps . To terminate the outer loop ( thus , the algorithm ) , we keep track how many steps the inner update for θ take . If the number steps is smaller than a preset threshold Tmin , we conclude that the algorithm has converged to the desired representation since the difference in representing s and s̃ is small , requiring little adaptation . We do not use the same logic to limit how long it takes to optimize the task head parameters φ , conditioned on the representation , as the goal is to use the derived representation to arrive at a strong performance .

Summary: This paper presents a transfer learning strategy for improving compositional generalization of semantic parsers based on pre-trained language models. Before fine-tuning the model on data from the target domain, the authors propose a pre-finetuning step, where models are trained on compositional splits of data from another source domain, with the goal to transfer the model's learned knowledge about language compositionality during this pre-finetuning step to the final learning stage on the target domain, therefore improving compositional generalization. To this end, the authors propose a pre-finetuning method which encourages the model to discover representations of natural language that are invariant against its compositional structures. This is achieved by iteratively freezing the encoder or decoder modules during pre-finetuning, and training the encoder and decoder modules on compositionally disjoint splits of the source data, such that the encoder learns representations that are robust against distributional shift of language compositionality.