Consistency Regularization Training for Compositional Generalization (2023)

Abstract: Existing neural models have difficulty generalizing to unseen combinations of seen components. To achieve compositional generalization, models are required to consistently interpret (sub)expressions across contexts. Without modifying model architectures, we improve the capability of Transformer on compositional generalization through consistency regularization training, which promotes representation consistency across samples and prediction consistency for a single sample. Experimental results on semantic parsing and machine translation benchmarks empirically demonstrate the effectiveness and generality of our method. In addition, we find that the prediction consistency scores on in-distribution validation sets can be an alternative for evaluating models during training, when commonly-used metrics are not informative.

## introduction compositional (systematic) generalization refers to the ability to understand and produce a potentially infinite number of novel combinations of known atoms @xcite . humans exhibit exceptional compositional generalization capability, easily producing and understanding unseen linguistic expressions by recombining the learned rules @xcite . therefore, it is also regarded as a desired property for neural networks. despite the impressive progress in language modeling @xcite @xcite , the sequence-to-sequence (seq2seq) models have been demonstrated inefficient in capturing the compositional rules, thus failing to generalize to novel compositions (lake and @xcite @xcite . achieving compositional generalization requires a model to perform consistently in the interpretation assigned to a (sub)expression across contexts @xcite . for example, the interpretation of a phrase "the book" is consistent whether it is described by a modifier "he likes", in both semantic parsing and machine translation domains @xcite . to improve the consistency, most existing work considers a change of neural architecture to suit particular composition or generalization test sets @xcite @xcite , which limits their potentials in real world applications. recently, the transformer architecture has become the standard for natural language processing (nlp), particularly in supporting large pretrained language models (plms) such as t5 and gpt-3 @xcite . the transformer-based plms have significantly improved few-shot fine-tuning and even made efficient zero-shot learning possible. as a result, there has been a trend towards developing data-centric ai @xcite , where the focus is on data preparation and training strategies rather than on the model architecture. however, it has recently been shown that the standard transformer is underestimated in its ability to handle compositionality @xcite , and there has been relatively little research done on how to improve this capability through training. we observe that limitation of compositional generalization in transformer can arise from the internal inconsistency under the standard training paradigm. first, transformer token representations have been shown to reside within a narrow range of the embedding space @xcite , which can easily be affected by context variations, especially from novel compositions (zheng the book on wildlife got a book write a book checked a book out book the book he likes the 𝒉 𝟏 𝒉 𝟐 𝒉 𝟑 𝒉 𝟒 m 𝑿 𝟐 𝑿 𝟏 𝒀 𝟐 𝒀′ 𝟐 𝒀 𝟏 𝒉 𝟏 𝒉 𝟐 𝒉 𝟑 𝒉 𝟒 book he likes 𝒉 𝟏 𝒉 𝟐 𝒉 𝟑 𝒉 𝟒 write a book … text (grouped by token type) model consistency regularization context token boundary contextualized representation . figure 1 : illustration of consistency regularization training. during training, we encourage the representation consistency of the same token (i.e., "book") across contexts, and enforce the instance-level output consistency when passing the input (e.g., x 2 ) to the model m twice (i.e., y 2 and y ' and lapata, 2022). second, internal uncertainties like dropout can lead to prediction variations of a single sample @xcite . such prediction inconsistency can limit the efficiency of learning patterns in training data @xcite . during inference, this defect is not significant when the models process in-distribution data; however, unseen compositions can magnify the negative influence, which degrades the final performance on compositional generalization. without modifying model architectures, we improve compositionality of transformer with consistency regularization training in terms of representation and prediction. for representation, we encourage the representations of the same token across contexts to be more consistent with each other, and the representations of different tokens to be separated, which can be achieved by contrastive learning @xcite . as shown in the right part of figure 1 , when combined with the modifier "he likes", the representation of "book" is pulled to be consistent with those in other contexts. such representations tolerate context changes better and meanwhile capture discriminative semantics. for prediction consistency, we feed each instance to the model multiple times and force the output distributions of a specific token to be close. in this way, the negative influence of internal uncertainties can be mitigated, which decreases fluctuation in output distributions while maintaining task-specific features. we conduct experiments on standard benchmarks for compositional generalization, including representative semantic parsing datasets (cogs @xcite and cfq @xcite )), and machine translation datasets (cognition @xcite and opus en-nl @xcite ). our method consistently improves upon standard transformer or pre-trained language models, achieving state-of-the-art performance on cogs, cognition, and opus en-nl, and competitive performance on cfq. specifically, we explore a consistency-based metric for model selection on cogs, as commonly-used metrics (e.g., accuracy) on the validation set are often not informative. the analysis of learning efficiency shows that our regularization enables the model to achieve an accuracy score of 18% with only 1.2k samples on cfq mcd1, which the baseline fails to learn. in addition, our analyses of representation variance and robustness to input noise demonstrate that our method delivers better consistency. foot_0 2 related work compositional generalization has attracted increasing attention with dedicated datasets (lake and @xcite @xcite @xcite . one line of research considers dedicated model architectures @xcite @xcite , which perform well on small scaled data but can face difficulties scaling to large or practical data. for example, @xcite propose a differentiable neural network to operate a symbolic stack machine. another line of research enhances the compositionality of standard architectures (i.e., transformer) by introducing new modules @xcite @xcite . however, significant architecture changes can bring about extra training cost or decoding latency. for example, edge transformer @xcite uses vectorbased attention weights, and dangle transformer @xcite re-encodes source representations at each decoding step, which increase model complexity to o(n 3 ). proto-transformer @xcite uses an additional attention module to incorporate prototype vectors obtained by clustering algorithms (e.g., k-means). different from them, we improve transformer from the perspective of regularization training without any architecture changes. recently, @xcite and @xcite empirically make slight changes of transformer components, and find its capability of compositionality is underestimated. meta-learning @xcite and data augmentation @xcite are also introduced to improve the base models, but the experiment results are limited. along the line of compositional generalization studies without modifying the model architectures, our method focuses on the internal consistency of transformer, and achieves better performance. regularization training has been shown effective in semi-supervised training @xcite , robust training @xcite , continual training @xcite , etc. to encourage compositional behavior, @xcite softly combine source/target sequence embeddings during training, and @xcite introduce gradient based meta learning to simulate distribution shift. in addition, contrastive learning serving as regularization has achieved success in various nlp tasks @xcite @xcite . different form them, we explore the effectiveness of the regularization training on the two different tasks in compositional generalization. ## method we propose to regularize the model training in two aspects, as illustrated in figure 1 : representation consistency of tokens across different contexts ( §3.1), and consistency of model prediction for a single sample ( §3.2). ## experiments: semantic parsing this section demonstrates empirical results on representative semantic parsing benchmarks for compositional generalization: cogs and cfq. ## experiments: machine translation unlike semantic parsing, the target of mt is also natural language and compositionality in natural domains is far more intricate. we further validate the effectiveness of our method on two dedicated machine translation datasets: cognition @xcite and opus en-nl @xcite . ## analysis in this section, we aim to provide a deeper understanding of how our consistency regularization improves compositional generalization by analyzing various aspects of the model's performance. ## conclusion we presented a regularization method to enhance compositional generalization, jointly encouraging the consistency of token representations across samples and sample-level prediction consistency. experiments on four dedicated datasets show the effectiveness of our method. the regularized transformer can be a strong baseline for future investigate of compositional generalization.