How Do In-Context Examples Affect Compositional Generalization? (2023)

Abstract: Compositional generalization–understanding unseen combinations of seen primitives–is an essential reasoning capability in human intelligence. The AI community mainly studies this capability by fine-tuning neural networks on lots of training samples, while it is still unclear whether and how in-context learning–the prevailing few-shot paradigm based on large language models–exhibits compositional generalization. In this paper, we present CoFe, a test suite to investigate in-context compositional generalization. We find that the compositional generalization performance can be easily affected by the selection of in-context examples, thus raising the research question what the key factors are to make good in-context examples for compositional generalization. We study three potential factors: similarity, diversity and complexity. Our systematic experiments indicate that in-context examples should be structurally similar to the test case, diverse from each other, and individually simple. Furthermore, two strong limitations are observed: in-context compositional generalization on fictional words is much weaker than that on commonly used ones; it is still critical that the in-context examples should cover required linguistic structures, even though the backbone model has been pre-trained on large corpus. We hope our analysis would facilitate the understanding and utilization of in-context learning paradigm.

## introduction compositional generalization is an essential capability of human intelligence. it means to understanding and producing novel expressions by recombining known components in language @xcite @xcite . taking examples in figure 1 , after learning the combination "baby in a room", human intelligence can easily generalize to "jackson in a room". on exploring this human-like capability in deep learning models, several benchmarks such as scan @xcite , cfq @xcite and cogs @xcite have been proposed based on semantic parsing foot_0 tasks. in these benchmarks, the training set cover all the primitives while lacking certain combinations, and the test set focuses on these missing combinations. by fine-tuning generic neural models on these benchmarks, much work reported that these models exhibit poor compositional generalization @xcite @xcite . recently, in-context learning with large language models exhibits impressive performance on various tasks @xcite @xcite . by conditioning on few-shot incontext examples, the pre-trained language model, with extremely large model size and pre-trained corpus, can perform downstream tasks without any update on pre-trained parameters. behind the impressive performance of in-context learning, we are curious whether this prevailing paradigm can take a step towards compositional generalization. to investigate this, we first take an initial exploration: for each test case in cogs, we select in-context examples from its training set and ensure that all primitives in each test case are covered by the equipped in-context examples. our initial exploration suggests that compositional generalization can be easily affected by in-context examples: with only covering primitives, davinci 175b lags behind fine-tuned gpt2-large with 24.2% accuracy (similar to the observation in @xcite ); with also covering some local structures (inspired by @xcite ), davinci outperforms fine-tuned gpt2-large with 3.9% accuracy. based on these initial observations, we raise and investigate the question: how do in-context examples affect compositional generalization? we construct the test suite cofe (based on cogs) to facilitate our systematic investigation. taking the coverage of primitives as a basic principle in cofe, we further define and inject three factors in selecting in-context examples: similarity, diversity, and complexity. similarity is considered as the matching of hidden structures behind concrete expressions. diversity reflects whether the context presents repeated patterns or not. complexity portrays the amount of information contained in each example. by controlling these factors in constructing cofe, we can systematically investigate how would in-context examples influence the performance on compositional generalization. our experiments demonstrate that all three factors matter for in-context compositional generalization. we leverage six large language models in gpt series: davinci, code-cushman-001, codecushman-002, text-davinci-002, text-chat-davinci-002, and code-davinci-002. the observations are consistent across models: to better perform compositional generalization, all backbone models prefer in-context examples with higher structural similarity to the test case, higher diversity among different examples, and lower complexity in each individual example. furthermore, beyond the influence from these factors, in-context compositional generalization still faces two challenges. one is that in-context learning has difficulty recombining fictional words (e.g., random tokens) rather than com-monly used ones. the other one is that in-context examples are still required to cover the linguistic structures in nl expressions, even though the backbone model has been pre-trained on large corpus. our contributions are three-fold: 1) to answer the research question posed, we investigate three factors in selecting in-context examples and draw consistent conclusions across models; 2) we construct cofe to conduct our systematic investigation, and will release it to facilitate further exploration of in-context compositional generalization; 3) we also point out two remaining challenges that in-context learning still struggles to handle. we hope our analysis would provide insights on how to select proper in-context examples, and to shed light on the future research of in-context compositional generalization. cofe is publicly available at https://github.com/microsoft/contextua lsp/tree/master/cofe . ## in-context compositional generalization in-context compositional generalization refers to understand and produce novel combinations through recombining the building blocks presented by in-context examples. we first introduce some basic settings for testing this desired capability, then show our initial observations. ## factors under in-context examples to facilitate our systematic investigation, we construct cofe (compositional generalization with few-shot examples), which is derived from cogs. for selecting in-context examples in constructing cofe, we identify, inject, and control three potential factors: similarity, diversity, and complexity. ## remaining challenges our investigation has revealed a huge potential of in-context learning on performing compositional generalization 5 . despite this potential, for achieving the ideal in-context compositional generalization, there remains the following two challenges. in-context examples are still required to match linguistic structures in nl expressions. since all backbone models have been pre-trained on large natural language corpus, we expect that these models could already handle the high variety in nl expressions without further hints from in-context examples. motivated by this, we conduct experiments on another variant of cofe: the source-side term match(x, x c ) is removed from equation 2, and the coverage of s(x) is limited (detailed in appendix c.6). figure 8 shows that on all five categories, the performance consistently drops if incontext examples do not match the nl-side structure. it suggests that even having been pre-trained on large corpus, in-context learning still struggles to effectively recognize the semantic equivalence among different linguistic structures behind nl expressions (detailed in appendix f.3). in-context learning has difficulty leveraging fictional words foot_3 . the ideal compositional generalization requires that the recombination of primitives should be independent of the surface form in primitives. in cofe, we set the target-side primitives as the uppercase of source-side ones (e.g., "cat"→"cat"). such case conversion is commonly used in semantic parsing tasks. to test whether in-context learning could use fictional words, we replace each target-side word with random characters (e.g., replace "cat" with "mxr", detailed in appendix c.7). figure 9 shows the huge drops after changing words. moreover, we investigate the structural accuracy by only keeping the structural terminals (e.g., parentheses and commas) in predictions. figure 9 shows that the structural accuracy is also affected by fictional words. it indicates that on performing in-context compositional generalization, the prediction of structural sketch is not decoupled with word-level patterns. compositional generalization (cg) has attracted much attention in nlp field. most existing benchmarks measured cg under fine-tuning with synthetic semantic parsing tasks, suggesting the limitations of general-purpose neural networks @xcite @xcite . many approaches were proposed to enhance the cg on general-purpose models @xcite @xcite @xcite or design task-specific methods @xcite ; herzig and berant, 2021; @xcite . some influential factors that affect cg have been revealed, such as the length bias @xcite , target-side format @xcite and local structures @xcite . most existing work explored cg under the fine-tuning paradigm, while our work advances the exploration under the in-context learning paradigm. in-context learning (icl) along with large language models (llms) has shown surprising performance in many nlp tasks @xcite @xcite @xcite @xcite @xcite . most related to our work, @xcite and @xcite also explored icl on cg challenges. @xcite utilized the targetside similarity on structural fragments and reported that llms still exhibited much poorer cg than finetuned small models on cogs, which is close to our initial observations. @xcite designed task-specific inference pipelines for performing cg under a least-to-most manner. our work provides more general understandings on how to improve cg performance by revealing several factors in selecting in-context examples. in addition, some more recent work has similar observations on the potential of llms on cg @xcite , gains from diversity @xcite , and challenges under fictional words @xcite selection of in-context examples is an essential part for the utilization of icl. most existing work considered the similarity as the major metric during selection. liu et al. (2022) selected k-nearest neighbors with similar sentence embeddings; shin et al. (2021) regarded the conditional probability from a pre-trained llm as the similarity score; rubin et al. (2021) and zhang et al. (2022b) separately trained a retriever to score the similarity; poesia et al. (2021) and madaan et al. (2022) estimated the target-side similarity. this work demonstrates the necessity of structural similarity in achieving cg, and also reveals the importance of two other factors beyond similarity, i.e., diversity and complexity. ## conclusion and future work this work investigates how in-context compositional generalization is affected by the selection of examples. the test suite cofe is constructed to study three factors. experiments show the effects of structural similarity, higher diversity and lower complexity. two challenges under in-context compositional generalization are further revealed. to apply our revealed factors outside the cofe test suite, one main challenge for future work is to determine the hidden structures behind expressions without knowing the exact generative grammar. here, we consider two potential approaches. one is to use a pre-trained parser to generate a parse tree for the input query and then measure tree similarity. the other approach is to pre-train an embedding model with a structure-aware training objective and then compute embedding similarity.