Distinguishing fair from unfair compositional generalization tasks (2025)

Abstract: Compositional generalization benchmarks seek to assess whether learning agents can successfully combine familiar concepts in novel ways. COGS (Kim & Linzen 2020, COGS, EMNLP) provides a suite of such tasks in the area of interpretive semantics (mapping sentences to logical forms). A noteworthy finding for COGS is that model performance varies widely across tasks. In this paper, we argue that these performance differences reflect deep properties of these tasks. We focus on two COGS tasks: an easy task (models are generally successful) and a hard task (no present-day models get any traction). Using both experiments and conceptual analysis, we argue that the easy task requires only a single distributional generalization that is well-supported by the training data, whereas the hard task involves a learning target that is ambiguous or even contradicted by the training data. We additionally argue that pretraining can disambiguate the hard task without compromising the goal of testing compositional generalization. Overall, our findings offer practical guidance to designers of compositional generalization benchmarks and also yield new insights into the nature of compositionality itself.

## introduction humans routinely produce and interpret novel sentences. the principle of compositionality @xcite @xcite seeks to explain this productivity by hypothesizing that the meanings of complex expressions are determined by the meanings of their parts, rescursively down to primitive lexical meanings. thus, even novel combinations of familiar elements have predictable and stable meanings. do language models (lms) process language in a similarly systematic way? compositional generalization benchmarks @xcite @xcite seek to address this question by creating train-test splits in which figure 1 : gen proper : the training data holds out a certain distribution for paula, as depicted by the missing red and green dotted lines for paula. evaluating on the held-out distribution, as in generalization target, leads to failure-unless the lexicon (prim-lf pairs) contains information that the learner can use for categorizing the distributionally constrained item with other distributionally free items in the training data. in the present paper, we argue that at least some of these performance differences are the result of fundamental differences in how the tasks themselves are designed. to develop the argument, we focus on two cogs tasks: primitive figure 2 : gen pp : the train set holds out the distribution for pps as modifiers of subjects, depicted by the discrepancy in the training and generalization distribution. pps can be categorized together by using the information in the lexicon (pp-lf pairs). this is not enough for generalization, for no pp has the distribution which can be used to learn that pps can occur within subjects. to subject/object (proper names) and ob-ject pp to subject pp. in the first (henceforth gen proper ), a subset of proper names p appear only as lexical primitives during training, while others appear as both subjects and objects. the test-time generalization task is to handle names in p as subjects and objects. this task proves to be straightforward for present-day models that are trained from scratch. in the second (henceforth gen pp ), pp modifiers appear only inside direct objects during training. the test-time generalization task is to handle pp modifiers inside subjects. this task is one where all present-day models, trained from scratch, score about 0. our core claim is that gen proper is a well-posed generalization task in a way that gen pp is not. figures 1 and 2 summarize the key difference. for gen proper , the primitive lfs for the held-out names convey to the model that they are members of the broader class of proper names. the train distribution covers all the test-time environments for names, and so the compositional generalization task essentially reduces to the category membership inference, for which lfs for the held-out names serve as evidence. for gen pp , no specific pps are held out for testing, but the train distribution covers only a subset of the test-time environments for pps. this is where models fail to generalize in the intended ways. in our experiments, we reproduce the core findings about performance disparity across tasks. we also substantiate our assessment of the differences with new experiments. in particular, we show that removing the lfs for the held-out names in the gen proper task leads to failures at roughly the same level as for gen pp ; the lfs are what support the category membership inference, and without them models cannot make that inferential leap. we also show that the inferential leap is supported by the comprehensive distribution of other proper names, which, if held-out, also independently leads to generalization failure. our findings also explain why the use of pretrained models results in substantial performance improvement on gen pp , a finding we reproduce for recogs. pretrained models effectively fill in the distributional gap for pps, which disambiguates the learning target and provides models with the needed distributional evidence. to further corroborate our assessment, we show that performance on gen pp can also be improved by minimally introducing the test-time distribution for a subset of pps (those headed by beside), which brings gen pp on a par with gen proper in terms of the evidence available to the learner. @xcite (henceforth k&s) is an experimental investigation of a task built in analogy with gen pp . focusing on human subjects, k&s seek to vindicate gen pp as a well-posed generalization task for lms. to create a productive dialectic, we carefully engage with k&s's findings. ## generalization task analysis in this section, we analyze our two target tasks, seeking to articulate the core hypotheses that we test experimentally in section 4. gen proper . figure 1 summarizes gen proper . the set of proper names is split into two groups: names in the distributionally-free group appear as both subjects and objects in the train set, while names in the held-out group appear only as primitives. the test-time task is to handle names from the held-out group as both subjects and objects. we hypothesize that this task involves only one compositional generalization step: inferring that the held-out names are in the same category as the distributionally-free ones. the lexical entries for proper names help to convey why this is achievable for models in practice: for recogs, all names (and common nouns) have lfs of the form lambda a . α ( a ) where α is the name. for cogs, all names are simply single tokens α, and they are the only entries with this format. we know that present-day lms are highly sensitive to such distributional alignment, and so we hypothesize that models successfully categorize all names together using these lf regularities. in section 4.2, we support this by showing that models completely fail at the task if we break the distributional alignment by giving held-out names a different lf. once a model has made this category inference for the held-out names, the generalization task is essentially an iid one, as no new structural environments are introduced at test time. in section 4.3, we show that performance drops to near 0 if we modify the task to include a structural generalization component as well (by having proper names appear only as objects during training and testing on subject occurrences). gen pp . as figure 2 indicates, the gen pp task is very different from the gen proper in terms of the generalization challenge it poses. for this task, as in the previous one, models do need to infer that all pps are distributionally alike. here, they get evidence from the relevant segments of lfs for sentences featuring pps. the much more significant challenge relates to the fact that an entirely new environment for ppsthe subject position -is introduced at test time. in other words, the (re)cogs training data contains a distributional gap for this task. a sentence with subject pps. in section 4.5, we test this by building on work linking surprisals with syntactic parsing @xcite . ## discussion in this section, in light of our analysis and results, we try to answer the following questions: what makes a structural generalization task well-posed, why is pretrained model use important for a better methodology, and can we reconcile our findings for lms with the human-subjects experiments of kim and smolensky (2024)? ## conclusion we focused on two tasks in cogs, gen proper and gen pp . we used our experiments to reveal the dis-tributional cues and gaps that help and preclude generalization for gen proper and gen pp , respectively. any compositional generalization task requires enough distributional evidence to reach the target generalization. in the absence of such evidence, the task is ill-posed; this, we take to be true for gen pp . using our conceptual analysis and results, we offered the methodological proposal of using pretrained models when comparing neural models with humans. a fair comparative study of compositional generalization should lead to a deeper understanding of compositionality and generalization separately as well.