Inductive Bias Is in the Eye of the Beholder (2023)

Abstract: Due to the finite nature of any evidence used in learning, systematic generalization is crucially reliant on the presence of inductive bias (Mitchell, 1980). We examine inductive biases in different types of sequence-to-sequence neural network models, including CNNs, LSTMs (with and without attention), and transformers, inspired by Kharitonov and Chaabouni (2021). Crucially, however, we consider a wider range of possible inductive biases than their study did. Investigating preferences for hierarchical generalization compared to other types of generalization, we find that, contrary to their results, transformers display no preference for hierarchical generalization, but instead prefer a counting strategy. We also investigate biases toward different types of compositionality. By controlling for a confound in Kharitonov and Chaabouni (2021)’s test set, we find much less consistent generalization overall, and find that a large number of responses were among types other than the two types of generalization they had considered. Nevertheless, we observe consistent compositional generalization to held out combinations of primitives and functions on a SCAN task (Lake and Baroni, 2017) by models of all types, but only when primitives occur with other functions in the training set. The pattern of success indicates generalization in models of these types is highly sensitive to distributional properties of their training data.

## introduction learners, both human and machine, systematically generalize from finite sets of data, and it is such generalization that makes them such effective agents. @xcite review the wealth of work focused on understanding the efficacy of different models for different types of generalization in nlp tasks. as @xcite notes, the kind of systematic generalization we hope our models will show is only possible in the presence of inductive bias, a preference for some generaliza-tion over others. inductive bias can derive from inherent properties of a model or from previous training. in this paper, we focus on the former. while such inherent inductive bias can be read off to a reasonable degree from the structure of a symbolic model, it is much less easy to understand the biases of a neural network architecture trained with some variant of backpropagation. work that documents variation among models in their ability to solve a certain nlp task can be understood as illuminating inductive biases in these models: ones that fare better are more biased toward the correct solution (modulo training differences). yet because of the complexity of most such tasks, it is typically difficult to identify the specific preferences that lead to a model's generalization behavior. the recent work of @xcite aims to avoid this issue by focusing on carefully controlled induction problems, which we can think of as "model induction organisms," where the range of solutions is limited. we aim to build on kc's important groundwork, focusing on two of the tasks that they proposed: hierarchical-or-linear and compositionor-memorization. we show that the evaluation of model behavior and assessment of inductive bias with even apparently trivial tasks requires great care. in particular, assessing inductive bias requires the consideration of the widest possible range of possible hypotheses, and failing to do so can lead to over-or under-estimating inductive bias for a certain type of hypothesis. in addition, we demonstrate that the process of constructing such model organism tasks must avoid the presence of quirks that lead even appropriately biased models astray. ## experiment 2: compositionality in experiment 2, we examine the question of whether different model architectures display a bias toward compositional generalizations of various types. by "compositional," we mean a process whose output is a function of its individual input symbols and their mode of combination @xcite . this question has been explored using a variety of tasks and datasets, e.g., scan (lake and baroni, 2017) and cogs @xcite , which both explore the problem of assigning structured semantic interpretations to english sentences. while offering useful measures of compositional generalization, the complexity of these datasets makes it difficult to assess the propensity for com-positional generalization in its simplest guise. here, we focus on a distilled task probing compositional generalization proposed by kc, which they call composition-or-memorization. this task targets compositional generalization in a way that is not specific to its role in natural language. ## related work in architectural inductive bias here, we briefly discuss work on inductive bias in neural network models that focuses particularly on those aspects of inductive bias that are traceable to network architecture; see @xcite for a more thorough review. in some cases, inductive biases have been intentionally built in to particular architectures. the parameter sharing and filter structure of cnns leads directly to a bias for translation invariance and feature locality, which are useful in a variety of tasks @xcite . the inclusion of gates with multiplicative interactions in lstms @xcite explicitly addressed deficiencies of rnns in modeling the long-distance dependencies found in natural language (elman, 1990). @xcite discuss the fact that unbounded growth in lstms' hidden state vectors leads to a counting bias. none of these biases relate explicitly to the questions of structural or compositional generalization that we have explored in this paper. research on language-related biases has explored structural generalization. @xcite train lstms and transformers on synthetic corpora that exhibit a range of word order patterns, some attested in natural languages, others not. lstms, unlike transformers, showed a preference for certain word order patterns over others, but neither showed a bias toward attested natural language patterns. mccoy et al. (2020) study the ability of different architectures to generalize structurally-defined mappings between sentences. lstms, rnns, and grus with different attention mechanisms failed to generalize structurally, preferring linear generalizations. @xcite find an even more extreme failure for transformer models. mccoy et al. only find a bias toward structural generalization in models with explicitly hierarchically-structured recurrence @xcite . to detect bias toward compositionality, researchers have explored tasks requiring the mapping of a natural language input to an interpretation of some sort, including scan (lake and baroni, 2017), pcfg set @xcite , cogs @xcite , and slog @xcite . success on these datasets is informative about the kinds of generalizations that networks are capable of. yet their complexity, compared to the tasks we explored, can make it difficult to identify reasons for success or failure. indeed, modifications to non-crucial properties of a dataset can yield quite different results @xcite . of course, considering complex cases is important, as we have indeed seen in section 3.3 above. how-ever, we see the study of simple tasks as providing complementary understanding. recently, it has been found that experiments aiming to identify inductive biases must also carefully control for training regimen. for example, training without early stopping can lead to qualitatively different patterns of generalization through "grokking" @xcite . it will be important to understand the range of applicability of grokking and the like, as well as the challenge such results pose for assessing inductive bias. ## conclusion a finite set of data may be consistent with an infinite number of possible generalizations @xcite . it is precisely for this reason that studies of inductive bias must proceed with great care to avoid adducing support for the existence of certain kinds of inductive biases prematurely. taking this idea to heart, we found that the kc's claim that transformers tout court display hierarchical bias to be premature. instead, we found their behavior to be most consistent with a countingbased strategy. similarly, we found that considering a more consistent test set when assessing compositionality led us to find less support for consistent generalization than kc. though the pattern of results we found was consistent with theirs when limiting our focus to the hypotheses they considered, we found that expanding the range of possible hypotheses revealed a more nuanced picture. nevertheless, we found there is hope for inducing compositional generalization: when we ensure that distributional properties of the training set are more like the those in the test set, compositional generalization appears easily achieved. this suggests low performance on a particular task may be due to irrelevant factors, like surfacelevel distributional properties of a training dataset. it may not be that a particular model cannot achieve high levels of success on certain tasks due to the inherent complexity of whatever pattern of behavior the task aims to measure, but because it has fixated on some accidental, irrelevant generalization in its training data that impedes its recognizing the correct generalization. alleviating models' propensity toward fixation on such irrelevancies, perhaps by altering their training data in systematic ways (as in our scan experiments), may prove useful in improving their performance in various domains.