Distinguishing rule- and exemplar-based generalization in learning systems

1 INTRODUCTION . Extrapolation or generalization—decisions on unseen datapoints—is always underdetermined by data ; which particular extrapolation behavior an algorithm exhibits is determined by the algorithm ’ s inductive biases ( Mitchell , 1980 ) . For modern deep learning systems , these inductive biases often deviate from those in humans . When the inductive biases of ML systems are opaque , and guarantees on extrapolation are not possible—as is often the case with many modern ML systems ( D ’ Amour et al. , 2020 ) —we can instead turn to empirical study of the behavior of a system to derive principles about the system ’ s operation . Cognitive psychology provides a rich basis for experimental designs to study the often-opaque human cognitive system via its external behavior . These can be leveraged to distinguish between competing hypotheses about a machine learning system ’ s inductive biases in the same manner ( Ritter et al. , 2017b ; Lake et al. , 2018 ; Dasgupta et al. , 2019 ) . In this paper , we draw on methods from cognitive psychology to define a protocol that teases apart the different inductive biases that go into informing how an opaque learning system extrapolates outside its training distribution . We focus in particular on combinatorial generalization for classification in the presence of spurious correlation . Our protocol goes significantly beyond existing work by controlling for various confounds . We isolate two distinct kinds of inductive bias—feature-level bias and exemplar-rule bias—that have different effects on model extrapolation . We examine these inductive biases across models in an expository points-in-a-plane setting , as well as in naturalistic image and language domains . Finally , we discuss the implications of these inductive biases and their relation to previous work on data augmentation and spurious correlation . Feature-level bias measures which features a system finds easier or harder to learn . This informs which feature a system will generalize on the basis of when both features are correlated or confounded . This kind of feature-level bias has been studied extensively in human cognition ( Landau et al. , 1988 ; Hudson Kam & Newport , 2005 ) . There has also been recent work—directly inspired by these observations ratio of predictions condition training examples extrapolation humans rule-based exemplar-based ( shape-biased ) ( no feature bias ) ( no feature bias ) cognitive psychology studies—that examines similar biases in artificial neural networks , most notably the “ shape-bias ” , the tendency to generalize image category labels according to shape rather than according to color or texture ( Ritter et al. , 2017a ; Hermann et al. , 2019 ; Geirhos et al. , 2018 ) . While there exists previous work examining specific feature biases in deep learning , we present a more general measure of feature-level bias as well as demonstrate how it interacts with—but is distinct from—another kind of inductive bias , viz . exemplar-vs-rule bias . Exemplar-vs-rule bias measures how a system uses features to inform decisions by trading off between exemplar- and rule-based generalization . A rule-based categorization decision is made on the basis of minimal features that support the category boundary ( e.g. , Ashby & Townsend , 1986 ) , while an exemplar-based decision-maker generalizes a category on the basis of similarity to category exemplars ( e.g. , Shepard & Chang , 1963 ) , and therefore may invoke many or all features that underlie a category . Extensive empirical work in cognitive psychology has found evidence of both kinds of generalization in humans ( Nosofsky et al. , 1989 ; Rips , 1989 ; Allen & Brooks , 1991 ; Rips & Collins , 1993 ; Smith & Sloman , 1994 ) . This trade-off can be understood intuitively as a continuum that varies the number of features employed to discriminate between categories ( Pothos , 2005 ) .1 This continuum also plays a role in representation learning systems such as deep neural networks ( Hinton & Salakhutdinov , 2006 ) , where feature selection is automated . 2 AN ILLUSTRATIVE EXAMPLE . We first examine an intuitive example that highlights the distinct inductive biases we care about . Consider the category learning paradigm in Fig . ( 1 ) . The stimuli vary along two feature dimensions , shape and color . Color determines the label of an object ( i.e. , green objects are “ dax ” ; purple are “ fep ” ) , and shape is unrelated to the underlying category structure and acts as a distractor . Participants ( either humans or artificial learning systems ) are independently placed in three different conditions— cue conflict , zero shot , and partial exposure—that vary in coverage of the feature space . After observing the training examples , the participant is presented with an extrapolation test consisting of an example outside the support of feature combinations observed during training ( i.e. , to classify the green circle as a “ dax ” or a “ fep , ” using arbitrary names to demonstrate that which feature is relevant to the category boundary is not given ) . We explain below how differences in classification behavior on this extrapolation isolate feature-level bias as well as exemplar-vs-rule bias . We encourage the reader to try the experiment themselves to examine their intuitions . Cue conflict ( CC , top row , Fig . ( 1 ) ) . The data presented in this condition confound color and shape ( i.e. , color and shape are equally predictive of the category boundary ) . How a system generalizes here directly measures its feature-level bias towards color or shape . 1We leave to future work the details of mathematically formalizing the properties of statistical learners that result in exemplar-vs-rule bias . We instead focus on the behavioral manifestations of this inductive bias and present an empirical protocol to measure it , even in opaque systems . Characteristic behavior ( right half of Fig . ( 1 ) ) . Since humans have an established shape bias ( Landau et al. , 1988 ) , we expect that humans will classify the test item according to the object that shares its shape , not its color ; in this case , as a “ fep. ” However , this inductive bias is not shared by ruleand exemplar-based reasoners that have no a priori propensity for features , and are equally likely to classify the test item as a “ dax ” or a “ fep. ” Zero shot ( ZS , middle row , Fig . ( 1 ) ) . This condition requires extrapolation to a new feature value “ zero-shot ” ( i.e. , without prior exposure ) . This setting is often used to examine out-of-domain ( OOD ) and compositional generalization in machine learning ( Xian et al. , 2018 ) . Behavior in this condition reveals whether the model has learned the discriminating features and whether it can extrapolate to new feature values , and thus acts as a baseline . Characteristic behavior ( right half , Fig . ( 1 ) ) . Rule- and exemplar-based behavior in this condition is confounded . A rule-based learner infers the minimal rule that color determines label , does not assign any predictive value to shape , and therefore classifies the extrapolation stimulus based on color as a “ dax. ” An exemplar-based learner categorizes based on the similarity along all feature dimensions of the extrapolation stimulus to category exemplars . Both training exemplars have no overlap with the test stimulus along the shape dimension , but the “ dax ” overlaps along the color dimension , and the learner categorizes it as a “ dax. ” Partial exposure ( PE , bottom row , Fig . ( 1 ) ) . Compared to zero shot , participants in this condition also receive “ partial exposure ” to a new feature value ( i.e. , circle ) along the shape dimension . This setting is most similar to combinatorial zero-shot generalization ( e.g. , Lake & Baroni , 2018a ) , where the learner is exposed independently to all feature values but has to generalize to a new combination . Characteristic behavior ( right half of Fig . ( 1 ) ) . This setting meaningfully distinguishes rule- and exemplar-based generalization . To understand the behavior of these two systems , we contrast this condition to the cue-conflict condition . The addition of the purple diamond-shaped “ fep ” means the learner has seen both a diamond and a circle labeled “ fep ” . A rule-based learner takes this as direct evidence that shape is not predictive of category label and classifies the extrapolation stimulus on the basis of color as a “ dax. ” This is typically also how humans extrapolate . This additional training example , however , does not impact an exemplar-based system , since it does not share any features with the extrapolation stimulus . The exemplar-based reasoner classifies on the basis of feature-overlap with training exemplars and is therefore indifferent , exactly as in the cue-conflict condition . From behavior to inductive bias . Feature-level bias is measured as deviation from chance performance in the CC condition . Exemplar-vs-rule bias is measured by the difference between performance in the PE and ZS conditions—-no difference indicates rule-based generalization , while the magnitude of the difference measures exemplar propensity . Pure exemplar-based reasoning implies no difference between the PE and CC conditions , while a non-zero difference indicates partial rule propensity . Implications . A purely exemplar-based system doesn ’ t learn decision boundaries that operate over minimal features . It instead favors a decision boundary that weights all features . This is undesirable in domains where not all feature combinations will be observed , and systematic generalization to unobserved combinations is expected ( Lake et al. , 2018 ; Marcus , 2018 ; Arjovsky et al. , 2019 ) . On the other hand , a rule-based system that applies the same category decision rules across all data regions might over-generalize , which is undesirable in some naturally occurring long-tailed distributions ( Feldman & Zhang , 2020 ; Feldman , 2020 ; Brown et al. , 2020 ) . In such cases , flexible exemplar-based learning that generalizes based on dense similarity is preferable ( Zhang et al. , 2016 ; Arpit et al. , 2017 ) . The protocol we present allows us to empirically measure the exemplar-vs-rule bias of a learner and therefore navigate the exemplar-vs-rule trade-off in various scenarios . 3 A PROTOCOL FOR EXAMINING INDUCTIVE BIAS . We embed the structure of the category learning problem discussed in Section ( 2 ) into a statistical learning problem that can be applied across domains to test black-box learners . Problem setting . We consider a setting where inputs are a composition of categorical attributes ( oracle setting in Andreas , 2019 ) with two latent binary features , zdisc , zdist , ∈ { 0 , 1 } that jointly determine the observation x via some mapping g : { 0 , 1 } 2 → X ; see Fig . ( 2 ) . These features can be derived from a richer set , e.g. , the median of a continuous feature ( see Appendix ) . We consider the training condition feature space cue conflict zero shot partial exposure extrapolation π0 = 0.5 , π1 = 0.5 π0 = 1.0 , π1 = 0.0 π0 = 0.0 , π1 = 0.0 π0 = 0.5 , π1 = 0.0 π1 = 1.0 binary classification task of fitting a model f̂ : X → { 0 , 1 } from a given model family F to predict a binary label for each observation . One of the underlying features , the discriminant , zdisc , defines the decision boundary ; the other one , the distractor , zdist , is not independently predictive of the label . This specifies a generative process x , zdisc , zdist ∼ p ( x | zdisc , zdist ) p ( zdisc , zdist ) . p ( x | zdisc , zdist ) is either generated ( e.g. , in Section ( 4 ) ) , or the empirical distribution of the subset of datapoints x with the corresponding underlying feature values ( assuming access to these annotations , e.g. , in Sections ( 5 ) and ( 6 ) ) . p ( zdisc , zdist ) is varied across training conditions , as outlined below . Training conditions . The upper-right quadrant in all subfigures of Fig . ( 2 ) , for which p ( zdisc = 1 , zdist = 1 ) = 1 , acts as a hold-out set on which we can evaluate generalization to an unseen combination of attribute values . We produce multiple training conditions with the remaining three quadrants of data by manipulating p ( zdisc , zdist ) . All the analyses in this paper compare model extrapolation to the held-out test quadrant across various training conditions . To equalize the class base rates we balance all training conditions across the discriminant ; i.e. , we enforce p ( zdisc = 0 ) = p ( zdisc = 1 ) = 0.5 . We also fix the number of datapoints across all conditions at N ; With these constraints , we can control p ( zdisc , zdist ) via two degrees of freedom : π0 = p ( zdist = 1 | zdisc = 0 ) ( this implicitly fixes p ( zdist = 0 | zdisc = 0 ) = 1 − π0 to balance the dataset ) ; and π1 = p ( zdist = 1 | zdisc = 1 ) .The three conditions in Section ( 2 ) , as well as the held-out test set , correspond to particular settings of π0 and π1 ( shown in Fig . ( 2 ) , more in Appendix ( A.2 ) ) . Measuring inductive bias . For a given model family F , let f̂ ZS denote the result of selecting a model from F by training in the zero-shot condition , and similarly f̂ PE and f̂ CC . Feature-level bias ( FLB ) and exemplar-vs-rule propensity ( EVR ) are measured as : FLB ( F ) = E [ ( A ( y , f̂ CC ( x ) ) ] − 0.5 , ( 1 ) EVR ( F ) = E [ A ( y , f̂ ZS ( x ) ) ] − E [ A ( y , f̂ PE ( x ) ) ] ( 2 ) where the expectation is taken with respect to the the data distribution under the extrapolation region ( i.e. , p ( x , y | π0 = 1 , π1 = 1 ) ) , A is the 0-1 accuracy . FLB takes values between -0.5 and 0.5 ( indicating bias toward zdist or zdisc , respectively ) ; 0 represents no feature bias . EVR takes values between 0 and 1 ( indicating rule-based and exemplar-based extrapolation , respectively ) . Related formalisms and spurious correlation . This binary formulation of discriminant and distractor features has previously been studied in the context of spurious correlation ( Sagawa et al. , 2020 ) . Rather than independently varying occupancy in the four quadrants , they directly manipulate the ( spurious ) linear correlation between the distractor and the discriminant features ( pmaj ) . In combinatorial feature spaces , a scalar spurious correlation insufficiently specifies the data distribution . The linear correlation coefficient ρ between zdisc and zdist—henceforth “ spurious correlation ” —can be written in terms of π0 and π1 : ρ ( π0 , π1 ) = α√ β ( 1− β ) ; α = π0 − π1 2 , β = π0 + π1 2 . ( 3 ) Different π0 and π1 combinations can give equal ρ ( see contours in Fig . ( 3 ) , dots indicate points along the equi-correlation contour that intersects with the PE condition : ( π0 = 0.5 , π1 = 0.0 , ρ = 0.58 ) ) . In this paper we demonstrate that different data distributions with the same spurious correlation can result in vastly different generalization behavior to the under-represented extrapolation quadrant . This indicates that sensitivity to spurious correlation is an underspecified inductive bias—we risk conflating conceptually distinct sources of inductive bias by focusing on this single metric . We argue for a formulation like ours—based on manipulating feature combinations—that can tease apart distinct inductive biases : at the level of what features a system finds easier to learn ( FLB ) as well as how to use these features to inform a decision boundary ( EVR ) . We discuss how these biases can be interdependent , but capture distinct behaviors that sensitivity to spurious correlation can not explain , thereby providing a more comprehensive picture of how a system generalizes .

Summary: The generalization of data distributions to unseen regions, i.e., extrapolation, remains one of the critical challenges in machine learning. The inductive biases of the learner determine such extrapolation. Unfortunately, machine learning systems often do not share the same inductive biases as humans and, as a result, may extrapolate in ways that do not match the analyst's expectations. The authors investigated two different types of such inductive bias: feature-level bias (differences in which features are more easily learned) and exemplar-based and rule-based bias (differences in how learned features are used for generalization). Inspired by these experimental approaches, we have proposed a protocol to investigate this trade-off in learning systems directly. We present empirical results for a range of models and the domains of explanatory images and language. We demonstrate that controlling for feature-level bias while measuring the trade-off between exemplars and rules provides a complete picture of extrapolative behavior than existing formalisms.