Distinguishing rule- and exemplar-based generalization in learning systems

1 INTRODUCTION . Extrapolation or generalization—decisions on unseen datapoints—is always underdetermined by data ; which particular extrapolation behavior an algorithm exhibits is determined by the algorithm ’ s inductive biases ( Mitchell , 1980 ) . For modern deep learning systems , these inductive biases often deviate from those in humans . When the inductive biases of ML systems are opaque , and guarantees on extrapolation are not possible—as is often the case with many modern ML systems ( D ’ Amour et al. , 2020 ) —we can instead turn to empirical study of the behavior of a system to derive principles about the system ’ s operation . Cognitive psychology provides a rich basis for experimental designs to study the often-opaque human cognitive system via its external behavior . These can be leveraged to distinguish between competing hypotheses about a machine learning system ’ s inductive biases in the same manner ( Ritter et al. , 2017b ; Lake et al. , 2018 ; Dasgupta et al. , 2019 ) . In this paper , we draw on methods from cognitive psychology to define a protocol that teases apart the different inductive biases that go into informing how an opaque learning system extrapolates outside its training distribution . We focus in particular on combinatorial generalization for classification in the presence of spurious correlation . Our protocol goes significantly beyond existing work by controlling for various confounds . We isolate two distinct kinds of inductive bias—feature-level bias and exemplar-rule bias—that have different effects on model extrapolation . We examine these inductive biases across models in an expository points-in-a-plane setting , as well as in naturalistic image and language domains . Finally , we discuss the implications of these inductive biases and their relation to previous work on data augmentation and spurious correlation . Feature-level bias measures which features a system finds easier or harder to learn . This informs which feature a system will generalize on the basis of when both features are correlated or confounded . This kind of feature-level bias has been studied extensively in human cognition ( Landau et al. , 1988 ; Hudson Kam & Newport , 2005 ) . There has also been recent work—directly inspired by these observations ratio of predictions condition training examples extrapolation humans rule-based exemplar-based ( shape-biased ) ( no feature bias ) ( no feature bias ) cognitive psychology studies—that examines similar biases in artificial neural networks , most notably the “ shape-bias ” , the tendency to generalize image category labels according to shape rather than according to color or texture ( Ritter et al. , 2017a ; Hermann et al. , 2019 ; Geirhos et al. , 2018 ) . While there exists previous work examining specific feature biases in deep learning , we present a more general measure of feature-level bias as well as demonstrate how it interacts with—but is distinct from—another kind of inductive bias , viz . exemplar-vs-rule bias . Exemplar-vs-rule bias measures how a system uses features to inform decisions by trading off between exemplar- and rule-based generalization . A rule-based categorization decision is made on the basis of minimal features that support the category boundary ( e.g. , Ashby & Townsend , 1986 ) , while an exemplar-based decision-maker generalizes a category on the basis of similarity to category exemplars ( e.g. , Shepard & Chang , 1963 ) , and therefore may invoke many or all features that underlie a category . Extensive empirical work in cognitive psychology has found evidence of both kinds of generalization in humans ( Nosofsky et al. , 1989 ; Rips , 1989 ; Allen & Brooks , 1991 ; Rips & Collins , 1993 ; Smith & Sloman , 1994 ) . This trade-off can be understood intuitively as a continuum that varies the number of features employed to discriminate between categories ( Pothos , 2005 ) .1 This continuum also plays a role in representation learning systems such as deep neural networks ( Hinton & Salakhutdinov , 2006 ) , where feature selection is automated . 2 AN ILLUSTRATIVE EXAMPLE . We first examine an intuitive example that highlights the distinct inductive biases we care about . Consider the category learning paradigm in Fig . ( 1 ) . The stimuli vary along two feature dimensions , shape and color . Color determines the label of an object ( i.e. , green objects are “ dax ” ; purple are “ fep ” ) , and shape is unrelated to the underlying category structure and acts as a distractor . Participants ( either humans or artificial learning systems ) are independently placed in three different conditions— cue conflict , zero shot , and partial exposure—that vary in coverage of the feature space . After observing the training examples , the participant is presented with an extrapolation test consisting of an example outside the support of feature combinations observed during training ( i.e. , to classify the green circle as a “ dax ” or a “ fep , ” using arbitrary names to demonstrate that which feature is relevant to the category boundary is not given ) . We explain below how differences in classification behavior on this extrapolation isolate feature-level bias as well as exemplar-vs-rule bias . We encourage the reader to try the experiment themselves to examine their intuitions . Cue conflict ( CC , top row , Fig . ( 1 ) ) . The data presented in this condition confound color and shape ( i.e. , color and shape are equally predictive of the category boundary ) . How a system generalizes here directly measures its feature-level bias towards color or shape . 1We leave to future work the details of mathematically formalizing the properties of statistical learners that result in exemplar-vs-rule bias . We instead focus on the behavioral manifestations of this inductive bias and present an empirical protocol to measure it , even in opaque systems . Characteristic behavior ( right half of Fig . ( 1 ) ) . Since humans have an established shape bias ( Landau et al. , 1988 ) , we expect that humans will classify the test item according to the object that shares its shape , not its color ; in this case , as a “ fep. ” However , this inductive bias is not shared by ruleand exemplar-based reasoners that have no a priori propensity for features , and are equally likely to classify the test item as a “ dax ” or a “ fep. ” Zero shot ( ZS , middle row , Fig . ( 1 ) ) . This condition requires extrapolation to a new feature value “ zero-shot ” ( i.e. , without prior exposure ) . This setting is often used to examine out-of-domain ( OOD ) and compositional generalization in machine learning ( Xian et al. , 2018 ) . Behavior in this condition reveals whether the model has learned the discriminating features and whether it can extrapolate to new feature values , and thus acts as a baseline . Characteristic behavior ( right half , Fig . ( 1 ) ) . Rule- and exemplar-based behavior in this condition is confounded . A rule-based learner infers the minimal rule that color determines label , does not assign any predictive value to shape , and therefore classifies the extrapolation stimulus based on color as a “ dax. ” An exemplar-based learner categorizes based on the similarity along all feature dimensions of the extrapolation stimulus to category exemplars . Both training exemplars have no overlap with the test stimulus along the shape dimension , but the “ dax ” overlaps along the color dimension , and the learner categorizes it as a “ dax. ” Partial exposure ( PE , bottom row , Fig . ( 1 ) ) . Compared to zero shot , participants in this condition also receive “ partial exposure ” to a new feature value ( i.e. , circle ) along the shape dimension . This setting is most similar to combinatorial zero-shot generalization ( e.g. , Lake & Baroni , 2018a ) , where the learner is exposed independently to all feature values but has to generalize to a new combination . Characteristic behavior ( right half of Fig . ( 1 ) ) . This setting meaningfully distinguishes rule- and exemplar-based generalization . To understand the behavior of these two systems , we contrast this condition to the cue-conflict condition . The addition of the purple diamond-shaped “ fep ” means the learner has seen both a diamond and a circle labeled “ fep ” . A rule-based learner takes this as direct evidence that shape is not predictive of category label and classifies the extrapolation stimulus on the basis of color as a “ dax. ” This is typically also how humans extrapolate . This additional training example , however , does not impact an exemplar-based system , since it does not share any features with the extrapolation stimulus . The exemplar-based reasoner classifies on the basis of feature-overlap with training exemplars and is therefore indifferent , exactly as in the cue-conflict condition . From behavior to inductive bias . Feature-level bias is measured as deviation from chance performance in the CC condition . Exemplar-vs-rule bias is measured by the difference between performance in the PE and ZS conditions—-no difference indicates rule-based generalization , while the magnitude of the difference measures exemplar propensity . Pure exemplar-based reasoning implies no difference between the PE and CC conditions , while a non-zero difference indicates partial rule propensity . Implications . A purely exemplar-based system doesn ’ t learn decision boundaries that operate over minimal features . It instead favors a decision boundary that weights all features . This is undesirable in domains where not all feature combinations will be observed , and systematic generalization to unobserved combinations is expected ( Lake et al. , 2018 ; Marcus , 2018 ; Arjovsky et al. , 2019 ) . On the other hand , a rule-based system that applies the same category decision rules across all data regions might over-generalize , which is undesirable in some naturally occurring long-tailed distributions ( Feldman & Zhang , 2020 ; Feldman , 2020 ; Brown et al. , 2020 ) . In such cases , flexible exemplar-based learning that generalizes based on dense similarity is preferable ( Zhang et al. , 2016 ; Arpit et al. , 2017 ) . The protocol we present allows us to empirically measure the exemplar-vs-rule bias of a learner and therefore navigate the exemplar-vs-rule trade-off in various scenarios . 3 A PROTOCOL FOR EXAMINING INDUCTIVE BIAS . We embed the structure of the category learning problem discussed in Section ( 2 ) into a statistical learning problem that can be applied across domains to test black-box learners . Problem setting . We consider a setting where inputs are a composition of categorical attributes ( oracle setting in Andreas , 2019 ) with two latent binary features , zdisc , zdist , ∈ { 0 , 1 } that jointly determine the observation x via some mapping g : { 0 , 1 } 2 → X ; see Fig . ( 2 ) . These features can be derived from a richer set , e.g. , the median of a continuous feature ( see Appendix ) . We consider the training condition feature space cue conflict zero shot partial exposure extrapolation π0 = 0.5 , π1 = 0.5 π0 = 1.0 , π1 = 0.0 π0 = 0.0 , π1 = 0.0 π0 = 0.5 , π1 = 0.0 π1 = 1.0 binary classification task of fitting a model f̂ : X → { 0 , 1 } from a given model family F to predict a binary label for each observation . One of the underlying features , the discriminant , zdisc , defines the decision boundary ; the other one , the distractor , zdist , is not independently predictive of the label . This specifies a generative process x , zdisc , zdist ∼ p ( x | zdisc , zdist ) p ( zdisc , zdist ) . p ( x | zdisc , zdist ) is either generated ( e.g. , in Section ( 4 ) ) , or the empirical distribution of the subset of datapoints x with the corresponding underlying feature values ( assuming access to these annotations , e.g. , in Sections ( 5 ) and ( 6 ) ) . p ( zdisc , zdist ) is varied across training conditions , as outlined below . Training conditions . The upper-right quadrant in all subfigures of Fig . ( 2 ) , for which p ( zdisc = 1 , zdist = 1 ) = 1 , acts as a hold-out set on which we can evaluate generalization to an unseen combination of attribute values . We produce multiple training conditions with the remaining three quadrants of data by manipulating p ( zdisc , zdist ) . All the analyses in this paper compare model extrapolation to the held-out test quadrant across various training conditions . To equalize the class base rates we balance all training conditions across the discriminant ; i.e. , we enforce p ( zdisc = 0 ) = p ( zdisc = 1 ) = 0.5 . We also fix the number of datapoints across all conditions at N ; With these constraints , we can control p ( zdisc , zdist ) via two degrees of freedom : π0 = p ( zdist = 1 | zdisc = 0 ) ( this implicitly fixes p ( zdist = 0 | zdisc = 0 ) = 1 − π0 to balance the dataset ) ; and π1 = p ( zdist = 1 | zdisc = 1 ) .The three conditions in Section ( 2 ) , as well as the held-out test set , correspond to particular settings of π0 and π1 ( shown in Fig . ( 2 ) , more in Appendix ( A.2 ) ) . Measuring inductive bias . For a given model family F , let f̂ ZS denote the result of selecting a model from F by training in the zero-shot condition , and similarly f̂ PE and f̂ CC . Feature-level bias ( FLB ) and exemplar-vs-rule propensity ( EVR ) are measured as : FLB ( F ) = E [ ( A ( y , f̂ CC ( x ) ) ] − 0.5 , ( 1 ) EVR ( F ) = E [ A ( y , f̂ ZS ( x ) ) ] − E [ A ( y , f̂ PE ( x ) ) ] ( 2 ) where the expectation is taken with respect to the the data distribution under the extrapolation region ( i.e. , p ( x , y | π0 = 1 , π1 = 1 ) ) , A is the 0-1 accuracy . FLB takes values between -0.5 and 0.5 ( indicating bias toward zdist or zdisc , respectively ) ; 0 represents no feature bias . EVR takes values between 0 and 1 ( indicating rule-based and exemplar-based extrapolation , respectively ) . Related formalisms and spurious correlation . This binary formulation of discriminant and distractor features has previously been studied in the context of spurious correlation ( Sagawa et al. , 2020 ) . Rather than independently varying occupancy in the four quadrants , they directly manipulate the ( spurious ) linear correlation between the distractor and the discriminant features ( pmaj ) . In combinatorial feature spaces , a scalar spurious correlation insufficiently specifies the data distribution . The linear correlation coefficient ρ between zdisc and zdist—henceforth “ spurious correlation ” —can be written in terms of π0 and π1 : ρ ( π0 , π1 ) = α√ β ( 1− β ) ; α = π0 − π1 2 , β = π0 + π1 2 . ( 3 ) Different π0 and π1 combinations can give equal ρ ( see contours in Fig . ( 3 ) , dots indicate points along the equi-correlation contour that intersects with the PE condition : ( π0 = 0.5 , π1 = 0.0 , ρ = 0.58 ) ) . In this paper we demonstrate that different data distributions with the same spurious correlation can result in vastly different generalization behavior to the under-represented extrapolation quadrant . This indicates that sensitivity to spurious correlation is an underspecified inductive bias—we risk conflating conceptually distinct sources of inductive bias by focusing on this single metric . We argue for a formulation like ours—based on manipulating feature combinations—that can tease apart distinct inductive biases : at the level of what features a system finds easier to learn ( FLB ) as well as how to use these features to inform a decision boundary ( EVR ) . We discuss how these biases can be interdependent , but capture distinct behaviors that sensitivity to spurious correlation can not explain , thereby providing a more comprehensive picture of how a system generalizes .

Summary: This paper studies the extrapolation of machine learning models to unseen regions, and specifically studies two types of biases: feature-level bias (differences in which features are more readily learned) and exemplar-vs-rule bias (differences in how these learned features are used for generalization). Motivated by the studies of exemplar vs. rule-based generalization in cognitive psychology, the authors present a protocol directly probing this trade-off in machine learning systems. The authors present empirical results across a range of models in both expository and real-world image and language domains and demonstrate that using the trad off provides a more complete picture of extrapolation behaviour than existing methods.