Distributional Generalization: A New Kind of Generalization

1 INTRODUCTION . We begin with an experiment motivating the need for a notion of generalization beyond test error . Experiment 1 . Consider a binary classification version of CIFAR-10 , where CIFAR-10 images x have binary labels Animal/Object . Take 50K samples from this distribution as a train set , but apply the following label noise : flip the label of cats to Object with probability 30 % . Now train a WideResNet f to 0 train error on this train set . How does the trained classifier behave on test samples ? Options below : 1 . The test error is low across all classes , since there is only 3 % label noise in the train set 2 . Test error is “ spread ” across the animal class , After all , the classifier is not explicitly told what a cat or a dog is , just that they are all animals . 3 . The classifier misclassifies roughly 30 % of test cats as “ objects ” , but all other types of animals are largely unaffected . In fact , reality is closest to option ( 3 ) . Figure 1 shows the results of this experiment with a WideResNet . The left panel shows the joint density of train inputs x with train labels Object/Animal . Since the classifier is interpolating , the classifier outputs on the train set are identical to the left panel . The right panel shows the classifier predictions f ( x ) on test inputs x . There are several notable things about this experiment . First , the error is localized to cats in test set as it was in the train set , even though no explicit cat labels were provided . Second , the amount of error on the cat class is close to the noise applied on the train set . Thus , the behavior of the classifier on the train set generalizes to the test set in a certain sense . This type of similarity in behavior would not be captured solely by average test error — it requires reasoning about the entire distribution of classifier outputs . In our work , we show that this experiment is just one instance of a different type of generalization , which we call “ Distributional Generalization ” . We now describe the mathematical form of this generalization . Then , through extensive experiments , we will show that this type of generalization occurs widely in existing machine learning methods : neural networks , kernel machines and decision trees . 1.1 DISTRIBUTIONAL GENERALIZATION . Supervised learning aims to learn a model that correctly classifies inputs x ∈ X from a given distribution D into classes y ∈ Y . We want a model with small test error on this distribution . In practice , we find such a classifier by minimizing the train error of a model on the train set . This procedure is justified when we expect a small generalization gap : the gap between the error on the train and test set . That is , the trained model f should have : ErrorTrainSet ( f ) ≈ ErrorTestSet ( f ) . We now re-write this classical notion of generalization in a form better suited for our extension . Classical Generalization : Let f be a trained classifier . Then f generalizes if : E x∼TrainSet ŷ←f ( x ) [ 1 { ŷ 6= y ( x ) } ] ≈ E x∼TestSet ŷ←f ( x ) [ 1 { ŷ 6= y ( x ) } ] ( 1 ) Above , y ( x ) is the true class of x and ŷ is the predicted class . The LHS of Equation 1 is the train error of f , and the RHS is the test error . Crucially , both sides of Equation 1 are expectations of the same function ( Terr ( x , ŷ ) : = 1 { ŷ 6= y ( x ) } ) under different distributions . The LHS of Equation 1 is the expectation of Terr under the “ Train Distribution ” Dtr , which is the distribution over ( x , ŷ ) given by sampling a train point x along with its classifier-label f ( x ) . Similarly , the RHS is under the “ Test Distribution ” Dte , which is this same construction over the test set . These two distributions are the central objects in our study , and are defined formally in Section 2.1 . We can now introduce Distributional Generalization , which is a property of trained classifiers . It is parameterized by a set of bounded functions ( “ tests ” ) : T ⊆ { T : X × Y → [ 0 , 1 ] } . Distributional Generalization : Let f be a trained classifier . Then f satisfies Distributional Generalization with respect to tests T if : ∀T ∈ T : E x∼TrainSet ŷ←f ( x ) [ T ( x , ŷ ) ] ≈ E x∼TestSet ŷ←f ( x ) [ T ( x , ŷ ) ] ( 2 ) We write this property as Dtr ≈T Dte . This states that the train and test distribution have similar expectations for all functions in the family T . For the singleton set T = { Terr } , this is equivalent to classical generalization , but it may hold for much larger sets T . For example in Experiment 1 , the train and test distributions match with respect to the test function “ Fraction of true cats labeled as object. ” In fact , it is best to think of Distributional Generalization as stating that the distributions Dtr and Dte are close as distributions . This property becomes especially interesting for interpolating classifiers , which fit their train sets exactly . Here , the Train Distribution ( xi , f ( xi ) ) is exactly equal1 to the original distribution ( x , y ) ∼ D , since f ( xi ) = yi on the train set . In this case , distributional generalization claims that the output distribution ( x , f ( x ) ) of the model on test samples is close to the true distribution ( x , y ) . The following conjecture specializes Distributional Generalization to interpolating classifiers , and will be the main focus of our work . Interpolating Indistinguishability Meta-Conjecture ( informal ) : For interpolating classifiers f , and a large family T of test functions , the distributions : ( x , f ( x ) ) x∈TestSet ≈T ( x , f ( x ) ) x∈TrainSet ≡ ( x , y ) x , y∼D 1The formal definition of Train Distribution , in Section 2.1 , includes the randomness of sampling the train set as well . We consider a fixed train set in the Introduction for sake of exposition . This is a “ meta-conjecture ” , which becomes a concrete conjecture once the family of tests T is specified . One of the main contributions of our work is formally stating two concrete instances of this conjecture— specifying exactly the family of tests T and their dependence on problem parameters ( the distribution , model family , training procedure , etc ) . It captures behaviors far more general than Experiment 1 , and applies to neural networks , kernels , and decision trees . We give empirical evidence for conjectures across a variety of natural settings in machine learning . 1.2 SUMMARY OF CONTRIBUTIONS . We extend the classical framework of generalization by introducing Distributional Generalization , in which the train and test behavior of models are close as distributions . Informally , for trained classifiers f , its outputs on the train set ( x , f ( x ) ) x∈TrainSet are close in distribution to its outputs on the test set ( x , f ( x ) ) x∈TestSet , where the form of this closeness depends on specifics of the model , training procedure , and distribution . This notion is more fine-grained than classical generalization , since it considers the entire distribution of model outputs instead of just the test error . We initiate the study of Distributional Generalization across various domains in machine learning . For interpolating classifiers , we state two formal conjectures which predict the form of distributional closeness that can be expected for a given model and task : 1 . Feature Calibration Conjecture ( Section 3 ) : Interpolating classifiers , when trained on samples from a distribution , will match this distribution up to all “ distinguishable features ” ( Definition 1 ) . 2 . Agreement Conjecture ( Section 4 ) : For two interpolating classifiers of the same type , trained independently on the same distribution , their agreement probability with each other on test samples roughly matches their test accuracy . We perform a number of experiments surrounding these conjectures , which reveal new behaviors of standard interpolating classifiers ( e.g . ResNets , MLPs , kernels , decision trees ) . We prove our conjectures for 1-Nearest-Neighbors ( Theorem 1 ) , which suggests some form of “ locality ” as the underlying mechanism . Finally , we discuss extending these results to non-interpolating methods in Section 5 . Our experiments and conjectures shed new light on the structure of interpolating classifiers , which are extensively studied in recent years yet still poorly understood . Related Work . Our work is inspired by the broader study of interpolating and overparameterized methods in machine learning ( e.g . Zhang et al . ( 2016 ) ; Belkin et al . ( 2018a ; b ; 2019 ) ; Liang and Rakhlin ( 2018 ) ; Nakkiran et al . ( 2020 ) ; Schapire et al . ( 1998 ) ; Breiman ( 1995 ) ) . In a similar vein to our work , Wyner et al . ( 2017 ) ; Olson and Wyner ( 2018 ) investigate decision trees , and show that random forests are equivalent to a Nadaraya–Watson smoother ( Nadaraya , 1964 ; Watson , 1964 ) with a certain smoothing kernel . Our conjectures also describe neural networks under label noise , which has been empirically and theoretically studied in the past ( Zhang et al. , 2016 ; Belkin et al. , 2018b ; Rolnick et al. , 2017 ; Natarajan et al. , 2013 ; Thulasidasan et al. , 2019 ; Ziyin et al. , 2020 ; Chatterji and Long , 2020 ) , though not formally characterized . The behaviors we consider are also similar to conditional density estimation ( e.g . Tsybakov ( 2008 ) ; Dutordoir et al . ( 2018 ) ) , though we consider samplers , not density estimators . We include a full discussion of related works in Appendix A . 2 PRELIMINARIES . Notation . We consider joint distributionsD on x ∈ X and discrete y ∈ Y = [ k ] . LetDn denote n iid samples from D and S = { ( xi , yi ) } denote a train set . Let F denote the classifier family ( including architecture and training algorithm for neural networks ) , and let f ← TrainF ( S ) denote training a classifier f ∈ F on train-set S. We consider classifiers which output hard decisions f : X → Y . Let NNS ( x ) = xi denote the nearest-neighbor to x in train-set S , with respect to a distance metric d. Our theorems will apply to any distance metric , and so we leave this unspecified . Let NN ( y ) S ( x ) denote the nearest-neighbor estimator itself , that is , NN ( y ) S ( x ) : = yi where xi = NNS ( x ) . Experimental Setup . Full experimental details are provided in Appendix B . Briefly , we train all classifiers to interpolation ( to 0 train error ) . Neural networks ( MLPs and ResNets ( He et al. , 2016 ) ) are trained with SGD . Interpolating decision trees are trained using the growth rule from Random Forests ( Breiman , 2001 ) . For kernel classification , we consider kernel regression on one-hot labels and kernel SVM , with small or 0 of regularization ( which is often optimal Shankar et al . ( 2020 ) ) . Distributional Closeness . We consider the following notion of closeness for two probability distributions : For two distributions P , Q over X × Y , let a “ test ” ( or “ distinguisher ” ) be a function T : X × Y → [ 0 , 1 ] which accepts a sample from either distribution , and is intended to classify the sample as either from distribution P or Q . For any set C ⊆ { T : X × Y → [ 0 , 1 ] } of tests , we say distributions P and Q are “ ε-indistinguishable up to C-tests ” if they are close with respect to all tests in class C. That is , P ≈Cε Q ⇐⇒ sup T∈C ∣∣∣∣ E ( x , y ) ∼P [ T ( x , y ) ] − E ( x , y ) ∼Q [ T ( x , y ) ] ∣∣∣∣ ≤ ε ( 3 ) Total-Variation distance is equivalent to closeness in all tests , i.e . C = { T : X × Y → [ 0 , 1 ] } , but we consider closeness for restricted families of tests C. P ≈ε Q denotes ε-closeness in TV-distance .

Summary: This paper proposes an extended notion of generalization. The new proposed notion asks that for a family of tests $T: X \times Y → [0,1]$, $T(x, f(x))$  will be similar for train/test examples. The paper proposes three interesting conjectures that are related to distributional generalization. The paper proves Conjecture 1 for nearest-neighbor classifiers. The paper also gives empirical evidence supporting their conjectures.