Distributional Generalization: A New Kind of Generalization

1 INTRODUCTION . We begin with an experiment motivating the need for a notion of generalization beyond test error . Experiment 1 . Consider a binary classification version of CIFAR-10 , where CIFAR-10 images x have binary labels Animal/Object . Take 50K samples from this distribution as a train set , but apply the following label noise : flip the label of cats to Object with probability 30 % . Now train a WideResNet f to 0 train error on this train set . How does the trained classifier behave on test samples ? Options below : 1 . The test error is low across all classes , since there is only 3 % label noise in the train set 2 . Test error is “ spread ” across the animal class , After all , the classifier is not explicitly told what a cat or a dog is , just that they are all animals . 3 . The classifier misclassifies roughly 30 % of test cats as “ objects ” , but all other types of animals are largely unaffected . In fact , reality is closest to option ( 3 ) . Figure 1 shows the results of this experiment with a WideResNet . The left panel shows the joint density of train inputs x with train labels Object/Animal . Since the classifier is interpolating , the classifier outputs on the train set are identical to the left panel . The right panel shows the classifier predictions f ( x ) on test inputs x . There are several notable things about this experiment . First , the error is localized to cats in test set as it was in the train set , even though no explicit cat labels were provided . Second , the amount of error on the cat class is close to the noise applied on the train set . Thus , the behavior of the classifier on the train set generalizes to the test set in a certain sense . This type of similarity in behavior would not be captured solely by average test error — it requires reasoning about the entire distribution of classifier outputs . In our work , we show that this experiment is just one instance of a different type of generalization , which we call “ Distributional Generalization ” . We now describe the mathematical form of this generalization . Then , through extensive experiments , we will show that this type of generalization occurs widely in existing machine learning methods : neural networks , kernel machines and decision trees . 1.1 DISTRIBUTIONAL GENERALIZATION . Supervised learning aims to learn a model that correctly classifies inputs x ∈ X from a given distribution D into classes y ∈ Y . We want a model with small test error on this distribution . In practice , we find such a classifier by minimizing the train error of a model on the train set . This procedure is justified when we expect a small generalization gap : the gap between the error on the train and test set . That is , the trained model f should have : ErrorTrainSet ( f ) ≈ ErrorTestSet ( f ) . We now re-write this classical notion of generalization in a form better suited for our extension . Classical Generalization : Let f be a trained classifier . Then f generalizes if : E x∼TrainSet ŷ←f ( x ) [ 1 { ŷ 6= y ( x ) } ] ≈ E x∼TestSet ŷ←f ( x ) [ 1 { ŷ 6= y ( x ) } ] ( 1 ) Above , y ( x ) is the true class of x and ŷ is the predicted class . The LHS of Equation 1 is the train error of f , and the RHS is the test error . Crucially , both sides of Equation 1 are expectations of the same function ( Terr ( x , ŷ ) : = 1 { ŷ 6= y ( x ) } ) under different distributions . The LHS of Equation 1 is the expectation of Terr under the “ Train Distribution ” Dtr , which is the distribution over ( x , ŷ ) given by sampling a train point x along with its classifier-label f ( x ) . Similarly , the RHS is under the “ Test Distribution ” Dte , which is this same construction over the test set . These two distributions are the central objects in our study , and are defined formally in Section 2.1 . We can now introduce Distributional Generalization , which is a property of trained classifiers . It is parameterized by a set of bounded functions ( “ tests ” ) : T ⊆ { T : X × Y → [ 0 , 1 ] } . Distributional Generalization : Let f be a trained classifier . Then f satisfies Distributional Generalization with respect to tests T if : ∀T ∈ T : E x∼TrainSet ŷ←f ( x ) [ T ( x , ŷ ) ] ≈ E x∼TestSet ŷ←f ( x ) [ T ( x , ŷ ) ] ( 2 ) We write this property as Dtr ≈T Dte . This states that the train and test distribution have similar expectations for all functions in the family T . For the singleton set T = { Terr } , this is equivalent to classical generalization , but it may hold for much larger sets T . For example in Experiment 1 , the train and test distributions match with respect to the test function “ Fraction of true cats labeled as object. ” In fact , it is best to think of Distributional Generalization as stating that the distributions Dtr and Dte are close as distributions . This property becomes especially interesting for interpolating classifiers , which fit their train sets exactly . Here , the Train Distribution ( xi , f ( xi ) ) is exactly equal1 to the original distribution ( x , y ) ∼ D , since f ( xi ) = yi on the train set . In this case , distributional generalization claims that the output distribution ( x , f ( x ) ) of the model on test samples is close to the true distribution ( x , y ) . The following conjecture specializes Distributional Generalization to interpolating classifiers , and will be the main focus of our work . Interpolating Indistinguishability Meta-Conjecture ( informal ) : For interpolating classifiers f , and a large family T of test functions , the distributions : ( x , f ( x ) ) x∈TestSet ≈T ( x , f ( x ) ) x∈TrainSet ≡ ( x , y ) x , y∼D 1The formal definition of Train Distribution , in Section 2.1 , includes the randomness of sampling the train set as well . We consider a fixed train set in the Introduction for sake of exposition . This is a “ meta-conjecture ” , which becomes a concrete conjecture once the family of tests T is specified . One of the main contributions of our work is formally stating two concrete instances of this conjecture— specifying exactly the family of tests T and their dependence on problem parameters ( the distribution , model family , training procedure , etc ) . It captures behaviors far more general than Experiment 1 , and applies to neural networks , kernels , and decision trees . We give empirical evidence for conjectures across a variety of natural settings in machine learning . 1.2 SUMMARY OF CONTRIBUTIONS . We extend the classical framework of generalization by introducing Distributional Generalization , in which the train and test behavior of models are close as distributions . Informally , for trained classifiers f , its outputs on the train set ( x , f ( x ) ) x∈TrainSet are close in distribution to its outputs on the test set ( x , f ( x ) ) x∈TestSet , where the form of this closeness depends on specifics of the model , training procedure , and distribution . This notion is more fine-grained than classical generalization , since it considers the entire distribution of model outputs instead of just the test error . We initiate the study of Distributional Generalization across various domains in machine learning . For interpolating classifiers , we state two formal conjectures which predict the form of distributional closeness that can be expected for a given model and task : 1 . Feature Calibration Conjecture ( Section 3 ) : Interpolating classifiers , when trained on samples from a distribution , will match this distribution up to all “ distinguishable features ” ( Definition 1 ) . 2 . Agreement Conjecture ( Section 4 ) : For two interpolating classifiers of the same type , trained independently on the same distribution , their agreement probability with each other on test samples roughly matches their test accuracy . We perform a number of experiments surrounding these conjectures , which reveal new behaviors of standard interpolating classifiers ( e.g . ResNets , MLPs , kernels , decision trees ) . We prove our conjectures for 1-Nearest-Neighbors ( Theorem 1 ) , which suggests some form of “ locality ” as the underlying mechanism . Finally , we discuss extending these results to non-interpolating methods in Section 5 . Our experiments and conjectures shed new light on the structure of interpolating classifiers , which are extensively studied in recent years yet still poorly understood . Related Work . Our work is inspired by the broader study of interpolating and overparameterized methods in machine learning ( e.g . Zhang et al . ( 2016 ) ; Belkin et al . ( 2018a ; b ; 2019 ) ; Liang and Rakhlin ( 2018 ) ; Nakkiran et al . ( 2020 ) ; Schapire et al . ( 1998 ) ; Breiman ( 1995 ) ) . In a similar vein to our work , Wyner et al . ( 2017 ) ; Olson and Wyner ( 2018 ) investigate decision trees , and show that random forests are equivalent to a Nadaraya–Watson smoother ( Nadaraya , 1964 ; Watson , 1964 ) with a certain smoothing kernel . Our conjectures also describe neural networks under label noise , which has been empirically and theoretically studied in the past ( Zhang et al. , 2016 ; Belkin et al. , 2018b ; Rolnick et al. , 2017 ; Natarajan et al. , 2013 ; Thulasidasan et al. , 2019 ; Ziyin et al. , 2020 ; Chatterji and Long , 2020 ) , though not formally characterized . The behaviors we consider are also similar to conditional density estimation ( e.g . Tsybakov ( 2008 ) ; Dutordoir et al . ( 2018 ) ) , though we consider samplers , not density estimators . We include a full discussion of related works in Appendix A . 2 PRELIMINARIES . Notation . We consider joint distributionsD on x ∈ X and discrete y ∈ Y = [ k ] . LetDn denote n iid samples from D and S = { ( xi , yi ) } denote a train set . Let F denote the classifier family ( including architecture and training algorithm for neural networks ) , and let f ← TrainF ( S ) denote training a classifier f ∈ F on train-set S. We consider classifiers which output hard decisions f : X → Y . Let NNS ( x ) = xi denote the nearest-neighbor to x in train-set S , with respect to a distance metric d. Our theorems will apply to any distance metric , and so we leave this unspecified . Let NN ( y ) S ( x ) denote the nearest-neighbor estimator itself , that is , NN ( y ) S ( x ) : = yi where xi = NNS ( x ) . Experimental Setup . Full experimental details are provided in Appendix B . Briefly , we train all classifiers to interpolation ( to 0 train error ) . Neural networks ( MLPs and ResNets ( He et al. , 2016 ) ) are trained with SGD . Interpolating decision trees are trained using the growth rule from Random Forests ( Breiman , 2001 ) . For kernel classification , we consider kernel regression on one-hot labels and kernel SVM , with small or 0 of regularization ( which is often optimal Shankar et al . ( 2020 ) ) . Distributional Closeness . We consider the following notion of closeness for two probability distributions : For two distributions P , Q over X × Y , let a “ test ” ( or “ distinguisher ” ) be a function T : X × Y → [ 0 , 1 ] which accepts a sample from either distribution , and is intended to classify the sample as either from distribution P or Q . For any set C ⊆ { T : X × Y → [ 0 , 1 ] } of tests , we say distributions P and Q are “ ε-indistinguishable up to C-tests ” if they are close with respect to all tests in class C. That is , P ≈Cε Q ⇐⇒ sup T∈C ∣∣∣∣ E ( x , y ) ∼P [ T ( x , y ) ] − E ( x , y ) ∼Q [ T ( x , y ) ] ∣∣∣∣ ≤ ε ( 3 ) Total-Variation distance is equivalent to closeness in all tests , i.e . C = { T : X × Y → [ 0 , 1 ] } , but we consider closeness for restricted families of tests C. P ≈ε Q denotes ε-closeness in TV-distance .

Summary: This paper proposes a new notion of generalization called distributional generalization which states that the outputs of the classifier for train and test are close as distributions not just their corresponding accuracy numbers. They propose conjectures about their the distributional closeness that they expect and how it depends on the model, number of data points and the algorithm. This paper gives experiments to support their conjectures for different model classes including neural networks, kernel methods and decision trees.