Original: D ISTRIBUTIONAL G ENERALIZATION : A N EW K IND OF G ENERALIZATION
Revised: D ISTRIBUTIONAL G ENERALIZATION : A N EW K IND OF G ENERALIZATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce a new notion of generalization— Distributional Generalization— which roughly states that outputs of a classiﬁer at train and test time are close as distributions , as opposed to close in just their average error.
Revised: We introduce a new notion of generalization— Distributional Generalization— which roughly states that outputs of a classiﬁer at train and test time are close as distributions , as opposed to close in just their average error.

Original: For example, if we mislabel 30% of dogs as cats in the train set of CIFAR-10, then a ResNet trained to interpolation will in fact mislabel roughly 30% of dogs as cats on the test set as well, while leaving other classes unaffected.
Revised: For example, if we mislabel 30% of dogs as cats in the train set of CIFAR-10, then a ResNet trained to interpolation will in fact mislabel roughly 30% of dogs as cats on the test set as well, while leaving other classes unaffected.

Original: This behavior is not captured by classical generalization, which would only consider the average error and not the distribution of errors over the input domain.
Revised: This behavior is not captured by classical generalization, which would only consider the average error and not the distribution of errors over the input domain.

Original: Our formal conjectures, which are much more general than this example, characterize the form of distributional generalization that can be expected in terms of problem parameters: model architecture, training procedure, number of samples, and data distribution.
Revised: Our formal conjectures, which are much more general than this example, characterize the form of distributional generalization that can be expected in terms of problem parameters: model architecture, training procedure, number of samples, and data distribution.

Original: We give empirical evidence for these conjectures across a variety of domains in machine learning, including neural networks, kernel machines, and decision trees.
Revised: We give empirical evidence for these conjectures across a variety of domains in machine learning, including neural networks, kernel machines, and decision trees.

Original: Our results thus advance our understanding of interpolating classiﬁers.
Revised: Our results thus advance our understanding of interpolating classiﬁers.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: We begin with an experiment motivating the need for a notion of generalization beyond test error.
Revised: We begin with an experiment motivating the need for a notion of generalization beyond test error.

Original: Experiment 1. Consider a binary classiﬁcation version of CIFAR-10, where CIFAR-10 images x have binary labels Animal/Object .
Revised: Experiment 1. Consider a binary classiﬁcation version of CIFAR-10, where CIFAR-10 images x have binary labels Animal/Object .

Original: Take 50K samples from this distribution as a train set, but apply the following label noise: ﬂip the label of cats to Object with probability 30%.
Revised: Take 50K samples from this distribution as a train set, but apply the following label noise: ﬂip the label of cats to Object with probability 30%.

Original: Now train a WideResNet f to 0 train error on this train set.
Revised: Now train a WideResNet f to 0 train error on this train set.

Original: How does the trained classiﬁer behave on test samples?
Revised: How does the trained classiﬁer behave on test samples?

Original: Options below:
Revised: Options below:

Original: The test error is low across all classes, since there is only 3% label noise in the train set 2.
Revised: The test error is low across all classes, since there is only 3% label noise in the train set 2.

Original: Test error is “spread” across the animal class, After all, the classiﬁer is not explicitly told what a cat or a dog is, just that they are all animals.
Revised: Test error is “spread” across the animal class, After all, the classiﬁer is not explicitly told what a cat or a dog is, just that they are all animals.

Original: The classiﬁer misclassiﬁes roughly 30% of test cats as “objects”, but all other types of animals
Revised: The classiﬁer misclassiﬁes roughly 30% of test cats as “objects”, but all other types of animals

Original: are largely unaffected.
Revised: are largely unaffected.

Original: In fact, reality is closest to option (3).
Revised: In fact, reality is closest to option (3).

Original: Figure 1 shows the results of this experiment with a WideResNet.
Revised: Figure 1 shows the results of this experiment with a WideResNet.

Original: The left panel shows the joint density of train inputs x with train labels Object/Animal .
Revised: The left panel shows the joint density of train inputs x with train labels Object/Animal .

Original: Since the classiﬁer is interpolating, the classiﬁer outputs on the train set are identical to the left panel.
Revised: Since the classiﬁer is interpolating, the classiﬁer outputs on the train set are identical to the left panel.

Original: The right panel shows the classiﬁer predictions f ( x ) on test inputs x .
Revised: The right panel shows the classiﬁer predictions f ( x ) on test inputs x .

Original: There are several notable things about this experiment.
Revised: There are several notable things about this experiment.

Original: First, the error is localized to cats in test set as it was in the train set, even though no explicit cat labels were provided.
Revised: First, the error is localized to cats in test set as it was in the train set, even though no explicit cat labels were provided.

Original: Second, the amount of error on the cat class is close to the noise applied on the train set.
Revised: Second, the amount of error on the cat class is close to the noise applied on the train set.

Original: Thus, the behavior of the classiﬁer on the train set generalizes to the test set in a certain sense.
Revised: Thus, the behavior of the classiﬁer on the train set generalizes to the test set in a certain sense.

Original: This type of similarity in behavior would not be captured solely by average test error — it requires reasoning about the entire distribution of classiﬁer outputs.
Revised: This type of similarity in behavior would not be captured solely by average test error — it requires reasoning about the entire distribution of classiﬁer outputs.

Original: In our work, we show that this experiment is just one instance of a different type of generalization , which we call “Distributional Generalization”.
Revised: In our work, we show that this experiment is just one instance of a different type of generalization , which we call “Distributional Generalization”.

Original: We now describe the mathematical form of this generalization.
Revised: We now describe the mathematical form of this generalization.

Original: Then, through extensive experiments, we will show that this type of generalization occurs widely in existing machine learning methods: neural networks, kernel machines and decision trees.
Revised: Then, through extensive experiments, we will show that this type of generalization occurs widely in existing machine learning methods: neural networks, kernel machines and decision trees.

Original: 1. D ISTRIBUTIONAL G ENERALIZATION
Revised: 1. D ISTRIBUTIONAL G ENERALIZATION

Original: Supervised learning aims to learn a model that correctly classiﬁes inputs x ∈ X from a given distribution D into classes y ∈ Y .
Revised: Supervised learning aims to learn a model that correctly classiﬁes inputs x ∈ X from a given distribution D into classes y ∈ Y .

Original: We want a model with small test error on this distribution.
Revised: We want a model with small test error on this distribution.

Original: In practice, we ﬁnd such a classiﬁer by minimizing the train error of a model on the train set.
Revised: In practice, we ﬁnd such a classiﬁer by minimizing the train error of a model on the train set.

Original: This procedure is justiﬁed when we expect a small generalization gap : the gap between the error on the train and test set.
Revised: This procedure is justiﬁed when we expect a small generalization gap : the gap between the error on the train and test set.

Original: That is, the trained model f should have: Error TrainSet ( f ) ≈ Error TestSet ( f ) .
Revised: That is, the trained model f should have: Error TrainSet ( f ) ≈ Error TestSet ( f ) .

Original: We now re-write this classical notion of generalization in a form better suited for our extension.
Revised: We now re-write this classical notion of generalization in a form better suited for our extension.

Original: Classical Generalization: Let f be a trained classiﬁer.
Revised: Classical Generalization: Let f be a trained classiﬁer.

Original: Then f generalizes if:
Revised: Then f generalizes if:

Original: [Equation]
Revised: [Equation]

Original: Above, y ( x ) is the true class of x and (cid:98) y is the predicted class.
Revised: Above, y ( x ) is the true class of x and (cid:98) y is the predicted class.

Original: The LHS of Equation 1 is the train error of f , and the RHS is the test error.
Revised: The LHS of Equation 1 is the train error of f , and the RHS is the test error.

Original: Crucially, both sides of Equation 1 are expectations of the same function ( T err ( x, (cid:98) y ) :
Revised: Crucially, both sides of Equation 1 are expectations of the same function ( T err ( x, (cid:98) y ) :

Original: = 1 { (cid:98) y (cid:54) =
Revised: = 1 { (cid:98) y (cid:54) =

Original: y ( x ) } ) under different distributions.
Revised: y ( x ) } ) under different distributions.

Original: The LHS of Equationis the expectation of T err under the “Train Distribution” D tr , which is the distribution over ( x, (cid:98) y ) given by sampling a train point x along with its classiﬁer-label f ( x ) .
Revised: The LHS of Equationis the expectation of T err under the “Train Distribution” D tr , which is the distribution over ( x, (cid:98) y ) given by sampling a train point x along with its classiﬁer-label f ( x ) .

Original: Similarly, the RHS is under the “Test Distribution” D te , which is this same construction over the test set.
Revised: Similarly, the RHS is under the “Test Distribution” D te , which is this same construction over the test set.

Original: These two distributions are the central objects in our study, and are deﬁned formally in Section 2.1.
Revised: These two distributions are the central objects in our study, and are deﬁned formally in Section 2.1.

Original: We can now introduce
Revised: We can now introduce

Original: Distributional Generalization, which is a property of trained classiﬁers.
Revised: Distributional Generalization, which is a property of trained classiﬁers.

Original: It is parameterized by a set of bounded functions (“tests”): T ⊆
Revised: It is parameterized by a set of bounded functions (“tests”): T ⊆

Original: { T : X × Y → [0 , 1] } .
Revised: { T : X × Y → [0 , 1] } .

Original: Distributional Generalization: Let f be a trained classiﬁer.
Revised: Distributional Generalization: Let f be a trained classiﬁer.

Original: Then f satisﬁes Distributional Generalization with respect to tests T if:
Revised: Then f satisﬁes Distributional Generalization with respect to tests T if:

Original: [Equation]
Revised: [Equation]

Original: We write this property as D tr ≈ T D te .
Revised: We write this property as D tr ≈ T D te .

Original: This states that the train and test distribution have similar expectations for all functions in the family T .
Revised: This states that the train and test distribution have similar expectations for all functions in the family T .

Original: For the singleton set T = { T err } , this is equivalent to classical generalization, but it may hold for much larger sets T .
Revised: For the singleton set T = { T err } , this is equivalent to classical generalization, but it may hold for much larger sets T .

Original: For example in Experiment 1, the train and test distributions match with respect to the test function “ Fraction of true cats labeled as object .”
Revised: For example in Experiment 1, the train and test distributions match with respect to the test function “ Fraction of true cats labeled as object .”

Original: In fact, it is best to think of Distributional Generalization as stating that the distributions D tr and D te are close as distributions .
Revised: In fact, it is best to think of Distributional Generalization as stating that the distributions D tr and D te are close as distributions .

Original: This property becomes especially interesting for interpolating classiﬁers , which ﬁt their train sets exactly.
Revised: This property becomes especially interesting for interpolating classiﬁers , which ﬁt their train sets exactly.

Original: Here, the Train Distribution ( x i , f ( x i )) is exactly equal 1 to the original distribution ( x, y ) ∼
Revised: Here, the Train Distribution ( x i , f ( x i )) is exactly equal 1 to the original distribution ( x, y ) ∼

Original: D , since f ( x i ) =
Revised: D , since f ( x i ) =

Original: y i on the train set.
Revised: y i on the train set.

Original: In this case, distributional generalization claims that the output distribution ( x, f ( x )) of the model on test samples is close to the true distribution ( x, y ) .
Revised: In this case, distributional generalization claims that the output distribution ( x, f ( x )) of the model on test samples is close to the true distribution ( x, y ) .

Original: The following conjecture specializes Distributional Generalization to interpolating classiﬁers, and will be the main focus of our work.
Revised: The following conjecture specializes Distributional Generalization to interpolating classiﬁers, and will be the main focus of our work.

Original: Interpolating Indistinguishability Meta-Conjecture (informal): For interpolating classiﬁers f , and a large family T of test functions, the distributions:
Revised: Interpolating Indistinguishability Meta-Conjecture (informal): For interpolating classiﬁers f , and a large family T of test functions, the distributions:

Original: [Equation]
Revised: [Equation]

Original: This is a “meta-conjecture”, which becomes a concrete conjecture once the family of tests T is speciﬁed.
Revised: This is a “meta-conjecture”, which becomes a concrete conjecture once the family of tests T is speciﬁed.

Original: One of the main contributions of our work is formally stating two concrete instances of this conjecture— specifying exactly the family of tests T and their dependence on problem parameters (the distribution, model family, training procedure, etc).
Revised: One of the main contributions of our work is formally stating two concrete instances of this conjecture— specifying exactly the family of tests T and their dependence on problem parameters (the distribution, model family, training procedure, etc).

Original: It captures behaviors far more general than
Revised: It captures behaviors far more general than

Original: Experiment 1, and applies to neural networks, kernels, and decision trees.
Revised: Experiment 1, and applies to neural networks, kernels, and decision trees.

Original: We give empirical evidence for conjectures across a variety of natural settings in machine learning.
Revised: We give empirical evidence for conjectures across a variety of natural settings in machine learning.

Original: 1. S UMMARY OF C ONTRIBUTIONS
Revised: 1. S UMMARY OF C ONTRIBUTIONS

Original: We extend the classical framework of generalization by introducing Distributional Generalization, in which the train and test behavior of models are close as distributions .
Revised: We extend the classical framework of generalization by introducing Distributional Generalization, in which the train and test behavior of models are close as distributions .

Original: Informally, for trained classiﬁers f , its outputs on the train set ( x, f ( x )) x ∈ TrainSet are close in distribution to its outputs on the test set ( x, f ( x )) x ∈ TestSet , where the form of this closeness depends on speciﬁcs of the model, training procedure, and distribution.
Revised: Informally, for trained classiﬁers f , its outputs on the train set ( x, f ( x )) x ∈ TrainSet are close in distribution to its outputs on the test set ( x, f ( x )) x ∈ TestSet , where the form of this closeness depends on speciﬁcs of the model, training procedure, and distribution.

Original: This notion is more ﬁne-grained than classical generalization, since it considers the entire distribution of model outputs instead of just the test error.
Revised: This notion is more ﬁne-grained than classical generalization, since it considers the entire distribution of model outputs instead of just the test error.

Original: We initiate the study of Distributional Generalization across various domains in machine learning.
Revised: We initiate the study of Distributional Generalization across various domains in machine learning.

Original: For interpolating classiﬁers, we state two formal conjectures which predict the form of distributional closeness that can be expected for a given model and task:
Revised: For interpolating classiﬁers, we state two formal conjectures which predict the form of distributional closeness that can be expected for a given model and task:

Original: Feature Calibration Conjecture (Section 3): Interpolating classiﬁers, when trained on samples from a distribution, will match this distribution up to all “distinguishable features” (Deﬁnition 1).
Revised: Feature Calibration Conjecture (Section 3): Interpolating classiﬁers, when trained on samples from a distribution, will match this distribution up to all “distinguishable features” (Deﬁnition 1).

Original: Agreement Conjecture (Section 4): For two interpolating classiﬁers of the same type, trained independently on the same distribution, their agreement probability with each other on test samples roughly matches their test accuracy .
Revised: Agreement Conjecture (Section 4): For two interpolating classiﬁers of the same type, trained independently on the same distribution, their agreement probability with each other on test samples roughly matches their test accuracy .

Original: We perform a number of experiments surrounding these conjectures, which reveal new behaviors of standard interpolating classiﬁers (e.g. ResNets, MLPs, kernels, decision trees).
Revised: We perform a number of experiments surrounding these conjectures, which reveal new behaviors of standard interpolating classiﬁers (e.g. ResNets, MLPs, kernels, decision trees).

Original: We prove our conjectures for 1-Nearest-Neighbors (Theorem 1), which suggests some form of “locality” as the underlying mechanism.
Revised: We prove our conjectures for 1-Nearest-Neighbors (Theorem 1), which suggests some form of “locality” as the underlying mechanism.

Original: Finally, we discuss extending these results to non-interpolating methods in
Revised: Finally, we discuss extending these results to non-interpolating methods in

Original: Section 5.
Revised: Section 5.

Original: Our experiments and conjectures shed new light on the structure of interpolating classiﬁers, which are extensively studied in recent years yet still poorly understood.
Revised: Our experiments and conjectures shed new light on the structure of interpolating classiﬁers, which are extensively studied in recent years yet still poorly understood.

Original: Related Work.
Revised: Related Work.

Original: Our work is inspired by the broader study of interpolating and overparameterized methods in machine learning (e.g. Zhang et al.
Revised: Our work is inspired by the broader study of interpolating and overparameterized methods in machine learning (e.g. Zhang et al.

Original: ; Belkin et al.
Revised: ; Belkin et al.

Original: (2018a;b; 2019); Liang and
Revised: (2018a;b; 2019); Liang and

Original: Rakhlin (2018); Nakkiran et al.
Revised: Rakhlin (2018); Nakkiran et al.

Original: Schapire et al.
Revised: Schapire et al.

Original: Breiman (1995)).
Revised: Breiman (1995)).

Original: In a similar vein to our work, Wyner et al.
Revised: In a similar vein to our work, Wyner et al.

Original: Olson and Wyner (2018) investigate decision trees, and show that random forests are equivalent to a Nadaraya–Watson smoother (Nadaraya, 1964; Watson, 1964) with a certain smoothing kernel.
Revised: Olson and Wyner (2018) investigate decision trees, and show that random forests are equivalent to a Nadaraya–Watson smoother (Nadaraya, 1964; Watson, 1964) with a certain smoothing kernel.

Original: Our conjectures also describe neural networks under label noise, which has been empirically and theoretically studied in the past (Zhang et al., 2016; Belkin et al., 2018b;
Revised: Our conjectures also describe neural networks under label noise, which has been empirically and theoretically studied in the past (Zhang et al., 2016; Belkin et al., 2018b;

Original: Rolnick et al., 2017; Natarajan et al., 2013; Thulasidasan et al., 2019; Ziyin et al., 2020; Chatterji and Long, 2020), though not formally characterized.
Revised: Rolnick et al., 2017; Natarajan et al., 2013; Thulasidasan et al., 2019; Ziyin et al., 2020; Chatterji and Long, 2020), though not formally characterized.

Original: The behaviors we consider are also similar to conditional density estimation (e.g. Tsybakov (2008); Dutordoir et al.
Revised: The behaviors we consider are also similar to conditional density estimation (e.g. Tsybakov (2008); Dutordoir et al.

Original: (2018)), though we consider samplers , not density estimators.
Revised: (2018)), though we consider samplers , not density estimators.

Original: We include a full discussion of related works in Appendix A.
Revised: We include a full discussion of related works in Appendix A.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Notation.
Revised: Notation.

Original: We consider joint distributions D on x ∈ X and discrete y ∈ Y =
Revised: We consider joint distributions D on x ∈ X and discrete y ∈ Y =

Original: [ k ] .
Revised: [ k ] .

Original: Let D n denote n iid samples from D and S =
Revised: Let D n denote n iid samples from D and S =

Original: { ( x i , y i ) } denote a train set.
Revised: { ( x i , y i ) } denote a train set.

Original: Let F denote the classiﬁer family (including architecture and training algorithm for neural networks), and let f ← Train F ( S ) denote training a classiﬁer f ∈ F on train-set S .
Revised: Let F denote the classiﬁer family (including architecture and training algorithm for neural networks), and let f ← Train F ( S ) denote training a classiﬁer f ∈ F on train-set S .

Original: We consider classiﬁers which output hard decisions f : X → Y .
Revised: We consider classiﬁers which output hard decisions f : X → Y .

Original: Let NN S ( x ) =
Revised: Let NN S ( x ) =

Original: x i denote the nearest-neighbor to x in train-set S , with respect to a distance metric d .
Revised: x i denote the nearest-neighbor to x in train-set S , with respect to a distance metric d .

Original: Our theorems will apply to any distance metric, and so we leave this unspeciﬁed.
Revised: Our theorems will apply to any distance metric, and so we leave this unspeciﬁed.

Original: Let NN ( y ) S
Revised: Let NN ( y ) S

Original: ( x ) denote the nearest-neighbor estimator itself, that is, NN ( y ) S ( x ) :
Revised: ( x ) denote the nearest-neighbor estimator itself, that is, NN ( y ) S ( x ) :

Original: = y i where x i = NN S ( x ) .
Revised: = y i where x i = NN S ( x ) .

Original: Experimental Setup.
Revised: Experimental Setup.

Original: Full experimental details are provided in Appendix B. Brieﬂy, we train all classiﬁers to interpolation (to 0 train error).
Revised: Full experimental details are provided in Appendix B. Brieﬂy, we train all classiﬁers to interpolation (to 0 train error).

Original: Neural networks (MLPs and ResNets (He et al., 2016)) are trained with SGD.
Revised: Neural networks (MLPs and ResNets (He et al., 2016)) are trained with SGD.

Original: Interpolating decision trees are trained using the growth rule from Random
Revised: Interpolating decision trees are trained using the growth rule from Random

Original: Forests (Breiman, 2001).
Revised: Forests (Breiman, 2001).

Original: For kernel classiﬁcation, we consider kernel regression on one-hot labels and kernel SVM, with small or 0 of regularization (which is often optimal Shankar et al.
Revised: For kernel classiﬁcation, we consider kernel regression on one-hot labels and kernel SVM, with small or 0 of regularization (which is often optimal Shankar et al.

Original: Distributional Closeness.
Revised: Distributional Closeness.

Original: We consider the following notion of closeness for two probability distributions: For two distributions P, Q over X × Y , let a “test” (or “distinguisher”) be a function
Revised: We consider the following notion of closeness for two probability distributions: For two distributions P, Q over X × Y , let a “test” (or “distinguisher”) be a function

Original: T : X × Y → [0 , 1] which accepts a sample from either distribution, and is intended to classify the sample as either from distribution P or Q .
Revised: T : X × Y → [0 , 1] which accepts a sample from either distribution, and is intended to classify the sample as either from distribution P or Q .

Original: For any set C ⊆ { T : X × Y → [0 , 1] } of tests, we say distributions P and Q are “ ε -indistinguishable up to C -tests” if they are close with respect to all tests in class C .
Revised: For any set C ⊆ { T : X × Y → [0 , 1] } of tests, we say distributions P and Q are “ ε -indistinguishable up to C -tests” if they are close with respect to all tests in class C .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: Total-Variation distance is equivalent to closeness in all tests,
Revised: Total-Variation distance is equivalent to closeness in all tests,

Original: i.e. C =
Revised: i.e. C =

Original: { T : X × Y → [0 , 1] } , but we consider closeness for restricted families of tests C .
Revised: { T : X × Y → [0 , 1] } , but we consider closeness for restricted families of tests C .

Original: P ≈ ε Q denotes ε -closeness in TV-distance.
Revised: P ≈ ε Q denotes ε -closeness in TV-distance.

Original: 2. F RAMEWORK FOR I NDISTINGUISHABILITY
Revised: 2. F RAMEWORK FOR I NDISTINGUISHABILITY

Original: We consider throughout the following three distributions over X × Y :
Revised: We consider throughout the following three distributions over X × Y :

Original: Source D :
Revised: Source D :

Original: [Equation]
Revised: [Equation]

Original: where x, y ∼
Revised: where x, y ∼

Original: [Equation]
Revised: [Equation]

Original: The Source Distribution D is simply the original distribution.
Revised: The Source Distribution D is simply the original distribution.

Original: To sample from the Train Distribution D tr , we ﬁrst sample a train set S ∼ D n , train a classiﬁer f on it, then output ( x tr , f ( x tr )) for a random train point x tr .
Revised: To sample from the Train Distribution D tr , we ﬁrst sample a train set S ∼ D n , train a classiﬁer f on it, then output ( x tr , f ( x tr )) for a random train point x tr .

Original: That is, D tr is the distribution of input and outputs of a trained classiﬁer f on its train set.
Revised: That is, D tr is the distribution of input and outputs of a trained classiﬁer f on its train set.

Original: To sample from the Test Distribution D te , do we this same procedure, but output ( x, f ( x )) for a random test point x .
Revised: To sample from the Test Distribution D te , do we this same procedure, but output ( x, f ( x )) for a random test point x .

Original: That is, the D te is the distribution of input and outputs of a trained classiﬁer f at test time.
Revised: That is, the D te is the distribution of input and outputs of a trained classiﬁer f at test time.

Original: The only difference between the Train Distribution and Test Distribution is that the point x is sampled from the train set or the test set, respectively.
Revised: The only difference between the Train Distribution and Test Distribution is that the point x is sampled from the train set or the test set, respectively.

Original: 2 For interpolating classiﬁers, f ( x tr ) =
Revised: 2 For interpolating classiﬁers, f ( x tr ) =

Original: y tr on the train set, and so the Source and Train distributions are equivalent:
Revised: y tr on the train set, and so the Source and Train distributions are equivalent:

Original: D ≡ D tr .
Revised: D ≡ D tr .

Original: Our general thesis is that the Train and Test Distributions are indistinguishable under a variety of test families T .
Revised: Our general thesis is that the Train and Test Distributions are indistinguishable under a variety of test families T .

Original: Formally, we argue that for certain families of tests T and interpolating classiﬁers F , the distributions: D ≡ D tr ≈ T ε D te .
Revised: Formally, we argue that for certain families of tests T and interpolating classiﬁers F , the distributions: D ≡ D tr ≈ T ε D te .

Original: Sections 3 and 4 give speciﬁc families of tests T for which these distributions are indistinguishable.
Revised: Sections 3 and 4 give speciﬁc families of tests T for which these distributions are indistinguishable.

Original:  F EATURE C ALIBRATION
Revised:  F EATURE C ALIBRATION

Original: The distributional closeness of Experiments 1 is subtle, and depends on the classiﬁer architecture, distribution, and training method.
Revised: The distributional closeness of Experiments 1 is subtle, and depends on the classiﬁer architecture, distribution, and training method.

Original: For example, Experiment 1 does not hold if we use a fullyconnected network (MLP) instead of a ResNet, or if we early-stop the ResNet instead of training to interpolation (see Appendix C.2).
Revised: For example, Experiment 1 does not hold if we use a fullyconnected network (MLP) instead of a ResNet, or if we early-stop the ResNet instead of training to interpolation (see Appendix C.2).

Original: Both these scenarios fail in different ways: An MLP cannot properly distinguish cats even when trained on real CIFAR-10 labels, and so (informally) it has no hope of behaving differently on cats in the setting of Experiment 1.
Revised: Both these scenarios fail in different ways: An MLP cannot properly distinguish cats even when trained on real CIFAR-10 labels, and so (informally) it has no hope of behaving differently on cats in the setting of Experiment 1.

Original: On the other hand, an early-stopped ResNet for Experiment 1 does not label 30% of cats as objects on the train set , since it does not interpolate, and thus has no hope of reproducing this behavior on the test set.
Revised: On the other hand, an early-stopped ResNet for Experiment 1 does not label 30% of cats as objects on the train set , since it does not interpolate, and thus has no hope of reproducing this behavior on the test set.

Original: We now characterize these behaviors, and their dependency on problem parameters, via a formal conjecture.
Revised: We now characterize these behaviors, and their dependency on problem parameters, via a formal conjecture.

Original: This conjecture characterizes a family of tests T for which the output distribution of a classiﬁer ( x, f ( x )) ∼ D te is “close” to the source distribution ( x, y ) ∼ D .
Revised: This conjecture characterizes a family of tests T for which the output distribution of a classiﬁer ( x, f ( x )) ∼ D te is “close” to the source distribution ( x, y ) ∼ D .

Original: At a high level, we argue that the distributions D te and D are statistically close if we ﬁrst “coarsen” the domain of x by some labelling L : X → [ M ] .
Revised: At a high level, we argue that the distributions D te and D are statistically close if we ﬁrst “coarsen” the domain of x by some labelling L : X → [ M ] .

Original: That is, for certain partitions L , the following distributions are statistically close: ( L ( x ) , f ( x ))
Revised: That is, for certain partitions L , the following distributions are statistically close: ( L ( x ) , f ( x ))

Original: ≈
Revised: ≈

Original: ε ( L ( x ) , y ) .
Revised: ε ( L ( x ) , y ) .

Original: Intuitively, allowable partitions are those which can be learnt from samples.
Revised: Intuitively, allowable partitions are those which can be learnt from samples.

Original: To formalize the set of allowable partitions L for a training algorithm, we deﬁne a distinguishable feature : a partition of the domain X that is learnable for a given family of models.
Revised: To formalize the set of allowable partitions L for a training algorithm, we deﬁne a distinguishable feature : a partition of the domain X that is learnable for a given family of models.

Original: For example, in Experiment 1, the partition into CIFAR-10 classes would be a distinguishable feature for ResNets, but not for MLPs.
Revised: For example, in Experiment 1, the partition into CIFAR-10 classes would be a distinguishable feature for ResNets, but not for MLPs.

Original: Deﬁnition 1 ( ( ε, F , D , n ) -Distinguishable Feature) .
Revised: Deﬁnition 1 ( ( ε, F , D , n ) -Distinguishable Feature) .

Original: For a distribution D over X × Y , number of samples n , family of models F , and small ε ≥ 0 , an ( ε, F , D , n ) -distinguishable feature is a partition
Revised: For a distribution D over X × Y , number of samples n , family of models F , and small ε ≥ 0 , an ( ε, F , D , n ) -distinguishable feature is a partition

Original: L : X → [ M ] of the domain X into M parts, such that training a model from F on n samples labeled by L works to classify L with high test accuracy.
Revised: L : X → [ M ] of the domain X into M parts, such that training a model from F on n samples labeled by L works to classify L with high test accuracy.

Original: Precisely, L is a ( ε, F , D , n ) -distinguishable feature if:
Revised: Precisely, L is a ( ε, F , D , n ) -distinguishable feature if:

Original: [Equation]
Revised: [Equation]

Original: This deﬁnition depends only on the marginal distribution of D on x , and not on the label distribution p D
Revised: This deﬁnition depends only on the marginal distribution of D on x , and not on the label distribution p D

Original: ( y | x ) .
Revised: ( y | x ) .

Original: To recap, this deﬁnition is meant to capture a labeling of the domain X that is learnable for a given training procedure.
Revised: To recap, this deﬁnition is meant to capture a labeling of the domain X that is learnable for a given training procedure.

Original: It must depend on the classiﬁer family F and number of samples n , since more powerful classiﬁers can distinguish more features.
Revised: It must depend on the classiﬁer family F and number of samples n , since more powerful classiﬁers can distinguish more features.

Original: Note that there could be many distinguishable features for a given setting ( ε, F , D , n ) — including features not implied by the class label such as the presence of grass in a CIFAR-10 image.
Revised: Note that there could be many distinguishable features for a given setting ( ε, F , D , n ) — including features not implied by the class label such as the presence of grass in a CIFAR-10 image.

Original: Our main conjecture in this section is that the test distribution ( x, f ( x )) ∼ D te is statistically close to the source distribution ( x, y ) ∼ D when the domain is “coarsened” by a distinguishable feature.
Revised: Our main conjecture in this section is that the test distribution ( x, f ( x )) ∼ D te is statistically close to the source distribution ( x, y ) ∼ D when the domain is “coarsened” by a distinguishable feature.

Original: Formally:
Revised: Formally:

Original: Conjecture 1 (Feature Calibration) .
Revised: Conjecture 1 (Feature Calibration) .

Original: For all natural distributions D , number of samples n , family of interpolating models F , and ε ≥ 0 , the following distributions are statistically close for all ( ε, F , D , n ) -distinguishable
Revised: For all natural distributions D , number of samples n , family of interpolating models F , and ε ≥ 0 , the following distributions are statistically close for all ( ε, F , D , n ) -distinguishable

Original: features L :
Revised: features L :

Original: [Equation]
Revised: [Equation]

Original: We claim that this holds for all distinguishable features L “automatically” – we simply train a classiﬁer, without specifying any particular partition.
Revised: We claim that this holds for all distinguishable features L “automatically” – we simply train a classiﬁer, without specifying any particular partition.

Original: As a trivial instance of the conjecture, suppose we have a distribution with deterministic labels, and consider the ε -distinguishable feature L ( x ) :
Revised: As a trivial instance of the conjecture, suppose we have a distribution with deterministic labels, and consider the ε -distinguishable feature L ( x ) :

Original: = y ( x ) , i.e. the label itself.
Revised: = y ( x ) , i.e. the label itself.

Original: The ε here is then simply the test error of f , and Conjecture 1 is true by deﬁnition.
Revised: The ε here is then simply the test error of f , and Conjecture 1 is true by deﬁnition.

Original: The formal statements of Deﬁnition 1 and Conjecture 1 may seem somewhat arbitrary, involving many quantiﬁers over ( ε, F , D , n ) .
Revised: The formal statements of Deﬁnition 1 and Conjecture 1 may seem somewhat arbitrary, involving many quantiﬁers over ( ε, F , D , n ) .

Original: However, we believe these statements are natural.
Revised: However, we believe these statements are natural.

Original: To support this, in Theorem 1 we prove that Conjecture 1 is formally true as stated for 1-Nearest-Neighbor classiﬁers.
Revised: To support this, in Theorem 1 we prove that Conjecture 1 is formally true as stated for 1-Nearest-Neighbor classiﬁers.

Original: Connection to Indistinguishability.
Revised: Connection to Indistinguishability.

Original: Conjecture 1 is an instantiation of our general Indistinguishably
Revised: Conjecture 1 is an instantiation of our general Indistinguishably

Original: Conjecture.
Revised: Conjecture.

Original: In particular, Conjecture 1 is equivalent to the statement D te ≈ L ε D where L is the family of all tests which depend on x only via a distinguishable feature L .
Revised: In particular, Conjecture 1 is equivalent to the statement D te ≈ L ε D where L is the family of all tests which depend on x only via a distinguishable feature L .

Original: In other words, D te is indistinguishable from D to any function that only sees the input x via a distinguishable feature L ( x ) .
Revised: In other words, D te is indistinguishable from D to any function that only sees the input x via a distinguishable feature L ( x ) .

Original: 3. E XPERIMENTS
Revised: 3. E XPERIMENTS

Original: We now give empirical evidence for our conjectures in a variety of settings in machine learning, including neural networks, kernel machines, and decision trees.
Revised: We now give empirical evidence for our conjectures in a variety of settings in machine learning, including neural networks, kernel machines, and decision trees.

Original: Selected experiments are summarized here, with full details and further experiments in Appendix C.
Revised: Selected experiments are summarized here, with full details and further experiments in Appendix C.

Original: Constant Partition: Consider the trivially-distinguishable constant feature L ( x ) = 0 .
Revised: Constant Partition: Consider the trivially-distinguishable constant feature L ( x ) = 0 .

Original: Then, Conjecture 1 states that the marginal distribution of class labels for any interpolating classiﬁer f ( x ) is close to the true marginals p ( y ) .
Revised: Then, Conjecture 1 states that the marginal distribution of class labels for any interpolating classiﬁer f ( x ) is close to the true marginals p ( y ) .

Original: We construct a variant of CIFAR-10 with class-imbalance and train classiﬁers with varying levels of test errors (9-41%) to interpolation on it.
Revised: We construct a variant of CIFAR-10 with class-imbalance and train classiﬁers with varying levels of test errors (9-41%) to interpolation on it.

Original: The marginals of the classiﬁer outputs are close to that of the train set, irrespective of the test error.
Revised: The marginals of the classiﬁer outputs are close to that of the train set, irrespective of the test error.

Original: (See Appendix C.3).
Revised: (See Appendix C.3).

Original: Class Partition: We now consider settings where the original class labels form a distinguishable feature (eg: CIFAR-10 classes are distinguishable by ResNets).
Revised: Class Partition: We now consider settings where the original class labels form a distinguishable feature (eg: CIFAR-10 classes are distinguishable by ResNets).

Original: The conjecture holds for arbitrary distributions p ( y | x ) .
Revised: The conjecture holds for arbitrary distributions p ( y | x ) .

Original: This includes the setting of Experiments 1 from the Introduction.
Revised: This includes the setting of Experiments 1 from the Introduction.

Original: We mislabel class 

Revised: We mislabel class

Original: → 1 with probability p in the CIFAR-10 train set and train a WideResNet-28-10 (WRN-28-10) on this distribution.
Revised: → 1 with probability p in the CIFAR-10 train set and train a WideResNet-28-10 (WRN-28-10) on this distribution.

Original: Now, we observe the fraction of samples mislabeled by this network (cid:98) p from 0 → 1 in the test set.
Revised: Now, we observe the fraction of samples mislabeled by this network (cid:98) p from 0 → 1 in the test set.

Original: Figure 2 shows p versus (cid:98) p .
Revised: Figure 2 shows p versus (cid:98) p .

Original: The Bayes optimal classiﬁer for this distribution behaves as a step function (in red), and a classiﬁer that obeys Conjecture 1 exactly would follow the diagonal (in green).
Revised: The Bayes optimal classiﬁer for this distribution behaves as a step function (in red), and a classiﬁer that obeys Conjecture 1 exactly would follow the diagonal (in green).

Original: The actual experiment (in blue) is close to the behavior predicted by Conjecture 1.
Revised: The actual experiment (in blue) is close to the behavior predicted by Conjecture 1.

Original: Appendix C.4 includes experiments for more distributions (including random joint density matrices) and other classiﬁers such as Decisions Trees.
Revised: Appendix C.4 includes experiments for more distributions (including random joint density matrices) and other classiﬁers such as Decisions Trees.

Original: Multiple features: Conjecture 1 states that the network should be automatically calibrated for all distinguishable features, without any explicit labels for them.
Revised: Multiple features: Conjecture 1 states that the network should be automatically calibrated for all distinguishable features, without any explicit labels for them.

Original: To verify this, we use the CelebA dataset (Liu et al., 2015), containing images with many binary attributes per image.
Revised: To verify this, we use the CelebA dataset (Liu et al., 2015), containing images with many binary attributes per image.

Original: We train a ResNet50 to classify one of the hard attributes (accuracy 80%) and conﬁrm that the output distribution is calibrated with respect to all attributes (Figure 2 that are themselves distinguishable by a ResNet-50.
Revised: We train a ResNet50 to classify one of the hard attributes (accuracy 80%) and conﬁrm that the output distribution is calibrated with respect to all attributes (Figure 2 that are themselves distinguishable by a ResNet-50.

Original: Coarse Partition: Consider AlexNet trained on ILSVRC-2012 ImageNet (Russakovsky et al., 2015), a 1000-class image classiﬁcation problem with 116 varieties of dogs.
Revised: Coarse Partition: Consider AlexNet trained on ILSVRC-2012 ImageNet (Russakovsky et al., 2015), a 1000-class image classiﬁcation problem with 116 varieties of dogs.

Original: The network achieves only 56.5% accuracy on the test set.
Revised: The network achieves only 56.5% accuracy on the test set.

Original: But it will at least classify most dogs as dogs (with 98.4% accuracy), making L ( x ) ∈ { dog, not-dog } a distinguishable feature.
Revised: But it will at least classify most dogs as dogs (with 98.4% accuracy), making L ( x ) ∈ { dog, not-dog } a distinguishable feature.

Original: Per Conjecture 1, the network is calibrated with respect to dogs: 22.4% of all dogs in ImageNet are Terriers, and the network classiﬁes 20.9% of all dogs as Terriers (though it has 9% error on which speciﬁc dogs it classiﬁes as Terriers).
Revised: Per Conjecture 1, the network is calibrated with respect to dogs: 22.4% of all dogs in ImageNet are Terriers, and the network classiﬁes 20.9% of all dogs as Terriers (though it has 9% error on which speciﬁc dogs it classiﬁes as Terriers).

Original: See
Revised: See

Original: Appendix Table 2 for details, and related experiments on ResNets and kernels in Appendix C.
Revised: Appendix Table 2 for details, and related experiments on ResNets and kernels in Appendix C.

Original: 3. D ISCUSSION
Revised: 3. D ISCUSSION

Original: Conjecture 1 claims that D te is close to D up to all tests which are themselves learnable .
Revised: Conjecture 1 claims that D te is close to D up to all tests which are themselves learnable .

Original: That is, if an interpolating method is capable of learning a certain partition of the domain, then it will also produce outputs that are calibrated with respect to this partition, when trained on any problem.
Revised: That is, if an interpolating method is capable of learning a certain partition of the domain, then it will also produce outputs that are calibrated with respect to this partition, when trained on any problem.

Original: This conjecture thus gives a way of quantifying the resolution with which classiﬁers approximate the source distribution D , via properties of the classiﬁcation algorithm itself.
Revised: This conjecture thus gives a way of quantifying the resolution with which classiﬁers approximate the source distribution D , via properties of the classiﬁcation algorithm itself.

Original: This is in contrast to many classical ways of quantifying the approximation of density estimators, which rely on analytic (rather than operational ) distributional assumptions (Tsybakov, 2008; Wasserman, 2006).
Revised: This is in contrast to many classical ways of quantifying the approximation of density estimators, which rely on analytic (rather than operational ) distributional assumptions (Tsybakov, 2008; Wasserman, 2006).

Original: Proper Scoring Rules.
Revised: Proper Scoring Rules.

Original: If the loss function used in training is a strictly-proper scoring rule such as cross-entropy (Gneiting and Raftery, 2007), then we may expect that in the limit of a largecapacity network and inﬁnite data, training on samples { ( x i , y i ) } will yield a good density estimate of p ( y | x ) at the softmax layer.
Revised: If the loss function used in training is a strictly-proper scoring rule such as cross-entropy (Gneiting and Raftery, 2007), then we may expect that in the limit of a largecapacity network and inﬁnite data, training on samples { ( x i , y i ) } will yield a good density estimate of p ( y | x ) at the softmax layer.

Original: However, this is not what is happening in our experiments: First, our experiments consider the hard-decisions, not the softmax outputs.
Revised: However, this is not what is happening in our experiments: First, our experiments consider the hard-decisions, not the softmax outputs.

Original: Second, we observe Conjectureeven in settings without proper scoring rules (e.g. kernel SVM and decision trees).
Revised: Second, we observe Conjectureeven in settings without proper scoring rules (e.g. kernel SVM and decision trees).

Original: 1-Nearest-Neighbors Connection.
Revised: 1-Nearest-Neighbors Connection.

Original: Here we show that the 1-Nearest-Neighbor classiﬁer provably satisﬁes Conjecture 1, under mild assumptions.
Revised: Here we show that the 1-Nearest-Neighbor classiﬁer provably satisﬁes Conjecture 1, under mild assumptions.

Original: This theorem applies generically to a wide class of distributions, with no assumptions on the ambient dimension of inputs or the underlying metric.
Revised: This theorem applies generically to a wide class of distributions, with no assumptions on the ambient dimension of inputs or the underlying metric.

Original: The only assumption is a weak regularity condition: sampling the nearest-neighbor train point to a random test point should yield (close to) a uniformly random test point.
Revised: The only assumption is a weak regularity condition: sampling the nearest-neighbor train point to a random test point should yield (close to) a uniformly random test point.

Original: Theorem 1.
Revised: Theorem 1.

Original: Let D be a distribution over X × Y , and let n ∈ N be the number of train samples.
Revised: Let D be a distribution over X × Y , and let n ∈ N be the number of train samples.

Original: Assume the following regularity condition holds:
Revised: Assume the following regularity condition holds:

Original: Sampling the nearest-neighbor train point to a random test point yields (close to) a uniformly random test point.
Revised: Sampling the nearest-neighbor train point to a random test point yields (close to) a uniformly random test point.

Original: That is, suppose that for some small δ ≥ 0 , the distributions: { NN S ( x ) } S ∼D n x ∼D ≈ δ { x } x ∼D .
Revised: That is, suppose that for some small δ ≥ 0 , the distributions: { NN S ( x ) } S ∼D n x ∼D ≈ δ { x } x ∼D .

Original: Then, Conjecture 1 holds.
Revised: Then, Conjecture 1 holds.

Original: That is, for all ( ε, NN , D , n ) -distinguishable partitions L , the following distributions are statistically close:
Revised: That is, for all ( ε, NN , D , n ) -distinguishable partitions L , the following distributions are statistically close:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The proof of Theorem 1 is straightforward, and provided in
Revised: The proof of Theorem 1 is straightforward, and provided in

Original: Appendix E. We view this theorem both as support for our formalism of Conjecture 1, and as evidence that the classiﬁers we consider in this work have local properties similar to 1-Nearest-Neighbors.
Revised: Appendix E. We view this theorem both as support for our formalism of Conjecture 1, and as evidence that the classiﬁers we consider in this work have local properties similar to 1-Nearest-Neighbors.

Original:  A GREEMENT P ROPERTY
Revised:  A GREEMENT P ROPERTY

Original: We now present an “agreement property” of interpolating classiﬁers.
Revised: We now present an “agreement property” of interpolating classiﬁers.

Original: This property is independent of the previous section, though both are special cases of our general indistinguishability conjecture.
Revised: This property is independent of the previous section, though both are special cases of our general indistinguishability conjecture.

Original: We claim that, informally, the test accuracy of a classiﬁer is close to the probability that it agrees with an identically-trained classiﬁer on a disjoint train set.
Revised: We claim that, informally, the test accuracy of a classiﬁer is close to the probability that it agrees with an identically-trained classiﬁer on a disjoint train set.

Original: Conjecture 2 (Agreement Property) .
Revised: Conjecture 2 (Agreement Property) .

Original: For certain classiﬁer families F and distributions D , the test accuracy of a classiﬁer is close to its agreement probability with an independently-trained classiﬁer.
Revised: For certain classiﬁer families F and distributions D , the test accuracy of a classiﬁer is close to its agreement probability with an independently-trained classiﬁer.

Original: That is, let S 1 , S 2 be disjoint train sets sampled from D n , and let f 1 , f 2 be classiﬁers trained on
Revised: That is, let S 1 , S 2 be disjoint train sets sampled from D n , and let f 1 , f 2 be classiﬁers trained on

Original: S 1 , S 2 respectively, then
Revised: S 1 , S 2 respectively, then

Original: [Equation]
Revised: [Equation]

Original: Moreover, this holds with high probability over training f 1 , f 2 : Pr ( x,y )
Revised: Moreover, this holds with high probability over training f 1 , f 2 : Pr ( x,y )

Original: ∼D [ f 1 ( x )
Revised: ∼D [ f 1 ( x )

Original: =
Revised: =

Original: y ]
Revised: y ]

Original: ≈
Revised: ≈

Original: Pr ( x,y ) ∼D [ f 1 ( x ) =
Revised: Pr ( x,y ) ∼D [ f 1 ( x ) =

Original: f 2 ( x )] .
Revised: f 2 ( x )] .

Original: The agreement property may be surprising for several reasons.
Revised: The agreement property may be surprising for several reasons.

Original: For example, suppose we have two classiﬁers f 1 , f 2 which were trained on independent train sets, and both achieve test accuracy say 50% on a 10-class problem.
Revised: For example, suppose we have two classiﬁers f 1 , f 2 which were trained on independent train sets, and both achieve test accuracy say 50% on a 10-class problem.

Original: Depending on our intuition, we may expect (1) They agree with each other much less than they agree with the true label, since each individual classiﬁer is an independently noisy version of the truth.
Revised: Depending on our intuition, we may expect (1) They agree with each other much less than they agree with the true label, since each individual classiﬁer is an independently noisy version of the truth.

Original: OR (2) They agree with each other much more than 50%, since classiﬁers tend to have “correlated” predictions.
Revised: OR (2) They agree with each other much more than 50%, since classiﬁers tend to have “correlated” predictions.

Original: However, in practice there is a surprising coincidence, and they agree with each other very close to 50%.
Revised: However, in practice there is a surprising coincidence, and they agree with each other very close to 50%.

Original: Conjecture 2 also provably holds for 1-Nearest-Neighbors in some settings, under stronger assumptions (Theorem 2 in Appendix E).
Revised: Conjecture 2 also provably holds for 1-Nearest-Neighbors in some settings, under stronger assumptions (Theorem 2 in Appendix E).

Revised: Finally, in Section D.3, we consider, and refute, several potential mechanisms which could explain the experimental results of

Revised: Conjecture 2.

Original: Connection to Indistinguishability.
Revised: Connection to Indistinguishability.

Original: Conjecture 2 is in fact a special case of our general Indistinguishability Conjecture.
Revised: Conjecture 2 is in fact a special case of our general Indistinguishability Conjecture.

Original: Formally, consider the speciﬁc test T agree : ( x, (cid:98) y )
Revised: Formally, consider the speciﬁc test T agree : ( x, (cid:98) y )

Original: (cid:55)→ 1 { f 1 ( x ) = (cid:98) y } where f 1 ← Train F ( D n ) .
Revised: (cid:55)→ 1 { f 1 ( x ) = (cid:98) y } where f 1 ← Train F ( D n ) .

Original: The expectation of this test under the Source Distribution D is exactly the
Revised: The expectation of this test under the Source Distribution D is exactly the

Original: LHS of Equation 6, while the expectation under the Test Distribution D te is exactly the RHS.
Revised: LHS of Equation 6, while the expectation under the Test Distribution D te is exactly the RHS.

Original: Thus,
Revised: Thus,

Original: Conjecture 2 can be equivalently stated as D ≈ T agree D te .
Revised: Conjecture 2 can be equivalently stated as D ≈ T agree D te .

Original: 4. E XPERIMENTS
Revised: 4. E XPERIMENTS

Original: In our experiments, we train a pair of classiﬁers f 1 , f 2 on random disjoint subsets of the train set for a given distribution.
Revised: In our experiments, we train a pair of classiﬁers f 1 , f 2 on random disjoint subsets of the train set for a given distribution.

Original: Both classiﬁers are otherwise trained identically, using the same architecture, number of trainsamples n , and optimizer.
Revised: Both classiﬁers are otherwise trained identically, using the same architecture, number of trainsamples n , and optimizer.

Original: We then plot the test error of f 1 against the agreement probability Pr x
Revised: We then plot the test error of f 1 against the agreement probability Pr x

Original: [ f 1 ( x ) = f 2 ( x )] .
Revised: [ f 1 ( x ) = f 2 ( x )] .

Original: Figure 3 shows experiments with ResNet18 on CIFAR-10 and
Revised: Figure 3 shows experiments with ResNet18 on CIFAR-10 and

Original: CIFAR-100, with varying train samples n .
Revised: CIFAR-100, with varying train samples n .

Original: The Agreement Property approximately holds for all pairs of identical classiﬁers, and continues to hold even for “weak” classiﬁers (
Revised: The Agreement Property approximately holds for all pairs of identical classiﬁers, and continues to hold even for “weak” classiﬁers (

Original: e.g. when f 1 , f 2 have high test error).
Revised: e.g. when f 1 , f 2 have high test error).

Original: Full experimental details are in Appendix D, including further experiments with RBF and
Revised: Full experimental details are in Appendix D, including further experiments with RBF and

Original: Laplace kernels, the Myrtle10 kernel (Shankar et al., 2020), and decision trees on UCI tasks.
Revised: Laplace kernels, the Myrtle10 kernel (Shankar et al., 2020), and decision trees on UCI tasks.

Original:  D ISTRIBUTIONAL G ENERALIZATION : B EYOND I NTERPOLATING M ETHODS
Revised:  D ISTRIBUTIONAL G ENERALIZATION : B EYOND I NTERPOLATING M ETHODS

Original: The previous sections have focused on interpolating classiﬁers, which ﬁt their train sets exactly.
Revised: The previous sections have focused on interpolating classiﬁers, which ﬁt their train sets exactly.

Original: For non-interpolating classiﬁers, their outputs on the train set ( x, f ( x )) x ∼ TrainSet will not match the original distribution ( x, y ) ∼ D .
Revised: For non-interpolating classiﬁers, their outputs on the train set ( x, f ( x )) x ∼ TrainSet will not match the original distribution ( x, y ) ∼ D .

Original: Thus, there is little hope that their outputs on the test set will match the original distribution, and we do not expect the Indistinguishability Conjecture to hold.
Revised: Thus, there is little hope that their outputs on the test set will match the original distribution, and we do not expect the Indistinguishability Conjecture to hold.

Original: However, Distributional Generalization does not require interpolation, and we could still expect that the the train and test distributions are close ( D tr ≈ T D te ) for some family of tests T .
Revised: However, Distributional Generalization does not require interpolation, and we could still expect that the the train and test distributions are close ( D tr ≈ T D te ) for some family of tests T .

Original: For example, the following is a possible generalization of Feature Calibration.
Revised: For example, the following is a possible generalization of Feature Calibration.

Original: Conjecture 3 (Generalized Feature Calibration, informal) .
Revised: Conjecture 3 (Generalized Feature Calibration, informal) .

Original: For trained classiﬁers f , the following distributions are statistically close for many partitions L of the domain:
Revised: For trained classiﬁers f , the following distributions are statistically close for many partitions L of the domain:

Original: [Equation]
Revised: [Equation]

Original: We leave unspeciﬁed the exact set of partitions L for which this holds, since we do not yet understand the appropriate notion of “distinguishable feature” in this setting.
Revised: We leave unspeciﬁed the exact set of partitions L for which this holds, since we do not yet understand the appropriate notion of “distinguishable feature” in this setting.

Original: However, we give experimental evidence suggesting some reﬁnement of Conjecture 3 is true.
Revised: However, we give experimental evidence suggesting some reﬁnement of Conjecture 3 is true.

Original: In
Revised: In

Original: Figure 4 we train Gaussian kernel regression on MNIST, with label noise determined by a random sparse confusion matrix.
Revised: Figure 4 we train Gaussian kernel regression on MNIST, with label noise determined by a random sparse confusion matrix.

Original: We vary the (cid:96) 2 regularization, and plot the confusion matrix of predictions on the train and test sets.
Revised: We vary the (cid:96) 2 regularization, and plot the confusion matrix of predictions on the train and test sets.

Original: With higher regularization, the kernel no longer interpolates the train set, but the test and train confusion matrices remain close.
Revised: With higher regularization, the kernel no longer interpolates the train set, but the test and train confusion matrices remain close.

Original: That is, regularization prevents the kernel from ﬁtting the noise on both the train and test sets in a similar way.
Revised: That is, regularization prevents the kernel from ﬁtting the noise on both the train and test sets in a similar way.

Original: Full experimental details are given in Appendix B, including an analogous experiment for neural networks on CIFAR-10, with early-stopping in place of regularization (Figure 19).
Revised: Full experimental details are given in Appendix B, including an analogous experiment for neural networks on CIFAR-10, with early-stopping in place of regularization (Figure 22).

Original: These experiments suggests that Distributional Generalization is a meaningful notion even for non-interpolating classiﬁers.
Revised: These experiments suggests that Distributional Generalization is a meaningful notion even for non-interpolating classiﬁers.

Original:  C ONCLUSION AND D ISCUSSION
Revised:  C ONCLUSION AND D ISCUSSION

Original: In this work, we presented a new set of empirical behaviors of standard interpolating classiﬁers.
Revised: In this work, we presented a new set of empirical behaviors of standard interpolating classiﬁers.

Original: We uniﬁed these under the framework of Distributional Generalization, which states that outputs of trained classiﬁers on the test set are “close” in distribution to their outputs on the train set.
Revised: We uniﬁed these under the framework of Distributional Generalization, which states that outputs of trained classiﬁers on the test set are “close” in distribution to their outputs on the train set.

Original: For interpolating classiﬁers, we stated several formal conjectures (Conjectures 1 and 2) to characterize the form of distributional closeness that can be expected.
Revised: For interpolating classiﬁers, we stated several formal conjectures (Conjectures 1 and 2) to characterize the form of distributional closeness that can be expected.

Original: Beyond Test Error.
Revised: Beyond Test Error.

Original: Our work proposes studying the entire distribution of classiﬁer outputs on test samples, beyond just its test error.
Revised: Our work proposes studying the entire distribution of classiﬁer outputs on test samples, beyond just its test error.

Original: We show that this distribution is often highly structured, and we take steps towards characterizing it.
Revised: We show that this distribution is often highly structured, and we take steps towards characterizing it.

Original: Surprisingly, modern interpolating classiﬁers appear to satisfy certain forms of distributional generalization “automatically,” despite being trained to simply minimize train error.
Revised: Surprisingly, modern interpolating classiﬁers appear to satisfy certain forms of distributional generalization “automatically,” despite being trained to simply minimize train error.

Original: This even holds in cases when satisfying distributional generalization is in conﬂict with satisfying classical generalization— that is, when a distributionally-generalizing classiﬁer must necessarily have high test error (e.g. Experiments 1 and 2).
Revised: This even holds in cases when satisfying distributional generalization is in conﬂict with satisfying classical generalization— that is, when a distributionally-generalizing classiﬁer must necessarily have high test error (e.g. Experiments 1 and 2).

Original: We thus hope that studying distributional generalization will be useful to better understand modern classiﬁers, and to understand generalization more broadly. 
Revised: We thus hope that studying distributional generalization will be useful to better understand modern classiﬁers, and to understand generalization more broadly. Classical Generalization.

Revised: Our framework of Distributional Generalization can be insightful even to study classical generalization.

Revised: That is, even if we ultimately want to understand test error, it may be easier to do so through distributional generalization.

Revised: This is especially relevant for understanding the success of interpolating methods, which pose challenges to classical theories of generalization.

Original: Limitations.
Revised: Our work shows new empirical behaviors of interpolating classiﬁers, as well as conjectures characterizing these behaviors. This sheds new light on these poorly understood methods, and could pave the way to better understanding their generalization. Interpolating vs. Non-interpolating Methods. Our work also suggests that interpolating classiﬁers should be viewed as conceptually different objects from non-interpolating ones, even if both have the same test error.

Revised: In particular, an interpolating classiﬁer will match certain aspects of the original distribution, which a non-interpolating classiﬁer will not.

Revised: This also suggests, informally, that interpolating methods should not be seen as methods which simply “memorize” their training data in a naive way (as in a look up table) – rather this “memorization” strongly inﬂuences the classiﬁer’s decision boundary (as in 1-Nearest-Neighbors).

Revised: Limitations.

Original: The conjectures presented in this work are not fully speciﬁed, since they do not exactly specify which classiﬁers or distributions for which they hold.
Revised: The conjectures presented in this work are not fully speciﬁed, since they do not exactly specify which classiﬁers or distributions for which they hold.

Original: We experimentally demonstrate instances of these conjectures in various “natural” settings in machine learning, but we do not yet understand which assumptions on the distribution or classiﬁer are required.
Revised: We experimentally demonstrate instances of these conjectures in various “natural” settings in machine learning, but we do not yet understand which assumptions on the distribution or classiﬁer are required.

Original: Some experiments also deviate slightly from the predicted behavior.
Revised: Some experiments also deviate slightly from the predicted behavior.

Original: Nevertheless, we believe our conjectures capture the essential aspects of the observed behaviors, at least to ﬁrst order.
Revised: Nevertheless, we believe our conjectures capture the essential aspects of the observed behaviors, at least to ﬁrst order.

Original: It is an important open question to reﬁne these conjectures and better understand their applications and limitations— both theoretically and experimentally. 
Revised: It is an important open question to reﬁne these conjectures and better understand their applications and limitations— both theoretically and experimentally. 6. O PEN Q UESTIONS

Revised: Our work raises a number of open questions and connections to other areas.

Original: A F ULL R ELATED W ORK
Revised: We brieﬂy collect some of them here. As described in the limitations, we do not precisely understand the set of distributions and interpolating classiﬁers for which our conjectures hold. We empirically tested a number of “realistic” settings, but it is open to state formal assumptions deﬁning these settings. It is open to theoretically prove versions of Distributional Generalization for models beyond 1-Nearest-Neighbors.

Revised: This is most interesting in cases where Distributional Generalization is at odds with classical generalization (e.g. Figure 2A).

Revised: It is open to understand the mechanisms behind the Agreement Property (Section 4), theoretically or empirically.

Revised: There are a number of works suggesting “local” behavior of neural networks, and these are somewhat consistent with our locality intuitions in this work.

Revised: However, it is open to formally understand whether these intuitions are justiﬁed in our setting.

Revised: We give two families of tests T for which our Interpolating Indistinguishability Metaconjecture empirically holds.

Revised: This may not be exhaustive – there may be other ways in which the source distribution D and test distribution D te are close.

Revised: It is open to explore more ways in which Distributional Generalization holds, beyond the tests presented here.

Original: Our work is inspired by the broader study of interpolating and overparameterized methods in machine learning; a partial list of works in this theme includes Zhang et al.
Revised: Our work is inspired by the broader study of interpolating and overparameterized methods in machine learning; a partial list of works in this theme includes Zhang et al.

Original: ; Belkin et al.
Revised: ; Belkin et al.

Original: (2018a;b; 2019); Liang and Rakhlin (2018); Nakkiran et al.
Revised: (2018a;b; 2019); Liang and Rakhlin (2018); Nakkiran et al.

Original: ; Mei and Montanari (2019); Schapire et al.
Revised: ; Mei and Montanari (2019); Schapire et al.

Original: (1998); Breiman (1995); Ghorbani et al.
Revised: (1998); Breiman (1995); Ghorbani et al.

Original: ; Hastie et al.
Revised: ; Hastie et al.

Original: ; Bartlett et al.
Revised: ; Bartlett et al.

Original: Advani and Saxe (2017); Geiger et al.
Revised: Advani and Saxe (2017); Geiger et al.

Original: ; Gerace et al.
Revised: ; Gerace et al.

Original: Chizat and Bach (2020); Goldt et al.
Revised: Chizat and Bach (2020); Goldt et al.

Original: (2019); Arora et al.
Revised: (2019); Arora et al.

Original: Allen-Zhu et al.
Revised: Allen-Zhu et al.

Original: ; Neyshabur et al.
Revised: ; Neyshabur et al.

Original: ; Dziugaite and Roy (2017); Muthukumar et al.
Revised: ; Dziugaite and Roy (2017); Muthukumar et al.

Original: ; Neal et al.
Revised: ; Neal et al.

Original: Interpolating Methods.
Revised: Interpolating Methods.

Original: Many of the best-performing techniques on high-dimensional tasks are interpolating methods, which ﬁt their train samples to 0 train error.
Revised: Many of the best-performing techniques on high-dimensional tasks are interpolating methods, which ﬁt their train samples to 0 train error.

Original: This includes neural-networks and kernels on images (He et al., 2016; Shankar et al., 2020), and random forests on tabular data (Fern´andez-Delgado et al., 2014).
Revised: This includes neural-networks and kernels on images (He et al., 2016; Shankar et al., 2020), and random forests on tabular data (Fern´andez-Delgado et al., 2014).

Original: Interpolating methods have been extensively studied both recently and in the past, since we do not theoretically understand their practical success (Schapire et al., 1998;
Revised: Interpolating methods have been extensively studied both recently and in the past, since we do not theoretically understand their practical success (Schapire et al., 1998;

Original: Schapire, 1999; Breiman, 1995; Zhang et al., 2016; Belkin et al., 2018a;b; 2019; Liang and Rakhlin, 2018; Mei and Montanari, 2019; Hastie et al., 2019; Nakkiran et al., 2020).
Revised: Schapire, 1999; Breiman, 1995; Zhang et al., 2016; Belkin et al., 2018a;b; 2019; Liang and Rakhlin, 2018; Mei and Montanari, 2019; Hastie et al., 2019; Nakkiran et al., 2020).

Original: In particular, much of the classical work in statistical learning theory (uniform convergence, VC-dimension, Rademacher complexity, regularization, stability) fails to explain the success of interpolating methods (Zhang et al., 2016; Belkin et al., 2018a;b; Nagarajan and Kolter, 2019).
Revised: In particular, much of the classical work in statistical learning theory (uniform convergence, VC-dimension, Rademacher complexity, regularization, stability) fails to explain the success of interpolating methods (Zhang et al., 2016; Belkin et al., 2018a;b; Nagarajan and Kolter, 2019).

Original: The few techniques which do apply to interpolating methods (e.g. margin theory (Schapire et al., 1998)) remain vacuous on modern neural-networks and kernels.
Revised: The few techniques which do apply to interpolating methods (e.g. margin theory (Schapire et al., 1998)) remain vacuous on modern neural-networks and kernels.

Original: Decision Trees.
Revised: Decision Trees.

Original: In a similar vein to our work, Wyner et al.
Revised: In a similar vein to our work, Wyner et al.

Original: Olson and Wyner (2018) investigate decision trees, and show that random forests are equivalent to a Nadaraya–Watson smoother Nadaraya (1964);
Revised: Olson and Wyner (2018) investigate decision trees, and show that random forests are equivalent to a Nadaraya–Watson smoother Nadaraya (1964);

Original: Watson (1964) with a certain smoothing kernel.
Revised: Watson (1964) with a certain smoothing kernel.

Original: Decision trees (Breiman et al., 1984) are often intuitively thought of as “adaptive nearest-neighbors,” since they are explicitly a spatial-partitioning method (Hastie et al., 2009).
Revised: Decision trees (Breiman et al., 1984) are often intuitively thought of as “adaptive nearest-neighbors,” since they are explicitly a spatial-partitioning method (Hastie et al., 2009).

Original: Thus, it may not be surprising that decision trees behave similarly to 1-Nearest-Neighbors.
Revised: Thus, it may not be surprising that decision trees behave similarly to 1-Nearest-Neighbors.

Original: Wyner et al.
Revised: Wyner et al.

Original: Olson and Wyner (2018) took steps towards characterizing and understanding this behavior – in particular, Olson and Wyner (2018) deﬁnes an equivalent smoothing kernel corresponding to a random forest, and empirically investigates the quality of the conditional density estimate.
Revised: Olson and Wyner (2018) took steps towards characterizing and understanding this behavior – in particular, Olson and Wyner (2018) deﬁnes an equivalent smoothing kernel corresponding to a random forest, and empirically investigates the quality of the conditional density estimate.

Original: Our work presents a formal characterization of the quality of this conditional density estimate (Conjecture 1), which is a novel characterization even for decision trees, as far as we know.
Revised: Our work presents a formal characterization of the quality of this conditional density estimate (Conjecture 1), which is a novel characterization even for decision trees, as far as we know.

Original: Kernel Smoothing.
Revised: Kernel Smoothing.

Original: The term kernel regression is sometimes used in the literature to refer to kernel smoothers , such as the Nadaraya–Watson kernel smoother (Nadaraya, 1964; Watson, 1964).
Revised: The term kernel regression is sometimes used in the literature to refer to kernel smoothers , such as the Nadaraya–Watson kernel smoother (Nadaraya, 1964; Watson, 1964).

Original: But in this work we use the term “kernel regression” to refer only to regression in a Reproducing Kernel
Revised: But in this work we use the term “kernel regression” to refer only to regression in a Reproducing Kernel

Original: Hilbert Space, as described in the experimental details.
Revised: Hilbert Space, as described in the experimental details.

Original: Label Noise.
Revised: Label Noise.

Original: Our conjectures also describe the behavior of neural networks under label noise, which has been empirically and theoretically studied in the past, though not formally characterized before (Zhang et al., 2016;
Revised: Our conjectures also describe the behavior of neural networks under label noise, which has been empirically and theoretically studied in the past, though not formally characterized before (Zhang et al., 2016;

Original: Belkin et al., 2018b; Rolnick et al., 2017; Natarajan et al., 2013; Thulasidasan et al., 2019; Ziyin et al., 2020; Chatterji and Long, 2020).
Revised: Belkin et al., 2018b; Rolnick et al., 2017; Natarajan et al., 2013; Thulasidasan et al., 2019; Ziyin et al., 2020; Chatterji and Long, 2020).

Original: Prior works have noticed that vanilla interpolating networks are sensitive to label noise (e.g. Figure 1 in Zhang et al.
Revised: Prior works have noticed that vanilla interpolating networks are sensitive to label noise (e.g. Figure 1 in Zhang et al.

Original: (2016), and Belkin et al.
Revised: (2016), and Belkin et al.

Original: (2018b))
Revised: (2018b))

Original: , and there are many works on making networks more robust to label noise via modiﬁcations to the training procedure or objective (Rolnick et al., 2017; Natarajan et al., 2013;
Revised: , and there are many works on making networks more robust to label noise via modiﬁcations to the training procedure or objective (Rolnick et al., 2017; Natarajan et al., 2013;

Original: Thulasidasan et al., 2019; Ziyin et al., 2020).
Revised: Thulasidasan et al., 2019; Ziyin et al., 2020).

Original: In contrast, we claim this sensitivity to label noise is not necessarily a problem to be ﬁxed, but rather a consequence of a stronger property: distributional generalization.
Revised: In contrast, we claim this sensitivity to label noise is not necessarily a problem to be ﬁxed, but rather a consequence of a stronger property: distributional generalization.

Original: Conditional Density Estimation.
Revised: Conditional Density Estimation.

Original: Our density calibration property is similar to the guarantees of a conditional density estimator.
Revised: Our density calibration property is similar to the guarantees of a conditional density estimator.

Original: More speciﬁcally, Conjecture 1 states that an interpolating classiﬁer samples from a distribution approximating the conditional density of p ( y | x ) in a certain sense.
Revised: More speciﬁcally, Conjecture 1 states that an interpolating classiﬁer samples from a distribution approximating the conditional density of p ( y | x ) in a certain sense.

Original: Conditional density estimation has been well-studied in classical nonparametric statistics (e.g. the
Revised: Conditional density estimation has been well-studied in classical nonparametric statistics (e.g. the

Original: Nadaraya–Watson kernel smoother (Nadaraya, 1964; Watson, 1964)).
Revised: Nadaraya–Watson kernel smoother (Nadaraya, 1964; Watson, 1964)).

Original: However, these classical methods behave poorly in high-dimensions, both in theory and in practice.
Revised: However, these classical methods behave poorly in high-dimensions, both in theory and in practice.

Original: There are some attempts to extend these classical methods to modern high-dimentional problems via augmenting estimators with neural networks (e.g. Rothfuss et al.
Revised: There are some attempts to extend these classical methods to modern high-dimentional problems via augmenting estimators with neural networks (e.g. Rothfuss et al.

Original: Random forests have also been known to exhibit properties similar to conditional density estimators.
Revised: Random forests have also been known to exhibit properties similar to conditional density estimators.

Original: This has been formalized in various ways, often only with asymptotic guarantees (Meinshausen, 2006; Pospisil and Lee, 2018; Athey et al., 2019).
Revised: This has been formalized in various ways, often only with asymptotic guarantees (Meinshausen, 2006; Pospisil and Lee, 2018; Athey et al., 2019).

Original: No prior work that we are aware of attempts to characterize the quality of the resulting density estimate via testable assumptions, as we do with our formulation of Conjecture 1.
Revised: No prior work that we are aware of attempts to characterize the quality of the resulting density estimate via testable assumptions, as we do with our formulation of Conjecture 1.

Original: Finally, our motivation is not to design good conditional density estimators, but rather to study properties of interpolating classiﬁers — which we ﬁnd happen to share properties of density estimators.
Revised: Finally, our motivation is not to design good conditional density estimators, but rather to study properties of interpolating classiﬁers — which we ﬁnd happen to share properties of density estimators.

Original: Uncertainty and Calibration.
Revised: Uncertainty and Calibration.

Original: The Agreement Property (Conjecture 2) bears some resemblance to uncertainty estimation (e.g. Lakshminarayanan et al.
Revised: The Agreement Property (Conjecture 2) bears some resemblance to uncertainty estimation (e.g. Lakshminarayanan et al.

Original: (2017)), since it estimates the the test error of a classiﬁer using an ensemble of 2 models trained on disjoint train sets.
Revised: (2017)), since it estimates the the test error of a classiﬁer using an ensemble of 2 models trained on disjoint train sets.

Original: However, there are important caveats: (1) Our Agreement Property only holds on-distribution, and degrades on off-distribution inputs.
Revised: However, there are important caveats: (1) Our Agreement Property only holds on-distribution, and degrades on off-distribution inputs.

Original: Thus, it is not as helpful to estimate out-of-distribution errors.
Revised: Thus, it is not as helpful to estimate out-of-distribution errors.

Original: (2) It only gives an estimate of the average test error, and does not imply pointwise calibration estimates for each sample.
Revised: (2) It only gives an estimate of the average test error, and does not imply pointwise calibration estimates for each sample.

Original: Feature Calibration (Conjecture 1) is also related to the concepts of calibration and multicalibration (Guo et al., 2017; Niculescu-Mizil and Caruana, 2005; H´ebert-Johnson et al., 2018).
Revised: Feature Calibration (Conjecture 1) is also related to the concepts of calibration and multicalibration (Guo et al., 2017; Niculescu-Mizil and Caruana, 2005; H´ebert-Johnson et al., 2018).

Original: In our framework, calibration is implied by Feature Calibration for a speciﬁc set of partitions L (determined by level sets of the classiﬁer’s conﬁdence).
Revised: In our framework, calibration is implied by Feature Calibration for a speciﬁc set of partitions L (determined by level sets of the classiﬁer’s conﬁdence).

Original: However, we are not concerned with a speciﬁc set of partitions (or “subgroups” in the algorithmic fairness literature) but we generally aim to characterize for which partitions Feature Calibration holds.
Revised: However, we are not concerned with a speciﬁc set of partitions (or “subgroups” in the algorithmic fairness literature) but we generally aim to characterize for which partitions Feature Calibration holds.

Original: Moreover, we consider only hard-classiﬁcation decisions and not conﬁdences, and we study only standard learning algorithms which are not given any distinguished set of subgroups/partitions in advance.
Revised: Moreover, we consider only hard-classiﬁcation decisions and not conﬁdences, and we study only standard learning algorithms which are not given any distinguished set of subgroups/partitions in advance.

Original: Our notion of distributional generalization is also related to the notion of “distributional subgroup overﬁtting” introduced recently by Yaghini et al.
Revised: Our notion of distributional generalization is also related to the notion of “distributional subgroup overﬁtting” introduced recently by Yaghini et al.

Original: (2019) to study algorithmic fairness.
Revised: (2019) to study algorithmic fairness.

Original: This can be seen as studying distributional generalization for a speciﬁc family of tests (determined by distinguished subgroups in the population).
Revised: This can be seen as studying distributional generalization for a speciﬁc family of tests (determined by distinguished subgroups in the population).

Original: Locality and Manifold Learning.
Revised: Locality and Manifold Learning.

Original: Our intuition for the behaviors in this work is that they arise due to some form of “locality” of the trained classiﬁers, in an appropriate space.
Revised: Our intuition for the behaviors in this work is that they arise due to some form of “locality” of the trained classiﬁers, in an appropriate space.

Original: This intuition is present in various forms in the literature, for example: the so-called called “manifold hypothesis,” that natural data lie on a low-dimensional manifold (e.g. Narayanan and Mitter (2010); Sharma and
Revised: This intuition is present in various forms in the literature, for example: the so-called called “manifold hypothesis,” that natural data lie on a low-dimensional manifold (e.g. Narayanan and Mitter (2010); Sharma and

Original: Kaplan (2020)), as well as works on local stiffness of the loss landscape (Fort et al., 2019), and works showing that overparameterized neural networks can learn hidden low-dimensional structure in high-dimensional settings (Gerace et al., 2020; Bach, 2017; Chizat and Bach, 2020).
Revised: Kaplan (2020)), as well as works on local stiffness of the loss landscape (Fort et al., 2019), and works showing that overparameterized neural networks can learn hidden low-dimensional structure in high-dimensional settings (Gerace et al., 2020; Bach, 2017; Chizat and Bach, 2020).

Original: It is open to more formally understand connections between our work and the above.
Revised: It is open to more formally understand connections between our work and the above.

Original: B E XPERIMENTAL D ETAILS
Revised: B E XPERIMENTAL D ETAILS

Original: Here we describe general background, and experimental details common to all sections.
Revised: Here we describe general background, and experimental details common to all sections.

Original: Then we provide section-speciﬁc details below.
Revised: Then we provide section-speciﬁc details below.

Original: B. D ATASETS
Revised: B. D ATASETS

Original: We consider the image datasets CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009), MNIST (LeCun et al., 1998), Fashion-MNIST (Xiao et al., 2017), CelebA (Liu et al., 2015), and ImageNet (Russakovsky et al., 2015).
Revised: We consider the image datasets CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009), MNIST (LeCun et al., 1998), Fashion-MNIST (Xiao et al., 2017), CelebA (Liu et al., 2015), and ImageNet (Russakovsky et al., 2015).

Original: We normalize images to x ∈ [0 , 1] C × W × H .
Revised: We normalize images to x ∈ [0 , 1] C × W × H .

Original: We also consider tabular datasets from the UCI repository Dua and Graff (2017).
Revised: We also consider tabular datasets from the UCI repository Dua and Graff (2017).

Original: For UCI data, we consider the 121 classiﬁcation tasks as standardized in Fern´andez-Delgado et al.
Revised: For UCI data, we consider the 121 classiﬁcation tasks as standardized in Fern´andez-Delgado et al.

Original: Some of these tasks have very few examples, so we restrict to the 92 classiﬁcation tasks from Fern´andez-Delgado et al.
Revised: Some of these tasks have very few examples, so we restrict to the 92 classiﬁcation tasks from Fern´andez-Delgado et al.

Original: (2014) which have at least 200 total examples.
Revised: (2014) which have at least 200 total examples.

Original: B. M ODELS
Revised: B. M ODELS

Original: We consider neural-networks, kernel methods, and decision trees.
Revised: We consider neural-networks, kernel methods, and decision trees.

Original: B.2. D ECISION T REES
Revised: B.2. D ECISION T REES

Original: We train interpolating decision trees using a growth rule from Random Forests (Breiman, 2001; Ho, 1995): selecting a split based on a random √ d subset of d features, splitting based on Gini impurity and growing trees until all leafs have a sam- bootstrap
Revised: We train interpolating decision trees using a growth rule from Random Forests (Breiman, 2001; Ho, 1995): selecting a split based on a random √ d subset of d features, splitting based on Gini impurity, and growing trees until all leafs have a single sample. This is as implemented by Scikit-learn Pedregosa et al. (2011) defaults with RandomForestClassifier(n_estimators=1, bootstrap=False) .

Original: B.2. K ERNELS
Revised: B.2. K ERNELS

Original: Throughout this work we consider classiﬁcation via kernel regression and kernel SVM.
Revised: Throughout this work we consider classiﬁcation via kernel regression and kernel SVM.

Original: For M -class classiﬁcation via kernel regression, we follow the methodology in e.g. Rahimi and Recht (2008);
Revised: For M -class classiﬁcation via kernel regression, we follow the methodology in e.g. Rahimi and Recht (2008);

Original: Belkin et al.
Revised: Belkin et al.

Original: (2018b)
Revised: (2018b)

Original: ; Shankar et al.
Revised: ; Shankar et al.

Original: We solve the following convex problem for training:
Revised: We solve the following convex problem for training:

Original: [Equation]
Revised: [Equation]

Original: where K ij = k
Revised: where K ij = k

Original: ( x i , x j ) is the kernel matrix of the training points for a kernel function k , y ∈ R N × M is the one-hot encoding of the train labels, and λ ≥ 0 is the regularization parameter.
Revised: ( x i , x j ) is the kernel matrix of the training points for a kernel function k , y ∈ R N × M is the one-hot encoding of the train labels, and λ ≥ 0 is the regularization parameter.

Original: The solution can be written
Revised: The solution can be written

Original: [Equation]
Revised: [Equation]

Original: which we solve numerically using SciPy linalg.solve (Virtanen et al., 2020).
Revised: which we solve numerically using SciPy linalg.solve (Virtanen et al., 2020).

Original: We use the explicit form of all kernels involved.
Revised: We use the explicit form of all kernels involved.

Original: That is, we do not use random-feature approximations (Rahimi and Recht, 2008), though we expect they would behave similarly.
Revised: That is, we do not use random-feature approximations (Rahimi and Recht, 2008), though we expect they would behave similarly.

Original: The kernel predictions on test points are then given by
Revised: The kernel predictions on test points are then given by

Original: [Equation]
Revised: [Equation]

Original: where g ( x ) ∈ R M are the kernel regressor outputs, and g ( x ) ∈
Revised: where g ( x ) ∈ R M are the kernel regressor outputs, and g ( x ) ∈

Original: [ M ] is the thresholded classiﬁcation decision.
Revised: [ M ] is the thresholded classiﬁcation decision.

Original: This is equivalent to training M separate binary regressors (one for each label), and taking the argmax for classiﬁcation.
Revised: This is equivalent to training M separate binary regressors (one for each label), and taking the argmax for classiﬁcation.

Original: We usually consider unregularized regression ( λ = 0 ), except in
Revised: We usually consider unregularized regression ( λ = 0 ), except in

Original: Section 5.
Revised: Section 5.

Original: For kernel SVM, we use the implementation provided by Scikit-learn (Pedregosa et al., 2011) sklearn.svm.
Revised: For kernel SVM, we use the implementation provided by Scikit-learn (Pedregosa et al., 2011) sklearn.svm.

Original: SVC with a precomputed kernel, for inverse-regularization parameter C ≥(larger C corresponds to smaller regularization).
Revised: SVC with a precomputed kernel, for inverse-regularization parameter C ≥(larger C corresponds to smaller regularization).

Original: Types of Kernels.
Revised: Types of Kernels.

Original: We use the following kernel functions k :
Revised: We use the following kernel functions k :

Original: R d × R d → R ≥ 0 .
Revised: R d × R d → R ≥ 0 .

Original: • Gaussian Kernel (RBF): k
Revised: • Gaussian Kernel (RBF): k

Original: ( x i , x j ) =
Revised: ( x i , x j ) =

Original: exp( − || x
Revised: exp( − || x

Original: i − x j || 22 2 (cid:101) σ) .
Revised: i − x j || 22 2 (cid:101) σ) .

Original: • Laplace Kernel: k
Revised: • Laplace Kernel: k

Original: ( x i , x j ) =
Revised: ( x i , x j ) =

Original: exp( − || x
Revised: exp( − || x

Original: i − x j || 2 (cid:101) σ ) .
Revised: i − x j || 2 (cid:101) σ ) .

Original: • Myrtle10 Kernel: This is the compositional kernel introduced by Shankar et al.
Revised: • Myrtle10 Kernel: This is the compositional kernel introduced by Shankar et al.

Original: We use their exact kernel for CIFAR-10.
Revised: We use their exact kernel for CIFAR-10.

Original: For the Gaussian and Laplace kernels, we parameterize bandwidth by σ := (cid:101) σ/ √ d .
Revised: For the Gaussian and Laplace kernels, we parameterize bandwidth by σ := (cid:101) σ/ √ d .

Original: We use the following bandwidths, found by cross-validation to maximize the unregularized test accuracy:
Revised: We use the following bandwidths, found by cross-validation to maximize the unregularized test accuracy:

Original: • MNIST: σ = 0 .
Revised: • MNIST: σ = 0 .

Original: 15 for RBF kernel.
Revised: 15 for RBF kernel.

Original: • Fashion-MNIST: σ = 0 .
Revised: • Fashion-MNIST: σ = 0 .

Original: 1 for RBF kernel.
Revised: 1 for RBF kernel.

Original: σ = 1 . 0 for Laplace kernel.
Revised: σ = 1 . 0 for Laplace kernel.

Original: • CIFAR-10:
Revised: • CIFAR-10:

Original: Myrtle10 Kernel from Shankar et al.
Revised: Myrtle10 Kernel from Shankar et al.

Original: (2020), and σ = 0 .
Revised: (2020), and σ = 0 .

Original: 1 for RBF kernel.
Revised: 1 for RBF kernel.

Original: B.2. N EURAL N ETWORKS
Revised: B.2. N EURAL N ETWORKS

Original: We use 4 different neural networks in our experiments.
Revised: We use 4 different neural networks in our experiments.

Original: We use a multi-layer perceptron, and three different Residual networks.
Revised: We use a multi-layer perceptron, and three different Residual networks.

Original: MLP
Revised: MLP

Original: We use a Multi-layer perceptron or a fully connected network with 3 hidden layers withneurons in each layer
Revised: We use a Multi-layer perceptron or a fully connected network with 3 hidden layers withneurons in each layer

Original: A hidden layer is followed by a BatchNormalization layer and ReLU activation function
Revised: A hidden layer is followed by a BatchNormalization layer and ReLU activation function

Original: WideResNet
Revised: WideResNet

Original: : We use the standard
Revised: : We use the standard

Original: WideResNet
Revised: WideResNet

Original: -28-10 described in Zagoruyko and Komodakis (2016).
Revised: -28-10 described in Zagoruyko and Komodakis (2016).

Original: Our code is based on this repository.
Revised: Our code is based on this repository.

Original: ResNet50:
Revised: ResNet50:

Original: We use a standard ResNet-50 from the PyTorch library (Paszke et al.,2017).
Revised: We use a standard ResNet-50 from the PyTorch library (Paszke et al.,

Original: MLP
Revised: MLP

Original: ResNet
Revised: ResNet

Original: WideResNet
Revised: WideResNet

Original: ResNet
Revised: ResNet

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Optimizer
Revised: Optimizer

Original: Adam ( β 1 = 0 .
Revised: Adam ( β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: SGD +
Revised: SGD +

Original: Momentum (0.9)
Revised: Momentum (0.9)

Original: SGD +
Revised: SGD +

Original: Momentum (0.9)
Revised: Momentum (0.9)

Original: SGD
Revised: SGD

Original: [Table]
Revised: [Table]

Original: Constant LR = 0 .
Revised: Constant LR = 0 .

Original: Inital LR = 0 .scale by 0.1 at epochs (80 , 120)
Revised: Inital LR = 0 .scale by 0.1 at epochs (80 , 120)

Original: Inital LR = 0 .scale by 0.2 at epochs (80 , 120 , 160)
Revised: Inital LR = 0 .scale by 0.2 at epochs (80 , 120 , 160)

Original: Initial LR = 0 .
Revised: Initial LR = 0 .

Original: 001 , scale by 0 .if training loss stagnant for 2000 gradient steps
Revised: 001 , scale by 0 .if training loss stagnant for 2000 gradient steps

Original: [Table]
Revised: [Table]

Original: Random ﬂips + RandomCrop(32, padding=4)
Revised: Random ﬂips + RandomCrop(32, padding=4)

Original: [Table]
Revised: [Table]

Original: -10 Error
Revised: -10 Error

Original: ∼ 37% ∼ 8% ∼ 4%
Revised: ∼ 37% ∼ 8% ∼ 4%

Original: N/A
Revised: N/A

Original: For Experiment 1 and Section 3, the hyperparameters used to train the above networks are given in
Revised: For Experiment 1 and Section 3, the hyperparameters used to train the above networks are given in

Original: Table 1.
Revised: Table 1.

Original: C F EATURE C ALIBRATION : A PPENDIX C. A GUIDE TO READING THE PLOTS
Revised: C F EATURE C ALIBRATION : A PPENDIX C. A GUIDE TO READING THE PLOTS

Original: All the experiments in support of Conjecture 1 (experiments in Section 3 and the Introduction) involve various quantities which we enumaerate here
Revised: All the experiments in support of Conjecture 1 (experiments in Section 3 and the Introduction) involve various quantities which we enumaerate here

Original: Inputs x : Each experiment involves inputs from a standard dataset like CIFAR-10 or MNIST.
Revised: Inputs x : Each experiment involves inputs from a standard dataset like CIFAR-10 or MNIST.

Original: We use the standard train/test splits for every dataset. 2.
Revised: We use the standard train/test splits for every dataset. 2.

Original: Distinguishable feature L ( x ) : This feature depends only on input x .
Revised: Distinguishable feature L ( x ) : This feature depends only on input x .

Original: We consider various features like the original classes itself, a superset of classes (as in coarse partition) or some secondary attributes (like the binary attributes provided with CelebA) 3.
Revised: We consider various features like the original classes itself, a superset of classes (as in coarse partition) or some secondary attributes (like the binary attributes provided with CelebA) 3.

Original: Output labels y :
Revised: Output labels y :

Original: The output label may be some modiﬁcation of the original labels.
Revised: The output label may be some modiﬁcation of the original labels.

Original: For instance, by adding some type of label noise, or a constructed binary task as in Experiment4.
Revised: For instance, by adding some type of label noise, or a constructed binary task as in Experiment4.

Original: Classiﬁer family F : We consider various types of classiﬁers like neural networks trained with gradient based methods, kernel and decision trees.
Revised: Classiﬁer family F : We consider various types of classiﬁers like neural networks trained with gradient based methods, kernel and decision trees.

Original: In each experiment, we are interested in two joint densities ( y, L ( x )) , which depends on our dataset and task and is common across train and test, and ( f ( x ) , L ( x )) which depends on the interpolating classiﬁers outputs on the test set.
Revised: In each experiment, we are interested in two joint densities ( y, L ( x )) , which depends on our dataset and task and is common across train and test, and ( f ( x ) , L ( x )) which depends on the interpolating classiﬁers outputs on the test set.

Original: Since y, L ( x ) and f ( x ) are discrete, we will look at their discrete joint distributions.
Revised: Since y, L ( x ) and f ( x ) are discrete, we will look at their discrete joint distributions.

Original: We sometimes refer to ( y, L ( x )) as the train joint density, as at interpolation ( y, L ( x ))
Revised: We sometimes refer to ( y, L ( x )) as the train joint density, as at interpolation ( y, L ( x ))

Original: =
Revised: =

Original: ( f ( x ) , L ( x )) for all training inputs x .
Revised: ( f ( x ) , L ( x )) for all training inputs x .

Original: We also refer to ( f ( x ) , L ( x )) as the test density, as we measure this only on the test set.
Revised: We also refer to ( f ( x ) , L ( x )) as the test density, as we measure this only on the test set.

Original: C.
Revised: C.

Original: E XPERIMENT
Revised: E XPERIMENT

Original: Experimental details: We now provide further details for Experiment 1.
Revised: Experimental details: We now provide further details for Experiment 1.

Original: We ﬁrst construct a dataset from CIFAR-10 that obeys the joint density ( y, L ( x )) shown in Figure 1 left panel.
Revised: We ﬁrst construct a dataset from CIFAR-10 that obeys the joint density ( y, L ( x )) shown in Figure 1 left panel.

Original: We then train a WideResNet-28-10 (WRN-28-10) on this modiﬁed dataset to zero training error.
Revised: We then train a WideResNet-28-10 (WRN-28-10) on this modiﬁed dataset to zero training error.

Original: The network is trained with the hyperparameters described in Table 1.
Revised: The network is trained with the hyperparameters described in Table 1.

Original: We then observe the joint density ( f ( x ) , L ( x )) on the test images and ﬁnd that the two joint densities are close as shown in Figure 5.
Revised: We then observe the joint density ( f ( x ) , L ( x )) on the test images and ﬁnd that the two joint densities are close as shown in Figure 5.

Original: We now consider a modiﬁcation of this experiment as follows:
Revised: We now consider a modiﬁcation of this experiment as follows:

Original: Experiment 2.
Revised: Experiment 2.

Original: Consider the following distribution over images x and binary labels y .
Revised: Consider the following distribution over images x and binary labels y .

Original: Sample x as a uniformly random CIFAR-10 image, and sample the label as p ( y | x ) =
Revised: Sample x as a uniformly random CIFAR-10 image, and sample the label as p ( y | x ) =

Original: Bernoulli ( CIFAR Class(x) / 10) .
Revised: Bernoulli ( CIFAR Class(x) / 10) .

Original: That is, if the CIFAR-10 class of x is k ∈ { 0 , 1 , . . .
Revised: That is, if the CIFAR-10 class of x is k ∈ { 0 , 1 , . . .

Original: 9 } , then the label is 1 with probability ( k/ 10) and 0 otherwise.
Revised: 9 } , then the label is 1 with probability ( k/ 10) and 0 otherwise.

Original: Figure 5 shows this joint distribution of ( x, y ) .
Revised: Figure 5 shows this joint distribution of ( x, y ) .

Original: As before, train a WideResNet to 0 training error on this distribution.
Revised: As before, train a WideResNet to 0 training error on this distribution.

Original: In this experiment too, we observe that the train and test joint densities are close as shown in Figure 5.
Revised: In this experiment too, we observe that the train and test joint densities are close as shown in Figure 5.

Original: Now, we repeat the same experiment, but with an MLP instead of WRN-28-10.
Revised: Now, we repeat the same experiment, but with an MLP instead of WRN-28-10.

Original: The training procedure is described in Table 1.
Revised: The training procedure is described in Table 1.

Original: This MLP has an error on 37% on the original CIFAR-10 dataset.
Revised: This MLP has an error on 37% on the original CIFAR-10 dataset.

Original: Since this MLP has poor accuracy on the original CIFAR-10 classiﬁcation task, it does not form a distinguishable partition for it.
Revised: Since this MLP has poor accuracy on the original CIFAR-10 classiﬁcation task, it does not form a distinguishable partition for it.

Original: As a result, the train and test joint densities (Figure 6) do not match as well as they did for WRN-28-10.
Revised: As a result, the train and test joint densities (Figure 6) do not match as well as they did for WRN-28-10.

Original: C. C ONSTANT P ARTITION
Revised: C. C ONSTANT P ARTITION

Original: Consider the trivially-distinguishable constant feature L ( x ) = 0 .
Revised: Consider the trivially-distinguishable constant feature L ( x ) = 0 .

Original: Then, Conjecture 1 states that the marginal distribution of class labels for any interpolating classiﬁer f ( x ) is close to the true marginals p ( y ) .
Revised: Then, Conjecture 1 states that the marginal distribution of class labels for any interpolating classiﬁer f ( x ) is close to the true marginals p ( y ) .

Original: That is, irrespective of the classiﬁer’s test accuracy, it outputs the “right” proportion of class labels on the test set, even when there is strong class imbalance.
Revised: That is, irrespective of the classiﬁer’s test accuracy, it outputs the “right” proportion of class labels on the test set, even when there is strong class imbalance.

Original: To show this, we construct a dataset based on CIFAR-10 that has class-imbalance.
Revised: To show this, we construct a dataset based on CIFAR-10 that has class-imbalance.

Original: For class k ∈ { 0 ... 9 } , sample ( k + 1) × 500 images from that class.
Revised: For class k ∈ { 0 ... 9 } , sample ( k + 1) × 500 images from that class.

Original: This will give us a dataset where classes will have marginal distribution p ( y = (cid:96) )
Revised: This will give us a dataset where classes will have marginal distribution p ( y = (cid:96) )

Original: ∝ (cid:96) + 1 for classes (cid:96) ∈ [10] , as shown in Figure 7.
Revised: ∝ (cid:96) + 1 for classes (cid:96) ∈ [10] , as shown in Figure 7.

Original: We do this both for the training set and the test set, to keep the distribution D ﬁxed.
Revised: We do this both for the training set and the test set, to keep the distribution D ﬁxed.

Original: We then train a variety of classiﬁers (MLPs, Kernels, ResNets) to interpolation on this dataset which have varying levels of test errors (9-41%).
Revised: We then train a variety of classiﬁers (MLPs, Kernels, ResNets) to interpolation on this dataset which have varying levels of test errors (9-41%).

Original: The class balance of classiﬁer outputs on the (rebalanced) test set is then close to the class balance on the train set, irrespective of the classiﬁer error.
Revised: The class balance of classiﬁer outputs on the (rebalanced) test set is then close to the class balance on the train set, irrespective of the classiﬁer error.

Original: Full experimental details and results are described in Appendix C. Note that a 1-nearest-neighbors classiﬁer would have this property.
Revised: Full experimental details and results are described in Appendix C. Note that a 1-nearest-neighbors classiﬁer would have this property.

Original: C. C LASS P ARTITION C.4. N EURAL N ETWORKS AND CIFAR- 
Revised: C. C LASS P ARTITION C.4. N EURAL N ETWORKS AND CIFAR-

Original: We now provide experiments in support of Conjecture 1 when the class itself is a distinguishable partition.
Revised: We now provide experiments in support of Conjecture 1 when the class itself is a distinguishable partition.

Original: A WRN-28-10 achieves an error of 4% on CIFAR-10.
Revised: A WRN-28-10 achieves an error of 4% on CIFAR-10.

Original: Hence, the original labels in
Revised: Hence, the original labels in

Original: CIFAR-10 form a distinguishable partition for this dataset.
Revised: CIFAR-10 form a distinguishable partition for this dataset.

Original: To demonstrate that Conjecture 1 holds, we consider different structured label noise on the CIFAR-10 dataset.
Revised: To demonstrate that Conjecture 1 holds, we consider different structured label noise on the CIFAR-10 dataset.

Original: To do so, we apply a variety of confusion matrices to the data.
Revised: To do so, we apply a variety of confusion matrices to the data.

Original: That is, for a confusion matrix C : 10 × 10 matrix, the element c ij gives the joint density that a randomly sampled image had original label j , but is ﬂipped to class i .
Revised: That is, for a confusion matrix C : 10 × 10 matrix, the element c ij gives the joint density that a randomly sampled image had original label j , but is ﬂipped to class i .

Original: For no noise, this would be an identity matrix.
Revised: For no noise, this would be an identity matrix.

Original: We begin by a simple confusion matrix where we ﬂip only one class 0 → 1 with varying probability p .
Revised: We begin by a simple confusion matrix where we ﬂip only one class 0 → 1 with varying probability p .

Original: Figure ??
Revised: Figure ??

Original: shows one such confusion matrix for p = 0 .
Revised: shows one such confusion matrix for p = 0 .

Original: We then train a WideResNet-28-10 to zero train error on this dataset.
Revised: We then train a WideResNet-28-10 to zero train error on this dataset.

Original: We use the hyperparameters described in B.2 We ﬁnd that the classiﬁer outputs on the test set closely track the confusion matrix that was applied to the distribution.
Revised: We use the hyperparameters described in B.2 We ﬁnd that the classiﬁer outputs on the test set closely track the confusion matrix that was applied to the distribution.

Original: Figureshows that this is independent of the value of p and continues to hold for p =
Revised: Figureshows that this is independent of the value of p and continues to hold for p =

Original: To show that this is not dependent on the particular class used, we also show that the same holds for a random confusion matrix.
Revised: To show that this is not dependent on the particular class used, we also show that the same holds for a random confusion matrix.

Original: We generate a sparse confusion matrix as follows.
Revised: We generate a sparse confusion matrix as follows.

Original: We set the diagonal to 0 .
Revised: We set the diagonal to 0 .

Original: Then, for every class j , we pick any two random classes for and set them to 0 .
Revised: Then, for every class j , we pick any two random classes for and set them to 0 .

Original: 2 and 0 .
Revised: 2 and 0 .

Original: We train a WRN-28-10 on it and report the test confusion matrix.
Revised: We train a WRN-28-10 on it and report the test confusion matrix.

Original: The resulting train and test densities are shown in Figure 9.
Revised: The resulting train and test densities are shown in Figure 9.

Original: As expected, the train and test confusion matrices are close, and share the same sparsity pattern.
Revised: As expected, the train and test confusion matrices are close, and share the same sparsity pattern.

Original: C.4. D ECISION T REES
Revised: C.4. D ECISION T REES

Original: Figure 10 shows a version of this experiment for decision trees on the molecular biology UCI task.
Revised: Figure 10 shows a version of this experiment for decision trees on the molecular biology UCI task.

Original: The molecular biology task is a 3-way classiﬁcation problem: to classify the type of a DNA splice junction (donor, acceptor, or neither), given the sequence of DNA (60 bases) surrounding the junction.
Revised: The molecular biology task is a 3-way classiﬁcation problem: to classify the type of a DNA splice junction (donor, acceptor, or neither), given the sequence of DNA (60 bases) surrounding the junction.

Original: We add varying amounts of label noise that ﬂips class 2 to class 1 with a certain probability, and we observe that interpolating decision trees reproduce this same structured label noise on the test set.
Revised: We add varying amounts of label noise that ﬂips class 2 to class 1 with a certain probability, and we observe that interpolating decision trees reproduce this same structured label noise on the test set.

Original: Similar results hold for decision trees; here we show experiments on two UCI tasks: molec-biol-splice and mushroom .
Revised: Similar results hold for decision trees; here we show experiments on two UCI tasks: wine and mushroom .

Original: The wine task is a 3-way classiﬁcation problem: to identify the cultivar of a given wine (out ofcultivars), given 13 physical attributes describing the wine.
Revised: The wine task is a 3-way classiﬁcation problem: to identify the cultivar of a given wine (out ofcultivars), given 13 physical attributes describing the wine.

Original: Figure 11 shows an analogous experiment with label noise taking class 1 to class 2 .
Revised: Figure 11 shows an analogous experiment with label noise taking class 1 to class 2 .

Original: The mushroom task is a 2-way classiﬁcation problem: to classify the type of edibility of a mushroom (edible vs poisonous) given 22 physical attributes (e.g. stalk color, odor, etc).
Revised: The mushroom task is a 2-way classiﬁcation problem: to classify the type of edibility of a mushroom (edible vs poisonous) given 22 physical attributes (e.g. stalk color, odor, etc).

Original: Figure 12 shows an analogous experiment with label noise ﬂipping class 0 to class 1 .
Revised: Figure 12 shows an analogous experiment with label noise ﬂipping class 0 to class 1 .

Original: C. M ULTIPLE F EATURES
Revised: C. M ULTIPLE F EATURES

Original: Conjecture 1 states that the network should be automatically calibrated for all distinguishable features, without any explicit labels for them.
Revised: Conjecture 1 states that the network should be automatically calibrated for all distinguishable features, without any explicit labels for them.

Original: To verify this, we use the CelebA dataset (Liu et al., 2015), containing images with various labelled binary attributes per-image (“male”, “blond hair”, etc).
Revised: To verify this, we use the CelebA dataset (Liu et al., 2015), containing images with various labelled binary attributes per-image (“male”, “blond hair”, etc).

Original: Some of these attributes form a distinguishable feature for ResNet50 as they are learnable to high accuracy (Jahandideh et al., 2018).
Revised: Some of these attributes form a distinguishable feature for ResNet50 as they are learnable to high accuracy (Jahandideh et al., 2018).

Original: We pick one of hard attributes as the target classiﬁcation task.
Revised: We pick one of hard attributes as the target classiﬁcation task.

Original: We train a
Revised: We train a

Original: ResNet-50 to predict the attribute { Attractive, Not Attractive } .
Revised: ResNet-50 to predict the attribute { Attractive, Not Attractive } .

Original: We choose this attribute because a
Revised: We choose this attribute because a

Original: ResNet-50 performs poorly on this task (test error ∼ 20% ) and has good class balance.
Revised: ResNet-50 performs poorly on this task (test error ∼ 20% ) and has good class balance.

Original: We choose an attribute with poor generalization because the conjecture would hold trivially for if the network generalizes well.
Revised: We choose an attribute with poor generalization because the conjecture would hold trivially for if the network generalizes well.

Original: We initialize the network with a pretrained ResNet-50 from the PyTorch library
Revised: We initialize the network with a pretrained ResNet-50 from the PyTorch library

Original: Paszke et al.
Revised: Paszke et al.

Original: and use the hyperparameters described in Section B
Revised: and use the hyperparameters described in Section B

Original: 2 to train on this attribute.
Revised: 2 to train on this attribute.

Original: We then check the train/test joint density with various other attributes like Male, Wearing Lipstick etc.
Revised: We then check the train/test joint density with various other attributes like Male, Wearing Lipstick etc.

Original: Note that the network is not given any label information for these additional attributes, but is calibrated with respect to them.
Revised: Note that the network is not given any label information for these additional attributes, but is calibrated with respect to them.

Original: That is, the network says ∼ 30% of images that have ’heavy makeup’ will be classiﬁed as ’Attractive’, even if the network makes mistakes on which particular inputs it chooses to do so.
Revised: That is, the network says ∼ 30% of images that have ’heavy makeup’ will be classiﬁed as ’Attractive’, even if the network makes mistakes on which particular inputs it chooses to do so.

Original: In this setting, the label distribution is deterministic, and not directly dependent on the distinguishable features, unlike the experiments considered before.
Revised: In this setting, the label distribution is deterministic, and not directly dependent on the distinguishable features, unlike the experiments considered before.

Original: Yet, as we see in Figure 13, the classiﬁer outputs are correctly calibrated for each attribute.
Revised: Yet, as we see in Figure 13, the classiﬁer outputs are correctly calibrated for each attribute.

Original: Loosely, this can be viewed as the network performing 1NN classiﬁcation in a metric space that is well separated for each of these distinguishable features.
Revised: Loosely, this can be viewed as the network performing 1NN classiﬁcation in a metric space that is well separated for each of these distinguishable features.

Original: C. C OARSE P ARTITION
Revised: C. C OARSE P ARTITION

Original: We now consider cases where the original classes do not form a distinguishable partition for the classiﬁer in consideration.
Revised: We now consider cases where the original classes do not form a distinguishable partition for the classiﬁer in consideration.

Original: That is, the classiﬁer is not powerful enough to obtain low error on the original dataset, but can perform well on a coarser division of the classes.
Revised: That is, the classiﬁer is not powerful enough to obtain low error on the original dataset, but can perform well on a coarser division of the classes.

Original: To verify this, we consider a division of the CIFAR-10 classes into Objects { airplane, automobile, ship, truck } vs Animals { cat, deer, dog, frog } .
Revised: To verify this, we consider a division of the CIFAR-10 classes into Objects { airplane, automobile, ship, truck } vs Animals { cat, deer, dog, frog } .

Original: An MLP trained on this problem has low error ( ∼ 8% ),
Revised: An MLP trained on this problem has low error ( ∼ 8% ),

Original: [Table]
Revised: [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: but the same network performs poorly on the  ∼ 37% 
Revised: but the same network performs poorly on the full dataset ( ∼ 37% error).

Original: Hence, Object vs Animals forms a distinguishable partition with MLPs.
Revised: Hence, Object vs Animals forms a distinguishable partition with MLPs.

Original: In Figure 14a, we show the results of training an  on the original CIFAR-10 classes.
Revised: In Figure 14a, we show the results of training an MLP on the original CIFAR-10 classes.

Original: We see that the network mostly classiﬁes objects as objects  animals as animals, even when it might mislabel a dog for a 
Revised: We see that the network mostly classiﬁes objects as objects and animals as animals, even when it might mislabel a dog for a cat.

Original: We perform a similar experiment for the RBF kernel on Fashion-MNIST, with partition { clothing, shoe, bag } , in Figure 14b.
Revised: We perform a similar experiment for the RBF kernel on Fashion-MNIST, with partition { clothing, shoe, bag } , in Figure 14b.

Original: ImageNet experiment.
Revised: ImageNet experiment.

Original: In Table 2 we provide results of the terrier experiment in the body, for various
Revised: In Table 2 we provide results of the terrier experiment in the body, for various

Original: ImageNet classiﬁers.
Revised: ImageNet classiﬁers.

Original: We use publicly available pretrained ImageNet models from this repository, and use their evaluations on the ImageNet test set.
Revised: We use publicly available pretrained ImageNet models from this repository, and use their evaluations on the ImageNet test set.

Original: C. D ISCUSSION : P ROPER S CORING R ULES
Revised: C. Q UANTITATIVE PREDICTIONS : TV DISTANCE VS ε

Revised: We now test the quantitative predictions made by Conjecture 1.

Revised: This conjecture states that

Revised: the

Revised: TV-distance between the joint distributions ( L ( x ) , f ( x )) and ( L ( x ) , y ) is at most ε , where ε is the error of the training procedure in learning L (see Deﬁnition 1).

Revised: [Equation]

Revised: To test this, we consider binary task similar to Experiment 1 where (Ship, Plane) are labeled as class 0 and (Cat, Dog) are labeled as class 1.

Revised: We then add add noise to p = 0 .

Revised: 3 fraction of cats to mislabel them as class 0.

Revised: Then, we train a convolutional network to interpolation on this task.

Revised: In this experiment, ( Cat, Dog, Plane, Ship ) form distinguishable features.

Revised: To vary the error ε on these

Revised: [Figure]

Revised: distinguishable features systematically, we train a networks to classify ( Cat, Dog, Plane, Ship ) with changing number of samples.

Revised: Networks with fewer samples have larger ε since they are worse at classifying the distinguishable features.

Revised: Then, we use the same setup and number of samples respectively to train networks on the binary task and measure the TV-distance between ( L ( x ) , f ( x )) and ( L ( x ) , y ) in this task.

Revised: The results are shown in Figure 15.

Revised: As predicted, the TV distance is upper bounded by ε .

Revised: C. D ISCUSSION : P ROPER S CORING R ULES

Original: Here we distinguish the density-estimation of Conjecture 1 from another setting where density estimation occurs.
Revised: Here we distinguish the density-estimation of Conjecture 1 from another setting where density estimation occurs.

Original: If (cid:96) ( (cid:98) p, y ) is a strictly-proper scoring rule 3 on predicted distribution (cid:98) p ∈ ∆( Y ) and sample y ∈ Y , then the population minimizer of (cid:96) ( F ( x ) , y ) is exactly the conditional density
Revised: If (cid:96) ( (cid:98) p, y ) is a strictly-proper scoring rule 3 on predicted distribution (cid:98) p ∈ ∆( Y ) and sample y ∈ Y , then the population minimizer of (cid:96) ( F ( x ) , y ) is exactly the conditional density

Original: F ( x ) = p ( y | x ) .
Revised: F ( x ) = p ( y | x ) .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: This suggests that in the limit of large-capacity network and very large data (to approximate population quantities), training neural nets with cross-entropy loss on samples ( x, y ) will yield a good density[Figure] estimate of p ( y | x ) at the softmax layer.
Revised: This suggests that in the limit of large-capacity network and very large data (to approximate population quantities), training neural nets with cross-entropy loss on samples ( x, y ) will yield a good density estimate of p ( y | x ) at the softmax layer.

Original: However, this is not what is happening in our experiments.
Revised: However, this is not what is happening in our experiments.

Original: First, our experiments consider the hard-thresholded classiﬁer, i.e. the argmax of the softmax layer.
Revised: First, our experiments consider the hard-thresholded classiﬁer, i.e. the argmax of the softmax layer.

Original: If the softmax layer itself was close to p ( y | x ) , then the classiﬁer itself will be close to argmax y p ( y | x ) – that is, close to the optimal classiﬁer.
Revised: If the softmax layer itself was close to p ( y | x ) , then the classiﬁer itself will be close to argmax y p ( y | x ) – that is, close to the optimal classiﬁer.

Original: However, this is not the case (since the classiﬁers make signiﬁcant errors).
Revised: However, this is not the case (since the classiﬁers make signiﬁcant errors).

Original: Second, we observe Conjecture 1 even in settings where we train with non-proper scoring rules (e.g. kernel regression, where the classiﬁer does not output a probability).
Revised: Second, we observe Conjecture 1 even in settings where we train with non-proper scoring rules (e.g. kernel regression, where the classiﬁer does not output a probability).

Original: D A GREEMENT P ROPERTY : A PPENDIX D. E XPERIMENTAL D ETAILS
Revised: D A GREEMENT P ROPERTY : A PPENDIX D. E XPERIMENTAL D ETAILS

Original: For ResNets on CIFAR-10 and CIFAR-100, we use the following training procedure.
Revised: For ResNets on CIFAR-10 and CIFAR-100, we use the following training procedure.

Original: For n ≤ 25000 , we sample two disjoint train sets S 1 , S 2 of size n from the 50K total train samples.
Revised: For n ≤ 25000 , we sample two disjoint train sets S 1 , S 2 of size n from the 50K total train samples.

Original: Then we train two ResNet18s f 1 , f 2 on S 1 , S 2 respectively.
Revised: Then we train two ResNet18s f 1 , f 2 on S 1 , S 2 respectively.

Original: We optimize using SGD on the cross-entropy loss, with batch size 128 , using learning rate schedule 0 .
Revised: We optimize using SGD on the cross-entropy loss, with batch size 128 , using learning rate schedule 0 .

Original: 1 for 40 (cid:98) 50000 n (cid:99) epochs, then 0 .
Revised: 1 for 40 (cid:98) 50000 n (cid:99) epochs, then 0 .

Original: 01 for 20 (cid:98) 50000 n (cid:99) epochs.
Revised: 01 for 20 (cid:98) 50000 n (cid:99) epochs.

Original: That is, we scale up the number of epoches for smaller train sizes, to keep the number of gradient steps constant.
Revised: That is, we scale up the number of epoches for smaller train sizes, to keep the number of gradient steps constant.

Original: We also early-stop optimization when the train loss reaches < 0 . 0001 , to save computational time.
Revised: We also early-stop optimization when the train loss reaches < 0 . 0001 , to save computational time.

Original: For experiments with data-augmentation, we use horizontal ﬂips and
Revised: For experiments with data-augmentation, we use horizontal ﬂips and

Original: RandomCrop(32, padding=4) .
Revised: RandomCrop(32, padding=4) .

Original: We estimate test accuracy and agreement probability on the
Revised: We estimate test accuracy and agreement probability on the

Original: CIFAR-10/100 test sets.
Revised: CIFAR-10/100 test sets.

Original: For the kernel experiments on Fashion-MNIST, we repeat the same procedure: we sample two disjoint train sets from all the train samples, train kernel regressors, and evaluate their agreement on the test set.
Revised: For the kernel experiments on Fashion-MNIST, we repeat the same procedure: we sample two disjoint train sets from all the train samples, train kernel regressors, and evaluate their agreement on the test set.

Original: Each point on the ﬁgures correspond to one trial.
Revised: Each point on the ﬁgures correspond to one trial.

Original: For UCI, some UCI tasks have very few examples, and so here we consider only the 92 classiﬁcation tasks from Fern´andez-Delgado et al.
Revised: For UCI, some UCI tasks have very few examples, and so here we consider only the 92 classiﬁcation tasks from Fern ´ andez-Delgado et al.

Original: (2014) which have at least 200 total examples.
Revised: (2014) which have at least 200 total examples.

Original: For each task, we randomly partition all the examples into a 40%-40%-20% split for 2 disjoint train sets, and 1 test set (20%).
Revised: For each task, we randomly partition all the examples into a 40%-40%-20% split for 2 disjoint train sets, and 1 test set (20%).

Original: We then train two interpolating decision trees, and compare their performance on the test set.
Revised: We then train two interpolating decision trees, and compare their performance on the test set.

Original: Decision trees are trained using a growth rule from Random Forests (Breiman, 2001; Ho, 1995): selecting a split based on a random √ d subset of d features, splitting based on Gini impurity, and growing trees until all leafs have a single sample.
Revised: Decision trees are trained using a growth rule from Random Forests (Breiman, 2001; Ho, 1995): selecting a split based on a random √ d subset of d features, splitting based on Gini impurity, and growing trees until all leafs have a single sample.

Original: This is as implemented by Scikit-learn Pedregosa et al.
Revised: This is as implemented by Scikit-learn Pedregosa et al.

Original: (2011) defaults with RandomForestClassifier(n_estimators=1, bootstrap=False) .
Revised: (2011) defaults with RandomForestClassifier(n_estimators=1, bootstrap=False) .

Original: D. A DDITIONAL P LOTS
Revised: D. A DDITIONAL P LOTS

Original: Figure 15 shows the Agreement Property for the Myrtle Kernel on CIFAR-10, and Figure 16 shows the RBF and Laplace kernels on Fashion-MNIST.
Revised: Figure 16 shows the Agreement Property for the Myrtle Kernel on CIFAR-10, and Figure 17 shows the RBF and Laplace kernels on Fashion-MNIST.

Original: Figure 17a shows interpolating decision trees on 90 classiﬁcation UCI tasks.
Revised: Figure 18a shows interpolating decision trees on 90 classiﬁcation UCI tasks.

Original: We plot the means of agreement probability and test accuracy when averaged over 100 random partitions for each task, and also include the corresponding plot for a single trial.
Revised: We plot the means of agreement probability and test accuracy when averaged over 100 random partitions for each task, and also include the corresponding plot for a single trial.

Original: Error bars on UCI tasks show 95%
Revised: Error bars on UCI tasks show 95%

Original: Clopper-Pearson conﬁdence intervals in estimating population quantities. 
Revised: Clopper-Pearson conﬁdence intervals in estimating population quantities. D. P OTENTIAL M ECHANISMS

Revised: We now consider, and refute, several potential mechanisms which could explain the experimental results of Conjecture 2 (“Agreement Property”).

Revised: In this section, we would like to understand the distribution of f ← Train( D n ) , in order to evaluate several proposed mechanisms.

Original:  [Figure]
Revised: Technically, sampling from this distribution requires training a classiﬁer on a fresh train set. Since we do not have inﬁnite samples for CIFAR-10, we construct empirical estimates by training an ensemble of classiﬁers on random subsets of CIFAR-10. Then, to approximate a sample f ← Train( D n ) , we simply sample from our ensemble f ← { f i } i . [Figure]

Original: E N EAREST -N EIGHBOR P ROOFS E. F EATURE C ALIBRATION P ROPERTY
Revised: D.3. B IMODAL S AMPLES

Revised: A simple model which would exhibit the Agreement Property is the following: Suppose test samples x come in two types: “easy” or “hard.” All classiﬁers get “easy” samples correct, but they output a uniformly random class on “hard” samples. That is, for a ﬁxed x , consider the probability that a freshly-trained classiﬁer gets x correct.

Revised: “Easy” samples are such that

Revised: [Equation]

Revised: while “hard” samples have a uniform distribution on output classes [ K ] :

Revised: [Equation]

Revised: Notice that for HARD samples x , a classiﬁer f 1 agrees with the true label y with exactly the same probability that it agrees with an independent classiﬁer f 2 (because both f 1 , f 2 are uniformly random on x ).

Revised: Thus, the agreement property (Conjecture 2) holds exactly under this model.

Revised: However, this strict decomposition of samples into “easy” and “hard” does not appear to be the case in the experiments, as shown below.

Revised: Figure 19 shows a histogram of

Revised: [Equation]

Revised: [Figure]

Revised: for test samples x in CIFAR-10, where f is a ResNet18 trained on 5000 samples 4 .

Revised: This quantity can be interpreted as the “easiness” of a given test sample ( x, y ) to a certain classiﬁer family.

Revised: If the EASY/HARD bimodal model were true, we would expect the distribution of h ( x ) to be concentrated on h ( x ) = 1 (easy samples) and h ( x ) = 0 .

Revised: 1 (hard samples).

Revised: But this is not the case in

Revised: Figure 19, and thus the bimodal model does not explain the Agreement Property.

Revised: D.3. P OINTWISE A GREEMENT

Revised: We could more generally posit that Conjecture 2 is true because the Agreement Property holds pointwise for most test samples x :

Revised: [Equation]

Revised: [Equation]

Revised: However, we ﬁnd (perhaps surprisingly) that this is not the case.

Revised: To see why this is surprising, observe that Conjecture 2 implies that the agreement probability is close to test accuracy, in expectation over the test sample and the classiﬁers f 1 , f 2 ← Train( D n ) :

Revised: [Equation]

Revised: Swapping the order of expectation, this implies

Revised: [Equation]

Revised: Now, we may expect that this means M ( x, y )

Revised: ≈ 0 pointwise, for most test samples ( x, y ) .

Revised: But this is not the case.

Revised: It turns out that M ( x, y ) takes on signiﬁcantly positive and negative values, and these effects “cancel out” in expectation over the distribution, to yield Conjecture 2.

Revised: For example, we compute M ( x, y ) for the Myrtle10 kernel on CIFAR-10 with 1000 train samples.1.

Revised: The agreement probability is within 0 .

Revised: 8% of the test error (as in Figure 16), and so

Revised: [Equation]

Revised: However, M ( x, y ) is not pointwise close to 0.

Revised: E.g,

Revised: [Equation]

Revised: Figure 20 plots the distribution of M ( x, y ) .

Revised: We see that some samples ( x, y ) have high agreement probability, and some low, and these happen to balance in expectation to yield the test accuracy.

Revised: Interestingly, 1-nearest neighbors can satisfy the agreement property of Claim 2 without satisfying the “pointwise agreement” of Equation 10.

Revised: It remains an open problem to understand the mechanisms behind the Agreement Property.

Revised: E N EAREST -N EIGHBOR P ROOFS E. F EATURE C ALIBRATION P ROPERTY

Original: Proof of Theorem 1.
Revised: Proof of Theorem 1.

Original: Recall that L being an ( ε, NN , D , n ) -distinguishable partition means that nearest-neighbors works to classify L ( x ) from x :
Revised: Recall that L being an ( ε, NN , D , n ) -distinguishable partition means that nearest-neighbors works to classify L ( x ) from x :

Original: [Equation]
Revised: [Equation]

Original: Now, we have
Revised: Now, we have

Original: [Equation] [Figure]
Revised: [Equation] [Equation]

Original: Line (12) is by distinguishability, since Pr[ L ( x ) (cid:54) =
Revised: Line (15) is by distinguishability, since Pr[ L ( x ) (cid:54) =

Original: L ( (cid:98) x i )] ≤ ε .
Revised: L ( (cid:98) x i )] ≤ ε .

Original: And Line (13) is by the regularity condition.
Revised: And Line (16) is by the regularity condition.

Original: E. A GREEMENT P ROPERTY
Revised: E. A GREEMENT P ROPERTY

Original: Theorem 2 (Agreement Property) .
Revised: Theorem 2 (Agreement Property) .

Original: For a given distribution D on ( x, y ) , and given number of train samples n ∈ N , suppose NN satisﬁes the following regularity condition: If we sample two independent train sets S 1 , S 2 , then the following two “couplings” are statistically close:
Revised: For a given distribution D on ( x, y ) , and given number of train samples n ∈ N , suppose NN satisﬁes the following regularity condition: If we sample two independent train sets S 1 , S 2 , then the following two “couplings” are statistically close:

Original: [Equation]
Revised: [Equation]

Original: The LHS is simply a random test point x i , along with its nearest-neighbor in the train set.
Revised: The LHS is simply a random test point x i , along with its nearest-neighbor in the train set.

Original: The RHS produces an ( x i , x j ) by sampling two independent train sets, sampling a test point x ∼ D , and producing the nearest-neighbor of x in S 1 and S 2 respectively.
Revised: The RHS produces an ( x i , x j ) by sampling two independent train sets, sampling a test point x ∼ D , and producing the nearest-neighbor of x in S 1 and S 2 respectively.

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Let the LHS of Equation (14) be denoted as distribution P over X × X .
Revised: Let the LHS of Equation (17) be denoted as distribution P over X × X .

Original: And let Q be the
Revised: And let Q be the

Original: RHS of Equation (14).
Revised: RHS of Equation (17).

Original: Let D x denote the marginal distribution on x of D , and let p ( y | x ) denote the conditional distribution with respect to D .
Revised: Let D x denote the marginal distribution on x of D , and let p ( y | x ) denote the conditional distribution with respect to D .

Original: The proof follows by considering the sampling of train set S in the following order: ﬁrst, sample all the x -marginals: sample test point x ∼ D x and train points S x ∼ D nx .
Revised: The proof follows by considering the sampling of train set S in the following order: ﬁrst, sample all the x -marginals: sample test point x ∼ D x and train points S x ∼ D nx .

Original: Then compute the nearestneighbors (cid:98) x ← NN S x ( x ) .
Revised: Then compute the nearestneighbors (cid:98) x ← NN S x ( x ) .

Original: And ﬁnally, sample the values y of all the points involved, according to the densities p ( y | x ) .
Revised: And ﬁnally, sample the values y of all the points involved, according to the densities p ( y | x ) .

Original: [Equation]
Revised: [Equation]

Original: as desired.
Revised: as desired.

Original: F N ON - INTERPOLATING C LASSIFIERS : A PPENDIX
Revised: F N ON - INTERPOLATING C LASSIFIERS : A PPENDIX

Original: Here we give an additional example of distributional generalization: in kernel SVM (as opposed to kernel regression, in the main text).
Revised: Here we give an additional example of distributional generalization: in kernel SVM (as opposed to kernel regression, in the main text).