Original: Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space
Revised: Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space

Original: Submission number:
Revised: Submission number:

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: Affilliation email
Revised: Affilliation email

Original: Abstract
Revised: Abstract

Original: Machine learning classifiers have been demonstrated, both empirically and theoretically, to be robust to label noise under certain conditions — notably the typicalassumption is that label noise is independent of the features given the class label.
Revised: Machine learning classifiers have been demonstrated, both empirically and theoretically, to be robust to label noise under certain conditions — notably the typicalassumption is that label noise is independent of the features given the class label.

Original: We provide a theoretical framework that generalizes beyond this typical assumptionby modeling label noise as a distribution over feature space.
Revised: We provide a theoretical framework that generalizes beyond this typical assumptionby modeling label noise as a distribution over feature space.

Original: We show that both thescale and the shape of the noise distribution influence the posterior likelihood; andthe shape of the noise distribution has a stronger impact on classification performance if the noise is concentrated in feature space where the decision boundarycan be moved.
Revised: We show that both thescale and the shape of the noise distribution influence the posterior likelihood; andthe shape of the noise distribution has a stronger impact on classification performance if the noise is concentrated in feature space where the decision boundarycan be moved.

Original: For the special case of uniform label noise (independent of featuresand the class label), we show that the Bayes optimal classifier for c classes isrobust to label noise until the ratio of noisy samples goes above c − 1 c
Revised: For the special case of uniform label noise (independent of featuresand the class label), we show that the Bayes optimal classifier for c classes isrobust to label noise until the ratio of noisy samples goes above c − 1 c

Original: (e.g. 90%for 10 classes), which we call the tipping point .
Revised: (e.g. 90%for 10 classes), which we call the tipping point .

Original: However, for the special caseof class-dependent label noise (independent of features given the class label), thetipping point can be as low as 50%.
Revised: However, for the special caseof class-dependent label noise (independent of features given the class label), thetipping point can be as low as 50%.

Original: Most importantly, we show that when thenoise distribution targets decision boundaries (label noise is directly dependenton feature space), classification robustness can drop off even at a small scale ofnoise.
Revised: Most importantly, we show that when thenoise distribution targets decision boundaries (label noise is directly dependenton feature space), classification robustness can drop off even at a small scale ofnoise.

Original: Even when evaluating recent label-noise mitigation methods we see reducedaccuracy when label noise is dependent on features.
Revised: Even when evaluating recent label-noise mitigation methods we see reducedaccuracy when label noise is dependent on features.

Original: These findings explain whymachine learning often handles label noise well if the noise distribution is uniformin feature-space; yet it also points to the difficulty of overcoming label noise whenit is concentrated in a region of feature space where a decision boundary can move.
Revised: These findings explain whymachine learning often handles label noise well if the noise distribution is uniformin feature-space; yet it also points to the difficulty of overcoming label noise whenit is concentrated in a region of feature space where a decision boundary can move.

Original:  Introduction
Revised:  Introduction

Original: An open question in machine learning is how the quality of training data, especially the labels, affectsthe learned model [2, 4, 13, 21, 23].
Revised: An open question in machine learning is how the quality of training data, especially the labels, affectsthe learned model [2, 4, 13, 21, 23].

Original: As an example shown in Figure 1, we imagine a classificationtask for detecting cancer based on observed biomarkers.
Revised: As an example shown in Figure 1, we imagine a classificationtask for detecting cancer based on observed biomarkers.

Original: The generative model assumes that thereis a hidden variable, called Y ∗ , denoting whether a person actually has cancer or not.
Revised: The generative model assumes that thereis a hidden variable, called Y ∗ , denoting whether a person actually has cancer or not.

Original: Dependingon the state of Y ∗ , biomarkers can be observed, denoted as a feature vector x , drawn from somedistribution x ∼ p ( X | Y ∗ ) .
Revised: Dependingon the state of Y ∗ , biomarkers can be observed, denoted as a feature vector x , drawn from somedistribution x ∼ P [ X | Y ∗ ] .

Original: The goal of machine learning (ML) is to predict the presence of cancer given biomarkers —  and evaluate the ML model, we have diagnoses, { y } , which may not always be correct (
Revised: The goal of machine learning (ML) is to predict the presence of cancergiven the biomarkers — yet to train and evaluate the ML model, we have diagnoses, { y } , which maynot always be correct (

Original: y ̸
Revised: y ̸

Original: = y ∗ for some samples).
Revised: = y ∗ for some samples).

Original: The question of label uncertainty is thus: Whathappens when the observed labels, y , do not match the hidden true labels y ∗ ?
Revised: The question of label uncertainty is thus: Whathappens when the observed labels, y , do not match the hidden true labels y ∗ ?

Original: Our theory concludes between the true label posterior p ( Y ∗ | X ) and the noisy distribution p ( Y | X, Y ∗ ) .
Revised: Our theory concludesthat the answer depends on the shape of the noisy distribution P [ Y | X ] with respect to feature space.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ExampleDisease ( cancer or not_cancer )
Revised: ExampleDisease ( cancer or not_cancer )

Original: Observed biomarkers
Revised: Observed biomarkers

Original: Diagnosis ( cancer or not_cancer )
Revised: Diagnosis ( cancer or not_cancer )

Original: [Table]
Revised: [Table]

Original: p ( Y =| x ﬂip ) p ( Y =| x ﬂip )
Revised: p ( Y =| x ﬂip ) p ( Y =| x ﬂip )

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: distributions: (b) uniform ,(c) class -dependent, and (d)-(g) feature -dependent.
Revised: distributions: b) uniform , class -dependent, and d) g feature -dependent.

Original: Empirical research suggests that ML is often able to generalize beyond label noise injected in trainingdata to make accurate predictions, particularly when the noisy label is sampled independently ofthe features and the true label, i.e. P ( Y | X, Y ∗ )
Revised: Empirical research suggests that ML is often able to generalize beyond label noise injected in trainingdata to make accurate predictions, particularly when the noisy label is sampled independently ofthe features and the true label, i.e. P [ Y | X, Y ∗ ]

Original: = P ( Y ) [2–5, 21, 23, 24].
Revised: = P [ Y ] [2–5, 21, 23, 24].

Original: This is a remarkablefinding considering how messy such data appears, as shown in Fig.
Revised: This is a remarkablefinding considering how messy such data appears, as shown in Fig.

Original: 2b.
Revised: 2b.

Original: However, this robustnessto label noise drops significantly when these approaches test their findings by synthesizing labelnoise with the assumption that the noise is independent of the features given the true class label, i.e.p ( Y | X, Y ∗ ) =
Revised: However, this robustnessto label noise drops significantly when these approaches test their findings by synthesizing labelnoise with the assumption that the noise is independent of the features given the true class label, i.e. P [ Y | X, Y ∗ ] =

Original: p ( Y | Y ∗ ) .
Revised: P [ Y | Y ∗ ] .

Original: In other words, for a given class, all samples in that class are equally likelyto have the same erroneous label, as demonstrated in Fig.
Revised: In other words, for a given class, all samples in that class are equally likelyto have the same erroneous label, as demonstrated in Fig.

Original: 2c.
Revised: 2c.

Original: We theoretically analyze the posteriorlikelihood of the true class given noisy training labels to demonstrate that both of these types of labelnoise (uniform and class-dependent) behave similarly but with very different tipping points of noiselevel beyond which the true labels are not recoverable.
Revised: We theoretically analyze the posteriorlikelihood of the true class given noisy training labels to demonstrate that both of these types of labelnoise (uniform and class-dependent) behave similarly but with very different tipping points of noiselevel beyond which the true labels are not recoverable.

Original: While the assumption of label noise being independent of features is convenient for generatingnoisy samples [3, 5, 13, 27], it is not clear that it is an appropriate assumption [4, 23].
Revised: While the assumption of label noise being independent of features is convenient for generatingnoisy samples [3, 5, 13, 27], it is not clear that it is an appropriate assumption [4, 23].

Original: Morerealistically, we expect that label noise occurs when the features are ambiguous; such as whencontradictory biomarkers are observed in an oncology patient, or when an image to be classified isblurry.
Revised: Morerealistically, we expect that label noise occurs when the features are ambiguous; such as whencontradictory biomarkers are observed in an oncology patient, or when an image to be classified isblurry.

Original: In other words, p ( Y | X, Y ∗ ) ̸ = p ( Y | Y ∗ ) .
Revised: In other words, P [ Y | X, Y ∗ ] ̸ = P [ Y | Y ∗ ] .

Original: Figs. 2d and 2f show data generated with two noisedistributions such that label noise is most likely to occur in regions of feature-space that are nearclass boundaries; while Figs.
Revised: Figs. 2d and 2f show data generated with two noisedistributions such that label noise is most likely to occur in regions of feature-space that are nearclass boundaries; while Figs.

Original: 2e and 2g show data where label flips are most likely to occur far fromtrue class boundaries.
Revised: 2e and 2g show data where label flips are most likely to occur far fromtrue class boundaries.

Original: Our theory and empirical work demonstrate that feature-dependent label flipsthat create noisy class boundaries far from the true class boundaries have the most negative impacton classification accuracy, even at levels of label noise much lower than the tipping points seen inuniform and class-dependent label noise.
Revised: Our theory and empirical work demonstrate that feature-dependent label flipsthat create noisy class boundaries far from the true class boundaries have the most negative impacton classification accuracy, even at levels of label noise much lower than the tipping points seen inuniform and class-dependent label noise.

Original: We demonstrate theoretically and empirically that classification is generally robust to uniform andclass-dependent label noise until the scale of the noise exceeds a threshold that depends on the“spread" of the noise distribution; but that beyond this tipping point, classification accuracy declinesrapidly.
Revised: We demonstrate theoretically and empirically that classification is generally robust to uniform andclass-dependent label noise until the scale of the noise exceeds a threshold that depends on the“spread" of the noise distribution; but that beyond this tipping point, classification accuracy declinesrapidly.

Original: Yet, we also demonstrate that such robustness to label noise is misleading; because ourintroduction of feature-dependent label noise shows that classification accuracy can be loweredsignificantly even for small amounts of label noise.
Revised: Yet, we also demonstrate that such robustness to label noise is misleading; because ourintroduction of feature-dependent label noise shows that classification accuracy can be loweredsignificantly even for small amounts of label noise.

Original: We evaluate, for the first time, the damagingeffect of feature-dependent label noise on recent strategies for mitigating label noise.
Revised: We evaluate, for the first time, the damagingeffect of feature-dependent label noise on recent strategies for mitigating label noise.

Original: We concludethat the shape of the label noise distribution with respect to feature space is a significant factor inlabel noise robustness.
Revised: We concludethat the shape of the label noise distribution with respect to feature space is a significant factor inlabel noise robustness.

Revised: Our theoretical findings should guide future theoretical and empirical work onlabel noise to focus studies on the most informative combinations of noise scale and shape; savingsignificant computational effort while improving the research community’s ability to gain insightabout how label noise affects learned models.

Original:  Preliminaries2.1 Problem Statement
Revised:  Preliminaries2.1 Problem Statement

Original: We consider the problem of label noise in which, for given data sample, ( X, Y ∗ ) ; for some fraction ofsamples, the true labels Y ∗ ∈ { 1 , . .
Revised: We consider the problem of label noise in which, for given data sample, ( X, Y ∗ ) ; for some fraction ofsamples, the true labels Y ∗ ∈ { 1 , . .

Original: . , c } are replaced with incorrect (noisy) labels Y .
Revised: . , c } are replaced with incorrect (noisy) labels Y .

Original: The fraction ofnoisy samples is called the noise level .
Revised: The fraction ofnoisy samples is called the noise level .

Original: The typical paradigm for modelling such noise is using labelflips [16, 21], where for each sample, there is some probability that the true label will be flipped to anincorrect label.
Revised: The typical paradigm for modelling such noise is using labelflips [16, 21], where for each sample, there is some probability that the true label will be flipped to anincorrect label.

Original: Which label it flips to is typically modelled in one of two ways: 1) uniform noise(also known as symmetric label noise): each incorrect class is equally likely; or 2) class-dependentnoise (also known as asymmetric label noise) in which some classes are more likely than others for agiven true class.
Revised: Which label it flips to is typically modelled in one of two ways: 1) uniform noise(also known as symmetric label noise): each incorrect class is equally likely; or 2) class-dependentnoise (also known as asymmetric label noise) in which some classes are more likely than others for agiven true class.

Original: In the case of class-dependent noise, the noise function is typically represented as ac × c label transition matrix describing the probably of a transition from Y ∗ to Y .
Revised: In the case of class-dependent noise, the noise function is typically represented as ac × c label transition matrix describing the probably of a transition from Y ∗ to Y .

Original: The number ofnon-zero off-diagonal elements in each row of this matrix (or inversely, the sparsity) characterizes thespread of the noise distribution [13].
Revised: The number ofnon-zero off-diagonal elements in each row of this matrix (or inversely, the sparsity) characterizes thespread of the noise distribution [13].

Original: In other words, when spread is c − 1 , the noise distribution isuniform, but the lower the sparsity, the further from uniform the distribution is.
Revised: In other words, when spread is c − 1 , the noise distribution isuniform, but the lower the sparsity, the further from uniform the distribution is.

Original: 2.2 Related Work
Revised: 2.2 Related Work

Original: Several methods have been developed to improve classifier robustness to noisy labels during training.
Revised: Several methods have been developed to improve classifier robustness to noisy labels during training.

Original: CleanLab removes noisy samples from training [13].
Revised: CleanLab removes noisy samples from training [13].

Original: Methods like CoTeaching weight the samples[5, 20].
Revised: Methods like CoTeaching weight the samples[5, 20].

Original: Approximate expectation-maximization infers the true label [4].
Revised: Approximate expectation-maximization infers the true label [4].

Original: MentorNet uses a teacherstudent learning approach [7].
Revised: MentorNet uses a teacherstudent learning approach [7].

Original: MixUp interpolates pairs of examples and their labels [27].
Revised: MixUp interpolates pairs of examples and their labels [27].

Original: SoftLabelstreats the labels as learnable parameters [23].
Revised: SoftLabelstreats the labels as learnable parameters [23].

Original: Symmetric cross-entropy (SCE) uses a robust lossfunction [24].
Revised: Symmetric cross-entropy (SCE) uses a robust lossfunction [24].

Original: Similar to our paper, [3] evaluates robustness of existing approaches to label noise.
Revised: Similar to our paper, [3] evaluates robustness of existing approaches to label noise.

Original: Chen et al.
Revised: Chen et al.

Original: [2] shows that the noisy test accuracy is a quadratic function of the noise level for the caseof uniform label noise.
Revised: [2] shows that the noisy test accuracy is a quadratic function of the noise level for the caseof uniform label noise.

Original: Yet, none of these approaches are yet evaluated on feature-dependent noise.
Revised: Yet, none of these approaches are yet evaluated on feature-dependent noise.

Original: A few papers argue for the need to consider feature-dependent label noise, but they do not evaluatetheir approaches in this setting [4, 23].
Revised: A few papers argue for the need to consider feature-dependent label noise, but they do not evaluatetheir approaches in this setting [4, 23].

Original: Notably, two papers replace the label-flip paradigm withreal-world webly-labeled noise (which is likely feature-dependent) [8, 10], but simultaneously inducea domain shift which makes analysis challenging; and the dependence on the features is unknown.
Revised: Notably, two papers replace the label-flip paradigm withreal-world webly-labeled noise (which is likely feature-dependent) [8, 10], but simultaneously inducea domain shift which makes analysis challenging; and the dependence on the features is unknown.

Original:  Theory
Revised:  Theory

Original: We show that the estimation of the true conditional distribution P [ Y ∗ | X ] depends on the level ofnoise, as well as the shape of the noise.
Revised: We show that the estimation of the true conditional distribution P [ Y ∗ | X ] depends on the level ofnoise, as well as the shape of the noise.

Original: We define the classification problem of c classes with labelflips; and examine different types of label noise: uniform, class-dependent, and feature-dependent.
Revised: We define the classification problem of c classes with labelflips; and examine different types of label noise: uniform, class-dependent, and feature-dependent.

Original: Proofs are given in the Supplement.
Revised: Proofs are given in the Supplement.

Original: 3.1 Problem Definition: Classification with Label Flips
Revised: 3.1 Problem Definition: Classification with Label Flips

Original: In a classification problem with c classes, we are given an n-sample of pairs ( X, Y ∗ )
Revised: In a classification problem with c classes, we are given an n-sample of pairs ( X, Y ∗ )

Original: ∈ R d ×{ 1 , . . .
Revised: ∈ R d ×{ 1 , . . .

Original: , c } , with joint distribution determined by
Revised: , c } , with joint distribution determined by

Original: [Equation]
Revised: [Equation]

Revised: and P [ Y

Original:  for each k ∈ { 1 , . . .
Revised: [Equation] = k ] = [Equation] 1) for each k ∈ { 1 , . . .

Original: , c } .
Revised: , c } .

Original: We want to estimate the clean posterior :
Revised: We want to estimate the clean posterior :

Original: [Equation]
Revised: [Equation]

Original: Rather than observing Y ∗ , we observe the noisy label Y with probability P
Revised: Rather than observing Y ∗ , we observe the noisy label Y with probability P

Original: [ Y = i | Y ∗
Revised: [ Y = i | Y ∗

Original: =
Revised: =

Original: k, X =x ]
Revised: k, X =x ]

Original: = η ki ( x ) where (cid:80) ci =
Revised: = η ki ( x ) where (cid:80) ci =

Original: 1 η ki ( x ) = 1 .
Revised: 1 η ki ( x ) = 1 .

Original: In other words, Y | Y ∗ , X is multinomial distributed such that1 − η kk ( x ) is the probability of a label flip and each η ki ( x ) influences which noisy label, i , may bechosen.
Revised: In other words, Y | Y ∗ , X is multinomial distributed such that1 − η kk ( x ) is the probability of a label flip and each η ki ( x ) influences which noisy label, i , may bechosen.

Original: The impact of label noise is that the estimate of the noisy posterior is
Revised: The impact of label noise is that the estimate of the noisy posterior is

Original: [Equation]
Revised: [Equation]

Original: 3.2 Uniform Label Noise
Revised: 3.2 Uniform Label Noise

Original: Classifiers are generally robust to uniform label noise up to a tipping point, as has been welldemonstrated empirically.
Revised: Classifiers are generally robust to uniform label noise up to a tipping point, as has been welldemonstrated empirically.

Original: Definition 3.1 (Uniform noise) For a classification problem with c classes, given the scale of labelnoise, 0 ≤ ϵ ≤ 1 , the distribution of noisy labels is constant such that:
Revised: Definition 3.1 (Uniform noise) For a classification problem with c classes, given the scale of labelnoise, 0 ≤ ϵ ≤ 1 , the distribution of noisy labels is constant such that:

Original: [Equation]
Revised: [Equation]

Original: Plugging the Definition 3.1 into the noisy posterior Eq 3; the noisy posterior under uniform noise is:
Revised: Plugging the Definition 3.1 into the noisy posterior Eq 3; the noisy posterior under uniform noise is:

Original: [Equation]
Revised: [Equation]

Original: Lemma 3.2 (Noisy accuracy)
Revised: Lemma 3.2 (Noisy accuracy)

Original: Given noisy train and test samples according to Definition 3.1, thenoisy test accuracy of a Bayes optimal classifier is quadratic with respect to the noise level ϵ :
Revised: Given noisy train and test samples according to Definition 3.1, thenoisy test accuracy of a Bayes optimal classifier is quadratic with respect to the noise level ϵ :

Original: [Equation]
Revised: [Equation]

Original: Corollary 3.2.1 The minimum noisy accuracy of 1 c occurs at ϵ = c − 1 c
Revised: Corollary 3.2.1 Given noisy train and test samples according to Definition 3.1, the minimum noisyaccuracy of 1 c occurs at ϵ = c − 1 c

Original: See Fig 3. Note that setting ¯ m ∗ k = 1 yields the special case given in Corollary 1.1 of [2].
Revised: Fig 3a plots the quadratic of Eq 6 which shows that models trained on noisy data will at first decreasein accuracy as tested on noisy labels, but then begin to fit the label noise rather than the true labels asthe fraction of noisy labels increases beyond the minimizing point.

Original: Theorem 3.3 (Clean accuracy)
Revised: Theorem 3.3 (Clean accuracy)

Original: Given noisy train samples according to Definition 3.1 and cleantest samples, the clean test accuracy of a Bayes optimal classifier is logistic with respect to the noiselevel ϵ :
Revised: Given noisy train samples according to Definition 3.1 and cleantest samples, the clean test accuracy of a Bayes optimal classifier is logistic with respect to the noiselevel ϵ :

Original: [Equation]
Revised: [Equation]

Original: where b is an arbitrary base of the exponent used in the softmax approximation to the argmax.
Revised: where b is an arbitrary base of the exponent used in the softmax approximation to the argmax.

Original: Thelarger the value of b , the closer to the true maximum function.
Revised: Thelarger the value of b , the closer to the true maximum function.

Original: Corollary 3.3.1 Clean accuracy{+3.3.
 Given noisy train samples according to Definition 3.1 and clean test samples, cleanaccuracy+} drops by half at ϵ = c − 1 c . 
Revised: Corollary {+3.3.
 Given noisy train samples according to Definition 3.1 and clean test samples, cleanaccuracy+} drops to ¯ m ∗ k 2 at ϵ = c − 1 c . [Figure]

Original: See Fig 3.
Revised: See Fig 3.

Original: For a dataset like ImageNet with 1000 classes, the clean accuracy would only drop offwhen the noise level is above 99 .
Revised: For a dataset like ImageNet with 1000 classes, the clean accuracy would only drop offwhen the noise level is above 99 .

Original: This finding is consistent with empirical studies [17] showingthat deep learning models are robust to this type of label noise; particularly when there are largenumbers of classes.
Revised: This finding is consistent with empirical studies [17] showingthat deep learning models are robust to this type of label noise; particularly when there are largenumbers of classes.

Original: 3.3 Class-Dependent Flips
Revised: 3.3 Class-Dependent Flips

Original: We consider the special case of class-dependent label noise that is also well-demonstrated empirically.
Revised: We consider the special case of class-dependent label noise that is also well-demonstrated empirically.

Original: Often, this class-dependent noise is defined such that η ki = 0 for several i .
Revised: Often, this class-dependent noise is defined such that η ki = 0 for several i .

Original: We refer to the numberof non-zero elements as the spread of the class-dependent noise function.
Revised: We refer to the numberof non-zero elements as the spread of the class-dependent noise function.

Original: In the least-spread case,η kj = ϵ for some j ∈ { 1 , . .
Revised: In the least-spread case,η kj = ϵ for some j ∈ { 1 , . .

Original: . , c } \ { k } while η ki = 0 for all other values of i ∈ { 1 , . .
Revised: . , c } \ { k } while η ki = 0 for all other values of i ∈ { 1 , . .

Original: . , c } \ { k, j } .
Revised: . , c } \ { k, j } .

Original: [Figure]

Original: Definition 3.4 (Class-dependent noise) For a classification problem with c classes, given the scaleof label noise, 0 ≤ ϵ ≤ 1 , and spread of noisy label distribution, 1 ≤ s ≤ c − 1 ; the distribution ofnoisy labels is constant within each class k ∈ { 1 , . .
Revised: Definition 3.4 (Class-dependent noise) For a classification problem with c classes, given the scaleof label noise, 0 ≤ ϵ ≤ 1 , and spread of noisy label distribution, 1 ≤ s ≤ c − 1 ; the distribution ofnoisy labels is constant within each class k ∈ { 1 , . .

Original: , c } :
Revised: , c } :

Revised: [Equation]

Revised: (cid:40)

Original: [Equation]
Revised: [Equation]

Revised: , for i = k ϵ · t ki , ∀ i ∈ { 1 , . . .

Revised: , c } \ { k } and (cid:80) i ̸

Original: {+= k | t ki | 
+}
Revised: {+= k | t ki | 
+}

Original:  Plugging Definition 3.4 into the noisy posterior Eq 3 we see that the noisy posterior under classdependent noise depends on the spread, s , rather than the total number of classes:
Revised: = s and (cid:80) i ̸ = k t ki =(8) Plugging Definition 3.4 into the noisy posterior Eq 3 we see that the noisy posterior under classdependent noise depends on the spread, s , rather than the total number of classes:

Original: [Equation]
Revised: [Equation]

Original: Lemma 3.5 (Noisy accuracy)
Revised: Lemma 3.5 (Noisy accuracy)

Original: Given noisy train and test samples according to Definition 3.4, thenoisy test accuracy of a Bayes optimal classifier is quadratic with respect to the noise level ϵ andspread s :
Revised: Given noisy train and test samples according to Definition 3.4, thenoisy test accuracy of a Bayes optimal classifier is quadratic with respect to the noise level ϵ andspread s :

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Theorem 3.6 (Clean accuracy)
Revised: Theorem 3.6 (Clean accuracy)

Original: Given noisy train samples according to Definition 3.4 and cleantest samples, the clean test accuracy of a Bayes optimal classifier is logistic with respect to the noiselevel ϵ :[Equation]
Revised: Given noisy train samples according to Definition 3.4 and cleantest samples, the clean test accuracy of a Bayes optimal classifier is logistic with respect to the noiselevel ϵ :

Original: [Equation]
Revised: [Equation]

Original: Class-dependent noise impacts classification accuracy in a similar way to uniform noise; however,the tipping-point in class-dependent noise is controlled by the spread rather than by the total numberof classes.
Revised: Class-dependent noise impacts classification accuracy in a similar way to uniform noise; however,the tipping-point in class-dependent noise is controlled by the spread rather than by the total numberof classes.

Original: Thus, even for a large number of classes, the tipping point can be as low as ϵ = 1 / 2 .
Revised: Thus, even for a large number of classes, the tipping point can be as low as ϵ = 1 / 2 .

Original: 3.4 Feature-Dependent Label Noise
Revised: 3.4 Feature-Dependent Label Noise

Original: We consider the more general paradigm of feature-dependent label noise which has not receivedmuch attention either theoretically or empirically.
Revised: We consider the more general paradigm of feature-dependent label noise which has not receivedmuch attention either theoretically or empirically.

Original: Consider the simplest feature-dependent label-flipscenario where a single sample ( X = x, Y ∗ = k ) for some fixed x with true label k will have noiseintroduced as ( X = x, Y = j ) for some j ∈ { 1 , . .
Revised: Consider the simplest feature-dependent label-flipscenario where a single sample ( X = x, Y ∗ = k ) for some fixed x with true label k will have noiseintroduced as ( X = x, Y = j ) for some j ∈ { 1 , . .

Original: , c } .
Revised: , c } .

Original: No matter which noisy label j appears,the posterior likelihood of the noisy data will be reduced to m k ( x ) =
Revised: No matter which noisy label j appears,the posterior likelihood of the noisy data will be reduced to m k ( x ) =

Original: (1 − p ( x )) m ∗ k
Revised: (1 − p ( x )) m ∗ k

Original: ( x ) ; while theposterior likelihood of the incorrect label will increase to m j ( x ) = m ∗ j ( x ) + p ( x )
Revised: ( x ) ; while theposterior likelihood of the incorrect label will increase to m j ( x ) = m ∗ j ( x ) + p ( x )

Original: m ∗ k ( x ) .
Revised: m ∗ k ( x ) .

Original: In the
Revised: In the

Original: Bayes optimal classifier, classification errors occur when:
Revised: Bayes optimal classifier, classification errors occur when:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: for true label k for given point x .
Revised: for true label k for given point x .

Original: Therefore, for a fixed x , the label-flip Y =
Revised: Therefore, for a fixed x , the label-flip Y =

Original: j that would have themost impact on classification (the greatest increase of the right-hand side of the inequality) is
Revised: j that would have themost impact on classification (the greatest increase of the right-hand side of the inequality) is

Original: [Equation]
Revised: [Equation]

Original: An immediate consequence of this observation is that worst-case label noise is a function of x ;therefore any evaluation of label noise (empirical or theoretical) that does not depend on features isoptimistic.
Revised: An immediate consequence of this observation is that worst-case label noise is a function of x ;therefore any evaluation of label noise (empirical or theoretical) that does not depend on features isoptimistic.

Original: Now, consider which samples, i.e. the shape of η kk ( x ) , would have the most impact on cleanclassification accuracy.
Revised: Now, consider which samples, i.e. the shape of η kk ( x ) , would have the most impact on cleanclassification accuracy.

Original: By re-arranging Eq 12, we see that the ratio m k ( x ) / max i ∈{ 1 ,...,c }\ k m i ( x )defines the optimal classification boundaries.
Revised: By re-arranging Eq 12, we see that the ratio m k ( x ) / max i ∈{ 1 ,...,c }\ k m i ( x )defines the optimal classification boundaries.

Original: By targeting label flips on samples x where this ratio ishigh, a new (noisy) classification boundary can be created; as illustrated in Fig.
Revised: By targeting label flips on samples x where this ratio ishigh, a new (noisy) classification boundary can be created; as illustrated in Fig.

Original: 1b.
Revised: 1b.

Original: Claim 3.7 (Worst-case noise)
Revised: Claim 3.7 (Worst-case noise)

Original: For a classification problem with c classes, the samples, x , that havethe greatest impact on clean label prediction are those that maximize the ratio between the likelihoodof the true class to the likelihood of the most-likely incorrect class: 
Revised: For a classification problem with c classes, the samples, x , that havethe greatest impact on clean label prediction are those that maximize the ratio between the likelihoodof the true class to the likelihood of the most-likely incorrect class: [Equation] for

Original: [Equation]
Revised: [Equation]

Original: k
Revised: k

Original: [Equation]
Revised: [Equation]

Original: 3.5 Discussion of theoretical results
Revised: 3.5 Discussion of theoretical results

Original: Relationship to existing theory It is known that label noise is not generally distinguishable fromother label uncertainty (such as class overlap) [18, 19].
Revised: Relationship to existing theory It is known that label noise is not generally distinguishable fromother label uncertainty (such as class overlap) [18, 19].

Original: Various approaches provide guarantees of labelnoise identification through constraints on m ∗ k ( x ) including “purity" of classes [1, 11, 15, 18, 19].
Revised: Various approaches provide guarantees of labelnoise identification through constraints on m ∗ k ( x ) including “purity" of classes [1, 11, 15, 18, 19].

Original: Other methods provide guarantees on robust learning given constraints on the level of label noise[1, 4, 25], or level of noise in classification [2, 13], or only for binary classification [12], or restrictedto generalized linear models [20].
Revised: Other methods provide guarantees on robust learning given constraints on the level of label noise[1, 4, 25], or level of noise in classification [2, 13], or only for binary classification [12], or restrictedto generalized linear models [20].

Original: In contrast, we consider the unconstrained problem to analyze theeffect of label flips for any noise level and feature-dependent noise.
Revised: In contrast, we consider the unconstrained problem to analyze theeffect of label flips for any noise level and feature-dependent noise.

Original: Rather than proving guaranteesabout effectiveness of models under constraints; we show that when such constraints do not hold,classification degrades suddenly—not gracefully.
Revised: Rather than proving guaranteesabout effectiveness of models under constraints; we show that when such constraints do not hold,classification degrades suddenly—not gracefully.

Revised: Setting ¯ m ∗ k = 1 in our Corollary 3.2.1 yields the special case given in Corollary 1.1 of [2].

Revised: Ourgeneralization beyond the perfect classifiers (defined in our notation as m ∗ k ( x ) =

Revised: ¯ m ∗ k = 1 ) assumedin [2] is important because if we apply our result to the problem of identifying noisy labels (aspresented in [2]); then we see that the accuracy of predicting noisy labels is a function of m ∗ k ( x ) —i.e. accurate prediction of label noise depends on the features — even when label noise is independentof the features.

Original: Implications The standard assumption of label noise being independent of features can lead tooverly-optimistic findings on the robustness of classification algorithms.
Revised: Implications The standard assumption of label noise being independent of features can lead tooverly-optimistic findings on the robustness of classification algorithms.

Original: We can see that if label noiseis applied uniformly randomly in feature space (even if it is non-uniform across classes), then evenas the noisy likelihood begins to decrease, classification accuracy remains high until the likelihoodestimate decreases below the tipping point. 
Revised: We can see that if label noiseis applied uniformly randomly in feature space (even if it is non-uniform across classes), then evenas the noisy likelihood begins to decrease, classification accuracy remains high until the likelihoodestimate decreases below the tipping point. [Figure] [Equation] [Figure]

Revised: [Figure]

Original:  Synthetic Data Experiments
Revised:  Synthetic Data Experiments

Original: The theory is validated empirically on synthetic data for which P [ Y ∗ | X ] and PY | X  are known.
Revised: The theory is validated empirically on synthetic data for which P [ Y ∗ | X ] and P [ Y | X ] are known.

Original: Datais 2-d Gaussian with up to 10 classes, with some classes overlapping and others easily-separable asshown in Fig.
Revised: Data is 2-d Gaussian with up to 10 classes, with some classes overlapping and others easily-separableas shown in Fig.

Original: 2a.
Revised: 2a.

Original: We also empirically evaluate 5-d Gaussian data with up to 50 classes.
Revised: We also empirically evaluate 5-d Gaussian data with up to 50 classes.

Revised: All codewill be made available as open-source.

Original: For uniform noise, we sample y ∼ η ki according to Eq.
Revised: For uniform noise, we sample y ∼ η ki according to Eq.

Original: 4 for varying numbers of classes, c ∈{ 2 , 4 , 6 , 8 , 9 , 10 } , and noise ratio, ϵ ∈ { 0 , 0 .
Revised: 4 for varying numbers of classes, c ∈{ 2 , 4 , 6 , 8 , 9 , 10 } , and noise ratio, ϵ ∈ { 0 , 0 .

Original: For class-dependent noise, we sampley ∼ η ki according to Eq. 8 for varying numbers of classes, c ∈ { 2 , 4 , 6 , 8 , 9 , 10 } ; noise ratio,ϵ ∈ { 0 , 0 . 1 , . . .
Revised: For class-dependent noise, we sampley ∼ η ki according to Eq. 8 for varying numbers of classes, c ∈ { 2 , 4 , 6 , 8 , 9 , 10 } ; noise ratio,ϵ ∈ { 0 , 0 . 1 , . . .

Original: , 0 . 9 , 1 } ; and spread ∈ { 1 , 2 , 5 } .
Revised: , 0 . 9 , 1 } ; and spread ∈ { 1 , 2 , 5 } .

Original: For feature-dependent noise, we sample y in fivedifferent ways as summarized in Table 1.
Revised: For feature-dependent noise, we sample y in fivedifferent ways as summarized in Table 1.

Original: (1) For uniform-x , the probability of a given sample flippingis constant: η kk ( x ) = 1 − ϵ ; while the choice of label to flip to is proportional to the most-likelyincorrect label:
Revised: (1) For uniform-x , the probability of a given sample flippingis constant: η kk ( x ) = 1 − ϵ ; while the choice of label to flip to is proportional to the most-likelyincorrect label:

Original: η kj ( x ) ∝ P
Revised: η kj ( x ) ∝ P

Original: [ Y ∗ =
Revised: [ Y ∗ =

Original: j | x ] .
Revised: j | x ] .

Original: (2) For resampling noise, η kk ( x ) ∝ P [ Y ∗ =
Revised: (2) For resampling noise, η kk ( x ) ∝ P [ Y ∗ =

Original: k | x ] , in otherwords, samples that are least likely in a true class are the most likely to flip.
Revised: k | x ] , in otherwords, samples that are least likely in a true class are the most likely to flip.

Original: (3) For inverse resampling ,η kk ( x ) ∝ P [ Y ∗ =
Revised: (3) For inverse resampling ,η kk ( x ) ∝ P [ Y ∗ =

Original: k | x ] − 1 , i.e., samples with the highest likelihood in the true class are most likelyto flip.
Revised: k | x ] − 1 , i.e., samples with the highest likelihood in the true class are most likelyto flip.

Original: (4) For minimum-gap noise, η kk ( x ) ∝
Revised: (4) For minimum-gap noise, η kk ( x ) ∝

Original: [Equation]
Revised: P [ Y ∗ =

Revised: k | x ] max j ̸ = k P [ Y ∗ =

Original:  ] , so that the samples nearest decisionboundaries are the most likely to flip.
Revised: j | x ] , so that the samples nearest decisionboundaries are the most likely to flip.

Original: (5) For maximum-gap noise, η kk 
Revised: (5) For maximum-gap noise, η kk ( x ) ∝ max j ̸ = k P

Original: [Equation]
Revised: [ Y ∗ =

Original:  x ) ∝ [Equation]
Revised: j | x ] P [ Y ∗ =

Original:  , sothat the samples farthest from decision boundaries are the most likely to flip.
Revised: k | x ] , sothat the samples farthest from decision boundaries are the most likely to flip.

Original: There are 100 samples per class in the training set and 100 samples per class in the test set.
Revised: There are 100 samples per class in the training set and 100 samples per class in the test set.

Original: Thefeature matrices X train and X test are the same across all trials.
Revised: Thefeature matrices X train and X test are the same across all trials.

Original: The corresponding noisy label vectors
Revised: The corresponding noisy label vectors

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation] [Figure] [Figure] Y train and Y test are sampled for each combination of noise settings.
Revised: Y train and Y test are sampled for each combination of noise settings.

Original: We also evaluate the accuracyof the held-out clean test labels Y ∗ test .
Revised: We also evaluate the accuracyof the held-out clean test labels Y ∗ test .

Original: The noise level ϵ varies from 0 to 1 in 0 .
Revised: The noise level ϵ varies from 0 to 1 in 0 .

Original: 1 increments.
Revised: 1 increments.

Original: Aneural network with 2 hidden layers is trained; and further details of the architecture is given in the
Revised: Aneural network with 2 hidden layers is trained; and further details of the architecture is given in the

Original: Supplement.
Revised: Supplement.

Original: The model is trained 5 times starting from a different random seed; with the mean andstandard deviation of the accuracies reported.
Revised: The model is trained 5 times starting from a different random seed; with the mean andstandard deviation of the accuracies reported.

Original:  Affirming existing results [2, 17, 21], and our theoretical analysis, the similarity of Figure 4 to thetheoretical expectation of Fig.
Revised: Results Affirming existing results [2, 17, 21], and our theoretical analysis, the similarity of Figure 4to the theoretical expectation of Fig.

Original: 3 is striking.
Revised: 3 is striking.

Original: The neural network is robust to uniform label noise up toa point, where that tipping point depends on the number of classes.
Revised: The neural network is robust to uniform label noiseup to a point, where that tipping point depends on the number of classes.

Original: Figure 4 gives insight into what ishappening during training.
Revised: Figure 4 gives insight intowhat is happening during training.

Original: We see by the very low accuracies on the noisy test labels that the modelis not able to fit the noise from this uninformative noise distribution.
Revised: We see by the very low accuracies on the noisy test labels thatthe model is not able to fit the noise from this uninformative noise distribution.

Original: Yet, there is enough signal stillin the data to learn the patterns that generalize to the clean test set. 
Revised: Yet, there is enoughsignal still in the data to learn the patterns that generalize to the clean test set. Beyond the tipping [Figure]

Original: Beyond the tipping point (theminimizing point in the quadratic function of the noisy accuracy), the neural network begins fittingthe noise instead of the true signal, and the clean accuracy drops off steeply.
Revised: [Figure] point (the minimizing point in the quadratic function of the noisy accuracy), the neural networkbegins fitting the noise instead of the true signal [Figure] and the clean accuracy drops off steeply.

Original: Results on class-dependent noise as shown in Figure 5, also affirm existing results [13, 24, 27] andour theoretical analysis.
Revised: Results on class-dependent noise as shown in Figure 5, also affirm existing results [13, 24, 27] andour theoretical analysis.

Original: We see that the tipping point depends on the spread rather than the totalnumber of classes in the classification task.
Revised: We see that the tipping point depends on the spread rather than the totalnumber of classes in the classification task.

Original: The most harmful class-dependent noise occurs withspread=1, that is when labels from a given class flip to one specific erroneous label, the tipping pointoccurs when the noise level is 0 . 5 .[Figure] noise.
Revised: The most harmful class-dependent noise occurs withspread=1, that is when labels from a given class flip to one specific erroneous label, the tipping pointoccurs when the noise level is 0 . 5 .

Original: [Figure] [Table] With the introduction of feature-dependent label noise as seen in Fig. 6, we see that the neuralnetwork is not always robust even at low levels of label noise.
Revised: With the introduction of feature-dependent label noise as seen in Fig. 6, we see that the neuralnetwork is not always robust even at low levels of label noise.

Original: In particular, the worst-case labelnoise, GapMax, of Claim 3.7 significantly lowers classification accuracy even at ϵ = 0 .
Revised: In particular, the worst-case labelnoise, GapMax, of Claim 3.7 significantly lowers classification accuracy even at ϵ = 0 .

Original: 1 and 0 . 2 .
Revised: 1 and 0 . 2 .

Original: Interestingly, the classification accuracy from uniform-x noise is statistically indistinguishable fromclass-dependent noise; indicating that the dependence of η kk ( x ) on x is important. 
Revised: Interestingly, the classification accuracy from uniform-x noise is statistically indistinguishable fromclass-dependent noise; indicating that the dependence of η kk ( x ) on x is important. Fig.

Revised: 7 highlights the clean test accuracy of two label-noise mitigation methods: CleanLab [13] and

Original: Image Benchmark Experiments
Revised: CoTeaching [5]. More methods are detailed in the Supplement. At ϵ = 0 . 1 both strategies improveresults compard with the baseline of Fig.

Revised: 6c.

Revised: But for higher levels of ϵ , CleanLab has little effectwhile CoTeaching performs worse than baseline.

Revised: [Figure]

Revised: [Table]

Revised: [Table]

Original: We compare methods for learning with noisy labels, including CleanLab with the confident joint[13], MixUp [27], symmetric cross entropy (SCE) [3], and CoTeaching [5]; using the public-licenseopen-source code provided by the authors.
Revised: We compare methods for learning with noisy labels, including CleanLab with the confident joint[13], MixUp [27], symmetric cross entropy (SCE) [3] [Table] and CoTeaching [5]; using the public-licenseopen-source code provided by the authors.

Original: For all methods the base architecture is ResNet-50 [6]with more details including computational costs, and extended empirical results, in the Supplement.
Revised: For all methods the base architecture is ResNet-32 [6]with more details including computational costs, and extended empirical results, in the Supplement.

Original: We use the classification benchmarks CIFAR-10 and CIFAR-100 of 32x32-pixel color images in 10or 100 classes, with 60,000 images per dataset [9].
Revised: We use the classification benchmarks CIFAR-10 and CIFAR-100 of 32x32-pixel color images in 10or 100 classes, with 60,000 images per dataset [9].

Original: Class-dependent noisy labels Y
Revised: Class-dependent noisy labels Y

Original: | Y ∗ are generatedaccording to Definition 3.4, assuming that the labels given in the dataset are the true labels Y ∗ .
Revised: | Y ∗ are generatedaccording to Definition 3.4, assuming that the labels given in the dataset are the true labels Y ∗ .

Original: Togenerate label noise Y
Revised: Togenerate label noise Y

Original: | X that is dependent on features but independent from the learned classificationfunction itself, we use non-learned similarity-preserving image hashes [14, 26].
Revised: | X that is dependent on features but independent from the learned classificationfunction itself, we use non-learned similarity-preserving image hashes [14, 26].

Original: For each image [Table] in the dataset, we compute the similarity-preserving hash; which puts visually-similar (rather thansemantically-similar) images near each other.
Revised: For each image inthe dataset, we compute the similarity-preserving hash, h ( x ) ; which puts visually-similar (rather thansemantically-similar) images near each other.

Original: Then for each class, we calculate the average hashwhich represents the “center" of the class.
Revised: Then for each class, we calculate the average hash, ¯ h k ,which represents the “center" of the class.

Original: For class-conditional noise, we generate label flips betweenclasses that are s -nearest neighbors based on the hash centers and spread s .
Revised: For class-conditional noise, we generate label flips betweenclasses that are s -nearest neighbors based on the hash centers and spread s .

Original: For feature-dependentnoise, we generate label flips  based on the distance from a particular sample’s hash to the s -nearestneighbor hash centers.
Revised: For feature-dependentnoise, we generate label flips using the GapMax function of Table 1 based on the distance from aparticular sample’s hash to the s -nearest neighbor hash centers, i.e. P [ Y ∗ = k | x ] ∝ 1 /δ k ( x ) where [Equation]

Original: 5.1 Results
Revised: Results

Original: The results for class-dependent label flips on CIFAR-10 in Table 2 show general patterns of whatour theory predicts: accuracy drops severely between ϵ = 0 .
Revised: The results for class-dependent label flips on CIFAR-10 in Table 2 show general patternsof what our theory predicts: accuracy drops severely between ϵ = 0 .

Original: 4 and ϵ = 0 .
Revised: 4 and ϵ = 0 .

Original: 6 when s = 1 .
Revised: 6 when s = 1 .

Original: For s = 2 ,the drop is less severe until ϵ = 0 . 8 .
Revised: For s = 2 , the drop is less severe until ϵ = 0 . 8 .

Original: Also supporting our theory, the results for feature-dependentlabel flips show more gradual decline in accuracy so that on average the accuracy is lower forfeature-dependent label flips than for class-dependent label flips when ϵ ∈ { 0 . 2 , 0 . 4 } , as well as forhigher values of ϵ when spread, s ∈ { 2 , 5 } .
Revised: Also supporting our theory, the results for featuredependent label flips show more gradual decline in accuracy so that on average the accuracy is lowerfor feature-dependent label flips than for class-dependent label flips when ϵ ∈ { 0 . 2 , 0 . 4 } , as wellas for higher values of ϵ when spread, s ∈ { 2 , 5 } .

Original: Table 3 shows that similar trends hold for  adataset witha much high number of classes. 
Revised: Table 3 shows that similar trends hold for adataset with a much high number of classes. It is interesting to note

Revised: that when a mitigation strategyis performing poorly (e.g. SCE on CIFAR10 at ϵ = 0 . 2 or CleanLab on CIFAR100 at ϵ = 0 . 2 )compared with the other methods; then it does not seem to matter whether the noise is featuredependent or class-dependent.

Original: Therefore, we see that  the robustness of classifiers with  label noisemitigation strategies behaves differently whether that noise is feature-dependent.
Revised: Therefore, we see that in general the robustness of classifiers with successful label noise mitigation strategies behave differently whether that noise is feature-dependent.

Original: Feature-dependentnoise reduces classification accuracy at lower levels of noise.
Revised: Feature-dependent noise reduces classification accuracy at lower levels of noise.

Original: CleanLab and MixUp show the strongest results, but there is no clear winner among mitigationstrategies for label noise; as all methods display weaknesses under various noise levels or noise types.
Revised: CleanLab and MixUp show the strongest results, but there is no clear winner among mitigationstrategies for label noise; as all methods display weaknesses under various noise levels or noise types.

Revised: One promising result is that the mitigation strategies tend to narrow the gap in performance betweenhandling class noise and feature noise; which is consistent with our findings on the synthetic data in

Original:  The strong performance of MixUp on CIFAR-100 is probably due to it being particularly well-suitedto the hierarchical nature of the classes [22, 27].
Revised: Figure 7 . The strong performance of MixUp on CIFAR-100 is probably due to it being particularlywell-suited to the hierarchical nature of the classes [22, 27].

Original: Whereas, CleanLab struggles with CIFAR-100probably because the low baseline accuracy even at low levels of noise makes it difficult to confidentlypredict clean versus noisy samples.
Revised: Whereas, CleanLab struggles with CIFAR-100 probably because the low baseline accuracy even at low levels of noise makes it difficultto confidently predict clean versus noisy samples.

Original:  Conclusion
Revised:  Conclusions

Revised: Limitations Our theoretical results are limited to the Bayes optimal classifier.

Revised: As with most theory,the degree to which these results apply in practice must be explored empirically.

Revised: Future Directions

Revised: The theory could be extended to explain why learning from massive, noisydatasets can be so successful, especially when the noisy dataset is used as a pre-training task: majorpatterns in data can be learned despite label noise as long as there is some signal within the noise.

Revised: Our theory could also be used to guide the curation of datasets as they are being collected.

Original:  In both the uniform and class-dependent label flip scenarios, classification is robust to label noise upto a tipping point; and beyond that tipping point, classification fails catastrophically. 
Revised: Not alllabels need to be clean, but enough labels in every class across the relevant feature space do needto be clean. Data-specific approaches may look for regions of feature space where labels tend to benoisy and focus more effort on cleaning labels for those regions. Conclusion In both the uniform and class-dependent label flip scenarios, classification is robust tolabel noise up to a tipping point; and beyond that tipping point, classification fails catastrophically. These theoretical findings are helpful in designing evaluation approaches for label noise detectionand mitigation.

Revised: Considering that label noise mitigation strategies can require training 2 models (suchas for CoTeaching) or even 6 models (such as for CleanLab); we can see why empirical studies arecomputationally limited to investigating just a few variations on noise level and shape.

Revised: We nowknow that levels of noise near where the tipping-point occurs will be most informative; that thereis little reason to evaluate various levels of spread of class-dependent noise, and that incorporatingfeature-dependent noise is most informative for practical applications.

Original: Classificationaccuracy is significantly lower at small levels of noise when the noise distribution depends on featuresand is targeted at samples with the highest gap between the likelihood of the true label and the nextmost-likely label.
Revised: Classification accuracy is significantly lower at small levels of noise when the noise distributiondepends on features and is targeted at samples with the highest gap between the likelihood of thetrue label and the next-most-likely label.

Original: While various label noise mitigation strategies provide guarantees about conditionsunder which they work; we see that the mitigation strategies on real data generally fail for similarscale and shape of label noise distributions.
Revised: While various label noise mitigation strategies provideguarantees about conditions under which they work; we see that the mitigation strategies on real datagenerally fail for similar scale and shape of label noise distributions.

Original: Realistically, we expect that label noise does depend onthe features; and this paper demonstrates theoretically and empirically that the old assumption oflabel noise—being independent of features given the class—leads to an overly optimistic expectationof classifier robustness in the face of label noise.
Revised: Realistically, we expect thatlabel noise does depend on the features; and this paper demonstrates theoretically and empirically thatthe old assumption of label noise—being independent of features given the class—leads to an overlyoptimistic expectation of classifier robustness in the face of label noise.

Original: While it is challenging to model feature-dependentlabel noise; it is clearly necessary as we see significantly lower classification accuracy when labelnoise is feature-dependent.
Revised: While it is challenging tomodel feature-dependent label noise; it is clearly necessary as we see significantly lower classificationaccuracy when label noise is feature-dependent.