Title: AN ENERGY-BASED FRAMEWORK FOR ARBITRARY LABEL NOISE CORRECTION Anonymous authors Paper under double-blind review

Abstract: We propose an energy-based framework for correcting mislabelled training examples in the context of binary classification. While existing work addresses random and class-dependent label noise, we focus on feature dependent label noise, which is ubiquitous in real-world data and difficult to model. Two elements distinguish our approach from others: 1) instead of relying on the original feature space, we employ an autoencoder to learn a discriminative representation and 2) we introduce an energy-based formalism for the label correction problem. We prove that a discriminative representation can be learned by training a generative model using a loss function comprised of the difference of energies corresponding to each class. The learned energy value for each training instance is compared to the original training labels and contradictions between energy assignment and training label are used to correct labels. We validate our method across eight datasets, spanning synthetic and realistic settings, and demonstrate the technique's state-of-the-art label correction performance. Furthermore, we derive analytical expressions to show the effect of label noise on the gradients of empirical risk.

1. INTRODUCTION:

Machine learning algorithms depend on reliable training labels or, when applicable, sufficiently robust learning models in order to produce generalizable predictions (Zhu & Wu, 2004; Frénay & Verleysen, 2014) . Many empirical datasets suffer from training label corruption, which can result from annotation error, human bias, or a noisy process for generating labels (Smyth, 1996; Brodley & Friedl, 1999) . In practice, it can be costly to obtain noise-free labels; hence, research on reducing the effects of label noise on learning has received considerable attention.

Most work, however, focuses on label noise that is independent of the input features (e.g., random label noise or class conditional random noise). The introduction of feature dependency significantly complicates mathematical analysis (Natarajan et al., 2013; Liu & Tao, 2014; Northcutt et al., 2017) . Although work in this domain has yielded impressive results and improved generalization (Rebbapragada & Brodley, 2007a; Natarajan et al., 2013; Patrini et al., 2017; Rolnick et al., 2017; Ren et al., 2018) , the simplicity of these noise assumptions fails to capture crucial mislabelling processes that arise in practice. In this work, we propose a semi-supervised framework for correcting the effects of feature dependent label noise on supervised learning algorithms.

Label noise processes are categorized into three types, as illustrated in Figure 1 . Type I refers to a noise model where any label in the training data is incorrect with probability γ. In the case of type II noise, the probability of label corruption is conditioned on the class, such that we have different noise rate γ 0 and γ 1 for each class. Finally, type III noise models describe how label noise depends explicitly on input features. Feature dependent label noise or equivalently type III noise is a more realistic type of label noise that is ubiquitous in empirical datasets (Lachenbruch, 1974; Schafer & Graham, 2002; Frénay & Verleysen, 2014) . As an example of type III noise, consider the diagnostic labels of Alzheimer's disease. In this setting, the probability of label noise depends on both age and sex (i.e., younger male patients are harder to diagnose) (Khachaturian, 1985; Murray et al., 2016) . Type III noise also includes the prevalence of unreliable labels for training instances in low density regions of feature space (Denoeux, 1995; 1997; 2000) or near classification boundaries (Lachenbruch, 1966; 1974; Chhikara & McKeon, 1984; Cohen, 1997; Beigman & Klebanov, 2009; Beigman Klebanov & Beigman, 2009; Kolcz & Cormack, 2009) .

Figure 1 : Different categories of label noise and their statistical dependencies, as depicted by the red arrow. In type I noise, all instances are equally likely to be mislabelled base on some probability γ ∈ [0, 1 2 ]. In the case of type II, this probability is different for each class: γ 0 ∈ [0, 1 2 ] and γ 1 ∈ [0, 1 2 ]. Type III label noise is the most realistic model and yet the least studied. In this case the probability of an error is a function of the input features: i.e. p error ∼ f (x).

In this paper, we present a practical framework for correcting label noise that extends beyond type I and type II noise. We train an energy-based generative model on a small subset of the training data with reliable labels, either manually annotated or automatically learned. This model is used to identify and correct mislabelled examples based on the whether the assigned energy of a given point is compatible with its original label. We empirically demonstrate our framework's ability to handle input dependent label noise on both simulated and real datasets.

2. RELATED WORK:

The full body of work on the problem of label noise is too extensive to review here; however, we direct interested readers to the detailed review by Frénay & Verleysen (2014) . In this section, we highlight key contributions in the label noise literature. We then briefly discuss existing theoretical work.

2.1. LABEL NOISE CORRECTION:

Existing work in label noise correction is designed for Type I and Type II noise and falls into three categories: relabeling, learning procedures and loss functions. We expand on each area below.

Relabeling: Earlier approaches to label noise focus on relabeling the noisy training set. Brodley & Friedl (1999) use the output of an ensemble of classifiers to identify mislabeled training examples. Sun et al. (2007) identify mislabeled instances based on the entropy of class probabilities outputted by a Bayesian classifier. These approaches, while robust to simpler label noise models, do not account for Type III noise.

Learning Procedures: Methods modifying the learning procedure include the perceptron algorithm with margin (PAM) Frénay & Verleysen (2014) . Crammer & Lee (2010) use a velocity-based learning procedure to learn the weight vector distribution, termed gaussian herding (NHERD). Sukhbaatar et al. (2014) introduce a noise layer to a neural network architecture to learn the noise function, while Rebbapragada & Brodley (2007b) weight each example by class confidence in the training procedure. These methods commonly suffer from overfitting to the noise and again, primarily focus on Type I and Type II noise.

Loss Functions: Long & Servedio (2010) have shown classification algorithms that optimize a convex potential over a linear class are not robust to random label noise. This has led to work which aims to modify convex loss functions in order to make them noise-tolerant in the presence of type I and type II noise. Ghosh et al. (2015) derived sufficient conditions for classification losses that render them robust to random noise; namely, the components of a loss (where each component is defined over a given class) must sum to a constant value. Rooyen et al. (2015) proposed a convex loss that avoids the negative result of Long & Servedio (2010) for type I noise by virtue of being negatively unbounded. Natarajan et al. (2013) have shown that risk minimization over the corrupted data is consistent with risk minimization over clean data provided that the standard convex loss (e.g., binary cross entropy) is modified appropriately to produce an unbiased loss estimator (ULE). Each of these loss functions is presented for and validated on Type I and Type II noise.

2.2 EXISTING THEORETICAL GUARANTEES (Bylander, 1994; Blum et al., 1998; Blum & Mitchell, 1998) offer guarantees for hypothesis generalization in the face of Type I and Type II noise. Angluin & Laird (1988); Bylander (1997; 1998) ; Servedio (1999) present guarantees for type III noise, where the probability of error depends on the distance to the margin. We introduce a gradient-based interpretation of label noise to these existing results. training examples (x 1 , y 1 ), (x 2 , y 2 ), . . . , (x n , y n ) have been drawn, where (x i , y i ) ∈ R d × {0, 1}. The clean and corrupted training datasets are denoted as

3. BINARY CLASSIFICATION IN THE PRESENCE OF LABEL NOISE:

EQUATION

respectively, where due to some label noise process y i → ỹi . Hence, we have access to the noisy data T during training instead of the clean data T . It can be assumed that the corrupted samples (x i , ỹi ) are drawn from a noisy distribution P .

In supervised binary classification, we aim to learn a discriminator f : R d → R that minimizes the risk with respect to a given loss function1 . Formally, we want to minimize the empirical risk

is R [ , f, T ] = 1 m m i=1 (f (x i , θ), y i ),

where the m, , θ denote the size of the mini-batch size, loss function, and the learnable model parameters respectively. Gradient based learning algorithms compute ∇ θ ( R [ , f, T ]) and update model parameters. For example, as in mini-batch gradient descent,

EQUATION

where η is the learning rate. In practice, we may not have access to the clean data T . Instead we must learn the discriminator using the noisy data T . Thus, the second term in Eq. 2 becomes ∇ θ ( R[ , f, T ]), which denotes the risk with respect to the noisy training data.

3.2. LABEL NOISE EFFECT ON RISK MINIMIZATION:

We can describe the effect of label noise on learning by expressing the noisy gradient ∇ θ ( R[ , f, T ]) in terms of the true gradient ∇ θ ( R[ , f, T ]) and a noise term. In the case of type I noise where any label y i can be flipped to 1 -y i with probability γ ∈ [0, 1/2], we have

EQUATION

where

E true ≡ E (x,ỹ)∼ P [∇ θ R[ , T ]]

is the true (i.e. noise-free) gradient, and E rand is the completely noisy term obtained by replacing all targets y i with a random value of either 0 or 1. For type II noise, we have an equation analogous to Eq. 3, where γ is replaced by the average γ = (γ 0 + γ 1 )/2 noise rate and we have an additional term reflecting the noise imbalance between each class when γ 0 = γ 1 . For more information on this and derivations of these equations, see Appendix A.

From Eq. 3 we deduce that for small γ the true gradients dominate the dynamics of searching θ space for optimal parameters. As γ grows, the true gradients are scaled down by the factor (1 -2γ), which means that the contribution of E true diminishes and the random perturbation term E rand starts to dominate as γ grows. A similar conclusion can be reached for type II noise (replacing γ with γ) where we have an additional term modifying the true gradients based on the class imbalance (see A.2).

The effect of E rand can be negated by increasing the amount of training data proportionately with γ as discussed by Rolnick et al. (2017) : i.e. as the probability of a random error occurring increases, the model needs more data to make progress in the direction of the true gradient. We can neutralize the effects (i.e. suppress E rand ) of random noise (type I) or class conditional random noise (type II) by increasing the number of training examples and by utilizing a robust loss function when possible (Natarajan et al., 2013) .

In contrast, Type III noise is substantially more difficult to address because the available training data T can have optimal discriminators that are significantly different from the true discriminator corresponding to the clean data T . Separating Type III noise into the true term and the perturbation term necessitates restrictive assumptions on the noise model (See Appendix A. In Figure 2 , we show this by visualizing SGD over the true training set (gold lines) and the noise training set (blue lines). As shown in Figure 2 , increasing the size of the training set is effective in address Type II noise. This strategy is not only ineffective for addressing type III noise, but they can also lead to our model learning the biased discriminator; thus, hindering any possibility of generalization.

4. AN ENERGY-BASED FRAMEWORK FOR BIAS CORRECTION:

Type III noise is particularly challenging because noise can depend on the input features in an arbitrarily complex way. Modeling this requires significant information about the noise process, which can be difficult to obtain in practice. In order to deal with this challenge we will avoid training directly on the mislabelled instances. Instead, we start our training procedure by assuming there exists a small subset of T that is noise-free (e.g. 1% of the data is sufficient the MNIST dataset). Such a set can be obtained via a manual labelling process or by using domain knowledge to identify the appropriate data points2 .

Our objective is to correct mislabelled training instances such that the corrected data leads to the improved generalization of a trained model. We will demonstrate in Section 5 that our framework extends beyond Type I and Type II noise to Type III noise, in realistic settings. In particular, we demonstrate the method's strength in correcting algorithmically assigned labels.

The proposed correction framework consists of three steps:

1. Obtain known labels: There are two ways of identifying instances that have correct labels.

The first approach is to use domain knowledge -frequently, one has access to a subset of clean labels and their instances. We can use this subset as our clean data for step 2. If this set is unknown or too small, then we can inference which examples are likely to be correctly labeled, as is done in Ding et al. (2018) . We report results on both approaches in Section 5.

2.. Train semi-supervised model::

The framework uses the filtered data from the previous step to train a semi-supervised algorithm that learns to classify instances based on similarity between features. In contrast with supervised learning, where the goal is to learn p(y|x), we want to learn which features are most compatible with a given label: i.e. p(x|y). Thus, we train a generative model (e.g. energy-based autoencoder) to use feature similarity (as quantified by an energy function) to identify which instances belong to the specific class. 3. Identify and correct mislabelled instances: The features of the full training data (with mislabeled targets) are injected into the previously trained semi-supervised model resulting in each instance being assigned an energy value based on the output of the model. The energy corresponding to each instance serves as a proxy for class assignment (i.e. low energy corresponds to y = 0 whereas high energy corresponds to y = 1). Contradictions between energy assignment and training labels are used to correct the training data such that all labels are compatible with their assigned energy.

We first motivate the design decisions involved in this framework. Namely, we explore our choice of contrastive divergence as a loss function and empirically justify our use of an autoencoder. In the subsequent section, we elaborate on our implementation of the framework.

4.1. FRAMEWORK MOTIVATION:

Given a small, clean dataset T ⊂ T , we aim to train a model which can be used to determine labels on the remaining, noisily labelled dataset. One obvious approach is to train a binary classifier directly and use it to correct labels. However, as discussed by Berthelot et al. (2017) , binary classification provides a relatively weak training signal which largely ignores the intricacies of the input feature distribution. See Fig. 3 where we compare the class separation achieved by a few standard binary classifiers with our proposed approach. Alternatively, one could train class conditional models. However, such models are trained without knowledge of the primary task, differentiating classes. Further, traditional unsupervised training regimes typically use a maximum likelihood formulation (i.e. forward KL divergence) which is prone to distributing probability mass broadly (the so-called "mean-seeking behaviour" (Murphy, 2012)) and further limit the discriminative ability of the learned model. 

EQUATION

) where p 0 (x) and p 1 (x) denote the target probability distributions conditioned on class 0 and 1 respectively and q(x; θ) denotes the learned model. Thus, training aims to produce a model q (x; θ) which assigns high probability to samples from class 0, while giving low probability to samples from class 1. In theorem 1, we show that a finite sample approximation of the optimization objective in Eq. 4 can be computed as

EQUATION

where T y , for y ∈ {0, 1}, contains instances from class y and T y ⊆ T , |T y | denotes the number of elements in T y and E(x; θ) is the energy of the model q (x; θ), i.e., q (x; θ) = exp (-E(x; θ)) /Z(θ) where Z(θ) is the normalizing partition function. Analogously, we can define Eq. 5 over each mini-batch. This contrastive loss have been applied successfully in adversarial training, resulting in faster and improved stability of training, enhanced generator quality, and improved generator diversity (Zhao et al., 2016; Berthelot et al., 2017) . As described in section 2.2 of (LeCun et al., 2006) , the negative log-likelihood function, E(x; θ) = -log q (x; θ), constitute a valid energy which has been used in numerous energy-based models [e.g. (Bengio et al., 2003; Zhao et al., 2016; Berthelot et al., 2017) ].

Theorem 1. The contrastive divergence CD (θ) in Equation 4has the finite approximation L (θ) up to some constant k which is independent of θ.

Proof. Using the definition of KL divergence, we expand Equation 4: where the two entropy terms x p 0 (x) log p 0 (x)dx and x p 1 (x) log p 1 (x)dx have been grouped in the constant k as they do not depend on the optimization parameters θ. Assume the Gibbs form for the model distribution,

EQUATION

EQUATION

where E(x; θ) is the energy and Z(θ) = x e -E(x;θ) dx is the partition function. Then, Equation 6becomes

EQUATION

where the partition function terms conveniently cancel out due to the normalization of p 0 and p 1 . Therefore,

CD (θ) = E p0(x) [E (x)] -E p1(x) [E (x)] + k (10)

and we conclude that CD (θ) has the finite approximation L (θ) as defined in Equation 5, up to a constant k.

4.2. FRAMEWORK IMPLEMENTATION: CONTRASTIVE AUTOENCODER:

Using the clean subset T , we implement step 2 of our framework by using an energy-based autoencoder (analogous to Zhao et al. (2016) ). In order to ensure stable training and to prevent the second term of Equation 5 from diverging, we balance our mini-batch to have an equal number of positive and negative class samples. After training is complete, we process the original training data T using the trained autoencoder. Specifically, we compute the energy using the resulting outputs: E (x; θ) = -log q (x; θ), which reduces to the reconstruction loss corresponding to the autoencoder (Goodfellow et al., 2016) . For example, if the underlying distribution is assumed to be Gaussian then, we have E (x; θ) = xx(θ) 2 , where x(θ) is the output of the autoencoder3 . Whereas, if the underlying distribution is assumed to be Bernoulli, we obtain

E (x; θ) = -[x log x(θ) + (1 -x) log(1 -x(θ))].

To execute our label correction protocol, we impose the following two conditions. (i) if E(x; θ) > Ē1 -a for x ∈ T0 , then change label of x. Here we denote T0 as the originals negative samples, Ē1 as the mean energy over the set of clean positive samples T 1 , and a is a tuneable hyperparameter that determines how aggressively we want to change 0 → 1. Analogously, we impose condition (ii) if E(x; θ) < Ē0 -b for x ∈ T1 , then change label of x. The rationale is to modify samples where the original label ỹ contradicts the energy assignment. When one is not able to tune the threshold hyperparameters a and b, an empirical heuristic is to set them equal to the standard deviation about the means Ē1 and Ē0 respectively. See Appendix B details and pseudo-code.

5. EXPERIMENTS:

We conduct experiments with simulated data and real-world data where three different type III noise models are studied: (i) Linear noise (p error ∼ αx i ): the probability of an error occurring depends linearly on a feature (Table 1 ). (ii) Quadratic noise (p error ∼ αx 2 i ): the probability of an error occurring depends on the square of a feature. We control the amount of noise added with α and we select x i if it has sufficient variance to make the noise model distinct from random noise, i.e. 0 variance implies random noise (Table 2 ). (iii) Boundary noise: the probability of label error depends on the distance from the class boundary, which is determined using the noise-free data. We report the average class-weighted F1-score along with the standard error over 10 runs with random train-test (80:20) splits (Table 3 ).

UCI benchmark datasets::

We train logistic regression on the corrected dataset resulting from the proposed algorithm. If a small clean dataset is not initially provided, then our results are labelled as AE (learned), otherwise they are labelled at AE (known) -see step 1 in Sec. 4. We compare to noise robust algorithms: NHERD Crammer & Lee (2010) , PAM Frénay & Verleysen (2014) , and ULE Natarajan et al. (2013) . Additionally, we report upper and lower bounds by training logistic regression on the noisy data (LR-N) and on the clean data (LR-C). All algorithms are tested on noisefree data. Even without access to any clean labels (i.e., we learn which labels are likely clean), our framework generally outperforms the other algorithms in the presence of linear noise and quadratic noise. In general, learning which labels are noisy following the method in Ding et al. (2018) is not possible for arbitrary label noise processes: e.g., when the optimal discriminators of T are very different than the optimal discriminators of T . For boundary noise, we show that using a small clean dataset T enables our methods to learn in the presence of boundary noise, where the other methods generally breakdown (Table 3 ). For a fair evaluation, the benchmark algorithms (which don't have access to clean data) should be compared to AE (learned).

MNIST: Next, we compare our method with another state-of-the-art noise robust algorithm that relies on clean data, i.e., the loss weighting scheme of Ren et al. (2018) . Both methods are given the same percentage of clean data for each noise setting. The task is to classify the distinguish "3" (class 0) vs. other digits (class 1). The noise process is input dependent as it changes 4, 5, 6 to class 0 depending on a specified noise rate (α). We demonstrate (Table 4 ) that our method generalizes better in the presence of type III noise, even when starting with only 1% of the data that is clean.

Arrhythmia We use the MIT-BIH Arrhythmia dataset Goldberger et al. (2000) to evaluate the proposed method's ability to correct algorithmically assigned labels. Algorithmically-assigned labels (AALs) are prevalent in domains with abundant unlabeled data and high labelling costs. Common applications include web page annotation, but the value of this approach extends to automatic annotation of images and natural language. We employ the data preprocessing procedure described by Mondéjar-Guerra et al. (2019) , where each arrhythmia is described by 59 features.

We divide the dataset into three parts: T 1 , T 2 and T 3 . T 1 and T 2 serve as training sets and T 3 is the test set. We train a classifier on T 1 and subsequently use that classifier to generate AALs for T 2 . Thus, the predictions of the trained classifier become ỹ and the original expert labels remain y. As demonstrated in 

6. CONCLUSION:

We have proposed an energy-based framework to correct mislabelled training instances. By minimizing a contrastive loss, the proposed method learns a representation that is valuable in relabeling noisy training sets. We evaluate the proposed model across six datasets and three noise models to demonstrate the method's empirical value in correcting feature-dependent label noise. Furthermore, we demonstrate our method's improvement upon existing work in making machine learning more robust to label noise processes.

APPENDICES A EFFECT OF LABEL NOISE ON EMPIRICAL RISK:

Here we show how label noise affects optimization of empirical risk. Recall that our task is to improve the labels of a training set in order to enable effective learning. In the following sections, we explore the Type I, Type II, and Type III noise. We first introduce the key variables.

We focus on binary classification, where the probability of class 0 is p and consequently, the probability of class 1 is (1 -p):

p(y = 0) = p p(y = 1) = 1 -p T represents the training set with correct labels, while T represents the training set with label noise.

In our setting, we are given T and propose a method to transform T into T . We will call D the true label distribution and P the noisy label distribution.

T = (X i , Y i ) for i = 1, 2, . . . , n T = (X i , Ỹi ) for i = 1, 2, . . . , n

In the following calculations, we describe the difference in empirical risks over each training set and the subsequent effect on stochastic gradient descent. We refer to the empirical risk over the clean training set as R [ , T ]. Similarly, we refer to the empirical risk over the corrupted training set as R , T .

R [ , T ] = 1 n n i=1 (Φ(X i , θ), Y i ) ,

In order to explain the effects of label noise on stochastic gradient descent (SGD), we are primarily interested in the gradient of the empirical risk with respect to θ. Below, we break the empirical risk term into two class-dependent terms and introduce short-hand to represent each of these terms.

E (X,Y )∼D ∇ θ R[ , T ] = E (X,Y )∼D ∇ θ 1 n n i=1 (Φ(X i , θ), Y i ) (11) = ∇ θ 1 n n i=1 E (X,Y )∼D [ (Φ(X i , θ), Y i )] (12) = ∇ θ 1 n n i=1 p(x i , y i ) (Φ(x i , θ), y i ) dx i dy i (13) = ∇ θ 1 n n i=1 p(x i |y i )p(y i ) (Φ(x i , θ), y i ) dx i dy i (14) = p(y i = 0)∇ θ 1 n n i=1 p(x i |0) 0 (Φ(x i , θ)) dx i (15) + p(y i = 1)∇ θ 1 n n i=1 p(x i |1) 1 (Φ(x i , θ)) dx i ,

We define shorthand based on Equation (5):

β 0 = ∇ θ 1 n n i=1 p(x i |0) 0 (Φ(x i , θ)) dx i (16

)

EQUATION

)

Rewriting the original equation in terms of the above, we arrive at the following form for empirical risk:

EQUATION

)

We revisit Equation 8 in the follow sections to interpret empirical risk under varying noise models. Note that β 0 and β 1 are the same in both the expected empirical risk with respect to the true label distribution (D) and the corrupted label distribution ( P ) for Type I and Type II noise. Additionally, we introduce shorthand for the true expectation of empirical risk (E true ) and the expectation of empirical risk when labels are assigned at random (E rand ).

EQUATION

A.1 TYPE I: RANDOM NOISE

In the random noise scenario, each label has an equal probability of being flipped. We parameterize this noise by γ, resulting in :

EQUATION

We may define both p(ỹ i = 0) in terms of γ and p:

EQUATION

Likewise, we define p(ỹ i = 1) as:

EQUATION

Below, we substitute these expressions into Equation 8:

EQUATION

A.2 TYPE II NOISE Type II noise implies class-dependent label noise. Thus, we define γ 0 and γ 1 , the class-dependent noise rates. In addition, we define γ * as the average class-dependent noise rate and as the absolute distance of γ 0 and γ 1 from γ * . Without loss of generality, we assume γ 0 > γ 1 .

EQUATION

We may redefine p(ỹ i = 0) and p(ỹ i = 1) using these terms:

EQUATION

Plugging this into Equation 8,

E (X, Ỹ )∼ P ∇ θ R[ , T ] = (p -γ * p -p + γ * -γ * p -+ p)β 0 (42) + (1 -γ * + -p + γ * p -p + γ * p + p)β 1 = (p -2γ * p + γ * -)β 0 (43) + (1 -2γ * + -p + 2γ * p + γ * )β 1

Similar to our approach with Type I noise, we aim to rephrase this equation in terms of E true and E rand .

EQUATION

A.3 TYPE III NOISE

In this section, we analyze a specific case of type III noise where label corruption is restricted to a given region of the feature space. As above, we want to study the effect on learning by determining how the true gradient is modified in the presence of noise. We assume a scalar input space and define the feature dependent noise as follows:

p(ỹ i |x) = γ x ∈ [a, b] 0 x / ∈ [a, b]

The noise model above asserts that label noise depends only on the feature space and is label- p(y i |x i )p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ i

B ALGORITHM PSEUDO-CODE:

In this section, we include a more detailed form of the algorithms proposed in the paper. B i = {x j , y j } ← GetBalancedBatch (T )

3: if y i = 1 and e i < Ē0 + σ 0 then 11:

O i = { xj } ← Forward (B i , θ i ) 4: i ← L (B i , O i ; θ i ) L (B i , O i ; θ i ) is

y i ← 0 12:

else if y i = 0 and e i > Ē1 -σ 0 then 13:

y i ← 1 14:

end if 15: end for

:

In practice the loss function is usually classification-calibrated to minimize the 0-1 loss given that the sample size is sufficiently large(Bartlett et al.,

2006).

This assumption can be relaxed in situations where we can learn which labels are likely to be clean. E.g. seeDing et al. (2018)

More precisely, E (x; θ) = xx(θ) 2 up to an irrelevant scaling factor that we can drop for simplicity.

Figures and Tables:

PROBLEM SETUP AND NOTATIONLet P denote the true distribution from which n i.i.d.

Figure 2: Plots showing the path traversed in θ space by a supervised model as learnable parameters are updated via a gradient based optimizer. The gold line shows the path followed by a noise-free model and the blue lines show the modified paths due to label noise. In the case of random label noise (left plot), the perturbations E rand to the true gradient direction are curbed by an increase in training set size. In contrast, for input dependent noise (right plot), supplying the model with more data perpetuates the label bias.

Instead, we prioritize learning to discriminate the underlying classes based on aspects of the input feature distributions. To do this, we propose to use a contrastive divergence(Carreira-Perpinan & Hinton, 2005) training loss: CD

Figure 3: In the first row, we visualize the separation achieved by classifiers trained on a training set with noisy labels. From left to right, we show the results of logistic regression, an SVM, a 1 layer feed-forward neural network and the proposed technique. In the second row, we visualize the training data. From left to right, we show the clean data, the noisy data and the autoencodercorrected data. We observe that standard binary classifiers are unable to achieve satisfactory class separation. The 1 hidden layer NN classifier accomplishes significant separation but it incorrectly identifies mislabelled instances. The proposed model is able to identify mislabelled instances based on the energy distribution of the training data.

agnostic -thus, examples from both class 1 and class 0 with x ∈ [a, b] are equally likely to suffer from a label flip.RevisitingE (X, Ỹ )∼ P ∇ θ R[ , T ] = ∇ θ i |x i )p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ i , i |x i )p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ i i |x i )p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ i .Now we add and subtract the following term:i |x i )p(x i ) (Φ(x i , θ), ) dx i dỹ i(47)In order to obtain:E (X, Ỹ )∼ P ∇ θ R[ , T ] = E true + ∇ θ i |x i )p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ i i |x i )p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ i = E true + ∇ θ ỹ i |x i ) -p(y i |x i )) p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ iWhere we have used the fact that:E true = ∇ θ i |x i )p(x i ) (Φ(x i , θ), ỹi ) dx i dỹ i

the proposed technique achieves a higher F1 score than competing ± 0.03 0.91 ± 0.01 0.76 ± 0.01 0.90 ± 0.02 0.94 ± 0.01 0.95 ± 0.01 0.96 ± 0.01 Diabetes 5 , 1.0 0.60 ± 0.03 0.50 ± 0.01 0.48 ± 0.03 0.60 ± 0.03 0.64 ± 0.01 0.70 ± 0.02 0.72 ± 0.02 German 1 , 1.2 0.67 ± 0.03 0.72 ± 0.01 0.49 ± 0.03 0.63 ± 0.01 0.67 ± 0.03 0.73 ± 0.01 0.76 ± 0.02 Image 1 , 0.7 0.61 ± 0.03 0.63 ± 0.01 0.54 ± 0.01 0.61 ± 0.02 0.67 ± 0.01 0.77 ± 0.01 0.77 ± 0.01 Twonorm 1 , 1.2 0.68 ± 0.04 0.50 ± 0.02 0.43 ± 0.03 0.82 ± 0.04 0.88 ± 0.01 0.91 ± 0.02 0.98 ± 0.01 Breast Cancer 5 , 1.0 0.66 ± 0.02 0.67 ± 0.02 0.52 ± 0.03 0.60 ± 0.03 0.60 ± 0.02 0.71 ± 0.02 0.70 ± 0.01

Table content: <table><tr><td>Dataset</td><td>Noise Parameters (col, α)</td><td>LR-N</td><td>NHERD</td><td>PAM</td><td>ULE</td><td colspan="2">AE (learned) AE (known)</td><td>LR-C</td></tr><tr><td colspan="7">Lin-Sep 0.86 Arrhythmia 1, 1.2 -0.79 ± 0.01 0.83 ± 0.01 0.81 ± 0.02 0.66 ± 0.04 0.85 ± 0.02</td><td>0.85 ± .01</td><td>0.85 ± 0.02</td></tr></table>

Noise model: probability of an error occurring depends linearly on an input feature. We report the class weighted mean f1 score on the noise-free test set along with the standard error.

Table content: <table><tr><td>Dataset</td><td>Noise Parameters (col, α)</td><td>LR-N</td><td>NHERD</td><td>PAM</td><td>ULE</td><td>AE (learned) AE (known)</td><td>LR-C</td></tr><tr><td>Lin-Sep</td><td>1, 1.2</td><td colspan="6">0.40 ± 0.01 0.53 ± 0.04 0.62 ± 0.03 0.61 ± 0.02 0.93 ± 0.01 0.96 ± 0.01 0.96 ± 0.01</td></tr><tr><td>Diabetes</td><td>5 , 1.2</td><td colspan="6">0.62 ± 0.01 0.59 ± 0.01 0.67 ± 0.01 0.71 ± 0.01 0.66 ± 0.03 0.69 ± 0.01 0.72 ± 0.02</td></tr><tr><td>German</td><td>1 , 1.2</td><td colspan="6">0.60 ± 0.02 0.72 ± 0.01 0.60 ± 0.01 0.67 ± 0.01 0.68 ± 0.03 0.73 ± 0.03 0.76 ± 0.02</td></tr><tr><td>Image</td><td>1 , 0.7</td><td colspan="6">0.55 ± 0.02 0.64 ± 0.01 0.57 ± 0.01 0.60 ± 0.03 0.74 ± 0.01 0.77 ± 0.01 0.77 ± 0.01</td></tr><tr><td>Twonorm</td><td>1 , 1.2</td><td colspan="6">0.90 ± 0.03 0.55 ± 0.02 0.86 ± 0.01 0.94 ± 0.01 0.96 ± 0.01 0.97 ± 0.01 0.98 ± 0.01</td></tr><tr><td>Breast Cancer</td><td>5 , 1.0</td><td colspan="6">0.60 ± 0.02 0.60 ± 0.02 0.63 ± 0.02 0.63 ± 0.03 0.65 ± 0.03 0.66 ± 0.01 0.70 ± 0.01</td></tr></table>

Noise model: probability of an error occurring depends quadratically on an input feature. We report the class weighted mean f1 score on the noise-free test set along with the standard error.

Table content: <table><tr><td>Dataset</td><td>Noise Parameters (α)</td><td>LR-N</td><td>NHERD</td><td>PAM</td><td>ULE</td><td>AE (known)</td><td>LR-C</td></tr><tr><td>Lin-Sep</td><td>0.7</td><td colspan="6">0.39 ± 0.01 0.41 ± 0.01 0.53 ± 0.02 0.85 ± 0.01 0.94 ± 0.01 0.96 ± 0.01</td></tr><tr><td>Diabetes</td><td>0.7</td><td colspan="6">0.56 ± 0.01 0.53 ± 0.03 0.50 ± 0.02 0.55 ± 0.02 0.68 ± 0.02 0.72 ± 0.02</td></tr><tr><td>German</td><td>0.7</td><td colspan="6">0.57 ± 0.01 0.70 ± 0.01 0.47 ± 0.01 0.60 ± 0.01 0.72 ± 0.01 0.76 ± 0.02</td></tr><tr><td>Image</td><td>0.7</td><td colspan="6">0.43 ± 0.01 0.61 ± 0.01 0.45 ± 0.02 0.43 ± 0.01 0.74 ± 0.03 0.77 ± 0.01</td></tr><tr><td>Twonorm</td><td>0.5</td><td colspan="6">0.52 ± 0.03 0.51 ± 0.03 0.52 ± 0.03 0.51 ± 0.04 0.88 ± 0.03 0.98 ± 0.01</td></tr><tr><td>Breast Cancer</td><td>0.7</td><td colspan="6">0.62 ± 0.02 0.62 ± 0.02 0.46 ± 0.02 0.63 ± 0.02 0.66 ± 0.04 0.70 ± 0.01</td></tr></table>

Noise model: probability of an error occurring depends linearly on the distance from class boundary (which is defined by the clean dataset before noise is synthetically introduced). We report the class weighted mean f1 score on the noise-free test set along with the standard error.

Table content: <table><tr><td colspan="2">% Clean data Noise Parameter (α)</td><td>AE(known)</td><td colspan="2">Ren et al. (2018) No noise model</td></tr><tr><td>1</td><td>0.1</td><td>98.28 ± 0.01</td><td>93.27 ± 0.01</td><td>90.87 ± 0.05</td></tr><tr><td>1</td><td>0.8</td><td>98.14 ± 0.00</td><td>91.27 ± 0.01</td><td>87.32 ± 0.03</td></tr><tr><td>5</td><td>0.1</td><td>98.77 ± 0.00</td><td>93.44 ± 0.01</td><td>91.01 ± 0.02</td></tr><tr><td>5</td><td>0.8</td><td>98.65 ± 0.00</td><td>91.29 ± 0.01</td><td>88.85 ± 0.02</td></tr><tr><td>10</td><td>0.1</td><td>98.93 ± 0.00</td><td>94.87 ± 0.03</td><td>90.99 ± 0.02</td></tr><tr><td>10</td><td>0.8</td><td>98.54 ± 0.01</td><td>91.77 ± 0.02</td><td>90.16 ± 0.03</td></tr></table>

We compare the label re-weighting scheme ofRen et al. (2018) with our proposed model. The models compared for different sizes of clean dataset and different noise rates α. The base model in each case is a standard LeNet as inRen et al. (2018) methods. The success of the model in this setting suggests its unique robustness to feature-dependent noise.

Table content: <table/>

Algorithm 1 Train energy-based autoencoder : for i in {1,2,. . . ,m} do where m is total number of batches

Table content: <table><tr><td>Require: T</td><td colspan="2">full training data</td></tr><tr><td>Require: T</td><td colspan="2">training data with known labels</td></tr><tr><td>Require: η</td><td colspan="2">learning rate</td></tr><tr><td>Require: Forward (x, θ)</td><td>autoencoder transformation that maps ∀x →</td><td>x</td></tr><tr><td>Require: E ( x)</td><td colspan="2">energy function, i.e. reconstruction loss</td></tr><tr><td>Require: L (B, O; θ)</td><td colspan="2">objective function as defined in eq. 5</td></tr><tr><td>Require: GetBalancedBatch (T )</td><td colspan="2">get mini-batch with an equal number</td></tr><tr><td/><td colspan="2">of instances from each class</td></tr></table>

defined in Eq. 5 5:∇ θ ( i ) ← Backward ( i ; θ i ) Step (θ i , η, ∇ θ ( i )) 7: end for 8: Initialize: E = {}empty dictionary that will store energies 9: for {x i , ỹi } in T do10: xi ← Forward (x i ; θ )where θ is the output of previous loop T c ← T T i.e. T c = (x i , y i ) ∈ T (x i , y i ) / ∈ T 8: for {x i , y i } in T c do

Table content: <table><tr><td colspan="3">6: θ i+1 ← 13: end for</td><td/><td/></tr><tr><td colspan="5">Algorithm 2 Correct training data</td></tr><tr><td colspan="2">Require: T</td><td/><td/><td/><td>full training data</td></tr><tr><td colspan="2">Require: T</td><td/><td/><td/><td>training data with known labels</td></tr><tr><td colspan="2">Require: E</td><td/><td/><td/><td>energies from algorithm 1</td></tr><tr><td colspan="3">Require: E ( x)</td><td/><td/><td>energy function, i.e. reconstruction loss</td></tr><tr><td colspan="6">Require: GetNegativeSamples (T )</td><td>get all samples from class 0</td></tr><tr><td colspan="6">Require: GetPositiveSamples (T )</td><td>get all samples from class 1</td></tr><tr><td colspan="6">1: T 0 ← GetNegativeSamples (T )</td></tr><tr><td colspan="6">2: T 1 ← GetPositiveSamples (T ) 3: Ē0 ← 1 |T0| x∈T0 E (x)</td><td>get mean w.r.t class 0</td></tr><tr><td colspan="3">4: Ē1 ← 1 |T1|</td><td colspan="2">x∈T1 E (x)</td><td>get mean w.r.t class 1</td></tr><tr><td colspan="2">5: σ 0 ←</td><td colspan="2">x∈T0</td><td>2 (E(x;θ)-Ē0) |T0|-1</td><td>1 2</td><td>get standard deviation w.r.t class 0</td></tr><tr><td colspan="2">6: σ 1 ←</td><td colspan="2">x∈T1</td><td>2 (E(x;θ)-Ē1) |T1|-1</td><td>1 2</td><td>get standard deviation w.r.t class 1</td></tr><tr><td>7: 9:</td><td colspan="2">e i ← E[i]</td><td/><td/><td>get energy corresponding to x i</td></tr><tr><td>10:</td><td/><td/><td/><td/></tr></table>

Peer review (iclr2019):
 This paper addresses Type III label noise correction problem in which the labeling noise depends on the features. They assume that we can obtain a small amount of cleanly labeled data, and use an energy-based semi-supervised learning approach to bootstrap the relabeling process. Pros: - Problem is well-motivated with a reasonably good overview of this research area. - Paper is generally well-written with enough details to follow and good experimental result discussion. Cons: - The energy-based approach based on contrastive divergence is pretty straightforwardly defined, but it will make the paper much stronger if the authors can have more analysis on this approach and/or provide theoretical guarantee on generalization. - It is not obvious to me how to extend the proposed approach to multi-class problems. - It will be beneficial to test the approach on more real-world problems on top of the toy-data-alike binary classification problems. Minor clarification questions: - What amount of cleanly labeled data is sufficiently required for the proposed approach to work? The authors have some pre-selected percentage in experiments but it is non-trivial to establish that for different applications. - Related to the previous comment, how much clean data were used in AE (known) columns in all experiments? - In Fig 2 between the two subgraphs, why is the left one showing positive thetas while the right one showing negative thetas? - Were the hyperparameters a & b chosen from cross-validation or from std of E terms in all experiment results? - In Table 1, for Breast Cancer dataset, how can AE (known) be better than the upper-bound LR-C? - It would be good to vary the noise parameters and show how robust the proposed approach is in dealing with different levels of noise.