Title: IS THE LABEL TRUSTFUL: TRAINING BETTER DEEP LEARNING MODEL VIA UNCERTAINTY MINING NET

Abstract: In this work, we consider a new problem of training deep neural network on partially labeled data with label noise. As far as we know, there have been very few efforts to tackle such problems. We present a novel end-to-end deep generative pipeline for improving classifier performance when dealing with such data problems. We call it Uncertainty Mining Net (UMN). During the training stage, we utilize all the available data (labeled and unlabeled) to train the classifier via a semi-supervised generative framework. During training, UMN estimates the uncertainly of the labels' to focus on clean data for learning. More precisely, UMN applies the samplewise label uncertainty estimation scheme. Extensive experiments and comparisons against state-of-the-art methods on several popular benchmark datasets demonstrate that UMN can reduce the effects of label noise and significantly improve classifier performance.

1. INTRODUCTION:

Deep Learning (DL), to learn powerful representations, it usually requires a large amount of training data. However, for many real world problems, it is not always possible to obtain sufficiently large training data. What we can usually get is limited training data with corrupted labels which heavily affect the model performance. Although acquiring large data is not hard, considering the information explosion on the internet, accurate labeling is usually an expensive and error-prone task which involves humans' interaction, especially experts with knowledge in the specific field. Most of the time, we have to build DL models using limited training data with corrupted data labels. It becomes very challenging to apply current popular deep learning frameworks to solve this problem.

Training a deep learning model via noisily labeled data is a challenging task (Patrini et al., 2017; Li et al., 2017; Ding et al., 2018; Menon et al., 2015; Goldberger & Ben-Reuven, 2016; Jiang et al., 2017) . Most of current explorations for dealing with such problems usually require large amount of labeled data as a prerequisite. It means that in these works, the designed pipelines and evaluations are based on having enough labeled training data, though they may include label noise. These approaches will suffer from the limited size of labeled training data as well as the label noise. As we know, the hierarchical representation learned by deep learning models mainly benefits from the amount of data. The deep learning model can easily overfit to the mislabeled samples especially when the data size is small (Tarvainen & Valpola, 2017; Ren et al., 2018) .

Unfortunately, there is no general solutions to solve the problem of training on limited annotated data with label noise. If we directly train the deep learning model only with the labeled data, the small portion of labeled training data will limit the learning capability. Meanwhile, the label noise makes it very challenging to utilize the labeled data directly. In this work, our framework can handle these problems simultaneously. UMN includes two major components. The first part is used to learn a latent feature representation via a generative framework (VAE) using all the training data. Then the learned embedding is applied as the input to its subsequent semi-supervised learning component by conditional variational autoencoder to train the classifier. To handle the label noise, we integrate label uncertainty estimation module where the reliable data contributes more to the model training and noisy data contributes less.

Though recent works (Hataya & Nakayama, 2019; Ren et al., 2018) discuss such problems, they usually assume to have a separate small clean dataset to begin with. It may not be possible to have an independent clean dataset in real scenarios. Current supervised robust learning approaches cannot be directly adopted to our case since the labeled data is small and not reliable. The limited labeled training data makes the supervised model overfit very easily to the small size of noisy data. This will heavily hurt the model performance. The limited data labels and noise can also affect other related state-of-the-art semi-supervised learning approaches. We will present the comparisons with these approaches in the experiment section.

In this paper, our major contributions can be summarized into the following aspects: First, our studied problem is very general in real scenarios: small mislabeled datasets with large amount of unlabeled data, and there are very few existing approaches to target this kind of problems. Second, instead of treating the label noise as the preknowledge, we explicitly use the deep generative architecture to model the noisy labels at the sample level precisely. This has been validated by the improved experimental results as well as the theoretical analysis. Third, we show how to use the moving average model to estimate the sample-wise uncertainty in labels. We have explained it in the theory analysis section and validated this assumption in the experiments. Furthermore, our experiments demonstrate that UMN can also help identify these mis-labeled data without any prior knowledge. Finally, we build one end-to-end deep learning pipeline to train against the noisy labeled data.

2. RELATED WORKS:

Along with the popularity of deep neural network, the problem of training deep neural networks with noisy label data has started to draw our attention. Natarajan et al. (Natarajan et al., 2013) propose the unbiased estimator of the surrogate loss function and calculate theoretical bounds for empirical risk optimization. Modeling the consistency as the regularization for deep neural network is another route for robust learning with noisy labeled data. Reed et al. (Reed et al., 2015) design a robust loss to model the prediction consistency. In (Li et al., 2017) , the authors propose a knowledge distillation framework where they train an auxiliary model on a small set of clean data samples and linearly combine its predictions with the observed labels to form the new targets to train. In (Ding et al., 2018) , a two-stage approach is proposed. First a DNN model is trained on the noisy data and used to filter out these noisy samples. Then another model is trained on the filtered dataset using a semi-supervised approach by treating these filtered out samples as unlabeled. Menon et al. model the corruption process of a dataset by learning the class-probability estimator (Menon et al., 2015) . Another way of regularization for training on noisy labels is to estimate the class conditional corruption ratio. Goldberger et al. (Goldberger & Ben-Reuven, 2016) propose a softmax layer along with the classifier to predict the class conditional corruption ratio. In (Jiang et al., 2017) , they propose MentorNet which learns a data driven dynamic curriculum to be followed by the StudentNet.

Semi-supervised learning with few labeled samples is also a well studied area. Some of the closely related works are (Gordon & Hernández-Lobato, 2017) , (Kingma et al., 2014) , (Tarvainen & Valpola, 2017) . Kingma et al (Kingma et al., 2014) propose a stacked deep generative semi-supervised model for training on partially labeled datasets. They first train a variational autoencoder on the labeled data, then stack on top of it a conditional variational autoencoder and continue training in semi-supervised fashion to get a robust classifier. In (Tarvainen & Valpola, 2017) , the authors train a classifier in semi-supervised way using temporal ensemble approach. They train a student network and maintain a teacher network as the exponential moving average of the student. They use classification loss for labeled data and enforce consistency loss between teacher and student for unlabeled data.

There have been limited works dealing with bi-quality data (Hataya & Nakayama, 2019) , (Langevin et al., 2018) . Bi-quality data is defined in (Hataya & Nakayama, 2019) as datasets with few labeled samples where the labels are potentially corrupted. In (Langevin et al., 2018) , the authors discuss to use the deep generative semi-supervised model to model noisy labels. They assume that the labels are also randomly corrupted with a probability ρ. Their idea relies on the predefined ρ to train the classifier. However, the corruption probability ρ is not known as a prior in reality.

Compared to these work, we present a new architecture for training a robust classifier with dataset including noisy labels. We assume that there is no preknowledge of the label corruption rate and we do not have access to a separate clean dataset which is more close to the real scenario. For simplicity, the KL loss for the predictions on unlabeled data with respect to an uniform prior is not shown. For simplicity, we omit the optional consistency loss term between the classifiers of θ and θ for unlabeled data in the figure.

3. METHOD:

The goal of our work is to learn a better model with the limited labeled data including the noise.

Since the labeled data only occupies a small portion of the training data, the labeled data alone is not sufficient to train a good model. At the same time, we also need to solve the label noise problem.

To deal with these issues, our pipeline consists of two major components. In the first part, we apply unsupervised learning scheme to learn a latent feature representation via all the available training data. Then we utilize the label via semi-supervised learning pipeline to train the classifier. At the same time, we incorporate the label uncertainty estimation to reduce the influences coming from these mislabeled data. We estimate the uncertainty of the given sample from the prediction of the updating model and its guider model (exponential moving average of the updating model), part of this idea is inspired by work (Polyak & Juditsky, 1992) . The estimated uncertainty is then used to assign weights to the training data samples. This is an end-to-end framework. We call it Uncertainty Mining Nets(UMN). In the following subsection, we will introduce UMN in more details.

3.1. LATENT-FEATURE LEARNING:

In this work, we represent the given data as the set of (X, Y ) = {(x 1 , y 1 ) , . . . , (x N , y N )} where x i is the observed data sample and y i is its corresponding label which may be potentially corrupted. If x i is unlabeled, then y i becomes empty. To be concise, we omit the index i in the rest of the paper.

Motivated by the recent success of generative models in semi-supervised learning related applications, firstly, we apply the variational autoencoder (VAE) on all the training data. VAE consists of two modules: an Encoder that maps the variables x to the latent variables z to approximate the prior distribution of p(z) and a Generator p θ (x|z) that samples the the input variables x given the latent variables z. It can be formulated as

EQUATION

where f (x; z, θ) is the likelihood function and N indicates the Gaussian distribution. θs are network parameters. The goal of VAE is to maximize the evidence variational lower bound (ELBO) of p θ (x)

as log p θ (x) E q φ (z|x) log p θ (x|z) -D KL (q φ (z|x)||p(x)).

The learned latent variables obtained from VAE can be used as the feature representation to train the classifier. Based on this, we can fully utilize the knowledge from the training data without the constraint of the labeled data size limitation. The low-dimensional embedding z can well cover the distribution of the input data x. Our hypothesis is that the unlabeled data helps training robustly against the noisy labels via the deep generative modeling. We will see this in the next subsection.

3.2. SEMI-SUPERVISED LEARNING:

After building the unsupervised learning framework via VAE, we apply the label information to improve the learning capability of the neural network. Instead of using the raw data x, we directly use the latent vector z a obtained from VAE of latent-feature learning part along with the pre-defined labels. We follow the idea of conditional generative model.

Our generative process can be formulated as,

EQUATION

)

where M is the multinomial distribution. y indicates the true label. z a is jointly generated from z b and y and z b is the latent representation of the input. In this work, we treat y as the latent variables. We use ŷ to denote the observed labels. To more accurately capture the reliability of labels, we estimate each labeled sample uncertainty via p θ (ŷ | y, z a ). the posterior distribution can be factorised as

EQUATION

Following this, the Evidence Lower Bound (ELBO) can be derived as follows:

-

EQUATION

where q φ (y|z a ), q φ (z a |x), q φ (z b |z a , y), p θ (z a |z b , y), p θ (x|z a ) indicate the classifier, encoder, conditional encoder, conditional decoder and decoder functions respectively. L indicates labeled data. U means unlabeled data and C denotes the set of used classes. The last term can be treated as labeled loss term as the supervised portion of Eq. 5.

For a more detailed derivation of the ELBO, please refer to the Appendix. As indicated in Eq. 5, except the last term, all other terms in the equation are calculated over all training data samples. The last term serves as the supervised factor for semi-supervised learning where the observed labels may be incorrect. It uses an uncertainty related function p θ (ŷ|y k , z a ) to modulate the sample weights during the gradient back-propagation. As indicated in Fig. 1 , p θ (ŷ|y k , z a ) is approximated based on the sample-wise uncertainty . More details are given in the following.

For simplicity, we can also represent the generative process of the observed ŷ by Fig. 2 .

One plausible way to estimate the uncertainty of the sample label is to model the class conditional corruption ratio p(ŷ|y), which is a mean field approximation. As discussed in (Langevin et al., 2018) , the class conditional probability can be computed as and the labeled loss term can be formulated as

EQUATION

EQUATION

where

f ( ) = log[ (C-1)(1-)

] is a function with constant . From Eq. 7, we can see that it assigns the same weight for all samples belonging to the same category, i.e., all labeled terms contribute equally to the gradient back propagated regardless of their label correctness as

EQUATION

Although this idea is simple and easy to follow, the assumption may not be the case in the real scenario. In general, the value of which is the class conditional corruption rate, remains unknown beforehand. Another issue is that it's not accurate to apply the same categorical corruption ratio to all data samples of a given class. Our approach solves these problems by estimating sample-wise uncertainty during the training stage.

We replace the class conditional in Eq. 9 with a point-wise estimation of the label uncertainty (z a ), where we follow the generative process of Eq. 3. The uncertainty of label is hence (z a ). Pointwise label uncertainty is calculated as

p(ŷ | y, z a ) = 1 -(z a ), if ŷ = y (z a )/(C -1). otherwise (9) So f ( ) in Eq. 7 becomes f ( ) = log[ (C-1)(1-(za)) (za)

]. That is, samples contribute relatively more to the gradient backpropagation when the estimated is small which indicates there is a high probability that the given label is correct. By focusing on these reliable targets during training, we can train a more robust classifier. From this, we can see that the key is to approximate (z a ) precisely. In the following subsection, we discuss the proposed approach to model (z a ) using the differences in predictions from the updating classifier model and its moving average model.

We want to emphasize that although UMN is based on VAE framework, its architecture can be flexible. Depending on the task, the encoder of its generative model-VAE can be implemented using various deep learning networks, e.g., AlexNet, GoogLeNet, ResNet, etc. Meanwhile, UMN is not limited to one specific task, like image classification. It can be easily extended to other classification problems.

3.2.1. UNCERTAINTY ESTIMATION:

As discussed above, we aim to train a more robust classifier in an end-to-end semi-supervised architecture. The key to this idea is to estimate sample-wise label noise ( (z a )) as this cannot be known as a priori. We are motivated by the initial work of the iterate average model (Polyak & Juditsky, 1992) where exponential moving average of the model weights can be used to stabilize the training in stochastic gradient based approaches. Exponential moving average of the model weights can be calculated as:

EQUATION

)

where θ is the set of weight parameters of the classifier model and θ is its exponential moving average. γ controls the smoothness of model updates. We name them Learner and Guider models respectively. t is the step index of the iterative optimization. Since this iterate average gives optimal bound for convergence rate (Polyak & Juditsky, 1992) and can be less sensitive to the noisy updates (Eq. 10), we adopt it as the Guider model to estimate the label uncertainty.

We approximate the label uncertainty via the absolute difference in the predicted probability of the Learner and the Guider for the observed class as following

EQUATION

where f and f indicate the classifier's predictions of the learner and guider's respectively. As the Learner learns from the noisy labeled data, the Guider model is used to weigh the samples based on uncertainty thus limiting the contributions of unreliable samples to the gradients back propagated.

In the following subsection, we provide the analysis of the reason why using the difference between the predictions of the Guider and Learner models is a good measure for the uncertainty estimation.

3.2.2. THEORETICAL ANALYSIS:

Our deep generative model allows us to model the correct labels via the observed labels and therefore leads to a plausible way to adjust sample weights during training. We will show in the following theorem that the label correctness is associated with the difference between the Learner and Guider models during training. Theorem 1. Suppose that θ * is the optimal solution of the optimization problem. θ L is the stochastic random variable for SGD (learner) and θ G is iterate average (guider

) of θ L , θ G (T ) = µ T ≡ 1 T T 0 θ L (t)dt.

The distance between θ L and θ * is consistent with the distance between θ L and θ G in the sense that

E ∂ ∂θ L θ L -θ * 2 (θ L -θ G ) > 0 (12)

which holds when the training epoch T satisfies the following relation

EQUATION

)

where λ is the constant learning rate.

We can see from Eq.12, during the early training stage, the distance between the learner and the optimal solution increases when the distance between the learner and guider increases and vice versa. This leads to preventing samples with uncertain labels to participate in training and thus reducing the overfitting on potentially wrong labels. The proof of the theorem is given in the Appendix.

4. EXPERIMENTS AND RESULTS:

Our experiments are designed to evaluate whether UMN is an effective approach to learn a good model with limited annotated training data which include label noise. We compare to popular approaches of supervised learning, semi-supervised learning and robust learning dealing with noisy labels. These methods represent state-of-the-art approaches for model learning with noisy data. Moreover, we also compare the results with the pipeline without using our uncertainty estimation module. Further, we also evaluate the performance of applying the uncertainty estimation module in identifying corrupted labels.

We experiment on a variety of image classification problems with varying degrees of label corruption rates. We use three popular datasets including MNIST, SVHN and CIFAR-10. For a comprehensive evaluation, we set up five different uniform labels corruption rates including [10%, 20%, 30%, 40%, 50%].

4.1. IMPLEMENTATION DETAILS:

In the experiments, supervised deep learning framework means all the label data are directly used for training. For SVHN and CIFAR-10, we experiment a CNN architecture with 13 convolutional neural network as well as the ResNet-101 architecture (He et al., 2015) as baselines for supervised learning approach. To compare with semi-supervised learning approaches, we select two recent popular works.

One is named as mean-teacher (MT) proposed by Tarvainen and Valpola (Tarvainen & Valpola, 2017) and the other one is Langevin et al. (Langevin et al., 2018) named Mislabeled-VAE (M-VAE). Langevin et al. (Langevin et al., 2018) only roughly discusses their idea without showing the details of the model architecture and experimental results on popular benchmarks. In this work, we implement the idea in (Langevin et al., 2018) and compare with UMN in all three different datasets. In our experiments, we use the same encoder and decoder architectures for UMN and M-VAE.

In comparison to robust learning approaches, we apply another two recent works including Mentor-Net (Jiang et al., 2017) and Reweight (Ren et al., 2018) . For a fair comparison with these works, we follow all the original setting in these papers in our experiments. For each corruption ratio, we ran 5 experiments by randomly choosing samples to corrupt each time, and report the mean error rate.

As illustrated in Fig. 1 , our framework (UMN) is composed of two encoders and two decoders (VAE and C-VAE) along with a classifier. For the experiments in MNIST, We adapt a multi-layer perceptron (MLP) architecture similar to the one proposed in (Kingma et al., 2014) , for the VAE, C-VAE and the classifier models. The encoders, decoders and the classifier of our model use a single hidden layer with ReLU activation and an output layer with no activation. For SVHN and CIFAR-10, we apply a 13-layer convolutional neural network (ConvNet) to build our framework. At the first stage of unsupervised learning, z a is generated via convolution and up-sampling layers. Similar as the first stage, z b is generated via the encoder framework which is composed of convolution and dense layers. We maintain a moving average of the learner and use the difference between the predictions of learner and guider model to calculate . The decoder in the semi-supervised learning stage is built with de-convolution and dense layers.

Due to the limitation in space, we detail the model architecture in the Appendix section. We use the Adam optimizer (Kingma & Ba, 2014) with an initial learning rate of 0.001, β 1 = 0.90 and β 2 = 0.99. Each experiment has 5 runs and each run has 350 epochs. In each epoch, UMN uses all the training data including labeled and unlabeled data. For these supervised learning frameworks, we only use the labeled data. Our implementation is based on deep learning framework TensorFlow with a single NVIDIA Tesla P100 GPU.

4.2. DATASETS AND RESULTS:



4.2.1. MNIST:

MNIST is a widely used dataset in machine learning community. It includes 60, 000 images with size 28 × 28 pixels. In our experiments, we use 50, 000 images for training and 10, 000 images for testing. 100 labeled data samples are used in the experiments. In semi-supervised training, each mini-batch have 5 labeled samples and 95 unlabeled examples. In our compared supervised model, we apply a Multi-Layer Perceptron (MLP) classifier with one hidden layer of 784 units with ReLU activation. All the results are listed in Table 1 .

From the results, we can find that UMN performs much better than other approaches. Besides UMN, M-VAE (Langevin et al., 2018 ) outperforms other benchmarks. One possible reason may be due to probablistic modeling of the uncertainty. However, M-VAE depends on the pre-defined label corruption ratio which can't be obtained in most real scenarios.

We did not run Reweight (Ren et al., 2018) on the MNIST dataset since at the time of running the experiments an implementation of the approach on noisy MNIST data was not available and our implementation of the approach would not be a fair comparison without having the right parameter settings. However, comparisons are shown on the other datasets as implementation for them was made available by the authors.

4.2.2. SVHN:

We experiment with the Street View House Numbers (SVHN) datasets. This dataset includes 73, 257 RGB images of 32×32 resolution belonging to ten different classes. Following the same experimental settings as in (Tarvainen & Valpola, 2017) , we use 500 labeled data samples from SVHN where 50 data samples per category and we use the rest of the images for unlabeled data in the semi-supervised learning setting. We randomly corrupt the sample labels within each category uniformly with our defined corruption ratio. We also compare with other five different models on SVHN dataset. In our experiment, each batch includes 5% labeled data.

The comparison results are given in Table 1 . As listed in this table, we can find that UMN behaves the best in general except for the case when the corruption ratio is 10%. However, we do not observe this phenomena in MNIST and CIFAR-10. One of the reasons may be that MT can handle the situation when the noisy data occupy a small portion. The performance of MT model drops significantly as we ingest more mislabeled data. From the result, we can also observe that other two robust learning approaches (MentorNet and Reweight) achieve better performance than supervised only but much lower than these Semi-supervised learning methods. One possible reason is that they do not utilize the unlabeled data.

As illustrated in Fig. 3 , overfitting is observed in the M-VAE at a higher ratio of corruption. As we discussed previously, one major reason is because of the pre-defined f ( ) which is used as the constant value during the training phase. In contrast, UMN applies the sample-wise uncertainty estimation. It is accurate and more close to the real scenario. Furthermore, the training of UMN also converges faster with better performance.

Moreover, we conduct the experiments and evaluate the model performances using varying numbers of labeled data with the SVHN. We compare the performance for a popular supervised robust learning approach MentorNet(MN) and UMN, for which the test errors shown in Table 2 . In UMN, we still utilize all the unlabeled data which are not utilized for labeled data. UMN shows better performance for labeled data less than 1000 labels per class, which suggests the important role of the semi-supervised learning in robust learning and hence justify the purpose of this work. Next, we run the comparisions on CIFAR-10 dataset (Krizhevsky, 2009) . This dataset contains 32 × 32 pixels RGB images belonging to ten different classes. To have a fair comparison, we follow the same experimental setting as used in (Tarvainen & Valpola, 2017) . We randomly select 100 data samples from each category. In total, 4, 000 labeled data samples are included in the training set. Circles" are used to represent true labels and "Cross" to represent mislabeled samples. We can see that similarly colored circles (same class) are better grouped by our proposed UMN approach when compared to supervised and semi-supervised approaches. For the experiment, we use the MNIST dataset with 100 labeled samples with 50% label corruption. The network architecture used for UMN is described in section 4.2.1. Using the same architecture, for the supervised experiment we train only the classifier branch of the model, and for the semi-supervised experiment we set the uncertainty estimate to be = 0. We start from the Jensen's inequality for the log probability of the observations, log P (X) ≥ E q(z|x) log P (X|Z) -KL(q(z|X) p(z))

5. CONCLUSION:

where the observed variables are X = {ŷ, X} and the latent variables z for the VAE structure we proposed with a stacked semi-supervised learning architecture. The posterior and likelihood factorizes as q(z a , z b , y|x, ŷ) = q(z a |x) q(y|z a ) q(z b |z a , y) p(x, ŷ|z a , z b , y) = p(x|z a ) p(ŷ|y, z a ).

The reconstruction loss on r.h.s of Eq. 14 then gives, y∈C E q θ (za|x) p φ (y|z a ) logP (x|z a ) + logP (ŷ|z a , y)

= E q φ (za|x) log P (x|z a ) + E q φ (za|x)

EQUATION

where q φ and p θ are the encoder and decoder respectively for the variational autoencoder and can be parametrized by deep neural network. We note that the second term in the last equality of Eq. 15 is summed over the labeled samples in Semi-supervised learning. The KL divergence in Eq. 14 is given as -KL(q(z a , z b , y|X, ŷ) p(z a , z b , y))

= y∈C p θ (y|z a ) E q φ (za|x) E q φ (z b |za,y) log q φ (z a |x, ŷ) + log q φ (y|z a ) + log q φ (z b |z a , y)

-log p(z a |z b , y) -log p(z b ) -log p(y)

= -E q φ (za|x) y∈C q φ (y|z a ) KL(q(z b |z a , y) p(z b ))

-E q φ (za|x) y∈C q φ (y|z a ) KL(q φ (y|z a ) p(y))

-E q φ (za|x) y∈C q φ (y|z a )E q φ (z b |za,y) log p θ (z a |z b , y) -log q φ (z a |x) .

(16)

To summarize, the ELBO for VAE of UMN is -E q φ (za|x) y∈C q φ (y|z a ) KL(q(z b |z a , y) p(z b ))

-E q φ (za|x) y∈C q φ (y|z a ) KL(q φ (y|z a ) p(y))

-E q φ (za|x) y∈C q φ (y|z a )E q φ (z b |za,y) log p θ (z a |z b , y) -log q φ (z a |x)

+ E q φ (za|x) log P (x|z a ) + E q φ (za|x)

EQUATION

where in the last term, estimations for probabilities of the observed label is given based on the true labels and the corresponding encoded sample z a . The loss given by the ELBO are summed over all the samples except for the last term is summed over the samples with the observed variable ŷ available.

6.2. ALGORITHM OF UMN:

Algorithm 1: UMN Algorithm while Traning() do Sample a minibatch from dataset S i ; Draw z i a from q φ (z i a |x i ) given x i ∈ S i ; Draw z i a from q φ (z i a |x i ) ; for all data x i ∈ S i : do y i ∼ q b φ (y i |z i a ) end for all labeled data x i L ∈ S i : do ŷi ∼ p θ (ŷ i |y, z i a ) Compute based on the uncertainty estimation Eq.( 9) via Learner and Guider end Compute the generative process via Eq.(3) ; Compute loss function L via Eq.(5) ; g θ = ∂L ∂θ ; g φ = ∂L ∂φ ; θ = θ -AdamUpdate(θ); φ = φ -AdamUpdate(φ); For φ in classifier y i ∼ q b φ (y i |x i ), maintain Polyak's moving average for the guider as φ G (t) = (1 -α)φ L (t -1) + α * φ L (t) end (24)

where θ L and θ G indicate the weights of the leaner and guider respectively. Next we prove the theorem 12 that in the presence of the noisy labels, when we update the weights of the learner model, the distance between the learner and the guider suggests the correctness of the labels assuming the invariance of the Hessian of the learner and guider by the lemma.

Proof.

EQUATION

In order to show Eq. 13, it is equivalent to show

EQUATION

We denote θ L = θS + θ * ; θ G = θT + θ *

These are two stochastic optimization process described above for the stochastic gradient descent and the iterate average. θS and θT can be seen as the deviation from the optimal point. Hence we need to show

EQUATION

The left hand side of Eq. 27 gives

EQUATION

In order to evaluate the right hand side of the equation, we recall the Green's function for the Ornstein-Uhlenbeck process, E(θ(t)θ (s)) = Σe -λH(s-t) , if t < s Σe -λH(t-s) Σ, if t ≥ s (29)

Then, the right hand side of Eq. 27 becomes, (30) 

:



Figures and Tables:

Figure 2: The graphical model generating the observed label

Figure 4: Illustration of latent feature distribution and progression of . (a) 2D embedding of the learned latent feature z a (output of encoder in the VAE which serves as the input to the classifier branch) obtained using t-SNE (Maaten & Hinton, 2008) for the MNIST test data. Lef t and Right plots show distributions obtained without and with the use of uncertainty estimate , respectively. Different colors in the plots represent different classes (total 10). (b) The distribution visualization of evolves along with the time.

Figure 5: Plots showing the distribution of MNIST training samples in the learned latent space for different approaches. We have used t-SNE to visualize the distribution in 2-dimensions. The different colors indicate the 10 different classes of MNIST. "Circles" are used to represent true labels and "Cross" to represent mislabeled samples. We can see that similarly colored circles (same class) are better grouped by our proposed UMN approach when compared to supervised and semi-supervised approaches. For the experiment, we use the MNIST dataset with 100 labeled samples with 50% label corruption. The network architecture used for UMN is described in section 4.2.1. Using the same architecture, for the supervised experiment we train only the classifier branch of the model, and for the semi-supervised experiment we set the uncertainty estimate to be = 0.

-λH(t-t ) Σdt = Tr 1 T U Λ -1 (I -e -λT Λ )U T Σ ≈ Tr 1 λT H -1 Σ

An overview of UMN framework: X and Y are the observed inputs and potentially corrupt labels. θ and θ are semi-supervised generative models. η and η are noise functions that perturb the input X. The red and green colored arrows represent the flow of unlabeled and labeled data through the models. θ is updated via a stochastic gradient descent approach so as to minimize the classification and VAE losses, and θ is the Exponential Moving Average (EMA) of θ. The sample-wise uncertainty ( ) between the predictions of θ and θ are used to re-weigh the gradients that are back propagated to θ via the classification loss. The general architecture for θ is adapted from(Kingma et al., 2014). VAE indicates Variational AutoEncoder and C-VAE represents Conditional-VAE. Note that in θ , when making predictions (on labeled data) only the weights from encoder of the VAE and the classifier are used.

Table content: <table><tr><td/><td/><td/><td colspan="2">Stop gradient</td><td/><td/><td/></tr><tr><td>Y</td><td colspan="4">Classification Cost</td><td>Prediction</td><td colspan="2">e Uncertainty Estimate</td><td>Prediction</td></tr><tr><td/><td/><td/><td/><td colspan="2">Prediction</td><td/><td/></tr><tr><td/><td>Recon Cost</td><td>+</td><td>KL cost</td><td>C-VAE</td><td>Classifier</td><td>EMA Update of</td><td>C-VAE</td><td>Classifier</td></tr><tr><td>Recon Cost</td><td/><td/><td/><td>VAE</td><td>q</td><td>weights</td><td>VAE</td><td>q'</td></tr><tr><td/><td/><td/><td/><td>h</td><td/><td/><td>h'</td></tr><tr><td>X</td><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>Figure 1:</td><td/><td/><td/><td/><td/><td/><td/></tr></table>

Comparison of results on different benchmarks. S-1 represents the supervised learning model via 13 conv layers and S-2 represents supervised learning model via ResNet-101. MN and RW indicate the MentorNet and Reweight approach respectively. Error rate percentage % is used as the measurement unit.

Table content: <table><tr><td>Dataset</td><td>Corruption Ratio</td><td>S-1</td><td>S-2</td><td>MT</td><td colspan="3">Approaches M-VAE MN RW UMN(ours)</td></tr><tr><td/><td>10%</td><td colspan="2">29.4 29.5</td><td>6.6</td><td>4.3</td><td>25.6</td><td>-</td><td>2.5 ± 0.14</td></tr><tr><td/><td>20%</td><td colspan="3">36.2 36.6 11.7</td><td>2.7</td><td>39.7</td><td>-</td><td>2.8 ± 0.37</td></tr><tr><td>MNIST</td><td>30%</td><td colspan="3">41.1 43.0 14.6</td><td>7.4</td><td>45.8</td><td>-</td><td>5.3 ± 3.9</td></tr><tr><td/><td>40%</td><td colspan="3">51.3 49.0 17.9</td><td>9.9</td><td>57.7</td><td>-</td><td>5.8 ± 3.8</td></tr><tr><td/><td>50%</td><td colspan="3">57.5 59.4 34.4</td><td>28.0</td><td>65.3</td><td>-</td><td>18.0 ± 12.0</td></tr><tr><td/><td>10%</td><td colspan="3">34.9 32.0 18.0</td><td>26.1</td><td colspan="2">29.1 27.3</td><td>23.9 ± 0.2</td></tr><tr><td/><td>20%</td><td colspan="3">46.3 46.0 45.5</td><td>35.2</td><td colspan="2">42.1 39.0 30.5 ± 0.3</td></tr><tr><td>SVHN</td><td>30%</td><td colspan="3">61.4 58.9 53.0</td><td>37.2</td><td colspan="2">53.8 51.9 30.9 ± 0.6</td></tr><tr><td/><td>40%</td><td colspan="3">61.8 60.5 63.9</td><td>40.1</td><td colspan="2">59.1 53.7 37.1 ± 0.9</td></tr><tr><td/><td>50%</td><td colspan="3">65.1 63.3 65.9</td><td>48.3</td><td colspan="2">62.0 57.9 43.2 ± 1.4</td></tr><tr><td/><td>10%</td><td colspan="3">43.2 41.3 39.2</td><td>41.2</td><td colspan="2">41.2 39.4 37.8 ± 0.5</td></tr><tr><td/><td>20%</td><td colspan="3">46.3 44.7 42.1</td><td>43.9</td><td colspan="2">42.4 41.9 39.8 ± 0.8</td></tr><tr><td>CIFAR-10</td><td>30%</td><td colspan="3">52.9 50.0 47.8</td><td>51.4</td><td colspan="2">51.6 49.9 43.5 ± 0.5</td></tr><tr><td/><td>40%</td><td colspan="3">57.1 56.9 52.2</td><td>52.1</td><td colspan="2">54.1 53.9 43.5 ± 1.2</td></tr><tr><td/><td>50%</td><td colspan="3">63.3 61.3 65.4</td><td>56.3</td><td colspan="2">57.1 58.3 51.9 ± 1.7</td></tr></table>

Test errors with Varying Number of Labels for SVHN

Table content: <table><tr><td colspan="2">Number of labels 1000</td><td>2000</td><td>3000</td><td>10000 20000</td></tr><tr><td>UMN</td><td colspan="4">23.9 ± 0.2% 18.1% 16.3% 14.8% 13.2%</td></tr><tr><td>MentorNet</td><td colspan="4">30.5 ± 0.3% 27.9% 25.1% 18.3% 11.2%</td></tr><tr><td>4.2.3 CIFAR-10</td><td/><td/><td/></tr></table>

The rest of the training data are used for unlabeled data. The results are summarized in Table 1. From the listed results, we can see that UMN achieves the best performance among all experiments.

Table content: <table/>

In this work, we target a new problem of learning on partially labeled data with noise and propose a novel framework (UMN). UMN is a semi-supervised deep generative model dealing with data including scarce and potentially corrupted labels. Moreover, UMN can explicitly estimate the label uncertainty for a given potentially mislabeled data sample. Furthermore, we integrate these frameworks into an end-to-end pipeline. Compared to previous works, we do not need the subset of clean label data or any preknowledge of the corruption ratio. UMN is able to estimate the label correctness based on the uncertainty calculation. Experimental results demonstrate the superiority of UMN over state-of-the-arts for training deep learning models with noisy labels.

Table content: <table><tr><td>6 APPENDIX</td></tr><tr><td>6.1 ELBO FOR VAE OF UNCERTAINTY MININING NET (UMN)</td></tr></table>

Peer review (iclr2020):
The paper describes a method for learning in semi-supervised settings with label noise. This is an interesting topic with a relatively scarce literature. The proposed method works by first postulating a generative model for the labelled/unlabeled/label-corrupted data. The model is then fitted using a standard variational lower bound maximization. The approach is elegant, and appears to work empirically well. Unfortunately, I do not seem to really understand the rationale behind the main novelty (even at an heuristic level) of the paper which is contained in Section 3.2.1, namely Equation (10) which describes the intensity of the label corruption. Why is it a sensible idea? There is only 3 lines of comments after equation 10, which seems a bit short since this is the crux of the paper. Also, the notations of equation (10) are very confusing (to me) since the function f(.) was used before to denote another quantity. Minor remarks: ============= 1. page4 was a bit difficult to digest at first reading since the author did not describe in the main text he form (i.e. factorization structure) of the variational distribution q. 2. second line of Equation 14 seems to be wrong, while the 1st and last line seems correct -- the authors may want to double check 3. the term *labeled loss term* was not properly defined, which makes the reading a bit difficult even if one eventually gets what the authors mean 4. In equation (6) I do not understand why f(eps) = log[(C-1)(1-eps)/eps]. Why isnt it equal to log[p(hat(y)|y)] ? 5. notations z_a, z_1, etc.. do not seem to be consistent throughout the text 6. it is quite surprising to me that, in the low data-corruption regime, the method appears to be competitive/better with consistency-based method such as the MT approach. 7. naive question: why directly modeling eps as a function of z_a or (z_a, y), possibly parametrized by a neural network, a bad idea? In conclusion, it feels like the method has a lot of potential (in views of the numerical simulations) -- if the authors could clarify the exposition, this could be a very good contribution to the field. The method appears to be conceptually simple (i.e. postulate a generative model + fit it by maximizing the ELBO + one trick to estimate eps), which is a good thing -- what is missing, I think, is a real discussion of why the proposed manner to estimate epsilon is sensible. Edit after reading [1] ====================== The proposed generative model is same -- the authors should make this very clear in the paper. Although is is acknowledged: *In this work, we implement the idea in (Langevin et al., 2018)* -- this is only in section *4.1 IMPLEMENTATION DETAILS*. After reading (1), it is clear that the novelty of the paper us much less than what I had initially thought. The only difference is in Section 3.2.1, and this Section is far from being satisfying. [1] *A Deep Generative Model for Semi-Supervised Classification with Noisy Labels*, Langevin & al