ME-MOMENTUM:  EXTRACTING HARD CONFIDENT EXAMPLES FROM NOISILY LABELED DATA

1 INTRODUCTION . As training datasets are growing big while accurately labeling them is often expensive or sometimes even infeasible , cheap datasets with label noise are ubiquitous in many real-world applications . Without any care , label noise will degenerate the performance of learning algorithms , especially for these based on deep neural networks ( Zhang et al. , 2017 ) . Learning with noisy labels ( Angluin & Laird , 1988 ) aims to reduce the side-effect of label noise and therefore has become an important topic in machine learning . Existing methods for learning with noisy labels can be divided into two categories : algorithms that result in statistically consistent or inconsistent classifiers . Methods in the first category intent to design classifier-consistent algorithms ( Zhang & Sabuncu , 2018 ; Kremer et al. , 2018 ; Liu & Tao , 2016 ; Scott , 2015 ; Natarajan et al. , 2013 ; Goldberger & Ben-Reuven , 2017 ; Patrini et al. , 2017 ; Thekumparampil et al. , 2018 ; Yu et al. , 2018 ; Liu & Guo , 2020 ; Xu et al. , 2019 ) , where classifiers learned by using the noisy data will statistically converge to the optimal classifiers defined by clean data . However , these methods rely heavily on the noise transition matrix ( Liu & Tao , 2016 ; Patrini et al. , 2017 ; Xia et al. , 2019 ) . In real-world applications , it is hard to learn the instance-independent noise transition matrix ( Cheng et al. , 2020 ) . To be free of estimating the noise transition matrix , methods in the second category employ heuristics to reduce the side-effect of label noise ( Yu et al. , 2019 ; Han et al. , 2018b ; Malach & Shalev-Shwartz , 2017 ; Ren et al. , 2018 ; Jiang et al. , 2018 ; Ma et al. , 2018 ; Kremer et al. , 2018 ; Tanaka et al. , 2018 ; Reed et al. , 2015 ; Han et al. , 2018a ; Guo et al. , 2018 ; Veit et al. , 2017 ; Vahdat , 2017 ; Li et al. , 2017 ; 2020 ) . These methods were reported to empirically work well , especially in the setting of instance-dependent label noise . One promising direction in the second category is to extract examples with clean labels—confident examples— ( Northcutt et al. , 2017 ; Cheng et al. , 2020 ; Chen et al. , 2019 ; Malach & Shalev-Shwartz , 2017 ; Han et al. , 2018b ; Yu et al. , 2019 ; Thulasidasan et al. , 2019 ; Nguyen et al. , 2020 ; Dredze et al. , 2008 ; Liu & Tao , 2016 ; Wang et al. , 2017 ; Ren et al. , 2018 ; Jiang et al. , 2018 ) . The idea is that compared with the original noisy training data , the extracted examples are less noisy and thus will lead to a classifier with better performance . Given only noisy data , state-of-the-art methods exploit the memorization effect ( Zhang et al. , 2017 ; Arpit et al. , 2017 ) to extract confident examples . The memorization effect will enable deep neural networks to first learn patterns that are shared by majority training examples . As clean labels are of majority in each noisy class ( Natarajan et al. , 2013 ; Xiao et al. , 2015 ) , deep neural networks would therefore first fit training data with clean labels , and then gradually fit the examples with incorrect labels ( Chen et al. , 2019 ) . Therefore , early stopping ( Li et al. , 2020 ; Song et al. , 2019 ) and the small loss trick ( Jiang et al. , 2018 ; Han et al. , 2018b ; Yu et al. , 2019 ) can be used to exploit confident examples . Examples that are close to the decision boundary are called hard examples . As illustrated in Figure 1 , hard ( confident ) examples play an important role in shaping the decision boundary . It has also been widely studied in the traditional classification problem that hard examples are essential to train accurate classifiers ( Vapnik , 2013 ; Bengio et al. , 2009 ; Huang et al. , 2010 ; He et al. , 2018 ) . Notwithstanding the importance of hard confident examples , none of the existing methods studies how to extract hard confident examples from noisy data . Note that extracting hard confident examples is non-effortless . Since hard examples are often of a small proportion and contain less discriminative information compared with the easy ones ( these that are far away from the decision boundary ) , they are often entangled with inaccurately labeled examples in the procedures of extraction . In this paper , by alternately updating the confident examples and refining the classifier , we propose a deep learning paradigm that is able to extract hard confident examples from the noisy training data , leading to better classification performance . Specifically , the idea is similar to the usage of momentum from physics . As stated in the statistical learning theory , with better training data , better classifier can be obtained ( Mohri et al. , 2018 ) . We can then think of the classifier as a particle traveling through the hypothesis space , getting acceleration from the confident data . Classifiers with better performance can be achieved by properly exploiting the previously extracted confident examples . This is similar to the momentum trick used in optimization that previous gradient information can be used to escape local minimum and achieve fast convergence rates ( Sutskever et al. , 2013 ) 1 . At a high level , the proposed method is built on the memorization effect of deep neural networks and on the intuition that better confident examples will result in a better classifier and that a better classifier will identify better confident examples ( and hard confident examples ) . The proposed method is therefore called the Momentum of Memorization ( Me-Momentum ) . We conduct experiments to show the effectiveness of the proposed Me-Momentum on noisy versions of MNIST , CIFAR10 , CIFAR100 , and a real-world label noise dataset Clothing1M . Specifically , on MNIST and CIFAR , we generate class- and instance-dependent label noise and visualize the extracted hard confident examples , which justifies why Me-Momentum consistently outperforms the baseline methods . Codes will be available online . 2 ME-MOMENTUM . In this section , by specifying the proposed method of momentum of memorization ( Me-Momentum ; summarized in Algorithm 1 ) , we would like to detail how to accomplish extracting hard confident 1In optimisation , the parameter vector can be thought of as a particle traveling through the parameter space , getting acceleration from the gradient of the loss . The momentum trick demonstrated that gradient in previous update can help escape local minimum and achieve fast convergence rates . Algorithm 1 Me-Momentum Input : Noisy training data , noisy validation data , iteration number Ninner and Nouter ; Output : Extracted confident examples and the classifier ; 1 : Initialize a classifier f0 by using the noisy training data and early stopping ; //memorization effect for i = 1 , . . . , Nouter do for j = 1 , . . . , Ninner do 2 : Update the extracted confident examples ; //i.e. , the training examples whose noisy labels are identical to the ones predicted by fj−1 3 : Obtain the classifier fj ; //initialize the network by using the parameters of fj−1 and train it by employing confident examples ; the classifier fj will be chosen with the highest noisy validation accuracy throughout the training procedure 4 : Break and output fj−1 if the highest validation accuracy is non-increasing in the loop ; end 5 : Re-initialize a classifier f0 ; //randomly initialize the network and train it by using confident examples ; the classifier f0 will be chosen with the highest noisy validation accuracy throughout the training procedure 6 : Break and output fj−1 if the highest validation accuracy is non-increasing in the loop ; end examples and boosting the classification performances . At the high level , by alternately updating the confident examples and refining the classifier , Me-Momentum fulfils a positive cycle that better confident examples will result in a better classifier and that a better classifier will identify better confident examples . Specifically , Me-Momentum has two loops , i.e. , an inner loop and an outer loop . In the inner loop , Me-Momentum alternates update of the confident examples and classifier ( Steps 2 and 3 ) . However , the inner loop continually refines a classifier and thus depends heavily on the initialization of the classifier ( Step 1 ) . It may lead to the memorization of noisy labels and the inferiority of sample-selection bias . To handle this problem , the outer loop re-initialize the classifier ( Step 5 ) while maintains the previously extracted confident examples . There are some points to be clarified for the proposed Algorithm 1 : Q1 . How to initialize a good classifier in Step 1 ? Q2 . How to extract confident examples in Step 2 ? Q3 . How to validate the learned classifiers in Steps 3 and 5 without a clean validation set ? Q4 . Why the proposed method is called Me-Momentum ? Q5 . Why hard confident examples can be extracted ? To answer the first question , we would like to mention that the aim of the initialization in Step 1 is to initialize a good classifier for the positive cycle : a better classifier will identify better confident examples and better confident examples will result in a better classifier . A good candidate should have a fairly high classification accuracy , e.g. , higher than random guessing . Otherwise , the positive cycle can not be evoked . Fortunately , the initialization can be made by exploiting the memorization effect of deep neural networks that they would first fit clean data ( Arpit et al. , 2017 ; Zhang et al. , 2017 ) . Note that this memorization effect is independent of training optimization or network backbones ( Arpit et al. , 2017 ) . Specifically , we use the early stopping trick . For easy understanding , we would like to introduce a definition of high-peak . A noisy validation accuracy at the i-th epoch is called an i-th high-peak if it achieves the highest accuracy in the epoch range { 1 , . . . , i } . Assume the i-th and j-th high-peaks occur next to each other , having noisy validation accuracies of a and b , respectively . The training early stops if ( b − a ) / ( j − i ) ≤ τ , where τ is a hyper-parameter . In the experiments , we set τ = 0.1 , which empirically works well across all datasets . In Appendix A , we compare the difference between the early stopping method and the traditional validation method ; We also study the sensitivity of the hyper-parameter . The answer to the second question is closely related to the memorization effect . Note that the classifier initialized in Step 1 would fit the clean data well but not the incorrectly labeled data because of the memorization effect and early stopping . Therefore , we can treat the training examples whose noisy labels are identical to the ones predicted by the classifier obtained in Step 1 as confident examples . This also applies for the classifiers in Step 3 to extract confident examples , which are iteratively trained by employing the updated confident data . Note that there are some other feasible methods to extract confident examples , e.g. , extract those who have a large class posterior . In Step 3 , we aim to learn a better classifier compared with the one in the previous round . This can be achieved because of two reasons : ( 1 ) we initialize the network with the parameters of the classifier learned in the previous round ; ( 2 ) we have a better set of confident examples as the training sample . This starts the positive cycle that better confident examples will result in a better classifier and that a better classifier will identify better confident examples . The third question is essential for identifying the classifiers in the cycle . Note that the accurately labeled examples are always assumed to be dominant in each class in the community of learning with noisy labels ( Natarajan et al. , 2013 ; Liu & Tao , 2016 ; Han et al. , 2018b ) . Otherwise , the true class label can not be identified by only exploiting the noisy data . This assumption implies that the performance on the noisy validation set ( split from the noisy training set ) and these on test set are positively correlated . The noisy validation set could be used as a surrogate to validate the classifiers if no clean validation set is available . We therefore validate the classifiers in Steps 3 and 5 with the highest noisy validation accuracies during the training . Empirical results show that it works well . To answer the fourth question , we would like to first mention that the proposed method heavily relies on the memorization effect of deep neural networks . Specifically , in Step 1 , a classifier is initialized by exploiting the memorization effect via early stopping , which is used to identity confident examples . Later , the classifier and confident examples are iteratively refined and updated , respectively , which is the positive cycle we have mentioned before . Note that this cycle also depends on the memorization effect to update confident examples and refine classifiers . Our method is named as momentum of memorization ( Me-Momentum ) because it also uses the trick of momentum to better exploit the memorization effect . Specifically , we can think of the classifier as a particle traveling through the hypothesis space , getting acceleration from the updated extracted confident data . We exploit the previously extracted confident examples to help learn a better classifier , i.e. , initializing the network by using the parameters of the classifier trained with the confident examples extracted in the previous round . The impact of the confident examples in a certain round will decay as we continue updating the classifier . We intuitively answer the fifth question by employing Figure 2 . Hard confident examples will be progressively extracted also due to the momentum trick , i.e. , previously extracted confident examples will help identify hard confident examples . Specifically , in Figure 2 ( a ) , we show the noisy training examples , where circles and triangles represent two classes respectively and the red ones denote the examples with incorrect labels . Let the dash line in Figure 2 ( b ) denote a classifier learned by Me-Momentum . We can use it to extract confident examples , e.g. , the green circles and blue triangles , according to Step 2 of Algorithm 1 . Compared with Figure 1 ( a ) , we can see that hard confident examples close to the optimal boundary , i.e. , the circle and triangles with red edges , will be extracted . Note that as better and better classifiers will be learned by Me-Momentum , more and more hard confident examples will be extracted . An visualization is shown in Figure 4 . Relation to existing work : The strategy of alternatively optimizing the classifier and updating the training examples is not new for dealing with label noise . For example , Joint Optim ( Tanaka et al. , 2018 ) , Co-teaching ( Han et al. , 2018b ; Yu et al. , 2019 ) , and SELF ( Nguyen et al. , 2020 ) are similar to ours . Specifically , Joint Optim and Co-teaching update the classifier with one step of stochastic gradient descent while SELF and the proposed method refine the classifier to be optimal with respect to the extracted confident examples . However , existing methods have not focused on extracting hard confident examples and thus are substantially different from this paper because they neglected the importance of avoiding the accumulated error caused by the single initialization of the classifier and the sample-selection bias . Experiments ( e.g. , Figures 3 and 4 ) show that Me-Momentum with the outer loop part ( i.e. , re-initialization of the classifier ) contributes significantly to extracting hard confident examples and achieving high label precision . Note that without employing the ensemble technique , SELF is a special case of Me-Momentum with only the inner loop part . We avoid to integrate the ensemble technique to the proposed method for simplicity and clarity . Me-Momentum is similar to curriculum learning as it also learns from easy to difficult . However , curriculum learning needs a predefined curriculum ( sample weighting scheme ) , e.g. , assigning big/small wights for confident/noisy data . If the curriculum is not available , some clean data is required to learn a mentornet to provide a curriculum ( Jiang et al. , 2018 ) or a latent variable could be introduced by self-paced learning ( Kumar et al. , 2010 ) to learn a curriculum . Differently , MeMomentum only based on noisy data and does not explicitly learn a curriculum . Me-Momentum also has a similar flavor to active learning which tends to choose and label hard examples to learn from at each iteration . However , for active learning , no label information is available before the data is chosen while Me-Momentum has noisy labels and have to consider the side-effect of label noise .

Summary: The authors introduce an interesting approach to handling hard "confident" samples in learning with label noises. At the heart of the proposed approach is an interactive method that jointly refines the classifier and the samples. The confident samples are initialized by utilizing the memorization effect of deep networks. Then, a classifier is learned from such samples.