Pumpout: A Meta Approach for Robustly Training Deep Neural Networks with Noisy Labels

1 INTRODUCTION . Learning from the industrial-level data is quite demanding , since labels of such data are heavily noisy ( i.e. , 50 % of noisy labels ) , and their label generation processes are usually agnostic ( Xiao et al. , 2015 ; Jiang et al. , 2018 ) . Essentially , noisy labels of such data are corrupted from groundtruth labels without any prior assumptions ( i.e. , class-conditional noise ( Natarajan et al. , 2013 ) ) , which degrades the robustness of learning models . It is noted that industrial-level data is frequently emerging in our daily life , such as social-network data ( Cha & Cho , 2012 ) , E-commerce data ( Xiao et al. , 2015 ) and crowdsourcing data ( Welinder et al. , 2010 ; Han et al. , 2018a ) . Due to the large data volume , industrial-level data can be well handled by deep neural networks ( Xiao et al. , 2015 ) . Thus , the key issue is how to train deep neural networks robustly on noisy labels of such data , since deep neural networks have the high capacity to fit noisy labels eventually ( Zhang et al. , 2017 ) . To handle noisy labels , one common direction focuses on estimating the noise transition matrix ( Goldberger & Ben-Reuven , 2017 ; Han et al. , 2018b ) . For instance , Patrini et al . ( 2017 ) first leveraged a two-step solution to estimate the noise transition matrix . Based on this estimated matrix , they conducted backward loss correction , which is used for training deep neural networks robustly . However , the noise transition matrix is not easy to be estimated accurately , especially when the number of classes is large . Motivated by the memorization effects of deep neural networks ( Arpit et al. , 2017 ) , one emerging direction focuses on training only on selected instances ( Jiang et al. , 2018 ; Ren et al. , 2018 ; Han et al. , 2018c ) , which does not require any prior assumptions on noisy labels . Specifically , deep learning models are known to learn easy instances first , then gradually adapt to hard instances when training epochs become large ( Arpit et al. , 2017 ) . Therefore , in the first few iterations , we may train deep neural networks on the whole dataset , and let them sufficiently learn clean instances in the noisy dataset . Then we may later conduct early stopping ( Goodfellow et al. , 2016 ) , which tries to stop the training on noisy instances ; or we may employ the small-loss trick ( Jiang et al. , 2018 ; Han et al. , 2018c ) , which tries to perform the training selectively on clean ( small-loss ) instances . However , when noisy labels indeed exist , no matter using early stopping or small-loss trick , deep networks inevitably memorize some noisy labels ( Arpit et al. , 2017 ) , which will lead to the poor generalization performance ( Zhang et al. , 2017 ) . In this paper , we design a meta algorithm called Pumpout , which allows us to overcome the issue of memorizing noisy labels . The main idea of Pumpout is to actively squeeze out the negative effects of noisy labels from the training model , instead of passively forgetting these effects by further training . Specifically , on clean labels , Pumpout conducts stochastic gradient descent typically ; while on noisy labels , Pumpout conducts scaled stochastic gradient ascent , instead of stopping gradient computation as usual . This aggressive policy can erase the negative effects of noisy labels actively and effectively . We leverage Pumpout to upgrade two representative but orthogonal approaches in the area of “ deep learning with noisy labels ” : MentorNet ( Jiang et al. , 2018 ) and Backward Correction ( Patrini et al. , 2017 ; van Rooyen & Williamson , 2018 ) . We conducted experiments on benchmark vision and text datasets , namely simulated noisy MNIST , CIFAR-10 and NEWS datasets . Empirical results demonstrated that , under both extremely noisy labels ( i.e. , 45 % and 50 % of noisy labels ) and low-level noisy labels ( i.e. , 20 % of noisy labels ) , the robustness of two upgraded approaches ( by Pumpout ) is obviously superior than that of original approaches . 2 PUMPOUT MEETS NOISY SUPERVISION . Meta algorithm . The original idea of Pumpout is to actively squeeze out the negative effects of noisy labels from the training model , instead of passively forgetting these effects . Intuitively , take “ heart-broken ” story as a supportive example . When you are disappointed in your former love , the best way to recover is to embrace the new love , which actively squeezes out the hurt from your mind , instead of repeatedly self-blaming . In the design of meta algorithm , we should consider how our meta algorithm can simultaneously benefit multiple orthogonal approaches in the area of deep learning with noisy labels , such as training on selected instances ( Jiang et al. , 2018 ) , estimating the noise transition matrix ( Patrini et al. , 2017 ) and designing regularization ( Miyato et al. , 2016 ) . For this purpose , we generalize noisy labels into “ not-fitting ” labels , and generalize clean labels into “ fitting ” labels ( details of the fitting condition will be discussed in Q1 below ) . In the high level , the meta algorithm Pumpout is to train deep neural networks by stochastic gradient descent on “ fitting ” labels , and train deep neural networks by scaled stochastic gradient ascent on “ not-fitting ” labels . In the low level , the proposed Algorithm 1 is named Pumpout . Specifically , we maintain deep neural network f ( with parameter wf ) . When a single point { xi , yi } is sequentially selected from noisy set D ( step 3 ) , we first check whether { xi , yi } is fitting the discriminative condition or not . If yes , we conduct stochastic gradient descent typically ( step 4 ) ; otherwise , we conduct scaled ( γ ) stochastic gradient ascent ( step 5 ) , which erases the negative effects of “ not-fitting ” labels . These “ not-fitting ” labels hinder us to train a robust model . The abstract algorithm arises three important questions . Algorithm 1 Meta Algorithm Pumpout . 1 : Input network parameterwf , learning rate η , maximum epoch Tmax , hyper parameter 0 ≤ γ ≤ 1 ; for t = 1 , 2 , . . . , Tmax do 2 : Shuffle training set D ; //noisy dataset for i = 1 , . . . , |D| do 3 : Select { xi , yi } from D sequentially ; if { xi , yi } is fitting then 4 : Update wf = wf − η∇f ( xi , yi ) ; //stochastic gradient descent end else 5 : Update wf = wf + γη∇f ( xi , yi ) ; //scaled stochastic gradient ascent end end end 6 : Output wf . Three important questions . Q1 . What is the fitting condition ? Q2 . Why do we need gradient ascent on non-fitting data , in addition to gradient descent on fitting data ? Q3 . Why do we need to scale the stochastic gradient ascent on non-fitting data ? To answer the first question , we need to emphasize a view that orthogonal approaches require different fitting conditions . Intuitively , if a single point { xi , yi } satisfies a discriminative fitting condition , it means that our training model will regard this data point as a useful knowledge , and fitting on this point will benefit training the robust model . Conversely , if a single point { xi , yi } does not satisfy the discriminative fitting condition , it means that , our training model will regard this data point as useless knowledge , and want to erase the negative effects of this point actively . To instantiate the fitting condition , we provide two concrete cases in Algorithm 2 and Algorithm 3 , respectively . The above answer motivates our second question : why can not we only conduct stochastic gradient descent on fitting data points ( step 4 ) . In other words , can we remove scaled stochastic gradient ascent ( step 5 ) in Algorithm 1 ? In this case ( removing step 5 ) , our algorithm degenerates to training only on selected instances . However , once some of the selected instances are found to be false positives 1 , our training model will fit on them , and thus the negative effects will inevitably occur ( i.e. , degrading the generalization ( test accuracy ) ) . Instead of passively forgetting these negative effects ( i.e. , further training over many epochs ) , we hope to actively squeeze out the negative effects from the training model by using scaled stochastic gradient ascent ( step 5 ) . Lastly , the third question closely connects with the second one . Namely , why do we need scaled instead of ordinary stochastic gradient ascent ? The answer can be intuitively explained . Assume that we view stochastic gradient ascent as correction to “ not-fitting ” labels , and view 0 ≤ γ ≤ 1 as a scale parameter . When γ = 1 , our Pumpout will squeeze out the negative effects with full fast rate ; while when γ = 0 , our Pumpout will not squeeze out any negative effects . Both cases are not optimal , since we empirically find that the best performance is usually chosen when 0 < γ < 1 by using the validation set ( Section 4 ) . For the first case , the fast squeezing rate will negatively affect the convergence of our algorithm . For the second case , no squeezing rate will inevitably let deep neural networks memorize some “ not-fitting ” labels , which degrades their generalization ( Zhang et al. , 2017 ; Neyshabur et al. , 2017 ) . 3 PUMPOUT BENEFITS STATE-OF-THE-ART ALGORITHMS . In this section , we employ the idea of Pumpout to upgrade MentorNet and Backward Correction as follows . First , we briefly introduce the background of MentorNet and Backward Correction . Then , by using Pumpout , we propose upgraded MentorNet ( PumpoutSL ) and upgraded Backward Correction ( PumpoutBC ) . Lastly , we explain the relations between MentorNet ( Backward Correction ) and PumpoutSL ( PumpoutBC ) . 3.1 UPGRADE MENTORNET . MentorNet . To handle noisy labels , an emerging direction focuses on training only on selected instances ( Jiang et al. , 2018 ; Ren et al. , 2018 ; Han et al. , 2018c ) , which is free of estimating the noise transition matrix , and also free of the class-conditional noise assumption ( Natarajan et al. , 2013 ) . These works try to select clean instances out of the noisy ones , and then use them to update the network . Among those works , a representative method is MentorNet ( Jiang et al. , 2018 ) , which employs the small-loss trick . Specifically , MentorNet pre-trains an extra network , and then uses the extra network for selecting small-loss instances as clean instances to guide the training . However , the idea of MentorNet is similar to the self-training approach ( Chapelle et al. , 2009 ) , thus MentorNet inherits the same drawback of accumulated error caused by the sample-selection bias . PumpoutSL . Algorithm 2 represents the upgraded MentorNet using Pumpout approach ( denoted as PumpoutSL ) , where MentorNet uses the small-loss trick . Specifically , we maintain deep neural network f ( with parameter wf ) . When a mini-batch D̄ is formed ( step 3 ) , we first let f select a small proportion of instances in this mini-batch D̄s that have small training losses ( step 4 ) . The number of instances is controlled by R ( T ) , and f only samples R ( T ) percentage of instances out of the minibatch . More importantly , we let f select a proportion of instances in this mini-batch D̄b that have big training losses ( step 5 ) . The number of instances is controlled by 1−R ( T ) , and f only samples 1https : //en.wikipedia.org/wiki/False_positives_and_false_negatives 1−R ( T ) percentage of instances out of the mini-batch . Then , we conduct stochastic gradient descent on small-loss instances D̄s ( step 6 ) ; while we conduct scaled stochastic gradient ascent on big-loss instances D̄b ( step 7 ) , which actively erases the negative effects of big-loss instances . The update of R ( T ) ( step 8 ) follows Han et al . ( 2018c ) , in which extensive discussion has been conducted . Algorithm 2 PumpoutSL . The fitting condition is whether a point belongs to small-loss instances . 1 : Input network parameter wf , learning rate η > 0 , estimated noise rate τ , maximum epoch Tmax , maximum iteration Nmax , hyper parameter 0 ≤ γ ≤ 1 ; for T = 1 , 2 , . . . , Tmax do 2 : Shuffle training set D ; //noisy dataset for N = 1 , . . . , Nmax do 3 : Draw mini-batch D̄ from D ; 4 : Sample D̄s = arg minD̄ ` ( f , D̄ , R ( T ) ) ; //sample R ( T ) % small-loss instances 5 : Sample D̄b = arg maxD̄ ` ( f , D̄ , 1−R ( T ) ) ; //sample 1−R ( T ) % big-loss instances 6 : Update wf = wf − η∇f ( D̄s ) ; //update wf by stochastic gradient descent on D̄s ; 7 : Update wf = wf + γη∇f ( D̄b ) ; //update wf by scaled stochastic gradient ascent on D̄b ; end 8 : Update R ( T ) = 1−min { T Tk τ , τ } ; end 9 : Output wf . Relations between MentorNet and PumpoutSL . If we remove step 5 and step 7 in Algorithm 2 , PumpoutSL algorithm will be reduced to the core version of MentorNet , namely self-paced MentorNet . It means that PumpoutSL algorithm is more aggressive than MentorNet in essence . Namely , PumpoutSL conducts not only stochastic gradient descent on small-loss instances ( like MentorNet ) , but also scaled stochastic gradient ascent on big-loss instances .

Summary: This paper presents a meta algorithm to improve the robustness of learning methods under noisy labels. The idea is to squeeze out the negative effects of noisy labels actively. The paper trains deep neural networks by stochastic gradient descent on “fitting” labels; while trains deep neural networks by scaled stochastic gradient ascent on “not-fitting” labels. Experimental results show the improvement on robustness.