PARS: PSEUDO-LABEL AWARE ROBUST SAMPLE SELECTION FOR LEARNING WITH NOISY LABELS

1 INTRODUCTION . Deep neural networks rely on large-scale training data with human annotated labels for achieving good performance ( Deng et al. , 2009 ; Everingham et al. , 2010 ) . Collecting millions or billions of labeled training data instances is very expensive , requires significant human time and effort , and can also compromise user privacy ( Zheng et al. , 2020 ; Bonawitz et al. , 2017 ) . Hence , there has been a paradigm shift in the interests of the research community from large-scale supervised learning ( Krizhevsky et al. , 2017 ; He et al. , 2016a ; Huang et al. , 2017 ) to Learning with Noisy Labels ( LNL ) ( Natarajan et al. , 2013 ; Goldberger & Ben-Reuven , 2016 ; Patrini et al. , 2017 ; Tanno et al. , 2019 ) and/or unlabeled data ( Berthelot et al. , 2019b ; a ; Sohn et al. , 2020 ) . This is largely due to the abundance of raw unlabeled data with weak user tags ( Plummer et al. , 2015 ; Xiao et al. , 2015 ) or caption descriptions ( Lin et al. , 2014 ) . However , it is not trivial to build models that are robust to these noisy labels as the deep convolutional neural networks ( CNNs ) trained with cross-entropy loss can quickly overfit to the noise in the dataset , harming generalization ( Zhang et al. , 2016 ) . Most of the existing approaches on LNL can be divided into three main categories . First , several noise robust loss functions ( Ghosh et al. , 2017 ; Wang et al. , 2019a ; Zhang & Sabuncu , 2018 ) were proposed that are inherently tolerant to label noise . Second , sample selection methods ( also referred to as loss correction in some literature ) ( Han et al. , 2018 ; Yu et al. , 2019 ; Arazo et al. , 2019 ) are a popular technique that analyzes the per-sample loss distribution and separates the clean and noisy samples . The identified noisy samples are then re-weighted so that they contribute less in the loss computation . A challenge in this direction is to design a reliable criterion for separation and hence prevent overfitting to highly confident noisy samples , a behavior known as self-confirmation bias . Third , label correction methods attempt to correct the noisy labels using class-prototypes ( Han et al. , 2019 ) or pseudo-labeling techniques ( Tanaka et al. , 2018 ; Yi & Wu , 2019 ) . However , in order to correct noisy labels , we typically need an extra ( usually small ) set of correctly labeled validation labels . In particular , these methods can fail when the noise ratio is high and estimating correct labels or high-quality pseudo-labels is non-trivial . More recently , the success of several state-of-the-art LNL methods is attributed to leveraging SemiSupervised Learning ( SSL ) based approaches ( Li et al. , 2020 ; Kim et al. , 2019 ) . Typically , a sample selection technique is applied to separate clean and noisy labels in the training data , then the noisy labels are deemed unreliable and hence treated as unlabeled in a SSL setting . Following the recent SSL literature ( Lee et al. , 2013 ; Arazo et al. , 2020 ) , estimated pseudo-labels are usually used to replace the filtered noisy labels during training . These approaches have shown to be highly tolerant to label-noise . However , the noisy labels are always discarded in favor of pseudo-labels in all the existing literature , but they may still contain useful information for training . Pseudo-labeling is in turn only applied to the filtered noisy subset while the rest of the raw labels are typically used as is , which makes it sensitive to the quality of the filtering algorithm . In particular , motivated by a simple principle of making the most of the signal contained in the noisy training data , we design PARS , short for Pseudo-Label Aware Robust Sample Selection . Our contributions are as follows : 1 . PARS proposes a novel , principled training framework for LNL . It trains on both the original labels and pseudo-labels . Unlike previous works , instead of filtering and then discarding the low-confident noisy labels , PARS uses the entire set of original labels , and applies self-training with pseudo-labeling and data augmentation for the entire dataset ( rather than the filtered noisy data only ) . 2 . PARS is able to learn useful information from all the available data samples through label-dependent noise-aware loss functions . Specifically , in order to prevent overfitting to inaccurate original labels ( or inaccurate pseudo-labels ) , PARS performs a simple confidencebased filtering technique by setting a high threshold on their predicted confidence , and applies robust/negative learning ( or positive/negative learning ) accordingly . 3 . We perform extensive experiments on multiple benchmark datasets i.e . noisy CIFAR-10 , noisy CIFAR-100 and Clothing1M . Results demonstrate that PARS outperforms previous state-of-the-art methods by a significant margin , in particular when high level of noise is present in the training data . We also conduct sufficient ablation studies to validate the importance of our contributions . 4 . We design a novel low-resource semi-supervised LNL setting where only a small subset of data is weakly labeled ( Section 4.3 ) . We show significant gains over state-of-the-art approaches using PARS . This setting is particularly interesting when it is hard to obtain large-scale noisy labeled data . In particular , we find that surprisingly none of the existing LNL methods outperform a baseline SSL model ( FixMatch ) ( Sohn et al. , 2020 ) that is not even designed to handle label noise , and yet PARS can achieve up to an absolute 27 % improvement in test accuracy in a controlled high-noise low-resource setting . 2 RELATED WORK . In recent literature on LNL , methods typically fall into three design categories to learn a noise robust model : noise robust loss functions , sample selection approaches , or label correction methods . Noise robust loss function based methods propose objective functions that are tolerant to label noise . A commonly used loss function that is identified to be robust to noisy labels is Mean Absolute Error ( MAE ) ( Ghosh et al. , 2017 ) . Wang et al . ( 2019a ) proposed Improved MAE which is a re-weighted version of MAE . Zhang & Sabuncu ( 2018 ) proposed Generalized Cross Entropy Loss ( GCE ) which is a generalization of MAE and Categorical Cross Entropy loss . More recently , Wang et al . ( 2019b ) designed a Symmetric Cross Entropy ( SCE ) loss which is similar in spirit to the symmetric KLdivergence and combines the cross-entropy loss with the reverse cross-entropy . Although SCE is robust to noisy labels , Ma et al . ( 2020 ) proposed a normalized family of loss functions which are shown to be more robust than SCE for extreme levels of label noise . Kim et al . ( 2019 ) and Kim et al . ( 2021 ) designed a framework that alternates between positive learning on accurate/clean labels and negative learning on complementary/wrong labels . Loss correction approaches explicitly modify the loss function during training to take into account the noise distribution by modeling a noise transition matrix ( Patrini et al. , 2017 ; Tanno et al. , 2019 ; Xia et al. , 2019 ; Goldberger & Ben-Reuven , 2016 ) or based on label-dependent weights ( Natarajan et al. , 2013 ) . Another family of methods focus primarily on sample selection , where the model selects small-loss samples as “ clean ” samples under the assumption that the model first fits to the clean samples before memorizing the noisy samples ( also known as the early-learning assumption ) ( Arpit et al. , 2017 ; Zhang et al. , 2016 ; Liu et al. , 2020 ) . Han et al . ( 2018 ) and Yu et al . ( 2019 ) proposed Co-teaching where sample selection is conducted using two networks to select clean and noisy samples , and then the clean samples are used for further training . MentorNet ( Jiang et al. , 2018 ) is a student-teacher framework where a pre-trained teacher network guides the learning of the student network with clean samples ( whose labels are deemed “ correct ” ) . In the decoupling training strategy proposed by Malach & Shalev-Shwartz ( 2017 ) , two networks are trained simultaneously and guide each other on when and how to update . One limitation of these approaches is that they ignore all the noisy/unclean samples during training and only leverage the expected-to-be clean samples for improving performance . Li et al . ( 2020 ) proposed DivideMix where sample selection is conducted based on per-sample loss distribution , and then the noisy samples are treated as unlabeled data in a SSL setting ( Berthelot et al. , 2019b ) . Nishi et al . ( 2021 ) further investigated the potential of using augmentation strategies in LNL , one used for loss analysis and another for learning , thus improving generalization of DivideMix . Compared to the above , label correction aims to improve the quality of the noisy labels by explicitly correcting the wrong labels . Tanaka et al . ( 2018 ) and Yi & Wu ( 2019 ) predicted the correct labels either as estimates of label probabilities ( soft labels ) or as one-hot class labels ( hard labels ) . Arazo et al . ( 2019 ) combined label correction with iterative sample selection by first filtering clean labels from noisy labels modeling a two-component Beta Mixture , and then estimating to correct labels for those noisy samples . Tanaka et al . ( 2018 ) combined label correction with additional regularization terms that were proved to be helpful for LNL . Song et al . ( 2019 ) also used label replacement to refurbish a subset of labels , thereby gradually increasing the number of available training samples . Liu et al . ( 2020 ) computed soft labels as model predictions and then exploit it to avoid memorization . 3 METHODOLOGY . 3.1 PRELIMINARIES . For a K-class classification problem with noisy labels , let D = { ( xi , yi ) } n i=1 with x ∈ X ⊂ Rd as the input features and y ∈ Y = { 1 , . . . , K } as the corresponding label , we aim to learn a classifier p ( · ; θ ) : X → Y parametrized by θ . For a clean labeled set , i.e.where yi is the true label for the i-th training sample xi , we can learn the model parameters θ by minimizing the Cross Entropy ( CE ) loss : min θ 1 |D| ∑ ( x , y ) ∈D LCE ( p ( x ; θ ) , y ) . ( 1 ) However , in the presence of noisy labels where yi is possibly incorrect for the i-th training sample xi , the cross-entropy loss can quickly overfit to the noise in the dataset ( Zhang et al. , 2016 ) . Robust Learning . Variants to the CE loss have been proposed to improve the classification performance under label noise . Examples of such noise robust loss functions include Mean Absolute Error LMAE ( Ghosh et al. , 2017 ) , Symmetric Cross Entropy LSCE ( Wang et al. , 2019b ) , Normalized Cross Entropy LNCE or Active Passive Loss LAPL ( Ma et al. , 2020 ) . We may simply replace the LCE with any of these noise robust loss , denoted by LRL , in the optimization Equation ( 1 ) to perform Robust Learning : LRL ∈ { LMAE , LSCE , LNCE , LAPL } . ( 2 ) For the sake of the limited space , definitions and further discussion about the robust losses are deferred to Appendix A.1 . Positive and Negative Learning . A particular variant to the CE loss that is designed to handle noisy/wrong labels is called Negative Learning ( Kim et al. , 2019 ) . Given a sample with a correct label ( x , y ) , Positive Learning ( PL ) is equivalent to optimizing the standard CE loss in Equation ( 1 ) , so that we optimize the output probabilities corresponding to the true label to be close to 1 : LPL ( p ( x ; θ ) , y ) : = LCE ( p ( x ; θ ) , y ) = K∑ k=1 [ y ] k log [ p ( x ; θ ) ] k , ( 3 ) where the class label y is one-hot encoded , and [ · ] k denotes the k-th dimension of a vector . Now for a data example with a wrong label ( x , ȳ ) , Negative Learning ( NL ) applies the CE loss function to fit the “ complement ” of the label in Equation ( 1 ) , so that the predicted probabilities corresponding to the given label is optimized to be far from 1 : LNL ( p ( x ; θ ) , ȳ ) : = − K∑ k=1 [ ȳ ] k log [ 1− p ( x ; θ ) ] k . ( 4 ) Sample Selection . In order to apply the Positive and Negative Learning framework in LNL , Kim et al . ( 2019 ) proposed to select samples based on a simple threshold of the per-sample loss value , i.e.a data point ( x , y ) is selected to be clean for PL if [ p ( x ; θ ) ] y > γ where γ is a noise-aware threshold . This simple yet effective procedure is based on the commonly used `` small-loss '' assumption that smaller-loss samples ( typically in early learning stages ) are usually easier to learn and hence expected to be “ clean ” , and it has been successfully applied in several other studies ( Jiang et al. , 2018 ; 2020 ) . 3.2 PARS : PSEUDO-LABEL AWARE ROBUST SAMPLE SELECTION We propose PARS : Pseudo-Label Aware Robust Sample Selection . The method is illustrated in Figure 1 ( and Algorithm 1 in Appendix A.2 ) , and each component is discussed in detail below . In particular , there are three major differences between PARS and previous works : 1 ) PARS does not treat the high-confidence samples as `` clean '' , but rather applies robust loss functions to address the relatively low yet non-negligible level of noise . 2 ) PARS does not discard the noisy ( or lowconfidence ) labels , but rather applies a negative learning framework to learn the “ complement ” of these labels ( Kim et al. , 2019 ) . This constitutes one major difference between the selective negative learning procedure of Kim et al . ( 2019 ) and PARS , that Kim et al . ( 2019 ) only apply it during the warm-up stage and still choose to discard the filtered noisy labels in favour of pseudo-labels in a SSL manner , and we unify the two stages and show improved performance . 3 ) We improve the convergence of PARS by self-training using pseudo-labels and data augmentation for the entire dataset as a proxy to enhance clean labels and correct noisy labels . Robust Warm-up Training . In the presence of label noise , the model overfits to noise when trained with CE loss and produces overconfident and wrong predictions after a few epochs . This problem is more severe with high levels of label noise . To overcome this , we perform “ robust warm-up ” using a robust loss for a few epochs for the model to produce reliable and confident predictions . Specifically , in the warm-up training phase , we minimize : min θ 1 |D| ∑ ( x , y ) ∈D LRL ( p ( x ; θ ) , y ) , ( 5 ) where LRL can be any loss from Equation ( 2 ) . In our experiments , we test different loss functions for warm-up training ( for more details see Appendix A.3 ) . Robust Sample Selection . As shown in Figure 2a , after the robust warm-up training , we observe that there is a weak trend that high-confidence samples tend to have clean labels . Given the model parameters θ̂ at the current iteration and a confidence threshold τ ∈ [ 0 , 1 ] , we divide the entire dataset D ( or similarly , a minibatch B ⊂ D ) into an ambiguous1 set DA and noisy set DN of samples by thresholding the maximum of the predicted confidence probabilities over all classes : DA = { ( x , y ) ∈ D|max k [ p ( x ; θ̂ ) ] k > τ } , DN = D \ DA . ( 6 ) A notable difference is that our confidence-based thresholding is label-free ( i.e.thresholding of the maximum confidence over all classes ) , while the widely used loss-based thresholding Kim et al . ( 2019 ) depends on the ground-truth ( noisy ) label ( i.e.thresholding the confidence corresponding to the given class ) . Our approach has two advantages : 1 ) our selective criterion depends on the predicted confidence probabilities alone , therefore it is straightforward to apply to cases where ground-truth labels are unavailable ( e.g.in a semi-supervised setting in Section 4.3 ) ; 2 ) our strategy helps to reduce the self-confirmation bias of propagating the mistake of overfitting to possibly noisy samples , because our model only selects samples that it is confident to make a prediction on regardless of the correctness of the raw label . Notably , previous works such as Jiang et al . ( 2018 ) ; Li et al . ( 2020 ) usually maintain two divergent networks ( one for sample selection and one for the main classification task ) in order to avoid the self-confirmation bias . Our method proves to be highly resilient to the overfitting issue with a single network simultaneously updated to perform both sample selection and classification , which drastically simplifies implementation and training complexity . Label-Dependent Noise-Aware Loss Functions . Despite the robust sample selection of dividing samples into ambiguous and noisy set , we propose to learn useful information from all the available training data instead of discarding the raw labels of the noisy samples as typically done in previous literature ( Jiang et al. , 2018 ; Kim et al. , 2019 ; Li et al. , 2020 ) . Specifically , we propose to use two separate loss functions , tailored for either the ambiguous or noisy labels respectively , which prevents overfitting to noise and enables further filtering of labels effectively ( Figure 2b ) . For the ambiguous set , DA , we train with the robust learning loss , LRL , ( same loss used during the warm-up training ) , where we deem that the given label in this set may still contain non-negligible noise . For the noisy set , DN , we train with the negative learning loss , LNL , to learn the complement of the given raw labels for the identified least-confident highly-noisy samples . The overall label-dependent noise-aware loss function for the proposed robust/negative learning framework using the given raw labels is thus : Lraw ( θ ) : = 1 |DA| ∑ ( x , y ) ∈DA LRL ( p ( x ; θ ) , y ) + λN 1 |DN | ∑ ( x , ȳ ) ∈DN LNL ( p ( x ; θ ) , ȳ ) , ( 7 ) where λN is the weight for the negative learning loss term , andDA , DN are defined as in Equation ( 6 ) . We set a high threshold ( e.g.0.95 throughout our experiments ) on the predicted confidence in the robust sample selection ( Equation ( 6 ) ) since we observe that this yields better performance as shown 1We choose the term “ ambiguous ” rather than the typically used term “ clean ” ( Jiang et al. , 2018 ; 2020 ) because there is still significant amount of noise in the expected-to-be clean samples when a simple thresholding method is used to select samples ( Arazo et al. , 2019 ) . in Appendix A.4 . We will show that , by using the noisy labels in the negative learning setting , we are able to consistently improve the model performance ( Table 4 ) . Self-Training with Pseudo-Labels . After the robust warm-up training , the initial convergence of the model leads to reliable predictions which can be used as labels to further improve model convergence via self-training . Therefore , in parallel to using the given raw labels during training , we use pseudolabels generated by the network itself to guide the learning process . Given the model parameters θ̂ at the current iteration , for each sample ( x , · ) ∈ D , we generate the underlying pseudo-label z corresponding to the model ’ s most confident class , and then randomly sample a complementary “ wrong ” pseudo-label z̄ from the remaining classes following Kim et al . ( 2019 ) : z : = arg max k [ p ( x ; θ̂ ) ] k , z̄ ∈ { 1 , . . . , K } \ { z } . ( 8 ) Similarly to how we use label-dependent noise-aware loss functions for the given raw labels in the ambiguous and noisy sets , we train with the pseudo-labels from the two sets using different losses too . Specifically , we apply positive/negative learning for the generated pseudo-labels in the ambiguous/noisy set respectively : Lpseudo ( θ ) : = 1 |DA| ∑ ( x , · ) ∈DA LPL ( p ( x ; θ ) , z ) +λN 1 |DN | ∑ ( x , · ) ∈DN LNL ( p ( x ; θ ) , z̄ ) , ( 9 ) where λN is the weight for the negative learning loss term ( equal to the weight in Equation ( 7 ) ) , DA , DN are defined as in Equation ( 6 ) , z , z̄ are defined as in Equation ( 8 ) . We find that it suffices to train with the standard CE loss for self-training with pseudo-labels from the ambiguous set , without the need for noise robust learning . One caveat of applying self-training is that , the model ’ s own predictions may become unreliable when most of the samples are primarily guided by the self-training , encouraging the network to predict the same class to minimize the loss . We observe that this is particularly the case under high levels of label noise . To reliably apply self-training , we regularize Equation ( 9 ) with a confidence penalty ( Tanaka et al. , 2018 ; Arazo et al. , 2019 ) : Lreg ( θ ) : = K∑ k=1 [ p ] k log ( [ p ] k [ h ( D ; θ ) ] k ) , ( 10 ) where [ p ] k is the prior probability distribution for class k , and [ h ( D ; θ ) ] k is shorthand for the mean predicted probability of the model for class k across all samples in the dataset D. In our experiments , we assume a uniform distribution for the prior probabilities ( i.e . [ p ] k = 1/K ) , while approximating h ( D ; θ ) by h ( B ; θ ) using mini-batches B ⊂ D as suggested by Tanaka et al . ( 2018 ) . The generated pseudo-labels become reliably confident with help of the regularization , which guides the robust sample selection to resemble the true distribution more closely ( Figure 2c ) . PARS : Pseudo-Label Aware Robust Sample Selection . After the robust warm-up training stage ( Equation ( 5 ) ) , our final model , PARS , is then trained using both the original labels and the selfgenerated pseudo-labels with robust sample selection . The final loss is given by : min θ Ltotal ( θ ) : = Lraw ( θ ) + λSLpseudo ( θ ) + λRLreg ( θ ) , ( 11 ) where Lraw , Lpseudo , Lreg are defined as in Equations ( 7 ) , ( 9 ) and ( 10 ) respectively , and λS , λR are the weights for the self-training loss and confidence penalty term . Data Augmentation . For image classification tasks in LNL , data augmentation techniques have recently been shown to significantly improve generalization ( Nishi et al. , 2021 ) . We adapt such data augmentation strategies to further improve PARS for benchmarking image classification performance in our experiments . Specifically , we apply weak augmentation ( e.g.standard random flip , crop , normalization ) to all data samples in the robust/negative learning with original raw labels in computing Equation ( 7 ) . We also apply strong augmentation ( e.g.AutoAugment ( Cubuk et al. , 2019 ) , RandAugment ( Cubuk et al. , 2020 ) ) to all data samples in the positive/negative learning with pseudo-labels in computing Equations ( 9 ) and ( 10 ) . We do not incorporate strong augmentation for training with the original labels as it leads to poor performance in presence of high label noise , as also observed by Nishi et al . ( 2021 ) . In this way , PARS can be seen as one example of how to extend the work of Nishi et al . ( 2021 ) by combining weak and strong data augmentation effectively with noisy labels and pseudo-labeling , as an effective technique to further advance the state of the art in LNL .

Summary: The paper combines three branches of approaches (1. sample selection, 2. noise robust loss, 3. label correction) to address label noise in classification in a single framework. Specifically, the method includes (1) warm up phase, (2) a novel label-free sample selection, (3) noise aware loss (as a standard technique) and (4) self-training with pseudo labels along with the given labels. The proposed method outperforms prior arts in CIFAR-10/100, especially in high noise regimes (80-90%). More interestingly, in the small sample training regime, gains by the proposed method increase significantly in evaluations with CIFAR-10/100.