Sample Selection with Uncertainty of Losses for Learning with Noisy Labels

1 INTRODUCTION . Learning with noisy labels is one of the most challenging problems in weakly-supervised learning , since noisy labels are ubiquitous in the real world ( Mirzasoleiman et al. , 2020 ; Yu et al. , 2019 ; Nishi et al. , 2021 ; Arazo et al. , 2019 ; Yang et al. , 2021a ; Bai & Liu , 2021 ) . For instance , both crowdsourcing and web crawling yield large numbers of noisy labels everyday ( Han et al. , 2018 ) . Noisy labels can severely impair the performance of deep neural networks with strong memorization capacities ( Zhang et al. , 2017 ; Zhang & Sabuncu , 2018 ; Pleiss et al. , 2020 ; Lukasik et al. , 2020 ; Chen et al. , 2022 ) . To reduce the influence of noisy labels , a lot of approaches have been recently proposed ( Natarajan et al. , 2013 ; Liu & Tao , 2016 ; Ma et al. , 2018 ; Yang et al. , 2021b ; Zheng et al. , 2020 ; Xia et al. , 2019 ; 2020 ; Tanaka et al. , 2018 ; Malach & Shalev-Shwartz , 2017 ; Li et al. , 2020b ; Menon et al. , 2018 ; Thekumparampil et al. , 2018 ; Xu et al. , 2019 ; Kim et al. , 2019 ; Jiang et al. , 2020 ; Harutyunyan et al. , 2020 ) . They can be generally divided into two main categories . The first one is to estimate the noise transition matrix ( Patrini et al. , 2017 ; Shu et al. , 2020 ; Hendrycks et al. , 2018 ; Yang et al. , 2021c ; Wu et al. , 2022 ) , which denotes the probabilities that clean labels flip into noisy labels . However , the noise transition matrix is hard to be estimated accurately , especially when the number of classes is large ( Yu et al. , 2019 ) . The second approach is sample selection , which is our focus in this paper . This approach is based on selecting possibly clean examples from a mini-batch for training ( Han et al. , 2018 ; Wang et al. , 2018 ; Yao et al. , 2020a ; Wang et al. , 2019 ; Yu et al. , 2019 ; Lee et al. , 2019 ; Wang et al. , 2019 ; Yao et al. , 2022 ) . Intuitively , if we can exploit less noisy data for network parameter updates , the network will be more robust . A major question in sample selection is what criteria can be used to select possibly clean examples . At the present stage , the selection based on the small-loss criteria is the most common method , and has been verified to be effective in many circumstances ( Han et al. , 2018 ; Jiang et al. , 2018 ; Yu et al. , †Corresponding author 2019 ; Wei et al. , 2020 ; Yao et al. , 2020a ) . Specifically , since deep networks learn patterns first ( Arpit et al. , 2017 ) , they would first memorize training data of clean labels and then those of noisy labels with the assumption that clean labels are of the majority in a noisy class . Small-loss examples can thus be regarded as clean examples with high probability . Therefore , in each iteration , prior methods ( Han et al. , 2018 ; Wei et al. , 2020 ) select the small-loss examples based on the predictions of the current network for robust training . However , such a selection procedure is debatable , since it arguably does not consider uncertainty in selection . The uncertainty comes from two aspects . First , this procedure has uncertainty about small-loss examples . Specifically , the procedure uses limited time intervals and only exploits the losses provided by the current predictions . For this reason , the estimation for the noisy class posterior is unstable ( Yao et al. , 2020b ) , which causes the network predictions to be equally unstable . It thus takes huge risks to only use losses provided by the current predictions ( Figure 1 , left ) . Once wrong selection is made , the inferiority of accumulated errors will arise ( Yu et al. , 2019 ) . Second , this procedure has uncertainty about large-loss examples . To be specific , deep networks learn easy examples at the beginning of training , but ignore some clean examples with large losses . Nevertheless , such examples are always critical for generalization . For instance , when learning with imbalanced data , distinguishing the examples with non-dominant labels are more pivotal during training ( Menon et al. , 2020 ) . Deep networks often give large losses to such examples ( Figure 1 , right ) . Therefore , when learning under the realistic scenes , e.g. , learning with noisy imbalanced data , prior sample selection methods can not address such an issue well . To relieve the above issues , we study the uncertainty of losses in the sample selection procedure to combat noisy labels . To reduce the uncertainty of small-loss examples , we extend time intervals and utilize the mean of training losses at different training iterations . In consideration of the bad influence of mislabeled data on training losses , we build two robust mean estimators from the perspectives of soft truncation and hard truncation w.r.t . the truncation level , respectively . Soft truncation makes the mean estimation more robust by holistically changing the behavior of losses . Hard truncation makes the mean estimation more robust by locally removing outliers from losses . To reduce the uncertainty of large-loss examples , we encourage networks to pick the sample that has not been selected in a conservative way . Furthermore , to address the two issues simultaneously , we derive concentration inequalities ( Boucheron et al. , 2013 ) for robust mean estimation and further employ statistical confidence bounds ( Auer , 2002 ) to consider the number of times an example was selected during training . The study of uncertainty of losses in learning with noisy labels can be justified as follows . In statistical learning , it is known that uncertainty is related to the quality of data ( Vapnik , 2013 ) . Philosophically , we need variety decrease for selected data and variety search for unselected data , which share a common objective , i.e. , reduce the uncertainty of data to improve generalization ( Moore , 1990 ) . This is our original intention , since noisy labels could bring more uncertainty because of the low quality of noisy data . Nevertheless , due to the harm of noisy labels for generalization , we need to strike a good balance between variety decrease and search . Technically , our method is specially designed for handling noisy labels , which robustly uses network predictions and conservatively seeks less selected examples meanwhile to reduce the uncertainty of losses and then generalize well . Before delving into details , we clearly emphasize our contributions in two folds . First , we reveal prior sample selection criteria in learning with noisy labels have some potential weaknesses and discuss them in detail . The new selection criteria are then proposed with detailed theoretical analyses . Second , we experimentally validate the proposed method on both synthetic noisy balanced/imbalanced datasets and real-world noisy datasets , on which it achieves superior robustness compared with the stateof-the-art methods in learning with noisy labels . The rest of the paper is organized as follows . In Section 2 , we propose our robust learning paradigm step by step . Experimental results are discussed in Section 3 . The conclusion is given in Section 4 . 2 METHOD . In this section , we first introduce the problem setting and some background ( Section 2.1 ) . Then we discuss how to exploit training losses at different iterations ( Section 2.2 ) . Finally , we introduce the proposed method , which exploits training losses at different iterations more robustly and encourages networks to pick the sample that is less selected but could be correctly labeled ( Section 2.3 ) . 2.1 PRELIMINARIES . Let X and Y be the input and output spaces . Consider a k-class classification problem , i.e. , Y = [ k ] , where [ k ] = { 1 , . . . , k } . In learning with noisy labels , the training data are all sampled from a corrupted distribution on X × Y . We are given a sample with noisy labels , i.e. , S̃ = { ( x , ỹ ) } , where ỹ is the noisy label . The aim is to learn a robust classifier that could assign clean labels to test data by only exploiting a training sample with noisy labels . Let f : X → Rk be the classifier with learnable parameters w. At the i-th iteration during training , the parameters of the classifier f can be denoted as wi . Let ` : Rk × Y → R be a surrogate loss function for k-class classification . We exploit the softmax cross entropy loss in this paper . Given an arbitrary training example ( x , ỹ ) , at the i-th iteration , we can obtain a loss ` i , i.e. , ` i = ` ( f ( wi ; x ) , ỹ ) . Hence , until the t-th iteration , we can obtain a training loss set Lt about the example ( x , ỹ ) , i.e. , Lt = { ` 1 , . . . , ` t } . In this paper , we assume that the training losses in Lt conform to a Markov process , which is to represent a changing system under the assumption that future states only depend on the current state ( the Markov property ) . More specifically , at the i-th iteration , if we exploit an optimization algorithm for parameter updates ( e.g. , the stochastic gradient descent algorithm ( Bottou , 2012 ) ) and omit other dependencies ( e.g. , S̃ ) , we will have P ( wi|wi−1 , . . . , w0 ) = P ( wi|wi−1 ) , which means that the future state of the classifier f only depends on the current state . Furthermore , given a training example and the parameters of the classifier f , we can determine the loss of the training example as discussed . Therefore , the training losses in Lt will also conform to a Markov process . 2.2 EXTENDED TIME INTERVALS . As limited time interval can not address the instability issue of the estimation for the noisy class posterior well ( Pleiss et al. , 2020 ) , we extend time intervals and exploit the training losses at different training iterations for sample selection . One straightforward idea is to use the mean of training losses at different training iterations . Hence , the selection criterion could be µ̃ = 1 t t∑ i=1 ` i . ( 1 ) It is intuitive and reasonable to use such a selection criterion for sample selection , since the operation of averaging can mitigate the risks caused by the unstable estimation for the noisy class posterior , following better generalization . Nevertheless , such a method could arguably achieve suboptimal classification performance for learning with noisy labels . The main reason is that , due to the great harm of mislabeled data , part of training losses are with too large uncertainty and could be seen as outliers . Therefore , it could be biased to use the mean of training losses consisting of such outliers ( Diakonikolas et al. , 2020 ) , which further influences sample selection . More evaluations for our claims are provided in Section 3 .

Summary: This paper discusses the potential weaknesses in previous sample selection criteria in learning with noisy labels. And then propose a new selection criterion by incorporating the uncertainty of losses, together with theoretical justification. Experiments on both synthetic noisy balanced/imbalanced datasets and real-world noisy datasets validate the effectiveness of the proposed approach.