Sample Selection with Uncertainty of Losses for Learning with Noisy Labels

1 INTRODUCTION . Learning with noisy labels is one of the most challenging problems in weakly-supervised learning , since noisy labels are ubiquitous in the real world ( Mirzasoleiman et al. , 2020 ; Yu et al. , 2019 ; Nishi et al. , 2021 ; Arazo et al. , 2019 ; Yang et al. , 2021a ; Bai & Liu , 2021 ) . For instance , both crowdsourcing and web crawling yield large numbers of noisy labels everyday ( Han et al. , 2018 ) . Noisy labels can severely impair the performance of deep neural networks with strong memorization capacities ( Zhang et al. , 2017 ; Zhang & Sabuncu , 2018 ; Pleiss et al. , 2020 ; Lukasik et al. , 2020 ; Chen et al. , 2022 ) . To reduce the influence of noisy labels , a lot of approaches have been recently proposed ( Natarajan et al. , 2013 ; Liu & Tao , 2016 ; Ma et al. , 2018 ; Yang et al. , 2021b ; Zheng et al. , 2020 ; Xia et al. , 2019 ; 2020 ; Tanaka et al. , 2018 ; Malach & Shalev-Shwartz , 2017 ; Li et al. , 2020b ; Menon et al. , 2018 ; Thekumparampil et al. , 2018 ; Xu et al. , 2019 ; Kim et al. , 2019 ; Jiang et al. , 2020 ; Harutyunyan et al. , 2020 ) . They can be generally divided into two main categories . The first one is to estimate the noise transition matrix ( Patrini et al. , 2017 ; Shu et al. , 2020 ; Hendrycks et al. , 2018 ; Yang et al. , 2021c ; Wu et al. , 2022 ) , which denotes the probabilities that clean labels flip into noisy labels . However , the noise transition matrix is hard to be estimated accurately , especially when the number of classes is large ( Yu et al. , 2019 ) . The second approach is sample selection , which is our focus in this paper . This approach is based on selecting possibly clean examples from a mini-batch for training ( Han et al. , 2018 ; Wang et al. , 2018 ; Yao et al. , 2020a ; Wang et al. , 2019 ; Yu et al. , 2019 ; Lee et al. , 2019 ; Wang et al. , 2019 ; Yao et al. , 2022 ) . Intuitively , if we can exploit less noisy data for network parameter updates , the network will be more robust . A major question in sample selection is what criteria can be used to select possibly clean examples . At the present stage , the selection based on the small-loss criteria is the most common method , and has been verified to be effective in many circumstances ( Han et al. , 2018 ; Jiang et al. , 2018 ; Yu et al. , †Corresponding author 2019 ; Wei et al. , 2020 ; Yao et al. , 2020a ) . Specifically , since deep networks learn patterns first ( Arpit et al. , 2017 ) , they would first memorize training data of clean labels and then those of noisy labels with the assumption that clean labels are of the majority in a noisy class . Small-loss examples can thus be regarded as clean examples with high probability . Therefore , in each iteration , prior methods ( Han et al. , 2018 ; Wei et al. , 2020 ) select the small-loss examples based on the predictions of the current network for robust training . However , such a selection procedure is debatable , since it arguably does not consider uncertainty in selection . The uncertainty comes from two aspects . First , this procedure has uncertainty about small-loss examples . Specifically , the procedure uses limited time intervals and only exploits the losses provided by the current predictions . For this reason , the estimation for the noisy class posterior is unstable ( Yao et al. , 2020b ) , which causes the network predictions to be equally unstable . It thus takes huge risks to only use losses provided by the current predictions ( Figure 1 , left ) . Once wrong selection is made , the inferiority of accumulated errors will arise ( Yu et al. , 2019 ) . Second , this procedure has uncertainty about large-loss examples . To be specific , deep networks learn easy examples at the beginning of training , but ignore some clean examples with large losses . Nevertheless , such examples are always critical for generalization . For instance , when learning with imbalanced data , distinguishing the examples with non-dominant labels are more pivotal during training ( Menon et al. , 2020 ) . Deep networks often give large losses to such examples ( Figure 1 , right ) . Therefore , when learning under the realistic scenes , e.g. , learning with noisy imbalanced data , prior sample selection methods can not address such an issue well . To relieve the above issues , we study the uncertainty of losses in the sample selection procedure to combat noisy labels . To reduce the uncertainty of small-loss examples , we extend time intervals and utilize the mean of training losses at different training iterations . In consideration of the bad influence of mislabeled data on training losses , we build two robust mean estimators from the perspectives of soft truncation and hard truncation w.r.t . the truncation level , respectively . Soft truncation makes the mean estimation more robust by holistically changing the behavior of losses . Hard truncation makes the mean estimation more robust by locally removing outliers from losses . To reduce the uncertainty of large-loss examples , we encourage networks to pick the sample that has not been selected in a conservative way . Furthermore , to address the two issues simultaneously , we derive concentration inequalities ( Boucheron et al. , 2013 ) for robust mean estimation and further employ statistical confidence bounds ( Auer , 2002 ) to consider the number of times an example was selected during training . The study of uncertainty of losses in learning with noisy labels can be justified as follows . In statistical learning , it is known that uncertainty is related to the quality of data ( Vapnik , 2013 ) . Philosophically , we need variety decrease for selected data and variety search for unselected data , which share a common objective , i.e. , reduce the uncertainty of data to improve generalization ( Moore , 1990 ) . This is our original intention , since noisy labels could bring more uncertainty because of the low quality of noisy data . Nevertheless , due to the harm of noisy labels for generalization , we need to strike a good balance between variety decrease and search . Technically , our method is specially designed for handling noisy labels , which robustly uses network predictions and conservatively seeks less selected examples meanwhile to reduce the uncertainty of losses and then generalize well . Before delving into details , we clearly emphasize our contributions in two folds . First , we reveal prior sample selection criteria in learning with noisy labels have some potential weaknesses and discuss them in detail . The new selection criteria are then proposed with detailed theoretical analyses . Second , we experimentally validate the proposed method on both synthetic noisy balanced/imbalanced datasets and real-world noisy datasets , on which it achieves superior robustness compared with the stateof-the-art methods in learning with noisy labels . The rest of the paper is organized as follows . In Section 2 , we propose our robust learning paradigm step by step . Experimental results are discussed in Section 3 . The conclusion is given in Section 4 . 2 METHOD . In this section , we first introduce the problem setting and some background ( Section 2.1 ) . Then we discuss how to exploit training losses at different iterations ( Section 2.2 ) . Finally , we introduce the proposed method , which exploits training losses at different iterations more robustly and encourages networks to pick the sample that is less selected but could be correctly labeled ( Section 2.3 ) . 2.1 PRELIMINARIES . Let X and Y be the input and output spaces . Consider a k-class classification problem , i.e. , Y = [ k ] , where [ k ] = { 1 , . . . , k } . In learning with noisy labels , the training data are all sampled from a corrupted distribution on X × Y . We are given a sample with noisy labels , i.e. , S̃ = { ( x , ỹ ) } , where ỹ is the noisy label . The aim is to learn a robust classifier that could assign clean labels to test data by only exploiting a training sample with noisy labels . Let f : X → Rk be the classifier with learnable parameters w. At the i-th iteration during training , the parameters of the classifier f can be denoted as wi . Let ` : Rk × Y → R be a surrogate loss function for k-class classification . We exploit the softmax cross entropy loss in this paper . Given an arbitrary training example ( x , ỹ ) , at the i-th iteration , we can obtain a loss ` i , i.e. , ` i = ` ( f ( wi ; x ) , ỹ ) . Hence , until the t-th iteration , we can obtain a training loss set Lt about the example ( x , ỹ ) , i.e. , Lt = { ` 1 , . . . , ` t } . In this paper , we assume that the training losses in Lt conform to a Markov process , which is to represent a changing system under the assumption that future states only depend on the current state ( the Markov property ) . More specifically , at the i-th iteration , if we exploit an optimization algorithm for parameter updates ( e.g. , the stochastic gradient descent algorithm ( Bottou , 2012 ) ) and omit other dependencies ( e.g. , S̃ ) , we will have P ( wi|wi−1 , . . . , w0 ) = P ( wi|wi−1 ) , which means that the future state of the classifier f only depends on the current state . Furthermore , given a training example and the parameters of the classifier f , we can determine the loss of the training example as discussed . Therefore , the training losses in Lt will also conform to a Markov process . 2.2 EXTENDED TIME INTERVALS . As limited time interval can not address the instability issue of the estimation for the noisy class posterior well ( Pleiss et al. , 2020 ) , we extend time intervals and exploit the training losses at different training iterations for sample selection . One straightforward idea is to use the mean of training losses at different training iterations . Hence , the selection criterion could be µ̃ = 1 t t∑ i=1 ` i . ( 1 ) It is intuitive and reasonable to use such a selection criterion for sample selection , since the operation of averaging can mitigate the risks caused by the unstable estimation for the noisy class posterior , following better generalization . Nevertheless , such a method could arguably achieve suboptimal classification performance for learning with noisy labels . The main reason is that , due to the great harm of mislabeled data , part of training losses are with too large uncertainty and could be seen as outliers . Therefore , it could be biased to use the mean of training losses consisting of such outliers ( Diakonikolas et al. , 2020 ) , which further influences sample selection . More evaluations for our claims are provided in Section 3 .

Summary: The paper proposes a novel sample selection method for learning with noisy labels (LNL). Based on the typical "small-loss" assumption, the motivation is to consider uncertainty about large-loss samples in order to distinguish the two confounding cases: truly mislabeled samples, or clean yet underrepresented samples that are less frequently selected or learned by the model so far. The proposed method explores robust mean estimation to summarize the per-sample loss trajectory, through soft truncation and hard truncation. Concentration inequalities are used to obtain the final selection criteria to perform conservative search. Results are overall promising, in particular showing very good results dealing with label noise + extreme class imbalance (Figure 2).