Robust Curriculum Learning: from clean label detection to noisy label self-correction

1 INTRODUCTION . The expressive power and high capacity of deep neural networks ( DNNs ) result in accurate modeling and promising generalization if provided with sufficient data and clean ( correct ) labels . However , recent studies show that the training process is fragile and can easily overfit on noisy labels ( Zhang et al. , 2017 ) , which commonly appear in real-world data since precise annotation is not always available or affordable . Hence , it is important to study the training dynamics affected by imperfect labels and develop robust learning strategies that ideally eliminate the negative impact of noisy labels while fully exploiting the information from all the available data . Numerous approaches have been developed to address this challenge from various perspectives , e.g. , loss correction ( Xiao et al. , 2015 ; Vahdat , 2017 ; Lee et al. , 2018 ; Veit et al. , 2017 ; Li et al. , 2017b ) , robust loss functions ( Ghosh et al. , 2017 ; Zhang & Sabuncu , 2018 ; Wang et al. , 2019 ; Ma et al. , 2020 ) with provable noise tolerance , sample re-weighting ( Patrini et al. , 2017 ) , curriculum learning ( Kumar et al. , 2010 ; Jiang et al. , 2018 ; Guo et al. , 2018 ) , model co-teaching ( Han et al. , 2018 ) , etc . A principal methodology behind a variety of methods is to detect clean labels while discard/downweigh the data with wrong labels , so the model mainly learns from correct labels . A broadly applied criterion is to select the samples with small losses and treat them as clean data . It is inspired by empirical observations that DNNs learn simple patterns first before overfitting on the noisy labels ( Zhang et al. , 2017 ; Arpit et al. , 2017 ) . Several curriculum learning methods utilize this criterion ( Kumar et al. , 2010 ; Jiang et al. , 2014 ) , and in each step , select/upweigh samples with small losses . Robust loss functions also suppress the large losses associated with the possibly wrong labels . More recent approaches use mixture models ( Arazo et al. , 2019 ) to estimate the distribution of losses for clean and noisy data . However , the instantaneous loss ( i.e. , the loss evaluated at the current step ) of an individual sample is an unstable signal that can rapidly fluctuate due to DNN training ’ s randomness . The error generated by such an unstable metric accumulates when the selected samples are used to train the model producing the losses . Co-teaching methods alleviate this problem by training two DNNs and using the loss computed on one model to guild the other . Also , as the model changes during training , each sample ’ s loss needs to be re-evaluated even when it is not selected , which requires extra inference cost . MentorNet ( Jiang et al. , 2018 ) and Data Parameters ( Saxena et al. , 2019 ) train an extra model to produce the sample weights or selection results without computing the loss . Furthermore , it may not be efficient to repeatedly train the model only on clean data that consistently have small losses , since the model have already learned , well memorized or overfitted to them . A primary drawback of training only on clean labels detected is that discarding the whole data pairs ( x , y ) with wrong labels y removes potentially useful information about the data distribution p ( x ) ( Arazo et al. , 2019 ) . Hence , there has been growing interest in leveraging noisy data . Loss correction methods aim to correct the predicted class probabilities based on an estimated mislabeling probability between classes . Some other methods seek to relabel them by using the model itself ( e.g. , bootstrapping loss ( Reed et al. , 2014 ) ) or another model/mechanism ( e.g. , directed graphical models , conditional random fields , or CNNs ) trained on an additional set of clean data , which , however , is not always available . Self-training and unsupervised learning techniques ( Rasmus et al. , 2015 ; Berthelot et al. , 2019 ) have also been employed to generate pseudo labels to replace noisy labels ( Arazo et al. , 2019 ) . The pseudo labels are optimized together with the model or generated by the model with data augmentations to encourage the output consistency on the same sample ’ s augmentations . Unfortunately , the pseudo labels ’ quality may vary across different samples and significantly degenerate when the noise ratio is high , or the model fails to produce stable and correct predictions . In such a case , the relabeling error on some samples can be accumulated during training . In this paper , we address the aforementioned problems of noise-label learning by developing a curriculum learning strategy called Robust Curriculum Learning ( RoCL ) that smoothly transitions between two phases : ( 1 ) detection and supervised training on clean data ; and ( 2 ) relabeling and self-supervision on noisy data . Specifically , we train the model for multiple episodes , each starting from phase ( 1 ) and gradually moving to phase ( 2 ) . Unlike existing approaches , we only select samples with accurate given/pseudo labels that are most informative to the current model training . Our data selection criterion takes both the dynamics of per-sample loss and output consistency ( across multiple data augmentations ) into account . Using an exponential moving average of the loss and consistency over training history , it overcomes the instability of instantaneous losses and does not incur any additional inference cost . In addition , by adjusting a temperature parameter , the criterion can interpolate between the two phases and keep the training focusing on the data that the model mostly needs to improve on , e.g. , clean data with unsatisfying output consistency or wrongly-labeled data with accurate pseudo labels . Thus , we can fully exploit both clean and noisy data more efficiently with less risk of introducing extra noise or error accumulation . We further show that our data selection can be derived from a novel optimization formulation for robust curriculum learning . We evaluate our method on multiple noisy learning benchmarks and show that our method outperforms a diverse set of recent noisy-label learning approaches . 1.1 RELATED WORK . Early curriculum learning ( CL ) ( Khan et al. , 2011 ; Basu & Christensen , 2013 ; Spitkovsky et al. , 2009 ; Zhou et al. , 2021 ) seeks an optimized sequence of training samples ( i.e. , a curriculum , which can be designed by human experts ) to improve model performance . Self-paced learning ( SPL ) ( Kumar et al. , 2010 ; Tang et al. , 2012a ; Supancic III & Ramanan , 2013 ; Tang et al. , 2012b ) selects easy samples with smaller losses . It starts with selecting a few samples of small loss and gradually increases the selection size to cover all the training data . Self-paced curriculum learning ( Jiang et al. , 2015 ) combines the human expert in CL and loss-adaptation in SPL . SPL with diversity ( SPLD ) ( Jiang et al. , 2014 ) applies a negative group sparse regularization to SPL to promote the diversity of selected samples . Minimax curriculum learning Zhou & Bilmes ( 2018 ) promotes the diversity of samples during early learning to encourage exploration and focus on hard samples in later stages . In the context of robust learning with noisy labels , label correction methods aim to identify the wrong labels and possibly correct them to get more consistent labels for training . Previous work often apply an extra noise model ( directed graphical model ( Xiao et al. , 2015 ) , conditional random fields ( Vahdat , 2017 ) , neural network ( Lee et al. , 2018 ; Veit et al. , 2017 ) , knowledge graph ( Li et al. , 2017b ) ) to correct the noisy labels , which often require extra clean data and as well as training/inference of the noise model . Another line of research focuses on loss correction , which modifies the loss or prediction probabilities during training to correct the misinformation from the noisy labels . Patrini et al . ( 2017 ) uses two noise transition ( backward and forward ) matrices to correct the prediction probabilities . Label Smoothing Regularization ( Szegedy et al. , 2016 ; Pereyra et al. , 2017 ) alleviates the overfitting to noisy labels by using soft labels instead of one-hot labels . Reed et al . ( 2014 ) augments the loss with a notion of perceptual consistency . Jiang et al . ( 2018 ) trains a mentor network to reweigh samples duri ‘ ng the training of a student network . Guo et al . ( 2018 ) designs a curriculum by ranking the complexity of data using its distribution density in a feature space . Ren et al . ( 2018 ) proposes a meta-learning algorithm that learns to assign weights to samples based on their gradients in training compared to those of validation data , which requires extra clean data . Co-teaching ( Han et al. , 2018 ) feeds in the network with the most confident samples of another network to reduce confirmation bias . Amid et al . ( 2019 ) generalizes the logistic loss and the exponents in the softmax by applying a temperature to each of them and makes the training more robust to noise . Hu et al . ( 2019 ) trains a network on noisy labels in the weakly supervised setting and uses it as a regularization term to improve the training on clean data . Some approaches focus on designing loss functions that have robust behaviors and provable tolerance to label noise . Ghosh et al . ( 2017 ) theoretically proves that the Mean Absolute Error ( MAE ) is a robust loss . The Generalized Cross Entropy ( Zhang & Sabuncu , 2018 ) uses a negative Box-Cox transformation to obtain a loss function that generalizes MAE and Cross Entropy loss . Wang et al . ( 2019 ) proposes a Symmetric Cross Entropy that combines Cross Entropy loss and Reverse Cross Entropy loss . Ma et al . ( 2020 ) proposes a loss normalization method and shows that any loss can be made robust to noisy labels . RoCL shares similar ideas with some CL methods in that RoCL starts with learning easy and clean samples and gradually moves to hard and noisy ones . RoCL is more related to the loss correction approach in noisy-label learning literature as RoCL generates a curriculum dynamically assigning weight ( probability ) to each sample . RoCL differs from existing methods in : ( 1 ) it only selects a subset of informative and reliable labels for training in each epoch ; ( 2 ) it is a smooth transition not only from clean data to noisy data but also from supervised learning to self-supervision ; ( 3 ) it runs multiple episodes of the curriculum to avoid getting in a local minimum dominated by a small set of clean/noisy data or a specific type of loss ; ( 4 ) it does not assume the availability of an extra set of clean data ; ( 5 ) it does not require extra computation or any modification to the model . 2 DYNAMIC PATTERNS OF CLEAN/NOISY LABELS IN TRAINING . 2.1 LOSS DYNAMICS AND CLEAN LABEL DETECTION . A key challenge for most noise-label learning methods is to design a reliable criterion to select/reweigh clean data and distinguish them from the noisy data , so all the clean data can be fully exploited while most noisy labels are filtered out of the training process . Loss computed at an instantaneous step have been widely used for this purpose according to the observation that the loss on clean data is usually smaller than noisy data . One important reason is that the clean labels are mutually consistent with each other in producing gradient updates , and therefore , the model can fit them better and faster . On the other hand , the noisy labels may contain mutually inconsistent information , creating a form of long-lasting “ tug of war ” amongst themselves . For example , it can be hard for the model to find consistent visual patterns from images with noisy labels to make the desired predictions . However , instantaneous loss suffers from high variance across training epochs ( as shown in the first plot of Figure 1 ) and is inaccurate for clean data detection under high noise ratio ( i.e. , the proportion of wrong labels is high ) and the randomness of DNN training , e.g. , random initialization , random data augmentation , etc . Moreover , it needs to evaluate the instantaneous loss for all samples in each step , resulting in extra inference cost on unselected samples . The dynamic patterns of losses ( Zhou et al. , 2020b ) over the course of training give us a new insight for better clean data detection even when the noise ratio ( proportion of wrong labels ) is high . In particular , we hypothesize that a sample ’ s label is more likely to be correct if its losses persistently retain low values over training steps . Given a sample ( xi , yi ) with xi being the features and yi being the label , we describe its loss dynamics using a simple exponential moving average ( EMA ) of the instantaneous loss ` ( f ( xi ; θt ) , yi ) ( where f ( xi ; θt ) denotes the model output and θt is the model parameters at step t * along the training history , which is defined and computed recursively as lt+1 ( i ) = { γ × ` ( f ( xi ; θt ) , yi ) + ( 1− γ ) × lt ( i ) if i ∈ St lt ( i ) else , ( 1 ) Where γ ∈ [ 0 , 1 ] is a discounting factor , V is the set of all n training samples , and St ⊆ V is the set of samples selected ( by a certain curriculum ) for training at epoch t. We only update the EMA loss for selected samples using the byproduct lt ( i ) of training without requiring extra inference . In the first and the third plots of Figure 1 , we show how the losses and EMA losses associated with clean/noisy data change throughout the training process . Specifically , we train a ResNet34 ( He et al. , 2016 ) model on CIFAR10 with 60 % of the original labels randomly changed to a wrong class . To avoid quick overfitting to the noise , we train the model for multiple episodes ( each composed of several epochs over all data with a cosine annealing learning rate ) and alternate between the supervised learning episode that minimizes the cross-entropy loss against the given noisy labels and the self-supervision episode that minimizes the consistency loss Eq . ( 2 ) against the pseudo labels . Comparing the shaded areas ( std ) of instantaneous loss and EMA loss and the gap between curves in the two plots , we see that EMA leads to smaller variance within each group and larger gap between the clean and noisy groups , demonstrating the effectiveness of EMA loss for clean data detection .

Summary: This article is concerned with the problem of training models under noisy data. The authors first adopt the loss and output consistency for data selection. EMA method is used for smoothing to obtain more accurate clean label detection. Meanwhile, through the introduction of temperature hyperparameters, the model gradually completes the transition from supervised learning using clean labels to self-supervised learning using noisy labels.