Semi-supervised learning by selective training with pseudo labels via confidence estimation

1 INTRODUCTION . Semi-supervised learning ( SSL ) is a powerful technique to deliver a full potential of complex models , such as deep neural networks , by utilizing unlabeled data as well as labeled data to train the model . It is especially useful in some practical situations where obtaining labeled data is costly due to , for example , necessity of expert knowledge . Since deep neural networks are known to be “ data-hungry ” models , SSL for deep neural networks has been intensely studied and has achieved surprisingly good performance in recent works ( Van Engelen & Hoos , 2020 ) . In this paper , we focus on SSL for a classification task , which is most commonly tackled in the literature . Many recent SSL methods adopt a common approach in which two processes are iteratively conducted : generating pseudo labels of unlabeled data by using a currently training model and updating the model by using both labeled and pseudo-labeled data . In the pioneering work ( Lee , 2013 ) , pseudo labels are hard ones , which are represented by one-hot vectors , but recent methods ( Tarvainen & Valpola , 2017 ; Miyato et al. , 2018 ; Berthelot et al. , 2019 ; 2020 ; Verma et al. , 2019 ; Wang et al. , 2019 ; Zhang & Qi , 2020 ) often utilize soft pseudo-labels , which may contain several nonzero elements within each label vector . One simple reason to adopt soft pseudo-labels is to alleviate confirmation bias caused by training with incorrectly pseudo-labeled data , and this attempt seems to successfully contribute to the excellent performance of those methods . However , since soft pseudolabels only provide weak supervisions , those methods often show slow convergence in the training ( Lokhande et al. , 2020 ) . For example , MixMatch ( Berthelot et al. , 2019 ) , which is one of the stateof-the-art SSL methods , requires nearly 1,000,000 iterations for training with CIFAR-10 dataset . On the other hand , in this paper , we aim to utilize hard pseudo-labels to design an easy-to-try SSL method in terms of computational efficiency . Obviously , the largest problem to be tackled in this approach is how to alleviate the negative impact caused by training with the incorrect pseudo-labels . In this work , we propose a novel SSL method that adopts selective training with pseudo labels . To avoid to train a model with incorrect pseudo-labels , we explicitly select which pseudo-labeled data should be used to update the model . Specifically , assuming that loss on incorrectly pseudo-labeled data sensitively increase against data augmentation , we select the data corresponding to relatively small loss after applying data augmentation . To effectively conduct this selective training , we estimate confidence of pseudo labels and utilize it not only for screening candidates of pseudo-labeled data to be selected but also for automatically deciding how many pseudo-labeled data should be selected within a mini-batch . For accurate estimation of the confidence , we also propose a new data augmentation method , called MixConf , that enables us to obtain confidence-calibrated models even when the number of training data is small . Experimental results with several benchmark datasets validate the advantage of our SSL method as well as MixConf . 2 PROPOSED METHOD . Figure 2 shows an overview of our method . Given a mini-batch from labeled data and that from unlabeled data , we first generate pseudo labels of the unlabeled data based on predictions of the current model . Let x ∈ Rm , y ∈ { 1 , 2 , ... C } , and f : Rm → RC denote input data , labels , and the classifier to be trained , respectively . Given the input unlabeled data xU , the pseudo label ŷU is generated by simply taking argmax of the classifier ’ s output f ( xU ) . Then , we conduct selective training using both the labeled data and the pseudo-labeled data . In this training , to alleviate negative effect caused by training with incorrect pseudo-labels , we explicitly select which data should be used to update the model . Below , we describe details of this selective training . 2.1 SELECTIVE TRAINING WITH PSEUDO LABELS BASED ON CONFIDENCE . As described previously , the pseudo labels are generated based on the predictions of the current model , and we assume that the confidence of those predictions can be also computed in addition to the pseudo labels . When we use a popular architecture of deep neural networks , it can be obtained by simply taking max of the classifier ’ s output ( Hendrycks & Gimpel , 2016 ) as : ci = max j∈ { 1,2 , ... , C } f ( xUi ) [ j ] , ( 1 ) where ci is the confidence of the classifier ’ s prediction on the i-th unlabeled data xUi , and f ( x ) [ j ] is the j-th element of f ( x ) . When the model is sufficiently confidence-calibrated , the confidence ci is expected to match the accuracy of the corresponding prediction f ( xUi ) ( Guo et al. , 2017 ) , which means it also matches the probability that the pseudo label ŷUi is correct . To avoid training with incorrect pseudo-labels , we explicitly select the data to be used to train the model based on the confidence . This data selection comprises two steps : thresholding the confidence and selecting relatively small loss calculated with augmented pseudo-labeled data . The first step is quite simple ; we pick up the pseudo-labeled data that have higher confidence than a certain threshold cthr and discard the remaining . In the second step , MixConf , which will be introduced later but is actually a variant of Mixup ( Zhang et al. , 2018 ) , is applied to both the labeled and unlabeled data to augment them . As conducted in ( Berthelot et al. , 2019 ) , we shuffle all data and mix them with the original labeled and pseudo-labeled data , which results in { ( x̃Li , p̃Li ) } BL i=1 and { ( x̃Uj , p̃Uj ) } BU j=1 , respectively , where p ∈ RC is a vector-style representation of the label that is adopted to represent a mixed label . Then , we calculate the standard cross entropy loss for each mixed data . Finally , we select the mixed data that result in relatively small loss among the all augmented data , and only the corresponding small-loss is minimized to train the model . Why does the small-loss selection work ? Our important assumption is that the loss calculated with incorrect labels tends to sensitively increase when the data is augmented . This assumption would be supported by effectiveness of the well-known technique , called test-time augmentation ( Simonyan & Zisserman , 2015 ) , in which incorrect predictions are suppressed by taking an average of the model ’ s outputs over several augmentations . Since we conduct the confidence thresholding , the loss corresponding to the pseudo-labeled data is guaranteed to be smaller than a certain loss level defined by the threshold cthr . However , when we apply data augmentation , that is MixConf , to the pseudo-labeled data , the loss related to incorrectly pseudo-labeled data becomes relatively large , if the above assumption is valid . It means that selecting relatively small loss after applying MixConf leads to excluding incorrect pseudo-labels , and we can safely train the model by using only the selected data . Han et al . ( 2018 ) and Lokhande et al . ( 2020 ) have presented similar idea , called small-loss trick ( Han et al. , 2018 ) or speed as a supervisor ( Lokhande et al. , 2020 ) , to avoid training with incorrect labels . However , their assumption is different from ours ; it is that loss of incorrectly labeled data decreases much slower than that of correctly labeled data during training . Due to this assumption , their methods require joint training of two distinct models ( Han et al. , 2018 ) or nested loop for training ( Lokhande et al. , 2020 ) to confirm which data show relatively slow convergence during training , which leads to substantially large computational cost . On the other hand , since our method focuses on change of loss values against data augmentation , not that during training , we can efficiently conduct the selective training by just utilizing data augmentation in each iteration . Since the confidence of the pseudo label represents the probability that the pseudo label is correct , we can estimate how many data we should select based on the confidence by calculating an expected number of the mixed data generated from two correctly labeled data . Specifically , when the averaged confidence within the unlabeled data is equal to cave , the number of the data to be selected can be determined as follows : nL = BL + caveBU BL +BU BL , ( 2 ) nU = min ( BL , BL + caveBU BL +BU caveBU ) , ( 3 ) where nL is for the data generated by mixing the labeled data and shuffled data , and nU is for those generated by mixing the unlabeled data and shuffled data . Here , to avoid too much contribution from the pseudo-labeled data , we restrict nU to be smaller than BL . Within this restriction , we can observe that , if we aim to perfectly balance nL and nU , BU should be set to BL/cave . However , cave can not be estimated before training and can fluctuate during training . Therefore , for stable training , we set BU = BL/cthr instead and fix it during training . Finally , the total loss L to be minimized in our method is formulated as the following equation : L = 1 BL nL∑ i=1 l ( x̃Ls [ i ] , p̃ L s [ i ] ) + λU 1 BL nU∑ j=1 l ( x̃Ut [ j ] , p̃ U t [ j ] ) , ( 4 ) where l is the standard cross entropy loss , s and t represent the sample index sorted by loss in an ascending order within each mini-batch , and λU is a hyper-parameter that balances the two terms . To improve the accuracy of pseudo labels as well as their confidence , we can average the model ’ s outputs over K augmentations to estimate pseudo labels as conducted in ( Berthelot et al. , 2019 ) . In that case , we conduct MixConf for all augmented pseudo-labeled data , which results in K minibatches each of which containsBU mixed data . Therefore , we need to modify the second term in the right-hand side of Eq . ( 4 ) to take the average of losses over all the mini-batches . In our experiments , we used K = 4 except for an ablation study . 2.2 MIXCONF TO OBTAIN BETTER CALIBRATED MODELS . In the previous section , we assumed that the model is sufficiently confidence-calibrated , but deep neural networks are often over-confident on their predictions in general ( Guo et al. , 2017 ) . This problem gets bigger in case of training with a small-scale dataset as we will show in our experiments . Consequently , it should occur in our SSL setting , because there are only a small amount of labeled training data in the early stage of the training . If the confidence is over-estimated , incorrect pseudolabels are more likely to be selected to calculate the loss due to loose confidence-thresholding and over-estimated ( nL , nU ) , which should significantly degrade the performance of the trained model . To tackle this problem , we propose a novel data augmentation method , called MixConf , to obtain well-calibrated models even when the number of training data is small . MixConf basically follows the scheme of Mixup , which is known to contribute to model ’ s calibration ( Thulasidasan et al. , 2019 ) , but is more carefully designed for confidence calibration . Figure 2 shows an overview of MixConf . In a similar way with Mixup , MixConf randomly picks up two samples { ( x0 , p0 ) , ( x1 , p1 ) } from the given training dataset and generates a new training sample ( x̃ , p̃ ) by linearly interpolating these samples as the following equations : x̃ = λax0 + ( 1− λa ) x1 , ( 5 ) p̃ = λbp0 + ( 1− λb ) p1 , ( 6 ) where λa ∈ [ 0 , 1 ] and λb ∈ [ 0 , 1 ] denote interpolation ratios for data and labels , respectively . Note that λa is not restricted to be equal to λb in MixConf , while λa = λb in Mixup . Since Mixup is not originally designed to obtain confidence-calibrated models , we have to tackle the following two questions to obtain better calibrated models by such a Mixup-like data augmentation method : • How should we set the ratio for the data interpolation ? ( In case of Mixup , λa is randomly sampled from the beta distribution ) • How should we determine the labels of the interpolated data ? ( In case of Mixup , λb is set to be equal to λa ) We first tackle the second question to clarify what kind of property the generated samples should have . Then , we derive how to set λa and λb so that the generated samples have this property .

Summary: The paper uses selective training with pseudo labels. Specifically, the method selects the pseudo-labeled data associated  with small loss after performing  the data augmentation, and then uses  the selected data for training the model. Here, the model computes the confidence of the pseudo labels and then puts a threshold to determine  the number of the selected samples and ignore inaccurate pseudo labels. Moreover, MixConf, a variation of mixup, for data augmentation is proposed  to train a more confidence calibrated model. Finally, experimental results on the standard datasets  show the effectiveness of the proposed model compared to SOA SSL methods.