In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning

1 INTRODUCTION . The recent extraordinary success of deep learning methods can be mostly attributed to advancements in learning algorithms and the availability of large-scale labeled datasets . However , constructing large labeled datasets for supervised learning tends to be costly and is often infeasible . Several approaches have been proposed to overcome this dependency on huge labeled datasets ; these include semi-supervised learning ( Berthelot et al. , 2019 ; Tarvainen & Valpola , 2017 ; Miyato et al. , 2018 ; Lee , 2013 ) , self-supervised learning ( Doersch et al. , 2015 ; Noroozi & Favaro , 2016 ; Chen et al. , 2020a ) , and few-shot learning ( Finn et al. , 2017 ; Snell et al. , 2017 ; Vinyals et al. , 2016 ) . Semi-supervised learning ( SSL ) is one of the most dominant approaches for solving this problem , where the goal is to leverage a large unlabeled dataset alongside a small labeled dataset . One common assumption for SSL is that decision boundaries should lie in low density regions ( Chapelle & Zien , 2005 ) . Consistency-regularization based methods achieve this by making the network outputs invariant to small input perturbations ( Verma et al. , 2019 ) . However , one issue with these methods is that they often rely on a rich set of augmentations , like affine transformations , cutout ( DeVries & Taylor , 2017 ) , and color jittering in images , which limits their capability for domains where these augmentations are less effective ( e.g . videos and medical images ) . Pseudo-labeling based methods select unlabeled samples with high confidence as training targets ( pseudo-labels ) ; this can be viewed as a form of entropy minimization , which reduces the density of data points at the decision boundaries ( Grandvalet & Bengio , 2005 ; Lee , 2013 ) . One advantage of pseudo-labeling over consistency regularization is that it does not inherently require augmentations and can be generally applied to most domains . However , recent consistency regularization approaches tend to outperform pseudo-labeling on SSL benchmarks . This work is in defense of pseudo-labeling : we demonstrate that pseudo-labeling based methods can perform on par with consistency regularization methods . Although the selection of unlabeled samples with high confidence predictions moves decision boundaries to low density regions in pseudo-labeling based approaches , many of these selected predictions are incorrect due to the poor calibration of neural networks ( Guo et al. , 2017 ) . Since , calibration measures the discrepancy between the confidence level of a network ’ s individual predictions and its overall accuracy ( Dawid , 1982 ; Degroot & Fienberg , 1983 ) ; for poorly calibrated networks , an incorrect prediction might have high confidence . We argue that conventional pseudo-labeling based methods achieve poor results because poor network calibration produces incorrectly pseudo-labeled samples , leading to noisy training and poor generalization . To remedy this , we empirically study the relationship between output prediction uncertainty and calibration . We find that selecting predictions with low uncertainty greatly reduces the effect of poor calibration , improving generalization . Motivated by this , we propose an uncertainty-aware pseudo-label selection ( UPS ) framework that leverages the prediction uncertainty to guide the pseudo-label selection procedure . We believe pseudolabeling has been impactful due to its simplicity , generality , and ease of implementation ; to this end , our proposed framework attempts to maintain these benefits , while addressing the issue of calibration to drastically improve PL performance . UPS does not require modality-specific augmentations and can leverage most uncertainty estimation methods in its selection process . Furthermore , the proposed framework allows for the creation of negative pseudo-labels ( i.e . labels which specify the absence of specific classes ) . If a network predicts the absence of a class with high confidence and high certainty , then a negative label can be assigned to that sample . This generalization is beneficial for both single-label and multi-label learning . In the single-label case , networks can use these labels for negative learning ( Kim et al. , 2019 ) 1 ; in the multi-label case , class presence is independent so both positive and negative labels are necessary for training . Our key contributions include the following : ( 1 ) We introduce UPS , a novel uncertainty-aware pseudo-label selection framework which greatly reduces the effect of poor network calibration on the pseudo-labeling process , ( 2 ) While prior SSL methods focus on single-label classification , we generalize pseudo-labeling to create negative labels , allowing for negative learning and multi-label classification , and ( 3 ) Our comprehensive experimentation shows that the proposed method achieves strong performance on commonly used benchmark datasets CIFAR-10 and CIFAR-100 . In addition , we highlight our method ’ s flexibility by outperforming previous state-of-the-art approaches on the video dataset , UCF-101 , and the multi-label Pascal VOC dataset . 2 RELATED WORKS . Semi-supervised learning is a heavily studied problem . In this work , we mostly focus on pseudolabeling and consistency regularization based approaches as currently , these are the dominant approaches for SSL . Following ( Berthelot et al. , 2019 ) , we refer to the other SSL approaches for interested readers which includes : “ transductive ” models ( Gammerman et al. , 1998 ; Joachims , 1999 ; 2003 ) , graph-based methods ( Zhu et al. , 2003 ; Bengio et al. , 2006 ; Liu et al. , 2019 ) , generative modeling ( Belkin & Niyogi , 2002 ; Lasserre et al. , 2006 ; Kingma et al. , 2014 ; Pu et al. , 2016 ) . Furthermore , several recent self-supervised approaches ( Grill et al. , 2020 ; Chen et al. , 2020b ; Caron et al. , 2020 ) , have shown strong performance when applied to the SSL task . For a general overview of SSL , we point to ( Chapelle et al. , 2010 ; Zhu , 2005 ) . Pseudo-labeling The goal of pseudo-labeling ( Lee , 2013 ; Shi et al. , 2018 ) and self-training ( Yarowsky , 1995 ; McClosky et al. , 2006 ) is to generate pseudo-labels for unlabeled samples with a model trained on labeled data . In ( Lee , 2013 ) , pseudo-labels are created from the predictions of a trained neural network . Pseudo-labels can also be assigned to unlabeled samples based on neighborhood graphs ( Iscen et al. , 2019 ) . Shi et al . ( 2018 ) extend the idea of pseudo-labeling by incorporating confidence scores for unlabeled samples based on the density of a local neighborhood . Inspired by noise correction work ( Yi & Wu , 2019 ) , Wang & Wu ( 2020 ) attempt to update the pseudo-labels through an optimization framework . Recently , ( Xie et al. , 2019 ) show self-training can be used to improve the performance of benchmark supervised classification tasks . A concurrent 1The motivations for using negative learning ( NL ) in this work differs greatly from Kim et al . ( 2019 ) . In this work , NL is used to incorporate more unlabeled samples into training and to generalize pseudo-labeling to the multi-label classification setting , whereas Kim et al . ( 2019 ) use negative learning primarily to obtain good network initializations to learn with noisy labels . Further discussion about NL can be found in Appendix K. work ( Haase-Schutz et al. , 2020 ) partitions an unlabeled dataset and trains re-initialized networks on each partition . They use previously trained networks to filter the labels used for training newer networks . However , most of their experiments involve learning from noisy data . Although previous pseudo-labeling based SSL approaches are general and domain-agnostic , they tend to under-perform due to the generation of noisy pseudo-labels ; our approach greatly reduces noise by minimizing the effect of poor network calibration , allowing for competitive state-of-the-art results . Consistency Regularization The main objective of consistency regularization methods is to obtain perturbation/augmentation invariant output distribution . In ( Sajjadi et al. , 2016 ) random max-pooling , dropout , and random data augmentation are used as input perturbations . In ( Miyato et al. , 2018 ) perturbations are applied to the input that changes the output predictions maximally . Temporal ensembling ( Laine & Aila , 2017 ) forces the output class distribution for a sample to be consistent over multiple epochs . Tarvainen & Valpola ( 2017 ) reformulate temporal ensembling as a teacher-student problem . Recently , the Mixup ( Zhang et al. , 2018 ) augmentation , has been used for consistency regularization in ( Verma et al. , 2019 ) . Several SSL works combine ideas from both consistency regularization and pseudo-labeling ( Berthelot et al. , 2019 ; 2020 ; Zhou et al. , 2020 ) . In ( Berthelot et al. , 2019 ) , pseudo-labels are generated by averaging different predictions of augmented versions of the same sample and the Mixup augmentation is used to train with these pseudo-labels . The authors in ( Berthelot et al. , 2020 ) extend this idea by dividing the set of augmentations into strong and weak augmentations . Also , ( Zhou et al. , 2020 ) incorporate a time-consistency metric to effectively select time-consistent samples for consistency regularization . The success of recent consistency regularization methods can be attributed to domain-specific augmentations ; our approach does not inherently rely on these augmentations , which allows for application to various modalities . Also , our pseudo-labeling method is orthogonal to consistency regularization techniques ; therefore , these existing techniques can be applied alongside UPS to further improve network performance . Uncertainty and Calibration Estimating network prediction uncertainty has been a deeply studied topic ( Graves , 2011 ; Blundell et al. , 2015 ; Louizos & Welling , 2016 ; Lakshminarayanan et al. , 2017 ; Malinin & Gales , 2018 ; Maddox et al. , 2019 ; Welling & Teh , 2011 ) . In the SSL domain , ( Yu et al. , 2019 ; Xia et al. , 2020 ) use uncertainty to improve consistency regularization learning for the segmentation of medical images . A concurrent work ( Mukherjee & Awadallah , 2020 ) , selects pseudo-labels predicted by a pretrained language model using uncertainty for a downstream SSL task . One difference between our works is the selection of hard samples . Whereas Mukherjee et al . select a certain amount of hard samples ( i.e . those which are not confident or certain ) and learn from these using positive learning , we decide to use negative learning on these samples which reduces the amount of noise seen by the network . Zheng & Yang ( 2020 ) show strong performance on the domain adaptive semantic segmentation task by leveraging uncertainty . However , to the best of our knowledge , uncertainty has not been used to reduce the effect of poor network calibration in the pseudo-labeling process . In this work , instead of improving the calibration of the network ( Guo et al. , 2017 ; Xing et al. , 2020 ) , we present a general framework which can leverage most uncertainty estimation methods to select a better calibrated subset of pseudo-labels . 3 PROPOSED METHOD . 3.1 PSEUDO-LABELING FOR SEMI-SUPERVISED LEARNING . Notation Let DL = { ( x ( i ) , y ( i ) ) } NL i=1 be a labeled dataset with NL samples , where x ( i ) is the input and y ( i ) = [ y ( i ) 1 , ... , y ( i ) C ] ⊆ { 0 , 1 } C is the corresponding label with C class categories ( note that multiple elements in y ( i ) can be non-zero in multi-label datasets ) . For a sample i , y ( i ) c = 1 denotes that class c is present in the corresponding input and y ( i ) c = 0 represent the class ’ s absence . Let DU = { x ( i ) } NUi=1 be an unlabeled dataset with NU samples , which does not contain labels corresponding to its input samples . For the unlabeled samples , pseudo-labels ỹ ( i ) are generated . Pseudo-labeling based SSL approaches involve learning a parameterized model fθ on the dataset D̃ = { ( x ( i ) , ỹ ( i ) ) } NL+NU i=1 , with ỹ ( i ) = y ( i ) for the NL labeled samples . Generalizing Pseudo-label Generation There are several approaches to create the pseudo-labels ỹ ( i ) , which have been described in Section 2 . We adopt the approach where hard pseudo-labels are obtained directly from network predictions . Let p ( i ) be the probability outputs of a trained network on the sample x ( i ) , such that p ( i ) c represents the probability of class c being present in the sample . Using these output probabilities , the pseudo-label can be generated for x ( i ) as : ỹ ( i ) c = 1 [ p ( i ) c ≥ γ ] , ( 1 ) where γ ∈ ( 0 , 1 ) is a threshold used to produce hard labels . Note that conventional single-label pseudo-labeling can be derived from equation 1 when γ = maxc p ( i ) c . For the multi-label case , γ = 0.5 would lead to binary pseudo-labels , in which multiple classes can be present in one sample .

Summary: This paper is in defense of simple semi-supervised learning (SSL) with pseudo-labeling (PL): authors demonstrate with experiments on 4 vision datasets (CIFAR-10, CIFAR-100, Pascal VOC and UCF-101) that pseudo-labeling can perform on par with consistency regularization methods. Authors argue that PL doesn't work well because of poor network calibration: because of that the high confident predictions are wrong leading to noisy training and poor generalization. The main contribution of the paper is the usage of prediction uncertainty selection in addition to the confidence-based selection which provides high accuracy of PL used in the further training. Besides this PL is generalized to create negative labels: with this authors perform and show effectiveness of negative learning and multi-label classification. Proposed approach performs in the same ballpark as state-of-the-art methods on CIFAR-10 and CIFAR-100, while it achieves the new state-of-the-art results on video dataset and multi-label task. It is worth to notice that proposed approach is independent from the domain while consistency regularization methods extensively are based on the specific augmentation techniques for the vision/datasets.