In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning

1 INTRODUCTION . The recent extraordinary success of deep learning methods can be mostly attributed to advancements in learning algorithms and the availability of large-scale labeled datasets . However , constructing large labeled datasets for supervised learning tends to be costly and is often infeasible . Several approaches have been proposed to overcome this dependency on huge labeled datasets ; these include semi-supervised learning ( Berthelot et al. , 2019 ; Tarvainen & Valpola , 2017 ; Miyato et al. , 2018 ; Lee , 2013 ) , self-supervised learning ( Doersch et al. , 2015 ; Noroozi & Favaro , 2016 ; Chen et al. , 2020a ) , and few-shot learning ( Finn et al. , 2017 ; Snell et al. , 2017 ; Vinyals et al. , 2016 ) . Semi-supervised learning ( SSL ) is one of the most dominant approaches for solving this problem , where the goal is to leverage a large unlabeled dataset alongside a small labeled dataset . One common assumption for SSL is that decision boundaries should lie in low density regions ( Chapelle & Zien , 2005 ) . Consistency-regularization based methods achieve this by making the network outputs invariant to small input perturbations ( Verma et al. , 2019 ) . However , one issue with these methods is that they often rely on a rich set of augmentations , like affine transformations , cutout ( DeVries & Taylor , 2017 ) , and color jittering in images , which limits their capability for domains where these augmentations are less effective ( e.g . videos and medical images ) . Pseudo-labeling based methods select unlabeled samples with high confidence as training targets ( pseudo-labels ) ; this can be viewed as a form of entropy minimization , which reduces the density of data points at the decision boundaries ( Grandvalet & Bengio , 2005 ; Lee , 2013 ) . One advantage of pseudo-labeling over consistency regularization is that it does not inherently require augmentations and can be generally applied to most domains . However , recent consistency regularization approaches tend to outperform pseudo-labeling on SSL benchmarks . This work is in defense of pseudo-labeling : we demonstrate that pseudo-labeling based methods can perform on par with consistency regularization methods . Although the selection of unlabeled samples with high confidence predictions moves decision boundaries to low density regions in pseudo-labeling based approaches , many of these selected predictions are incorrect due to the poor calibration of neural networks ( Guo et al. , 2017 ) . Since , calibration measures the discrepancy between the confidence level of a network ’ s individual predictions and its overall accuracy ( Dawid , 1982 ; Degroot & Fienberg , 1983 ) ; for poorly calibrated networks , an incorrect prediction might have high confidence . We argue that conventional pseudo-labeling based methods achieve poor results because poor network calibration produces incorrectly pseudo-labeled samples , leading to noisy training and poor generalization . To remedy this , we empirically study the relationship between output prediction uncertainty and calibration . We find that selecting predictions with low uncertainty greatly reduces the effect of poor calibration , improving generalization . Motivated by this , we propose an uncertainty-aware pseudo-label selection ( UPS ) framework that leverages the prediction uncertainty to guide the pseudo-label selection procedure . We believe pseudolabeling has been impactful due to its simplicity , generality , and ease of implementation ; to this end , our proposed framework attempts to maintain these benefits , while addressing the issue of calibration to drastically improve PL performance . UPS does not require modality-specific augmentations and can leverage most uncertainty estimation methods in its selection process . Furthermore , the proposed framework allows for the creation of negative pseudo-labels ( i.e . labels which specify the absence of specific classes ) . If a network predicts the absence of a class with high confidence and high certainty , then a negative label can be assigned to that sample . This generalization is beneficial for both single-label and multi-label learning . In the single-label case , networks can use these labels for negative learning ( Kim et al. , 2019 ) 1 ; in the multi-label case , class presence is independent so both positive and negative labels are necessary for training . Our key contributions include the following : ( 1 ) We introduce UPS , a novel uncertainty-aware pseudo-label selection framework which greatly reduces the effect of poor network calibration on the pseudo-labeling process , ( 2 ) While prior SSL methods focus on single-label classification , we generalize pseudo-labeling to create negative labels , allowing for negative learning and multi-label classification , and ( 3 ) Our comprehensive experimentation shows that the proposed method achieves strong performance on commonly used benchmark datasets CIFAR-10 and CIFAR-100 . In addition , we highlight our method ’ s flexibility by outperforming previous state-of-the-art approaches on the video dataset , UCF-101 , and the multi-label Pascal VOC dataset . 2 RELATED WORKS . Semi-supervised learning is a heavily studied problem . In this work , we mostly focus on pseudolabeling and consistency regularization based approaches as currently , these are the dominant approaches for SSL . Following ( Berthelot et al. , 2019 ) , we refer to the other SSL approaches for interested readers which includes : “ transductive ” models ( Gammerman et al. , 1998 ; Joachims , 1999 ; 2003 ) , graph-based methods ( Zhu et al. , 2003 ; Bengio et al. , 2006 ; Liu et al. , 2019 ) , generative modeling ( Belkin & Niyogi , 2002 ; Lasserre et al. , 2006 ; Kingma et al. , 2014 ; Pu et al. , 2016 ) . Furthermore , several recent self-supervised approaches ( Grill et al. , 2020 ; Chen et al. , 2020b ; Caron et al. , 2020 ) , have shown strong performance when applied to the SSL task . For a general overview of SSL , we point to ( Chapelle et al. , 2010 ; Zhu , 2005 ) . Pseudo-labeling The goal of pseudo-labeling ( Lee , 2013 ; Shi et al. , 2018 ) and self-training ( Yarowsky , 1995 ; McClosky et al. , 2006 ) is to generate pseudo-labels for unlabeled samples with a model trained on labeled data . In ( Lee , 2013 ) , pseudo-labels are created from the predictions of a trained neural network . Pseudo-labels can also be assigned to unlabeled samples based on neighborhood graphs ( Iscen et al. , 2019 ) . Shi et al . ( 2018 ) extend the idea of pseudo-labeling by incorporating confidence scores for unlabeled samples based on the density of a local neighborhood . Inspired by noise correction work ( Yi & Wu , 2019 ) , Wang & Wu ( 2020 ) attempt to update the pseudo-labels through an optimization framework . Recently , ( Xie et al. , 2019 ) show self-training can be used to improve the performance of benchmark supervised classification tasks . A concurrent 1The motivations for using negative learning ( NL ) in this work differs greatly from Kim et al . ( 2019 ) . In this work , NL is used to incorporate more unlabeled samples into training and to generalize pseudo-labeling to the multi-label classification setting , whereas Kim et al . ( 2019 ) use negative learning primarily to obtain good network initializations to learn with noisy labels . Further discussion about NL can be found in Appendix K. work ( Haase-Schutz et al. , 2020 ) partitions an unlabeled dataset and trains re-initialized networks on each partition . They use previously trained networks to filter the labels used for training newer networks . However , most of their experiments involve learning from noisy data . Although previous pseudo-labeling based SSL approaches are general and domain-agnostic , they tend to under-perform due to the generation of noisy pseudo-labels ; our approach greatly reduces noise by minimizing the effect of poor network calibration , allowing for competitive state-of-the-art results . Consistency Regularization The main objective of consistency regularization methods is to obtain perturbation/augmentation invariant output distribution . In ( Sajjadi et al. , 2016 ) random max-pooling , dropout , and random data augmentation are used as input perturbations . In ( Miyato et al. , 2018 ) perturbations are applied to the input that changes the output predictions maximally . Temporal ensembling ( Laine & Aila , 2017 ) forces the output class distribution for a sample to be consistent over multiple epochs . Tarvainen & Valpola ( 2017 ) reformulate temporal ensembling as a teacher-student problem . Recently , the Mixup ( Zhang et al. , 2018 ) augmentation , has been used for consistency regularization in ( Verma et al. , 2019 ) . Several SSL works combine ideas from both consistency regularization and pseudo-labeling ( Berthelot et al. , 2019 ; 2020 ; Zhou et al. , 2020 ) . In ( Berthelot et al. , 2019 ) , pseudo-labels are generated by averaging different predictions of augmented versions of the same sample and the Mixup augmentation is used to train with these pseudo-labels . The authors in ( Berthelot et al. , 2020 ) extend this idea by dividing the set of augmentations into strong and weak augmentations . Also , ( Zhou et al. , 2020 ) incorporate a time-consistency metric to effectively select time-consistent samples for consistency regularization . The success of recent consistency regularization methods can be attributed to domain-specific augmentations ; our approach does not inherently rely on these augmentations , which allows for application to various modalities . Also , our pseudo-labeling method is orthogonal to consistency regularization techniques ; therefore , these existing techniques can be applied alongside UPS to further improve network performance . Uncertainty and Calibration Estimating network prediction uncertainty has been a deeply studied topic ( Graves , 2011 ; Blundell et al. , 2015 ; Louizos & Welling , 2016 ; Lakshminarayanan et al. , 2017 ; Malinin & Gales , 2018 ; Maddox et al. , 2019 ; Welling & Teh , 2011 ) . In the SSL domain , ( Yu et al. , 2019 ; Xia et al. , 2020 ) use uncertainty to improve consistency regularization learning for the segmentation of medical images . A concurrent work ( Mukherjee & Awadallah , 2020 ) , selects pseudo-labels predicted by a pretrained language model using uncertainty for a downstream SSL task . One difference between our works is the selection of hard samples . Whereas Mukherjee et al . select a certain amount of hard samples ( i.e . those which are not confident or certain ) and learn from these using positive learning , we decide to use negative learning on these samples which reduces the amount of noise seen by the network . Zheng & Yang ( 2020 ) show strong performance on the domain adaptive semantic segmentation task by leveraging uncertainty . However , to the best of our knowledge , uncertainty has not been used to reduce the effect of poor network calibration in the pseudo-labeling process . In this work , instead of improving the calibration of the network ( Guo et al. , 2017 ; Xing et al. , 2020 ) , we present a general framework which can leverage most uncertainty estimation methods to select a better calibrated subset of pseudo-labels . 3 PROPOSED METHOD . 3.1 PSEUDO-LABELING FOR SEMI-SUPERVISED LEARNING . Notation Let DL = { ( x ( i ) , y ( i ) ) } NL i=1 be a labeled dataset with NL samples , where x ( i ) is the input and y ( i ) = [ y ( i ) 1 , ... , y ( i ) C ] ⊆ { 0 , 1 } C is the corresponding label with C class categories ( note that multiple elements in y ( i ) can be non-zero in multi-label datasets ) . For a sample i , y ( i ) c = 1 denotes that class c is present in the corresponding input and y ( i ) c = 0 represent the class ’ s absence . Let DU = { x ( i ) } NUi=1 be an unlabeled dataset with NU samples , which does not contain labels corresponding to its input samples . For the unlabeled samples , pseudo-labels ỹ ( i ) are generated . Pseudo-labeling based SSL approaches involve learning a parameterized model fθ on the dataset D̃ = { ( x ( i ) , ỹ ( i ) ) } NL+NU i=1 , with ỹ ( i ) = y ( i ) for the NL labeled samples . Generalizing Pseudo-label Generation There are several approaches to create the pseudo-labels ỹ ( i ) , which have been described in Section 2 . We adopt the approach where hard pseudo-labels are obtained directly from network predictions . Let p ( i ) be the probability outputs of a trained network on the sample x ( i ) , such that p ( i ) c represents the probability of class c being present in the sample . Using these output probabilities , the pseudo-label can be generated for x ( i ) as : ỹ ( i ) c = 1 [ p ( i ) c ≥ γ ] , ( 1 ) where γ ∈ ( 0 , 1 ) is a threshold used to produce hard labels . Note that conventional single-label pseudo-labeling can be derived from equation 1 when γ = maxc p ( i ) c . For the multi-label case , γ = 0.5 would lead to binary pseudo-labels , in which multiple classes can be present in one sample .

Summary: As noted in (Guo et al., 2017), modern neural networks are often miscalibrated. Pseudo-labeling based Semi-Supervised learning schemes are predicated on high confidence predictions from these neural networks. This paper posits that this miscalibration may lead to inferior results in confidence-based pseudo-labeling approaches. By taking into account the uncertainty of models and only using pseudo labels from high-confidence instances with low uncertainty this work presents a model that significant improves on other PL strategies and is competitive with consistency-based regularization strategies that comprise the current state-of-the-art.