The Needle in the haystack: Out-distribution aware Self-training in an Open-World Setting

1 INTRODUCTION . In past years we have seen tremendous progress in image recognition based on deep learning ( Krizhevsky et al. , 2012 ; He et al. , 2016 ) . However , this success required large labeled datasets that are expensive to generate . On the other hand , large amounts of unlabeled data are broadly available , in particular in image recognition . The promise of semi-supervised learning ( Chapelle et al. , 2006 ) is to leverage unlabeled data in order to improve prediction performance . However , the underlying assumption of traditional and modern ( Berthelot et al. , 2019 ; Sohn et al. , 2020 ) SSL algorithms is that the unlabeled data comes from the same distribution or at least contains data from the same classes ( closed world assumption ) . Oliver et al . ( 2018 ) criticized this as being unrealistic as the assumption is hard to check when retrieving large amounts of unlabeled data from the web . SSL in an open world setting , where the unlabeled data contains task-relevant but also non-related out-of-distribution ( OOD ) images , has recently attracted attention as a more realistic approach to SSL ( Guo et al. , 2020 ; Chen et al. , 2020 ; Yu et al. , 2020 ) . While these methods could outperform standard SSL techniques when the unlabeled data contains out-of-distribution samples , they have mostly been evaluated in settings where the unlabeled data contains relatively few non-task-related samples . It is thus an open question whether existing open world SSL methods can scale to large unlabeled data bases where the ratio of task-relevant to non-related samples is much smaller . In this paper , we demonstrate that existing methods suffer from severe performance degradations when the ratio of non-related images in the unlabeled dataset increases . Moreover , even if they achieve high accuracy , they are not OOD aware , that is they systematically assign high confidence to non-related inputs , e.g . a CIFAR10 model classifies images containing humans as dogs or horses . Our contributions are : i ) we propose our scalable , iterative out-distribution-aware selftraining ( ODST ) which enforces low-confidence predictions on non-task-related samples and selects unlabeled samples for the in-distribution task via a novel class-adaptive selection scheme . This confidence based selection scheme allows us to deal with strongly unbalanced in-distribution classes in the unlabeled dataset , ii ) we outperform state-of-the-art SSL techniques ( Berthelot et al. , 2019 ; Sohn et al. , 2020 ) , as well as recent open world SSL techniques ( Guo et al. , 2020 ; Yu et al. , 2020 ) in terms of prediction performance and out-of-distribution detection and show that all other approaches suffer from distribution shifts , iii ) we show strong performance gains when using ODST with the full training sets of CIFAR10 and CIFAR100 as labeled data together with the 80 Million Tiny Image ( 80MTI ) dataset as unlabeled dataset . Thus we show that even with large amounts of labeled data , ODST can still leverage unlabeled data to improve prediction and OOD performance . 2 RELATED WORK . Semi-supervised learning ( SSL ) is an established technique ( Zhu , 2005 ; Zhu & Goldberg , 2009 ) for leveraging information from unlabeled data to improve predictive performance . In self-training ( Riloff , 1996 ; Riloff & Wiebe , 2003 ; Scudder , 1965 ) , a teacher model is trained in a fully-supervised fashion on a labeled dataset . The teacher model is then used to label a set of unlabeled examples , typically drawn from the original data distribution ( closed world assumption ) , which is then used in combination with the labeled samples to train a new student model . Various extensions of this protocol have been proposed , including the use of an ensemble of teacher models ( Zhou et al. , 2018 ) and Co-training ( Blum & Mitchell , 1998 ) . Recently , Xie et al . ( 2020 ) and Yalniz et al . ( 2019 ) used self-training to improve performance on ImageNet ( Russakovsky et al. , 2015 ) by using large image databases consisting of millions of taskrelevant and out-of-distribution samples . On CIFAR10 , Carmon et al . ( 2019 ) were able to significantly improve model robustness to adversarial perturbations by adding unlabeled samples from 80 million tiny images ( 80MTI ) ( Torralba et al. , 2008 ) using self-training . The distinctive feature of self-training in comparison to other SSL methods is that the training of the teacher model is separated from the labeling process . In contrast , in pseudo-labeling ( Lee , 2013 ; Iscen et al. , 2019 ; Shi et al. , 2018 ) labels are generated during training by the model itself . Similarly , consistencybased SSL-methods like Π-models ( Laine & Aila , 2016 ; Sajjadi et al. , 2016 ) , mean-teacher ( Tarvainen & Valpola , 2017 ) and virtual adversarial training ( Miyato et al. , 2018 ) enforce an invariance of the model ’ s output on the unlabeled data under a specific set of perturbations . Methods like MixMatch ( Berthelot et al. , 2019 ) and FixMatch ( Sohn et al. , 2020 ) combine consistency regularization with strong augmentation e.g . RandAugment ( Cubuk et al. , 2020 ) . A related technique is entropy-minimization ( Grandvalet & Bengio , 2005 ) , which penalizes low-confidence predictions on unlabeled samples during training . Oliver et al . ( 2018 ) found that SSL can improve the model ’ s performance in the traditional SSL setting where the unlabeled data is sampled from the same distribution as the training data ( closed world assumption ) but can degrade the performance when the unlabeled data contains non-task-related samples ( open world setting ) . The open world SSL setting has has been explored only recently , by combining elements of consistency regularization with online OOD detection and sample filtering ( Yu et al. , 2020 ; Chen et al. , 2020 ) or soft per-sample weighting ( Guo et al. , 2020 ) . While they demonstrate performance improvements when the unlabeled data contains non-task-related samples , their evaluation is restricted to settings where the unlabeled data contains mostly task-related samples . We later demonstrate that in more challenging settings , where the ratio of task-related samples in the unlabeled data is small , these methods show severe performance degradations . Out-of-distribution detection : Deep Neural networks ( DNN ) have empirically and theoretically been shown to produce overconfident predictions for inputs not related to the task , e.g . noise or other classes not contained in the labeled dataset ( Nguyen et al. , 2015 ; Hendrycks & Gimpel , 2017 ; Hein et al. , 2019 ) , i.e . the confidence of a DNN is not reliable for the detection of out-of-distribution ( OOD ) samples . Approaches for OOD detection include ODIN ( Liang et al. , 2018 ) or using the Mahalanobis distance of higher-order features ( Lee et al. , 2018 ) . Hendrycks et al . ( 2019 ) introduced Outlier exposure ( OE ) , see Hein et al . ( 2019 ) for the related CEDA , and show that the confidence can be used as a reliable OOD detector when enforcing low confidence on an OOD training set , even when tested on other OOD test datasets . OOD detection is also related to open set recognition ( Boult et al. , 2019 ) which is beyond the scope of this paper . 3 METHOD . In contrast to previous works which focused on problems where the ratio of non-task-related to taskrelated images is small , our goal is to show that our ODST in combination with a careful sample selection strategy yields a self-training scheme that can leverage large unlabeled datasets to improve performance on the CIFAR10 and CIFAR100 test set over a fully-supervised baseline trained on the entire training set and additionally has excellent OOD detection performance . First , we introduce self-training as in the noisy student self-training ( NSST ) of Xie et al . ( 2020 ) which serves a baseline . Then we introduce our out-distribution aware self-training ODST . In the following T = ( xi , yi ) ni=1 denotes the set of labeled examples , where xi ∈ Rd and yi ∈ { 1 , . . . , K } and U = ( zi ) mi=1 is a collection of unlabeled samples . The traditional SSL literature makes the assumption that the unlabeled samples U are drawn from the same distribution as the labeled examples T , or at least belong to the same set of classes which we explicitly do not do in this paper . Given the logits of a neural network f : Rd → R the predicted probability distribution for a point x is computed via the softmax as : p̂f ( s|x ) = efs ( x ) / ∑K l=1 e fl ( x ) . The confidence in the decision for x is then given by maxs=1 , ... , K p̂f ( s|x ) , and the cross-entropy loss between ( soft ) -labels p ∈ RK ( ∑ i pi = 1 , pi ≥ 0 ) and prediction p̂ is defined as : L ( p , p̂ ) = − ∑K i=1 pi log p̂i . 3.1 NOISY STUDENT SELF-TRAINING ( NSST ) . In NSST ( Xie et al. , 2020 ) the first teacher is a base model f ( 0 ) obtained by minimizing the crossentropy loss on the labeled set only . The iterative scheme of NSST starting at t = 0 is : 1 ) pseudo-label all unlabeled samples in U with current teacher f ( t ) 2 ) select a subset I ⊂ U of the pseudo-labeled examples by choosing per class the top-k with highest confidence and which are above a fixed threshold 3 ) train new model f ( t+1 ) by minimizing the loss on labeled and pseudo-labeled samples in I with AutoAugment ( Cubuk et al. , 2019 ) as data augmentation : 1 n n∑ i=1 L ( yi , p̂f ( t+1 ) ( xi ) ) + 1 |I| ∑ z∈I L ( p̂f ( t ) ( z ) , p̂f ( t+1 ) ( z ) ) 4 ) t← t+ 1 and go back to step 1 Xie et al . ( 2020 ) call their self-training noisy due to the very strong data augmentation which avoids overfitting via noise injection . The main difficulty in self-training in a closed world setting is the propagation of labeling mistakes which leads to a degradation of prediction performance which is taken care of by step 2 ) . While Xie et al . ( 2020 ) chose a fixed threshold , we choose it according to the false positive rate on an in-distribution validation set . However , in an open-world setting , an equally severe problem is that a large fraction of the unlabeled samples is not task-relevant such that including them leads to a distribution shift which happens for NSST ( see Figure 2 ) and other SSL methods ( see Figure 1 ) . However , more importantly , the classifier is highly confident on unrelated classes which is a problem for AI safety and this undesired behavior is even hard to notice as predictive performance might appear to improve when evaluated only on the test set . 3.2 OUT-DISTRIBUTION AWARE SELF-TRAINING ( ODST ) . We first provide an overview over our algorithmic scheme and then explain the individual steps in more detail . The base ODST model is initialized with an out-distribution aware base teacher model f ( 0 ) , trained by minimizing : 1 n n∑ i=1 L ( yi , p̂f ( 0 ) ( xi ) ) + 1 |U| ∑ z∈U L ( 1 K 1 , p̂f ( 0 ) ( z ) ) . ( 1 ) We then iterate the following steps starting from t = 0 : A ) calibrate f ( t ) on the in-distribution validation set B ) pseudo-label all unlabeled samples in U with current teacher f ( t ) C ) for each class c : select the top-k unlabeled instances with highest confidence classified as c that lie above the threshold . The selected samples for all classes are denoted as I D ) determine new pseudo-labels for the unlabeled instances . We use q ( z ) = p̂f ( t ) ( z ) for z ∈ I ( selected samples in step C ) ) and v ( z ) = 1 2 ( 1 K + p̂f ( t ) ( z ) ) , for z ∈ U\I . ( 2 ) E ) train a new model f ( t+1 ) by minimizing the loss on labeled and pseudo-labeled samples with AutoAugment ( Cubuk et al. , 2019 ) as strong data augmentation : 1 n+ |I| [ n∑ i=1 L ( yi , p̂f ( t+1 ) ( xi ) ) + ∑ z∈I L ( q ( z ) , p̂f ( t+1 ) ( z ) ) ] + 1 |U \ I| ∑ z∈U\I L ( v ( z ) , p̂f ( t+1 ) ( z ) ) ( 3 ) F ) t← t+ 1 and go to step A ) The Base classifier is essentially an Outlier Exposure ( OE ) model ( Hendrycks et al. , 2019 ) ( see also ( Hein et al. , 2019 ; Papadopoulos et al. , 2019 ) for related losses ) where the unlabeled set U is our training out-distribution on which we enforce uniform confidence . OE is known to be one of the best methods for out-of-distribution detection . As in our case , a crucial assumption is that the unlabeled samples are partially task-related , it might appear odd to enforce uniform confidence on all of U . However , we show in Section 3.3 that this asymptotically only leads to a down-weighting of the confidence for task-related samples but preserves the Bayes optimal decision and , in particular , enforces close-to-uniform confidence for all unrelated samples . A ) Calibration : while normal neural networks are known to be overconfident on in- ( Guo et al. , 2017 ) and out-distribution ( Nguyen et al. , 2015 ; Hein et al. , 2019 ) , the models resulting from enforcing low confidence on unlabeled points ( such as OE ) tend to be underconfident on the in-distribution . As we use the predictions of the teacher f ( t ) as new pseudo-labels for the unlabeled data , we calibrate f ( t ) by minimizing the expected calibration error using temperature rescaling Guo et al . ( 2017 ) . Thus the teacher model assigns the correct uncertainty score to its predictions on in-distribution samples which improves pseudo-label quality and and stabilizes the training procedure . C ) Sample Selection : The most important problem in self-training is to integrate the right samples into the pseudo-labeled set I . While our out-distribution aware teacher is better at discriminating between the in- and out-distribution based on confidence , there are still many samples with highly confident predictions due to the sheer size of the unlabeled dataset ( ≥ 106 ) . Note that we select at most the top-k samples ( where k = 5N ( t + 1 ) /K ) , but this might still be too much if not sufficiently many task-related examples of a class exist in the unlabeled dataset . We thus need to determine class-specific confidence thresholds to limit the selection . Class-specific selection has been neglected in the literature but is particularly important in practice as the number of task-related examples in the unlabeled dataset typically varies widely between the different classes . Using the in-distribution validation set , we define the in-distribution threshold for class c as the smallest predicted probability for class c such that the precision for all images which are above this threshold is greater than or equal to α ( binary classification problem : class c versus all other classes ) . An in-distribution precision threshold is especially important if the classification task contains similar classes , as learning from wrongly labeled in-distribution images ( a dog that is classified as a cat in CIFAR10 ) is likely to hurt predictive performance on the in-distribution task even more than the inclusion of an unrelated out-distribution image . If there exist less than k samples above the threshold for a particular class , we randomly repeat the accepted samples above the threshold to maintain a class-balanced training scheme . Note that it is much easier and also more interpretable to fix a precision value rather than the choice of a confidence threshold ( in particular if the model is not calibrated ) as done in Xie et al . ( 2020 ) . D ) Pseudo Labels : for the original labeled dataset we always use one-hot labels . For unlabeled data points that have been selected in I , we determine pseudo-labels q according to the predicted probability distribution over the classes by the calibrated teacher model . Due to the calibration , this should reflect the “ correct ” uncertainty about these labels . For all remaining images in our unlabeled dataset U\I , we use a weak form of knowledge distillation by defining pseudo-labels v as the average of the predicted probability distribution of the teacher model and the uniform distribution , given in ( 2 ) . This has two reasons : i ) a purely uniform distribution on U\I , which in the first iterations might still contain a lot of task-relevant images , leads to a bias as it does not distinguish between task-relevant and irrelevant images , ii ) only using soft-labels from the teacher model leads to overconfident predictions as one can observe in the non-out-distribution aware NSST method . Thus a trade-off between these opposing goals is their average which leads to heavy damping of the confidence ( note that the pseudo-labels have a maximal confidence of 12 + 1 K on U\I ) . E ) Training : For the final objective in ( 3 ) , the selected pseudo-labeled samples in I and the original samples in T are assigned the same weight . This is quite aggressive as for iteration t we add up to 5t as much pseudo-labeled data as labeled training data . However , this also enables larger performance gains given that the sample selection process is successful . Note that the losses on I ∪ T and on U\I have equal weight as the damping of confidences on U\I is crucial for the sample selection process . As pseudo-labels are computed on non-augmented images and we use heavy augmentation during training , our scheme can be regarded as offline consistency learning , where the model is encouraged to replicate the teacher ’ s output independent of the randomly selected augmentation . This allows us to limit the number of passes through the entire unlabeled pool to the number of self-training iterations , do model calibration and results in more stable targets . The entire scheme is repeated multiple times until the validation accuracy starts to degrade or a fixed maximum number of iterations is reached . ODST+ : In addition to ODST , we provide the variant ODST+ for scenarios where AI safety is critical . It differs from ODST only in step C ) where we calculate the final threshold as maximum over ODST ’ s in-distribution threshold and an additional out-distribution threshold . This class-specific out-distribution threshold controls the number of task-irrelevant samples that are falsely added into our pseudo-labeled sample pool I . This is done using an extra out-distribution validation set , i.e . a set of natural images that does not contain any class relevant images ( we discuss this choice in Section 4 ) . For each class c , we compute the α-quantile of the predicted probabilities for class c on the out-distribution images which we define as the out-distribution threshold for class c .

Summary: The paper assumes an open world setting for semi-supervised learning approaches. In comparison to OE method, the proposed ODST method further dampness the usage of pseudo labels. Specifically, this work further assumes that unlabeled data with lower confidence do come from a mixture model having two components with equal probabilities: one fully uncertain model with label $\frac{1}{K}$ evenly across all classes; and one model with pseudo label determined by the teacher model.