The Needle in the haystack: Out-distribution aware Self-training in an Open-World Setting

1 INTRODUCTION . In past years we have seen tremendous progress in image recognition based on deep learning ( Krizhevsky et al. , 2012 ; He et al. , 2016 ) . However , this success required large labeled datasets that are expensive to generate . On the other hand , large amounts of unlabeled data are broadly available , in particular in image recognition . The promise of semi-supervised learning ( Chapelle et al. , 2006 ) is to leverage unlabeled data in order to improve prediction performance . However , the underlying assumption of traditional and modern ( Berthelot et al. , 2019 ; Sohn et al. , 2020 ) SSL algorithms is that the unlabeled data comes from the same distribution or at least contains data from the same classes ( closed world assumption ) . Oliver et al . ( 2018 ) criticized this as being unrealistic as the assumption is hard to check when retrieving large amounts of unlabeled data from the web . SSL in an open world setting , where the unlabeled data contains task-relevant but also non-related out-of-distribution ( OOD ) images , has recently attracted attention as a more realistic approach to SSL ( Guo et al. , 2020 ; Chen et al. , 2020 ; Yu et al. , 2020 ) . While these methods could outperform standard SSL techniques when the unlabeled data contains out-of-distribution samples , they have mostly been evaluated in settings where the unlabeled data contains relatively few non-task-related samples . It is thus an open question whether existing open world SSL methods can scale to large unlabeled data bases where the ratio of task-relevant to non-related samples is much smaller . In this paper , we demonstrate that existing methods suffer from severe performance degradations when the ratio of non-related images in the unlabeled dataset increases . Moreover , even if they achieve high accuracy , they are not OOD aware , that is they systematically assign high confidence to non-related inputs , e.g . a CIFAR10 model classifies images containing humans as dogs or horses . Our contributions are : i ) we propose our scalable , iterative out-distribution-aware selftraining ( ODST ) which enforces low-confidence predictions on non-task-related samples and selects unlabeled samples for the in-distribution task via a novel class-adaptive selection scheme . This confidence based selection scheme allows us to deal with strongly unbalanced in-distribution classes in the unlabeled dataset , ii ) we outperform state-of-the-art SSL techniques ( Berthelot et al. , 2019 ; Sohn et al. , 2020 ) , as well as recent open world SSL techniques ( Guo et al. , 2020 ; Yu et al. , 2020 ) in terms of prediction performance and out-of-distribution detection and show that all other approaches suffer from distribution shifts , iii ) we show strong performance gains when using ODST with the full training sets of CIFAR10 and CIFAR100 as labeled data together with the 80 Million Tiny Image ( 80MTI ) dataset as unlabeled dataset . Thus we show that even with large amounts of labeled data , ODST can still leverage unlabeled data to improve prediction and OOD performance . 2 RELATED WORK . Semi-supervised learning ( SSL ) is an established technique ( Zhu , 2005 ; Zhu & Goldberg , 2009 ) for leveraging information from unlabeled data to improve predictive performance . In self-training ( Riloff , 1996 ; Riloff & Wiebe , 2003 ; Scudder , 1965 ) , a teacher model is trained in a fully-supervised fashion on a labeled dataset . The teacher model is then used to label a set of unlabeled examples , typically drawn from the original data distribution ( closed world assumption ) , which is then used in combination with the labeled samples to train a new student model . Various extensions of this protocol have been proposed , including the use of an ensemble of teacher models ( Zhou et al. , 2018 ) and Co-training ( Blum & Mitchell , 1998 ) . Recently , Xie et al . ( 2020 ) and Yalniz et al . ( 2019 ) used self-training to improve performance on ImageNet ( Russakovsky et al. , 2015 ) by using large image databases consisting of millions of taskrelevant and out-of-distribution samples . On CIFAR10 , Carmon et al . ( 2019 ) were able to significantly improve model robustness to adversarial perturbations by adding unlabeled samples from 80 million tiny images ( 80MTI ) ( Torralba et al. , 2008 ) using self-training . The distinctive feature of self-training in comparison to other SSL methods is that the training of the teacher model is separated from the labeling process . In contrast , in pseudo-labeling ( Lee , 2013 ; Iscen et al. , 2019 ; Shi et al. , 2018 ) labels are generated during training by the model itself . Similarly , consistencybased SSL-methods like Π-models ( Laine & Aila , 2016 ; Sajjadi et al. , 2016 ) , mean-teacher ( Tarvainen & Valpola , 2017 ) and virtual adversarial training ( Miyato et al. , 2018 ) enforce an invariance of the model ’ s output on the unlabeled data under a specific set of perturbations . Methods like MixMatch ( Berthelot et al. , 2019 ) and FixMatch ( Sohn et al. , 2020 ) combine consistency regularization with strong augmentation e.g . RandAugment ( Cubuk et al. , 2020 ) . A related technique is entropy-minimization ( Grandvalet & Bengio , 2005 ) , which penalizes low-confidence predictions on unlabeled samples during training . Oliver et al . ( 2018 ) found that SSL can improve the model ’ s performance in the traditional SSL setting where the unlabeled data is sampled from the same distribution as the training data ( closed world assumption ) but can degrade the performance when the unlabeled data contains non-task-related samples ( open world setting ) . The open world SSL setting has has been explored only recently , by combining elements of consistency regularization with online OOD detection and sample filtering ( Yu et al. , 2020 ; Chen et al. , 2020 ) or soft per-sample weighting ( Guo et al. , 2020 ) . While they demonstrate performance improvements when the unlabeled data contains non-task-related samples , their evaluation is restricted to settings where the unlabeled data contains mostly task-related samples . We later demonstrate that in more challenging settings , where the ratio of task-related samples in the unlabeled data is small , these methods show severe performance degradations . Out-of-distribution detection : Deep Neural networks ( DNN ) have empirically and theoretically been shown to produce overconfident predictions for inputs not related to the task , e.g . noise or other classes not contained in the labeled dataset ( Nguyen et al. , 2015 ; Hendrycks & Gimpel , 2017 ; Hein et al. , 2019 ) , i.e . the confidence of a DNN is not reliable for the detection of out-of-distribution ( OOD ) samples . Approaches for OOD detection include ODIN ( Liang et al. , 2018 ) or using the Mahalanobis distance of higher-order features ( Lee et al. , 2018 ) . Hendrycks et al . ( 2019 ) introduced Outlier exposure ( OE ) , see Hein et al . ( 2019 ) for the related CEDA , and show that the confidence can be used as a reliable OOD detector when enforcing low confidence on an OOD training set , even when tested on other OOD test datasets . OOD detection is also related to open set recognition ( Boult et al. , 2019 ) which is beyond the scope of this paper . 3 METHOD . In contrast to previous works which focused on problems where the ratio of non-task-related to taskrelated images is small , our goal is to show that our ODST in combination with a careful sample selection strategy yields a self-training scheme that can leverage large unlabeled datasets to improve performance on the CIFAR10 and CIFAR100 test set over a fully-supervised baseline trained on the entire training set and additionally has excellent OOD detection performance . First , we introduce self-training as in the noisy student self-training ( NSST ) of Xie et al . ( 2020 ) which serves a baseline . Then we introduce our out-distribution aware self-training ODST . In the following T = ( xi , yi ) ni=1 denotes the set of labeled examples , where xi ∈ Rd and yi ∈ { 1 , . . . , K } and U = ( zi ) mi=1 is a collection of unlabeled samples . The traditional SSL literature makes the assumption that the unlabeled samples U are drawn from the same distribution as the labeled examples T , or at least belong to the same set of classes which we explicitly do not do in this paper . Given the logits of a neural network f : Rd → R the predicted probability distribution for a point x is computed via the softmax as : p̂f ( s|x ) = efs ( x ) / ∑K l=1 e fl ( x ) . The confidence in the decision for x is then given by maxs=1 , ... , K p̂f ( s|x ) , and the cross-entropy loss between ( soft ) -labels p ∈ RK ( ∑ i pi = 1 , pi ≥ 0 ) and prediction p̂ is defined as : L ( p , p̂ ) = − ∑K i=1 pi log p̂i . 3.1 NOISY STUDENT SELF-TRAINING ( NSST ) . In NSST ( Xie et al. , 2020 ) the first teacher is a base model f ( 0 ) obtained by minimizing the crossentropy loss on the labeled set only . The iterative scheme of NSST starting at t = 0 is : 1 ) pseudo-label all unlabeled samples in U with current teacher f ( t ) 2 ) select a subset I ⊂ U of the pseudo-labeled examples by choosing per class the top-k with highest confidence and which are above a fixed threshold 3 ) train new model f ( t+1 ) by minimizing the loss on labeled and pseudo-labeled samples in I with AutoAugment ( Cubuk et al. , 2019 ) as data augmentation : 1 n n∑ i=1 L ( yi , p̂f ( t+1 ) ( xi ) ) + 1 |I| ∑ z∈I L ( p̂f ( t ) ( z ) , p̂f ( t+1 ) ( z ) ) 4 ) t← t+ 1 and go back to step 1 Xie et al . ( 2020 ) call their self-training noisy due to the very strong data augmentation which avoids overfitting via noise injection . The main difficulty in self-training in a closed world setting is the propagation of labeling mistakes which leads to a degradation of prediction performance which is taken care of by step 2 ) . While Xie et al . ( 2020 ) chose a fixed threshold , we choose it according to the false positive rate on an in-distribution validation set . However , in an open-world setting , an equally severe problem is that a large fraction of the unlabeled samples is not task-relevant such that including them leads to a distribution shift which happens for NSST ( see Figure 2 ) and other SSL methods ( see Figure 1 ) . However , more importantly , the classifier is highly confident on unrelated classes which is a problem for AI safety and this undesired behavior is even hard to notice as predictive performance might appear to improve when evaluated only on the test set . 3.2 OUT-DISTRIBUTION AWARE SELF-TRAINING ( ODST ) . We first provide an overview over our algorithmic scheme and then explain the individual steps in more detail . The base ODST model is initialized with an out-distribution aware base teacher model f ( 0 ) , trained by minimizing : 1 n n∑ i=1 L ( yi , p̂f ( 0 ) ( xi ) ) + 1 |U| ∑ z∈U L ( 1 K 1 , p̂f ( 0 ) ( z ) ) . ( 1 ) We then iterate the following steps starting from t = 0 : A ) calibrate f ( t ) on the in-distribution validation set B ) pseudo-label all unlabeled samples in U with current teacher f ( t ) C ) for each class c : select the top-k unlabeled instances with highest confidence classified as c that lie above the threshold . The selected samples for all classes are denoted as I D ) determine new pseudo-labels for the unlabeled instances . We use q ( z ) = p̂f ( t ) ( z ) for z ∈ I ( selected samples in step C ) ) and v ( z ) = 1 2 ( 1 K + p̂f ( t ) ( z ) ) , for z ∈ U\I . ( 2 ) E ) train a new model f ( t+1 ) by minimizing the loss on labeled and pseudo-labeled samples with AutoAugment ( Cubuk et al. , 2019 ) as strong data augmentation : 1 n+ |I| [ n∑ i=1 L ( yi , p̂f ( t+1 ) ( xi ) ) + ∑ z∈I L ( q ( z ) , p̂f ( t+1 ) ( z ) ) ] + 1 |U \ I| ∑ z∈U\I L ( v ( z ) , p̂f ( t+1 ) ( z ) ) ( 3 ) F ) t← t+ 1 and go to step A ) The Base classifier is essentially an Outlier Exposure ( OE ) model ( Hendrycks et al. , 2019 ) ( see also ( Hein et al. , 2019 ; Papadopoulos et al. , 2019 ) for related losses ) where the unlabeled set U is our training out-distribution on which we enforce uniform confidence . OE is known to be one of the best methods for out-of-distribution detection . As in our case , a crucial assumption is that the unlabeled samples are partially task-related , it might appear odd to enforce uniform confidence on all of U . However , we show in Section 3.3 that this asymptotically only leads to a down-weighting of the confidence for task-related samples but preserves the Bayes optimal decision and , in particular , enforces close-to-uniform confidence for all unrelated samples . A ) Calibration : while normal neural networks are known to be overconfident on in- ( Guo et al. , 2017 ) and out-distribution ( Nguyen et al. , 2015 ; Hein et al. , 2019 ) , the models resulting from enforcing low confidence on unlabeled points ( such as OE ) tend to be underconfident on the in-distribution . As we use the predictions of the teacher f ( t ) as new pseudo-labels for the unlabeled data , we calibrate f ( t ) by minimizing the expected calibration error using temperature rescaling Guo et al . ( 2017 ) . Thus the teacher model assigns the correct uncertainty score to its predictions on in-distribution samples which improves pseudo-label quality and and stabilizes the training procedure . C ) Sample Selection : The most important problem in self-training is to integrate the right samples into the pseudo-labeled set I . While our out-distribution aware teacher is better at discriminating between the in- and out-distribution based on confidence , there are still many samples with highly confident predictions due to the sheer size of the unlabeled dataset ( ≥ 106 ) . Note that we select at most the top-k samples ( where k = 5N ( t + 1 ) /K ) , but this might still be too much if not sufficiently many task-related examples of a class exist in the unlabeled dataset . We thus need to determine class-specific confidence thresholds to limit the selection . Class-specific selection has been neglected in the literature but is particularly important in practice as the number of task-related examples in the unlabeled dataset typically varies widely between the different classes . Using the in-distribution validation set , we define the in-distribution threshold for class c as the smallest predicted probability for class c such that the precision for all images which are above this threshold is greater than or equal to α ( binary classification problem : class c versus all other classes ) . An in-distribution precision threshold is especially important if the classification task contains similar classes , as learning from wrongly labeled in-distribution images ( a dog that is classified as a cat in CIFAR10 ) is likely to hurt predictive performance on the in-distribution task even more than the inclusion of an unrelated out-distribution image . If there exist less than k samples above the threshold for a particular class , we randomly repeat the accepted samples above the threshold to maintain a class-balanced training scheme . Note that it is much easier and also more interpretable to fix a precision value rather than the choice of a confidence threshold ( in particular if the model is not calibrated ) as done in Xie et al . ( 2020 ) . D ) Pseudo Labels : for the original labeled dataset we always use one-hot labels . For unlabeled data points that have been selected in I , we determine pseudo-labels q according to the predicted probability distribution over the classes by the calibrated teacher model . Due to the calibration , this should reflect the “ correct ” uncertainty about these labels . For all remaining images in our unlabeled dataset U\I , we use a weak form of knowledge distillation by defining pseudo-labels v as the average of the predicted probability distribution of the teacher model and the uniform distribution , given in ( 2 ) . This has two reasons : i ) a purely uniform distribution on U\I , which in the first iterations might still contain a lot of task-relevant images , leads to a bias as it does not distinguish between task-relevant and irrelevant images , ii ) only using soft-labels from the teacher model leads to overconfident predictions as one can observe in the non-out-distribution aware NSST method . Thus a trade-off between these opposing goals is their average which leads to heavy damping of the confidence ( note that the pseudo-labels have a maximal confidence of 12 + 1 K on U\I ) . E ) Training : For the final objective in ( 3 ) , the selected pseudo-labeled samples in I and the original samples in T are assigned the same weight . This is quite aggressive as for iteration t we add up to 5t as much pseudo-labeled data as labeled training data . However , this also enables larger performance gains given that the sample selection process is successful . Note that the losses on I ∪ T and on U\I have equal weight as the damping of confidences on U\I is crucial for the sample selection process . As pseudo-labels are computed on non-augmented images and we use heavy augmentation during training , our scheme can be regarded as offline consistency learning , where the model is encouraged to replicate the teacher ’ s output independent of the randomly selected augmentation . This allows us to limit the number of passes through the entire unlabeled pool to the number of self-training iterations , do model calibration and results in more stable targets . The entire scheme is repeated multiple times until the validation accuracy starts to degrade or a fixed maximum number of iterations is reached . ODST+ : In addition to ODST , we provide the variant ODST+ for scenarios where AI safety is critical . It differs from ODST only in step C ) where we calculate the final threshold as maximum over ODST ’ s in-distribution threshold and an additional out-distribution threshold . This class-specific out-distribution threshold controls the number of task-irrelevant samples that are falsely added into our pseudo-labeled sample pool I . This is done using an extra out-distribution validation set , i.e . a set of natural images that does not contain any class relevant images ( we discuss this choice in Section 4 ) . For each class c , we compute the α-quantile of the predicted probabilities for class c on the out-distribution images which we define as the out-distribution threshold for class c .

Summary: This papers studies semi-supervised learning with OOD samples appearing in unlabeled data as well as at test time, which is practical in real-world tasks. To handle OOD samples, this paper proposes a new framework based on OE and self-trainning. Extensive experiments are conducted to verify the superiority of the proposed method.