If your data distribution shifts, use self-learning

1 INTRODUCTION . Deep Neural Networks ( DNNs ) can reach human-level performance in complex cognitive tasks ( Brown et al. , 2020 ; He et al. , 2016a ; Berner et al. , 2019 ) if the distribution of the test data is sufficiently similar to the training data . However , DNNs are known to struggle if the distribution of the test data is shifted relatively to the training data ( Geirhos et al. , 2018 ; Dodge & Karam , 2017 ) . Two largely distinct communities aim to increase the performance of models under test-time distribution shifts : The robustness community generally considers ImageNet-scale datasets and evaluates models in an ad-hoc scenario . Models are trained on a clean source dataset like ImageNet , using heavy data augmentation ( Hendrycks et al. , 2020a ; Rusak et al. , 2020 ; Geirhos et al. , 2019 ) and/or large-scale pre-training ( Xie et al. , 2020a ; Mahajan et al. , 2018 ) . The trained models are not adapted in any way to test-time distribution shifts . This evaluation scenario is relevant for applications in which very different distribution shifts are encountered in an unpredictable order , and hence misses out on the gains of adaptation to unlabeled samples of the target distribution . The unsupervised domain adaptation ( UDA ) community often considers smaller-scale datasets and assumes that both the source and the ( unlabeled ) target dataset are known . Models are trained on both datasets ( e.g. , with an adversarial domain objective , Ganin et al. , 2016 ) before evaluation on the target domain data . This evaluation scenario provides optimal conditions for adaptation , but the reliance on the source dataset makes UDA more computationally expensive , more impractical and prevents the use of pre-trained models for which the source dataset is unknown or simply too large . In this work , we consider the source-free domain adaptation setting , a middle ground between the classical ad-hoc robustness setting and UDA in which models can adapt to the target distribution but without using the source dataset ( Kundu et al. , 2020 ; Kim et al. , 2021 ; Li et al. , 2020 ; Liang et al. , 2020 ) . This evaluation scenario is interesting for many practitioners and applications as an extension of the ad-hoc robustness scenario . It evaluates the possible performance of a deployed model on a systematic , unseen distribution shift at inference time : an embedded computer vision system in an autonomous car should adapt to changes without being trained on all available training data ; an image-based quality control software may not necessarily open-source the images it has been trained on , but still has to be adapted to the lighting conditions at the operation location ; a computer vision system in a hospital should perform robustly when tested on a scanner different from the training images—importantly , it might not be known at development time which scanner it will be tested on , and it might be prohibited to share images from many hospitals to run UDA . Can self-learning methods like pseudo-labeling and entropy-minimization also be used in this source-free domain adaptation setting ? To answer this question , we perform an extensive study of several self-learning variants , and find consistent and substantial gains in test-time performance across several robustness and out-of-domain benchmarks and a wide range of models and pretraining methods , including models trained with UDA methods that do not use self-learning . We also find that self-learning outperforms state-of-the-art source-free domain adaptation methods , namely Test-Time Training which is based on a self-supervised auxiliary objective and continual training ( Sun et al. , 2019b ) , test-time entropy minimization ( Wang et al. , 2020 ) and ( gradient-free ) BatchNorm adaptation ( Schneider et al. , 2020 ; Nado et al. , 2020 ) . We perform a large number of ablations to study important design choices for self-learning methods in source-free domain adaptation . Furthermore , we show that a variant of pseudo-labeling with a robust loss function consistently outperforms entropy minimization on ImageNet-scale datasets . We theoretically analyze and empirically verify the influence of the temperature parameter in self-learning and provide guidelines how this single parameter should be chosen . Our approach is visualized in Figure 1 . We do not consider test-time adaptation in an online setting like is studied e.g. , by Zhang et al . ( 2021 ) , where the model is adapted to one example at a time , and reset after each example . Related Work . Variants of self-learning have been used for UDA ( Berthelot et al. , 2021 ) , for example using auxiliary information ( Xie et al. , 2020b ) , consistency ( Wei et al. , 2020 ; Cai et al. , 2021 ; Prabhu et al. , 2021 ) or confidence ( Zou et al. , 2019 ) regularization . The main difference from these works to ours is that they 1 ) utilize both source and target data for self-learning whereas we only require access to unlabeled target data , 2 ) train their models from scratch whereas we merely fine-tune pretrained checkpoints on the unlabeled target data , and 3 ) are generally more complicated than our approach due to using more than one term in the objective function . Our work is conceptually most similar to virtual adversarial domain adaptation in the fine-tuning phase of DIRT-T ( Shu et al. , 2018 ) ) and Test-time entropy minimization ( TENT ; Wang et al. , 2020 ) . In contrast to DIRT-T , our objective is simpler and we scale the approach to considerably larger datasets on ImageNet scale . TENT , on the other hand , only evaluated a single method ( entropy minimization ) on a single vanilla model ( ResNet-50 ) on IN-C. We substantially expand this analysis to show that self-learning almost universally increases test-time performance under distribution shifts , regardless of the type of distribution shift , the model architecture or the pre-training method . Self-learning has also been applied to UDA for semantic segmentation ( Zou et al. , 2018 ) , for gradual domain adaptation ( Kumar et al. , 2020 ) , for semi-supervised learning ( Rizve et al. , 2021 ; Mukherjee & Awadallah , 2020 ) , for learning in biased datasets ( Chen et al. , 2020b ) and for automated data annotation ( De Sousa Ribeiro et al. , 2020 ) . Zoph et al . ( 2020 ) show that self-learning outperforms pretraining when stronger data augmentation is used and more labeled data is present . A more detailed discussion of related work alongside with the main differences to our work can be found in Appendix F. Our main contribution beyond these works is to show the effectiveness of self-learning on top of both robust , large scale , and domain adapted models , at scale . 2 SELF-LEARNING FOR TEST-TIME ADAPTATION . Different variants of self-learning have been used in both unsupervised domain adaptation ( French et al. , 2018 ; Shu et al. , 2018 ) , self-supervised representation learning ( Caron et al. , 2021 ) , and in semi-supervised learning ( Xie et al. , 2020a ) . In a typical self-learning setting a teacher network f t trained on the source domain predicts labels on the target domain . Then , a student model fs is fine-tuned on the predicted labels . In the following , let f t ( x ) denote the logits for sample x and let pt ( j|x ) ≡ σj ( f t ( x ) ) denote the probability for class j obtained from a softmax function σj ( · ) . Similarly , fs ( x ) and ps ( j|x ) denote the logits and probabilities for the student model fs . For all techniques , one can optionally only admit samples where the probability maxj pt ( j|x ) exceeds some threshold . We consider three popular variants of self-learning : Pseudo-labeling with hard or soft labels , as well as entropy minimization . Hard Pseudo-Labeling ( Lee , 2013 ; Galstyan & Cohen , 2007 ) . We generate labels using the teacher and train the student on pseudo-labels i using the standard cross-entropy loss , ` H ( x ) : = − log ps ( i|x ) , i = argmaxj pt ( j|x ) ( 1 ) Usually , only samples with a confidence above a certain threshold are considered for training the student . We test several thresholds but note that thresholding means discarding a potentially large portion of the data which leads to a performance decrease in itself . The teacher is updated after each epoch . Soft Pseudo-Labeling ( Lee , 2013 ; Galstyan & Cohen , 2007 ) . In contrast to the hard pseudolabeling variant , we here train the student on class probabilities predicted by the teacher , ` S ( x ) : = − ∑ j pt ( j|x ) log ps ( j|x ) . ( 2 ) Soft pseudo-labeling is typically not used in conjunction with thresholding , since it already incorporates the certainty of the model . The teacher is updated after each epoch . Entropy Minimization ( ENT ; Grandvalet & Bengio , 2004 ) . This variant is similar to soft pseudolabeling , but we no longer differentiate between a teacher and student network . It corresponds to an “ instantaneous ” update of the teacher . The training objective becomes ` E ( x ) : = − ∑ j ps ( j|x ) log ps ( j|x ) . ( 3 ) Intuitively , self-training with entropy minimization leads to a sharpening of the output distribution for each sample , making the model more confident in its predictions . Robust Pseudo-Labeling ( RPL ) . Virtually all introduced self-training variants use the standard cross-entropy classification objective . However , the standard cross-entropy loss has been shown to be sensitive to label noise ( Zhang & Sabuncu , 2018 ; Zhang et al. , 2017 ) . In the setting of domain adaptation , inaccuracies in the teacher predictions and , thus , the labels for the student , are inescapable , with severe repercussions for training stability and hyperparameter sensitivity as we show in the results . As a straight-forward solution to this problem , we propose to replace the cross-entropy loss by a robust classification loss designed to withstand certain amounts of label noise ( Ghosh et al. , 2017 ; Song et al. , 2020 ; Shu et al. , 2020 ; Zhang & Sabuncu , 2018 ) . A popular candidate is the Generalized Cross Entropy ( GCE ) loss which combines the noise-tolerant Mean Absolute Error ( MAE ) loss ( Ghosh et al. , 2017 ) with the CE loss . We only consider the hard labels and use the robust GCE loss as the training loss for the student , i = argmaxj p t ( j|x ) , ` GCE ( x , i ) : = q−1 ( 1− ps ( i|x ) q ) , ( 4 ) with q ∈ ( 0 , 1 ] . For the limit case q → 0 , the GCE loss approaches the CE loss and for q = 1 , the GCE loss is the MAE loss ( Zhang & Sabuncu , 2018 ) . We test updating the teacher both after every update step of the student ( RPL ) and once per epoch ( RPLep ) . 3 EXPERIMENT DESIGN . Datasets . IN-C ( Hendrycks & Dietterich , 2019 ) contains corrupted versions of the 50 000 images in the IN validation set . There are fifteen test and four hold-out corruptions , and there are five severity levels for each corruption . The established metric to report model performance on IN-C is the mean Corruption Error ( mCE ) where the error is normalized by the AlexNet error , and averaged over all corruptions and severity levels , see Eq . 20 , Appendix C.1 . IN-R ( Hendrycks et al. , 2020a ) contains 30 000 images with artistic renditions of 200 classes of the IN dataset . IN-A ( an , 2019 ) is composed of 7500 unmodified real-world images on which standard IN-trained ResNet50 ( He et al. , 2016b ) models yield chance level performance . CIFAR10 ( Krizhevsky et al. , 2009 ) and STL10 ( Coates et al. , 2011 ) are small-scale image recognition datasets with 10 classes each , and training sets of 50 000/5000 images and test sets of 10 000/8000 images , respectively . The digit datasets MNIST ( Deng , 2012 ) and MNIST-M ( Ganin et al. , 2016 ) both have 60 000 training and 10 000 test images . Hyperparameters . The different self-learning variants have a range of hyperparameters such as the learning rate or the stopping criterion . Our goal is to give a realistic estimation on the performance to be expected in practice .. To this end , we optimize hyperparameters for each variant of pseudolabeling on a hold-out set of IN-C that contains four types of image corruptions ( “ speckle noise ” , “ Gaussian blur ” , “ saturate ” and “ spatter ” ) with five different strengths each , following the procedure suggested in Hendrycks & Dietterich ( 2019 ) . We refer to the hold-out set of IN-C as our dev set . Models for ImageNet-scale datasets . We consider four popular model architectures : ResNet50 ( He et al. , 2016b ) , DenseNet161 ( Huang et al. , 2017 ) , ResNeXt101 ( Xie et al. , 2017 ) and EfficientNet-L2 ( Tan & Le , 2019 ) ( see Appendix B.1 for details on the used models ) . For ResNet50 , DenseNet and ResNeXt101 , we include a simple vanilla version trained on IN only . For ResNet50 and ResNeXt101 , we additionally include a state-of-the-art robust version trained with DeepAugment and Augmix ( DAug+AM , Hendrycks et al. , 2020a ) 1 . For the ResNeXt model , we also include a version that was trained on 3.5 billion weakly labeled images ( IG-3.5B , Mahajan et al. , 2018 ) . Finally , for EfficientNet-L2 we select the current state of the art on IN-C which was trained on 300 million images from JFT-300M ( Chollet , 2017 ; Hinton et al. , 2014 ) using a noisy studentteacher protocol ( Xie et al. , 2020a ) . We validate the IN and IN-C performance of all considered models and match the originally reported scores ( Schneider et al. , 2020 ) . For EfficientNet-L2 , we match IN top-1 accuracy up to 0.1 % points , and IN-C up to 0.6 % mCE . Models for CIFAR10/MNIST-scale datasets . For CIFAR10-C experiments , we use two WideResNets ( WRN , Zagoruyko & Komodakis , 2016 ) : the first one is trained on CIFAR10 and has a depth of 28 and a width of 10 and the second one is trained with AugMix ( Hendrycks et al. , 2020b ) and has a depth of 40 and a width of 2 . The remaining small-scale models are trained with unsupervised domain adaptation ( UDA ) methods . We propose to regard any UDA method which requires joint training with source and target data as a pre-training step , similar to regular pretraining on IN , and use self-learning on top of the final checkpoint . We consider two popular UDA methods : self-supervised domain adaptation ( UDA-SS ; Sun et al. , 2019a ) and Domain-Adversarial Training of Neural Networks ( DANN ; Ganin et al. , 2016 ) . In UDA-SS , the authors seek to align the representations of both domains by performing an auxiliary self-supervised task on both domains simultaneously . In all UDA-SS experiments , we use a WideResNet with a depth of 26 and a width of 16 . In DANN , the authors learn a domain-invariant embedding by optimizing a minimax objective . For all DANN experiments except for MNIST→MNIST-M , we use the same WRN architecture as above . For the MNIST→MNIST-M experiment , the training with the larger model diverged and we used a smaller WideResNet version with a width of 2 . We note that DANN training involves optimizing a minimax objective and is generally harder to tune .

Summary: The paper studies effectiveness of self-training to improve test time performance when the distribution of test data is not similar to the training data. The paper more specifically focuses on source-free domain adaptation settings where the source target data is not available. In this setup self-training has been tested as an additional step on top of  different robustness and adaptation approaches such as robust pretraining, unsupervised domain adaptation and self-supervised pretraining. The paper shows improvement on multiple ImageNet variants and CIFAR10-C, and also introduces ImageNet-D dataset as a new benchmark. ImageNet-D has been produced by matching label space of IN datasets with DomainNet data provided in Visual domain adaptation challenges. The main contribution of this paper is to perform a systematic and large study of self-training as a method to deal with distribution shifts.