On-Target Adaptation

1 INTRODUCTION . Deep networks achieve tremendous success on various visual tasks at the expense of massive data collection and annotation efforts . Even more data is needed when training ( source ) and testing ( target ) data differ , as the model must be adapted on the new data to maintain accuracy . To reduce the annotation effort on new data , unsupervised domain adaptation ( UDA ) approaches transfer knowledge from labeled source data to unlabeled target data . Standard UDA requires simultaneous optimization on the source and target data to do so . However , this requirement may not be entirely practical , in that shifted or future target data may not be available during training . Furthermore , ( re- ) processing source data during testing may be limited by computation , bandwidth , and privacy . Most importantly , it is the target data that ultimately matters for testing . In this work , we therefore turn our attention from source to target , and how to learn more from it . Recent work adapts to the target data without the source data or even adapts during testing . However , these “ source-free ” and “ test-time ” approaches still rely heavily on the source parameters for finetuning . Source-free adaptation initializes from source parameters then optimizes on target data without the joint use of source data ( Liang et al. , 2020 ; Li et al. , 2020 ; Kundu et al. , 2020 ) . Testtime adaptation partially updates source parameters on the target data while testing ( Sun et al. , 2019 ; Schneider et al. , 2020 ; Wang et al. , 2021 ) . Such approaches reduce reliance on the source data , and can even improve accuracy , but have they made full use of the target data ? Many of the model parameters are fixed ( Liang et al. , 2020 ; Schneider et al. , 2020 ; Wang et al. , 2021 ) or regularized toward the source parameters ( Li et al. , 2020 ; Kundu et al. , 2020 ) . We investigate whether more can be learned from target , and more accuracy gained , by not transferring the source parameters . We propose on-target adaptation to unshackle the target representation from the source representation . To do so , we ( 1 ) factorize the representation from the classifier and ( 2 ) separate the source parameters from the source predictions . By factorizing the representation from the classifier , we can train the representation entirely on the target data by self-supervision . Given this on-target representation , we can then supervise a new classifier from source predictions by distillation ( Hinton et al. , 2015 ) , without transferring the source parameters . Not transferring parameters frees our target model from the constraints of the source architecture , so that we can experiment with distinct target architectures . In this way , we can even change the model size to optimize a target-specific model that is more accurate and more efficient . In contrast to prior work on adaptation , this uniquely allows for learning 100 % of the target model parameters on target data , as illustrated by Figure 1 . To realize our proposed factorization and separation , we employ contrastive learning , sourcefree adaptation , and teacher-student distillation . We initialize the target representation by selfsupervision with contrastive learning . We turn the source model into a teacher model by sourcefree adaptation , and then generate pseudo-labels to supervise distillation . We lastly train the student model on the teacher supervision , starting from the target representation and new classifier parameters , and repeat this teacher-student cycle by resetting the student classifier parameters between epochs . Contrastive learning has recently enabled self-supervised representations to compete with or even surpass supervised representations ( Chen & He , 2020 ; Caron et al. , 2020 ; He et al. , 2020 ; Chen et al. , 2020 ; Grill et al. , 2020 ; Zbontar et al. , 2021 ) . We show it provides a sufficient target representation . Our experiments show on-target adaptation achieves state-of-the-art accuracy and computational efficiency on common domain adaptation benchmarks . For model accuracy , our method brings ∼3 % absolute improvement compared to state-of-the-art unsupervised and source-free domain adaptation methods on VisDA-C ( Peng et al. , 2017 ) and ImageNet Sketch ( Wang et al. , 2019a ) while reducing 50 % + of parameters . For computation , our method reduces FLOPs by 50+ % and memory by 75+ % for each forward pass of the target model . In the long-tailed classification setting , on-target class distribution learning equals the state-of-the-art learnable weight scaling ( Kang et al. , 2019 ) without needing source data . Ablation experiments support the generality of on-target representation learning across architectures , contrastive learning methods , losses , and amount of optimization . Our contribution is to investigate whether the source data should be the primary source of target model parameters , and to propose an alternative : on-target adaptation . Our insight is that the source representation can be fully decoupled from source supervision . Domain adaptation normally emphasizes the representation of source data , by either jointly optimizing on source data or transferring source parameters . On-target adaptation emphasizes the representation of target data instead , by distilling source predictions into a self-supervised target representation . We are the first to show this is feasible , as a new kind of source-free adaptation . Furthermore we show it improves accuracy and reduces computation on standard benchmarks like VisDA-C . 2 RELATED WORK . Adaptation On-target adaptation is unique in its decoupling of the target representation from the source representation . Prior adaptation approaches transfer the source representation to the target , either by joint optimization or by initialization . To transfer the source model to a visually different target domain , unsupervised domain adaptation ( UDA ) learns a joint representation for both domains for visual recognition tasks , such as image classification ( Tzeng et al. , 2014 ) , object detection ( Chen et al. , 2018 ) , semantic segmentation ( Hoffman et al. , 2016 ) . Some of the most representative unsupervised domain adaptation ideas are 1 ) maximum mean discrepancy ( Long et al. , 2015 ; 2017 ) ; 2 ) moment/correlation matching ( Sun et al. , 2016 ; Zellinger et al. , 2017 ) ; 3 ) domain confusion ( Ganin & Lempitsky , 2015 ; Tzeng et al. , 2017 ) ; 4 ) GAN-based alignment ( Liu et al. , 2017 ; Hoffman et al. , 2018 ) . All these UDA methods need simultaneous access to both source and target data . In practice , it might be impossible to meet this requirement due to limited bandwidth , computational power , or privacy concerns . Therefore , test-time training ( Sun et al. , 2019 ) , source-free adaptation ( Liang et al. , 2020 ) , and fully test-time adaptation ( Wang et al. , 2021 ) settings focus on adapting a source model by fine-tuning on the target data without source data . Exciting concurrent work even adapts without the source model by only using source predictions ( Liang et al. , 2021 ; Zhang et al. , 2021 ; Wu et al. , 2021 ) . These “ black-box ” adaptation methods exclusively optimize teacher and student predictions , with distillation losses and output regularizers . While we likewise apply teacher-student learning , our work is complementary in using contrastive learning as a loss on the input for the student representation . Semi-supervised learning Many UDA methods follow the practice of semi-supervised learning , especially pseudo labeling ( Lee , 2013 ) which is to utilize the model prediction to generate supervision for the unlabeled images . The typical setup of unsupervised domain adaptation methods is to jointly optimize with ground truths on the source and pseudo labels on the target ( Zhang et al. , 2018 ; Choi et al. , 2019 ; Long et al. , 2017 ; Zou et al. , 2018 ) . When source data annotations are not available , DeepCluster ( Caron et al. , 2018 ) and SHOT ( Liang et al. , 2020 ) further leverage weighted k-means clustering to reduce the side effects on noisy pseudo labels . Similarly , our method does not require access to labeled source data , while only relying on the target images with generated pseudo labels . In addition , our method heavily benefits from the contrastive learned target domain representation , which is treated as initialization to overcome the misleading of noisy pseudo labels . Long-tailed recognition Long-tailed recognition tackles imbalanced class distributions in realworld data . Existing work divides into three groups : 1 ) re-balancing the data distribution ( Chawla et al. , 2002 ; Han et al. , 2005 ; Shen et al. , 2016 ; Mahajan et al. , 2018 ) ; 2 ) designing class-balanced losses ( Cui et al. , 2019 ; Khan et al. , 2017 ; Cao et al. , 2019 ; Khan et al. , 2019 ; Huang et al. , 2019 ; Lin et al. , 2017 ; Shu et al. , 2019 ; Ren et al. , 2018 ; Hayat et al. , 2019 ) ; 3 ) transfer learning across classes ( Yin et al. , 2019 ; Liu et al. , 2019 ) . All of these methods address imbalance by altering training , so that the model may learn more balanced features , and a classifier that covers common ( head ) and rare ( tail ) classes . We instead adapt the classifier for long-tailed recognition during testing . 3 METHOD : ON-TARGET ADAPTATION . The goal of the proposed on-target adaptation is to tackle domain shift during the test-time with only a source model , without the access of annotation and source data . Specifically , the supervised model with source parameter f ( · ; θs ) trained on source images xs and labels ys needs to generalize on unlabeled target data xt when an unneglectable domain shift happened . Our on-target adaptation ( Figure 2 ) is proposed to obtain target model parameter θt purely during test-time . Stage 0 ( source ) : train model with labeled source data We train a deep ConvNet and learn source parameter θs by minimizing vanilla cross-entropy loss L ( ŷs , ys ) on labeled source data ( xs , ys ) . Specifically , L ( ŷs , ys ) = −Σcp ( ysc ) log ( p ( ŷsc ) ) for the predicted probability ŷsc of class c , where target probability ysgt is 1 for the ground truth class gt and 0 for the rest . Stage 1 ( teacher ) : adapt source model without source data We update the source parameter θs during testing to minimize information maximization ( InfoMax ) loss ( Gomes et al. , 2010 ) . Specifically , InfoMax loss augment entropy loss Lent = −Σcp ( ŷtc ) log ( p ( ŷtc ) with diversity objective Ldiv = DKL ( ŷt || 1C1C ) − log ( C ) . where DKL indicates the KullbackLeibler divergence , 1C is an all-one vector with C dimensions . Here 1C1C indicates the target label vector with evenly distributed 1C probabilities , where Ldiv is propose to enforce the global diversity over classes . As for the parameters to optimize over , we follow the motivation of decoupling the representation and classifier . When the classifier is frozen , the goal of optimization is to mitigate domain shift by deriving proper target features from the source model . In particular , we keep the classifier the same on both source and target domain , and obtain ∆ by the gradient of the test-time objective ( InfoMax ) , to update the representation part of model parameter θs . Stage 2 ( student ) : initialize target model with contrastive learning Instead of fine-tuning from source model , we choose to initialize the target feature purely from target data . Benefiting from the recent advances in contrastive learning methods , we train an unsupervised model with purely unlabeled target images . Specifically , we initialized target representation via improved momentum contrast learning ( MoCo v2 ) ( He et al. , 2020 ; Chen et al. , 2020 ) . It is worth noting that our method does not require a specific contrastive learning method . In other words , the default MoCo v2 could be easily replaced by a more recent self-supervised learning model , such as SwAV ( Caron et al. , 2020 ) , SimSiam ( Chen & He , 2020 ) , Barlow Twins ( Zbontar et al. , 2021 ) . Such a modular design makes it easier to benefit from the latest advance in contrastive learning . We have performed an ablation study on the choice of contrastive learning method in Section 4.5 . Stage 3 ( teacher-student ) : transfer knowledge from teacher to student We use the adapted source model f ( · ; θs +∆ ) as the initial teacher model to generate pseudo labels y′t on unannotated target images xt . Then we fine-tune the student model f ( · ; θt ) initialized by contrastive learning on target data with cross-entropy loss L ( ŷt , y′t ) = −Σcp ( y′tc ) log ( p ( ŷtc ) ) . Specifically , we use normal distribution with a mean of zero and standard deviation of 0.01 for the classification head , since contrastive learned model does not contain classifier . The teacher would be replaced with the latest student to gradually denoise pseudo labels for the subsequent phase . Meanwhile , the contrastive learned model would re-initialize the student feature to eliminate the accumulated errors from imperfect pseudo labels . In other words , the student model would start over one more time with only newer pseudo labels for the next transferring phase . Figure 3 illustrates the procedure of transferring the knowledge from teacher to student . Specifically , the interaction between teacher and student models benefits from consistency regularization and pseudo-labeling , inspired by a recent semi-supervised learning approach called FixMatch ( Sohn et al. , 2020 ) . During the transferring , we augment the target images with random cropping , random flipping , and AutoAugment with ImageNet policy , as “ strong ” augmentation , while the “ weak ” augmentation is the combination of resizing and center cropping when generating pseudo labels . Relying on the assumption that the model should generate similar predictions on data-augmented versions of the same image ( Bachman et al. , 2014 ; Sajjadi et al. , 2016 ; Laine & Aila , 2016 ) , consistency regularization enforces the cross-entropy loss between student output on strongly-augmented images and teacher output on weakly-augmented images .

Summary: The paper proposes a multi-stage approach for source-free domain adaptation, namely when source data is not available and one can only use the model pre-trained on the source and adapt it based on unlabelled target domain data. The paper advocates for learning on target not via fine-tuning the source model, but rather distill from it initializing on source using contrastive learning (the authors use Moco V2). The source model is also adapted on target via InfoMax loss before being used as the teacher for the distillation. The authors propose to use the FixMatch strategy (with strong and weak aug taken from AutoAugment) for the distillation. The authors show some gains when combining with SOTA source-free adaptation methods: TENT and SHOT. The authors also show that improved accuracy can be obtained with much smaller models trained on target using their method (e.g. replacing Res50 or Res101 with Res18 on Visda-C. Extensive experiments, ablations, and results are provided.