On-Target Adaptation

1 INTRODUCTION . Deep networks achieve tremendous success on various visual tasks at the expense of massive data collection and annotation efforts . Even more data is needed when training ( source ) and testing ( target ) data differ , as the model must be adapted on the new data to maintain accuracy . To reduce the annotation effort on new data , unsupervised domain adaptation ( UDA ) approaches transfer knowledge from labeled source data to unlabeled target data . Standard UDA requires simultaneous optimization on the source and target data to do so . However , this requirement may not be entirely practical , in that shifted or future target data may not be available during training . Furthermore , ( re- ) processing source data during testing may be limited by computation , bandwidth , and privacy . Most importantly , it is the target data that ultimately matters for testing . In this work , we therefore turn our attention from source to target , and how to learn more from it . Recent work adapts to the target data without the source data or even adapts during testing . However , these “ source-free ” and “ test-time ” approaches still rely heavily on the source parameters for finetuning . Source-free adaptation initializes from source parameters then optimizes on target data without the joint use of source data ( Liang et al. , 2020 ; Li et al. , 2020 ; Kundu et al. , 2020 ) . Testtime adaptation partially updates source parameters on the target data while testing ( Sun et al. , 2019 ; Schneider et al. , 2020 ; Wang et al. , 2021 ) . Such approaches reduce reliance on the source data , and can even improve accuracy , but have they made full use of the target data ? Many of the model parameters are fixed ( Liang et al. , 2020 ; Schneider et al. , 2020 ; Wang et al. , 2021 ) or regularized toward the source parameters ( Li et al. , 2020 ; Kundu et al. , 2020 ) . We investigate whether more can be learned from target , and more accuracy gained , by not transferring the source parameters . We propose on-target adaptation to unshackle the target representation from the source representation . To do so , we ( 1 ) factorize the representation from the classifier and ( 2 ) separate the source parameters from the source predictions . By factorizing the representation from the classifier , we can train the representation entirely on the target data by self-supervision . Given this on-target representation , we can then supervise a new classifier from source predictions by distillation ( Hinton et al. , 2015 ) , without transferring the source parameters . Not transferring parameters frees our target model from the constraints of the source architecture , so that we can experiment with distinct target architectures . In this way , we can even change the model size to optimize a target-specific model that is more accurate and more efficient . In contrast to prior work on adaptation , this uniquely allows for learning 100 % of the target model parameters on target data , as illustrated by Figure 1 . To realize our proposed factorization and separation , we employ contrastive learning , sourcefree adaptation , and teacher-student distillation . We initialize the target representation by selfsupervision with contrastive learning . We turn the source model into a teacher model by sourcefree adaptation , and then generate pseudo-labels to supervise distillation . We lastly train the student model on the teacher supervision , starting from the target representation and new classifier parameters , and repeat this teacher-student cycle by resetting the student classifier parameters between epochs . Contrastive learning has recently enabled self-supervised representations to compete with or even surpass supervised representations ( Chen & He , 2020 ; Caron et al. , 2020 ; He et al. , 2020 ; Chen et al. , 2020 ; Grill et al. , 2020 ; Zbontar et al. , 2021 ) . We show it provides a sufficient target representation . Our experiments show on-target adaptation achieves state-of-the-art accuracy and computational efficiency on common domain adaptation benchmarks . For model accuracy , our method brings ∼3 % absolute improvement compared to state-of-the-art unsupervised and source-free domain adaptation methods on VisDA-C ( Peng et al. , 2017 ) and ImageNet Sketch ( Wang et al. , 2019a ) while reducing 50 % + of parameters . For computation , our method reduces FLOPs by 50+ % and memory by 75+ % for each forward pass of the target model . In the long-tailed classification setting , on-target class distribution learning equals the state-of-the-art learnable weight scaling ( Kang et al. , 2019 ) without needing source data . Ablation experiments support the generality of on-target representation learning across architectures , contrastive learning methods , losses , and amount of optimization . Our contribution is to investigate whether the source data should be the primary source of target model parameters , and to propose an alternative : on-target adaptation . Our insight is that the source representation can be fully decoupled from source supervision . Domain adaptation normally emphasizes the representation of source data , by either jointly optimizing on source data or transferring source parameters . On-target adaptation emphasizes the representation of target data instead , by distilling source predictions into a self-supervised target representation . We are the first to show this is feasible , as a new kind of source-free adaptation . Furthermore we show it improves accuracy and reduces computation on standard benchmarks like VisDA-C . 2 RELATED WORK . Adaptation On-target adaptation is unique in its decoupling of the target representation from the source representation . Prior adaptation approaches transfer the source representation to the target , either by joint optimization or by initialization . To transfer the source model to a visually different target domain , unsupervised domain adaptation ( UDA ) learns a joint representation for both domains for visual recognition tasks , such as image classification ( Tzeng et al. , 2014 ) , object detection ( Chen et al. , 2018 ) , semantic segmentation ( Hoffman et al. , 2016 ) . Some of the most representative unsupervised domain adaptation ideas are 1 ) maximum mean discrepancy ( Long et al. , 2015 ; 2017 ) ; 2 ) moment/correlation matching ( Sun et al. , 2016 ; Zellinger et al. , 2017 ) ; 3 ) domain confusion ( Ganin & Lempitsky , 2015 ; Tzeng et al. , 2017 ) ; 4 ) GAN-based alignment ( Liu et al. , 2017 ; Hoffman et al. , 2018 ) . All these UDA methods need simultaneous access to both source and target data . In practice , it might be impossible to meet this requirement due to limited bandwidth , computational power , or privacy concerns . Therefore , test-time training ( Sun et al. , 2019 ) , source-free adaptation ( Liang et al. , 2020 ) , and fully test-time adaptation ( Wang et al. , 2021 ) settings focus on adapting a source model by fine-tuning on the target data without source data . Exciting concurrent work even adapts without the source model by only using source predictions ( Liang et al. , 2021 ; Zhang et al. , 2021 ; Wu et al. , 2021 ) . These “ black-box ” adaptation methods exclusively optimize teacher and student predictions , with distillation losses and output regularizers . While we likewise apply teacher-student learning , our work is complementary in using contrastive learning as a loss on the input for the student representation . Semi-supervised learning Many UDA methods follow the practice of semi-supervised learning , especially pseudo labeling ( Lee , 2013 ) which is to utilize the model prediction to generate supervision for the unlabeled images . The typical setup of unsupervised domain adaptation methods is to jointly optimize with ground truths on the source and pseudo labels on the target ( Zhang et al. , 2018 ; Choi et al. , 2019 ; Long et al. , 2017 ; Zou et al. , 2018 ) . When source data annotations are not available , DeepCluster ( Caron et al. , 2018 ) and SHOT ( Liang et al. , 2020 ) further leverage weighted k-means clustering to reduce the side effects on noisy pseudo labels . Similarly , our method does not require access to labeled source data , while only relying on the target images with generated pseudo labels . In addition , our method heavily benefits from the contrastive learned target domain representation , which is treated as initialization to overcome the misleading of noisy pseudo labels . Long-tailed recognition Long-tailed recognition tackles imbalanced class distributions in realworld data . Existing work divides into three groups : 1 ) re-balancing the data distribution ( Chawla et al. , 2002 ; Han et al. , 2005 ; Shen et al. , 2016 ; Mahajan et al. , 2018 ) ; 2 ) designing class-balanced losses ( Cui et al. , 2019 ; Khan et al. , 2017 ; Cao et al. , 2019 ; Khan et al. , 2019 ; Huang et al. , 2019 ; Lin et al. , 2017 ; Shu et al. , 2019 ; Ren et al. , 2018 ; Hayat et al. , 2019 ) ; 3 ) transfer learning across classes ( Yin et al. , 2019 ; Liu et al. , 2019 ) . All of these methods address imbalance by altering training , so that the model may learn more balanced features , and a classifier that covers common ( head ) and rare ( tail ) classes . We instead adapt the classifier for long-tailed recognition during testing . 3 METHOD : ON-TARGET ADAPTATION . The goal of the proposed on-target adaptation is to tackle domain shift during the test-time with only a source model , without the access of annotation and source data . Specifically , the supervised model with source parameter f ( · ; θs ) trained on source images xs and labels ys needs to generalize on unlabeled target data xt when an unneglectable domain shift happened . Our on-target adaptation ( Figure 2 ) is proposed to obtain target model parameter θt purely during test-time . Stage 0 ( source ) : train model with labeled source data We train a deep ConvNet and learn source parameter θs by minimizing vanilla cross-entropy loss L ( ŷs , ys ) on labeled source data ( xs , ys ) . Specifically , L ( ŷs , ys ) = −Σcp ( ysc ) log ( p ( ŷsc ) ) for the predicted probability ŷsc of class c , where target probability ysgt is 1 for the ground truth class gt and 0 for the rest . Stage 1 ( teacher ) : adapt source model without source data We update the source parameter θs during testing to minimize information maximization ( InfoMax ) loss ( Gomes et al. , 2010 ) . Specifically , InfoMax loss augment entropy loss Lent = −Σcp ( ŷtc ) log ( p ( ŷtc ) with diversity objective Ldiv = DKL ( ŷt || 1C1C ) − log ( C ) . where DKL indicates the KullbackLeibler divergence , 1C is an all-one vector with C dimensions . Here 1C1C indicates the target label vector with evenly distributed 1C probabilities , where Ldiv is propose to enforce the global diversity over classes . As for the parameters to optimize over , we follow the motivation of decoupling the representation and classifier . When the classifier is frozen , the goal of optimization is to mitigate domain shift by deriving proper target features from the source model . In particular , we keep the classifier the same on both source and target domain , and obtain ∆ by the gradient of the test-time objective ( InfoMax ) , to update the representation part of model parameter θs . Stage 2 ( student ) : initialize target model with contrastive learning Instead of fine-tuning from source model , we choose to initialize the target feature purely from target data . Benefiting from the recent advances in contrastive learning methods , we train an unsupervised model with purely unlabeled target images . Specifically , we initialized target representation via improved momentum contrast learning ( MoCo v2 ) ( He et al. , 2020 ; Chen et al. , 2020 ) . It is worth noting that our method does not require a specific contrastive learning method . In other words , the default MoCo v2 could be easily replaced by a more recent self-supervised learning model , such as SwAV ( Caron et al. , 2020 ) , SimSiam ( Chen & He , 2020 ) , Barlow Twins ( Zbontar et al. , 2021 ) . Such a modular design makes it easier to benefit from the latest advance in contrastive learning . We have performed an ablation study on the choice of contrastive learning method in Section 4.5 . Stage 3 ( teacher-student ) : transfer knowledge from teacher to student We use the adapted source model f ( · ; θs +∆ ) as the initial teacher model to generate pseudo labels y′t on unannotated target images xt . Then we fine-tune the student model f ( · ; θt ) initialized by contrastive learning on target data with cross-entropy loss L ( ŷt , y′t ) = −Σcp ( y′tc ) log ( p ( ŷtc ) ) . Specifically , we use normal distribution with a mean of zero and standard deviation of 0.01 for the classification head , since contrastive learned model does not contain classifier . The teacher would be replaced with the latest student to gradually denoise pseudo labels for the subsequent phase . Meanwhile , the contrastive learned model would re-initialize the student feature to eliminate the accumulated errors from imperfect pseudo labels . In other words , the student model would start over one more time with only newer pseudo labels for the next transferring phase . Figure 3 illustrates the procedure of transferring the knowledge from teacher to student . Specifically , the interaction between teacher and student models benefits from consistency regularization and pseudo-labeling , inspired by a recent semi-supervised learning approach called FixMatch ( Sohn et al. , 2020 ) . During the transferring , we augment the target images with random cropping , random flipping , and AutoAugment with ImageNet policy , as “ strong ” augmentation , while the “ weak ” augmentation is the combination of resizing and center cropping when generating pseudo labels . Relying on the assumption that the model should generate similar predictions on data-augmented versions of the same image ( Bachman et al. , 2014 ; Sajjadi et al. , 2016 ; Laine & Aila , 2016 ) , consistency regularization enforces the cross-entropy loss between student output on strongly-augmented images and teacher output on weakly-augmented images .

Summary: This work addresses the problem of source-free domain adaptation. Instead of fine-tuning *the source model* on the target data, this work proposes to fine-tune *a representation learned on the target data alone* using e.g. self-supervised contrastive learning. This permits distinct target architectures that can improve performance and/or reduce memory and computational cost.