Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data
Year: 2021

Abstract:
Unsupervised domain adaptation aims to align a labeled source domain and an unlabeled target domain, but it requires to access the source data which often raises concerns in data privacy, data portability and data transmission efficiency. We study unsupervised model adaptation (UMA), or called Unsupervised Domain Adaptation without Source Data, an alternative setting that aims to adapt source-trained models towards target distributions without accessing source data. To this end, we design an innovative historical contrastive learning (HCL) technique that exploits historical source hypothesis to make up for the absence of source data in UMA. HCL addresses the UMA challenge from two perspectives. First, it introduces historical contrastive instance discrimination (HCID) that learns from target samples by contrasting their embeddings which are generated by the currently adapted model and the historical models. With the historical models, HCID encourages UMA to learn instance-discriminative target representations while preserving the source hypothesis. Second, it introduces historical contrastive category discrimination (HCCD) that pseudo-labels target samples to learn category-discriminative target representations. Specifically, HCCD re-weights pseudo labels according to their prediction consistency across the current and historical models. Extensive experiments show that HCL outperforms and state-of-the-art methods consistently across a variety of visual tasks and setups.

heading: 1 Introduction
text: Deep neural networks (DNNs) [28, 73, 23] have achieved great success in various computer vision tasks [8, 55, 60, 59, 28, 73, 23] but often generalize poorly to new domains due to the inter-domain discrepancy [1]. Unsupervised domain adaptation (UDA) [78, 51, 76, 64, 66, 79, 77, 103, 102, 25, 71, 44, 26, 86] addresses the inter-domain discrepancy by aligning the source and target data distributions, but it requires to access the source-domain data which often raises concerns in data privacy, data portability, and data transmission efficiency.
In this work, we study unsupervised model adaptation (UMA), an alternative setting that aims to adapt source-trained models to fit target data distribution without accessing the source-domain data. Under the UMA setting, the only information carried forward is a portable source-trained model which is usually much smaller than the source-domain data and can be transmitted more efficiently [45, 42, 43, 72, 48] as illustrated in Table 1. Beyond that, the UMA setting also alleviates the concern of data privacy and intellectual property effectively. On the other hand, the absence of the labeled source-domain data makes domain adaptation much more challenging and susceptible to collapse.
∗Corresponding author.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
To this end, we develop historical contrastive learning (HCL) that aims to make up for the absence of source data by adapting the source-trained model to fit target data distribution without forgetting source hypothesis, as illustrated in Fig. 1. HCL addresses the UMA challenge from two perspectives. First, it introduces historical contrastive instance discrimination (HCID) that learns target samples by comparing their embeddings generated by the current model (as queries) and those generated by historical models (as keys): a query is pulled close to its positive keys while pushed apart from its negative keys. HCID can thus be viewed as a new type of instance contrastive learning for the task of UMA with historical models, which learns instance-discriminative target representations without forgetting source-domain hypothesis. Second, it introduces historical contrastive category discrimination (HCCD) that pseudo-labels target samples for learning category-discriminative target representations. Specifically, HCCD re-weights the pseudo labels according to their consistency across the current and historical models.
The proposed HCL tackles UMA with three desirable features: 1) It introduces historical contrast and achieves UMA without forgetting source hypothesis; 2) The HCID works at instance level, which encourages to learn instance-discriminative target representations that generalize well to unseen data [98]; 3) The HCCD works at category level (i.e., output space) which encourages to learn category-discriminative target representation that is well aligned with the objective of down-stream tasks.
The contributions of this work can be summarized in three aspects. First, we investigate memorybased learning for unsupervised model adaptation that learns discriminative representations for unlabeled target data without forgetting source hypothesis. To the best of our knowledge, this is the first work that explores memory-based learning for the task of UMA. Second, we design historical contrastive learning which introduces historical contrastive instance discrimination and category discrimination, the latter is naturally aligned with the objective of UMA. Third, extensive experiments show that the proposed historical contrastive learning outperforms state-of-the-art methods consistently across a variety of visual tasks and setups.

heading: 2 Related Works
text: Our work is closely related to several branches of research in unsupervised model adaptation, domain adaptation, memory-based learning and contrastive learning.
Unsupervised model adaptation aims to adapt a source-trained model to fit target data distributions without accessing source-domain data. This problem has attracted increasing attention recently with a few pioneer studies each of which focuses on a specific visual task. For example, [45, 46] freezes the classifier of source-trained model and performs information maximization on target data for classification model adaptation. [42] tackles classification model adaptation with a conditional GANs that generates training images with target-alike styles and source-alike semantics. [43] presents a self-entropy descent algorithm to improve model adaptation for object detection. [72] reduces the uncertainty of target predictions (by source-trained model) for segmentation model adaptation. [48] introduces data-free knowledge distillation to transfer source-domain knowledge for segmentation model adaptation. Despite the different designs for different tasks, the common motivation of these studies is to make up for the absence of source data in domain adaptation. [40] and [88] tackle source-free domain adaptation from a generative manner by generating samples from the source classes and generating reference distributions, respectively.
We tackle the absence of source data by a memory mechanism that encourages to memorize source hypothesis during model adaptation. Specifically, we design historical contrastive learning that learns target representations by contrasting historical and currently evolved models. To the best of our knowledge, this is the first work that explores memory mechanism for UMA.
Domain adaptation is related to UMA but it requires to access labeled source data in training. Most existing work handles UDA from three typical approaches. The first exploits adversarial training to align source and target distributions in the feature, output or latent space [78, 51, 76, 14, 96, 64, 66, 79, 34, 77, 35, 21, 94, 29, 10, 9, 80, 20]. The second employs self-training to generate pseudo labels to learn from unlabeled target data iteratively [103, 69, 100, 102, 31, 19, 90, 91]. The third leverages image translation to modify image styles to reduce domain gaps [25, 71, 44, 95, 26, 86, 32, 33, 93, 92]. In addition, [30] proposes a categorical contrastive learning for domain adaptation.
Memory-based learning has been studied extensively. Memory networks [81] as one of early efforts explores to use external modules to store memory for supervised learning. Temporal ensemble [41], as well as a few following works [74, 12] extend the memory mechanism to semi-supervised learning. It employs historical hypothesis/models to regularize the current model and produces stable and competitive predictions. Mean Teacher [74] leverages moving-average models as the memory model to regularize the training, and similar idea was extended for UDA [16, 99, 4, 52]. Mutual learning [97] has also been proposed for learning among multiple peer student models.
Most aforementioned methods require labeled data in training. They do not work very well for UMA due to the absence of supervision from the labeled source data, by either collapsing in training or helping little in model adaptation performance. We design innovative historical contrastive learning to make up for the absence of the labeled source data, more details to be presented in the ensuing subsections.
Contrastive learning [82, 87, 22, 54, 101, 24, 56, 75, 36, 11] learns discriminative representations from multiple views of the same instance. It works with certain dictionary look-up mechanism [22], where a given image x is augmented into two views, query and key, and the query token q should match its designated key k+ over a set of negative keys k− from other images. Existing work can be broadly classified into three categories based on dictionary creation strategies. The first creates a memory bank [82] to store all the keys in the previous epoch. The second builds an end-to-end dictionary [87, 75, 36, 11] that generates keys using samples from the current mini-batch. The third employs a momentum encoder [22] that generates keys on-the-fly by a momentum-updated encoder.
Other related source-free adaptation works. [7] considers supervised continual learning from previously learned tasks to a new task, which learns representations using the contrastive learning objective and preserves learned representations using a self-supervised distillation step, where the contrastively learned representations are more robust against the catastrophic forgetting for supervised continual learning. [38] addresses a source-free universal domain adaptation problem that does not guarantee that the classes in the target domain are the same as in the source domain. [39] propose a simple yet effective solution to realize inheritable models suitable for open-set source-free DA problem.

heading: 3 Historical Contrastive Learning
text: This section presents the proposed historical contrastive learning that memorizes source hypothesis to make up for the absence of source data as illustrated in Fig. 2. The proposed HCL consists of two key designs. The first is historical contrastive instance discrimination which encourages to learn instance-discriminative target representations that generalize well to unseen data [98]. The second is historical contrastive category discrimination that encourages to learn category-discriminative target representations which is well aligned with the objective of visual recognition tasks. More details to be described in the ensuring subsections.

heading: 3.1 Historical Contrastive Instance Discrimination
text: The proposed HCID learns from unlabeled target samples via contrastive learning over their embeddings generated from current and historical models: the positive pairs are pulled close while negative pairs are pushed apart. It is a new type of contrastive learning for UMA, which preserves sourcedomain hypothesis by generating positive keys from historical models. HCID works at instance level and encourages to learn instance-discriminative target representations that generalize well to unseen data [98].
HCID loss. Given a query sample xq and a set of key samples Xk = {xk0 , xk1 , xk2 , ..., xkN }, HCID employs current model Et to encode the query qt = Et(xq), and historical encoders Et−m to encode the keys kt−mn = E
t−m(xkn), n = 0, · · · , N . With the encoded embeddings, HCID is achieved via a historical contrastive loss LHisNCE, minimization of which pulls q close to its positive key kt−m+ while pushing it apart from all other (negative) keys:
LHisNCE = ∑
xq∈Xtgt
− log exp(qt·kt−m+ /τ)rt−m+∑N i=0 exp(q t·kt−mi /τ)r t−m i
(1)
where τ is a temperature parameter [82], r indicates the reliability of each key kt−mn , with which we reweight the similarity loss of each key to encourage to memorize well-learnt instead of poorly-learnt historical embeddings. In this work, we use the classification entropy to estimate the reliability of each key. The positive key sample is the augmentation of the query sample[22, 36], and all the rest are negative keys.
Remark 1 Note LHisNCE in Eq.1 has a similar form as the InfoNCE loss[56, 22]. InfoNCE can actually be viewed as a special case of HisNCE, where all the query and keys are encoded by the current model (m = 0) and the reliability is fixed (rt−mi = 1,∀i). For HisNCE, we assign each key a
reliability score to encourage to memorize the well-learnt historical embeddings only. It is also worth noting that Eq. 1 only shows historical contrast with one historical model for simplicity. In practice, we could employ multiple historical models to comprehensively distill (memorize) the well-learnt embeddings from them. It could be achieved by computing Eq.1 multiple times.

heading: 3.2 Historical contrastive category discrimination
text: We design HCCD that generates pseudo labels and learns them conditioned on a historical consistency, i.e., the prediction consistency across the current and historical models. HCCD can be viewed as a new type of self-training, where pseudo labels are re-weighted by the historical consistency. It works at category level and encourages to learn category-discriminative target representations that are aligned with the objective of visual recognition tasks in UMA.
Historical contrastive pseudo label generation. Given an unlabeled sample x, the current and historical models predict pt = Gt(x) (as the query) and pt−m = Gt−m(x) (as the keys). The pseudo label and the historical consistency of the sample are computed by:
ŷ = Γ(pt),
hcon = 1− Sigmoid(||pt − pt−m||1),
where p is a K-class probability vector, Γ is the pseudo label generation function [103, 102] and ŷ = (ŷ(1), ŷ(2), ..., ŷ(C)) is the predicted category label.
HCCD loss. Given the unlabeled data x and its historical contrastive pseudo label (ŷ, hcon), HCCD performs self-training on target data x via a weighted cross-entropy loss:
LHisST = − ∑
x∈Xtgt
hcon × ŷ log px, (2)
where hcon is the per-sample historical consistency and we use it to re-weight the self-training loss. If the predictions of a sample across the current and historical models are consistent, we consider it as a well-learnt sample and increase its influence in self-training. Otherwise, we consider it as pooly-learnt sample and decrease its influence in self-training.

heading: 3.3 Theoretical Insights
text: The two designs in Historical Contrastive Learning (HCL) are inherently connected with some probabilistic models and convergent under certain conditions:
Proposition 1 The historical contrastive instance discrimination (HCID) can be modelled as a maximum likelihood problem optimized via Expectation Maximization.
Proposition 2 The HCID is convergent under certain conditions.
Proposition 3 The historical contrastive category discrimination (HCCD) can be modelled as a classification maximum likelihood problem optimized via Classification Expectation Maximization.
Proposition 4 The HCCD is convergent under certain conditions.
The proofs of Proposition 1, Proposition 2, Proposition 3 and Proposition 4 are provided in the appendix, respectively.

heading: 4 Experiments
text: This section presents experiments including datasets, implementation details, evaluations of the proposed HCL in semantic segmentation, object detection and image classification tasks as well as the discussion of its desirable features.

heading: 4.1 Datasets
text: UMA for semantic segmentation is evaluated on two challenging tasks GTA5 [61]→ Cityscapes [15] and SYNTHIA [62] → Cityscapes. GTA5 has 24, 966 synthetic images and shares 19 categories with Cityscapes. SYNTHIA contains 9, 400 synthetic images and shares 16 categories with Cityscapes. Cityscapes has 2975 and 500 real-world images for training and validation, respectively.
UMA for object detection is evaluated on tasks Cityscapes→ Foggy Cityscapes [68] and Cityscapes → BDD100k [89]. Foggy Cityscapes is derived by applying simulated fog to the 2, 975 Cityscapes images. BDD100k has 70k training images and 10k validation images, and shares 7 categories with Cityscapes. We evaluate a subset of BDD100k (i.e., daytime) as in [84, 65, 13] for fair comparisons.
UMA for image classification is evaluated on benchmarks VisDA17 [58] and Office-31 [63]. VisDA17 has 152, 409 synthetic images as the source domain and 55, 400 real images of 12 shared categories as the target domain. Office-31 has 4110 images of three sources including 2817 from Amazon, 795 from Webcam and 498 from DSLR (with 31 shared categories). Following [102, 63, 70], the evaluation is on six adaptation tasks A→W, D→W, W→D, A→D, D→A, and W→A.

heading: 4.2 Implementation Details
text: Semantic segmentation: Following [79, 103], we employ DeepLab-V2 [8] as the segmentation model. We adopt SGD [3] with momentum 0.9, weight decay 1e − 4 and learning rate 2.5e − 4, where the learning rate is decayed by a polynomial annealing policy [8].
Object detection: Following [84, 65, 13], we adopt Faster R-CNN [60] as the detection model. We use SGD [3] with momentum 0.9 and weight decay 5e− 4. The learning rate is 1e− 3 for first 50k iterations and then decreased to 1e− 4 for another 20k iterations. Image classification: Following [102, 63, 70], we adopt ResNet-101 and ResNet-50 [23] as the classification models for VisDA17 and Office-31, respectively. We use SGD [3] with momentum 0.9, weight decay 5e− 4, learning rate 1e− 3 and batch size 32.

heading: 4.3 Unsupervised Domain Adaption for Semantic Segmentation
text: We evaluated the proposed HCL in UMA-based semantic segmentation tasks GTA5→ Cityscapes and SYNTHIA → Cityscapes. Tables 2 and 3 show experimental results in mean Intersectionover-Union (mIoU). We can see that HCL outperforms state-of-the-art UMA methods by large margins. In addition, HCL is complementary to existing UMA methods and incorporating it as
denoted by “+HCL” improves the existing UMA methods clearly and consistently. Furthermore, HCL even achieves competitive performance as compared with state-of-the-art UDA methods (labeled by 7in the column SF) which require to access the labeled source data in training. Further, We conduct ablation studies of the proposed HCL over the UMA-based semantic segmentation task GTA5→ Cityscapes. As the bottom of Table 2 shows, either HCID or HCCD achieves comparable performance. In addition, HCID and HCCD offer orthogonal self-supervision signals where HCID focuses on instance-level discrimination between queries and historical keys and HCCD focuses on category-level discrimination among samples with different pseudo category labels. The two designs are thus complementary and the combination of them in HCL produces the best segmentation.

heading: 4.4 Unsupervised Domain Adaptation for Object Detection
text: We evaluated the proposed HCL over the UMA-based object detection tasks Cityscapes→ Foggy Cityscapes and Cityscapes → BDD100k. Tables 4 and 5 show experimental results. We can observe that HCL outperforms state-of-the-art UMA method SFOD clearly. Similar to the semantic segmentation experiments, HCL achieves competitive performance as compared with state-of-the-art UDA methods (labeled by 7in column SF) which require to access labeled source data in training.

heading: 4.5 Unsupervised Domain Adaptation for Image Classification
text: We evaluate the proposed HCL over the UMA-based image classificat tasks VisDA17 and Office-31. Tables 6 and 7 show experimental results. We can observe that HCL outperforms state-of-the-art UMA methods clearly. Similar to the semantic segmentation and object detection experiments, HCL achieves competitive performance as compared with state-of-the-art UDA methods (labeled by 7) which require to access labeled source data in training.

heading: 4.6 Discussion
text: Generalization across computer vision tasks: We study how HCL generalizes across computer vision tasks by evaluating it over three representative tasks on semantic segmentation, object detection and image classification. Experiments in Tables 2- 7 show that HCL achieves competitive performance consistently across all three visual tasks, demonstrating the generalization ability of HCL across computer vision tasks.
Complementarity studies: We study the complementarity of our proposed HCL by combining it with existing UMA methods. Experiments in Table 2 (the row highlighted by “+HCL”) shows that incorporating HCL boosts the existing UMA methods consistently.
Feature visualization: This paragraph presents the t-SNE [53] visualization of feature representation on GTA→ Cityscapes model adaptation task. We compare HCL with two state-of-the-art UMA methods, i.e., “UR" [72] and “SFDA" [48], and Fig.3 shows the visualization. We can observe that HCL can learn desirable instance-discriminative yet category-discriminative representations because it incorporates two key designs that work in a complementary manner: 1) HCID works at instance level, which encourages to learn instance-discriminative target representations that generalize well to unseen data [98]; 2) HCCD works at category level which encourages to learn category-discriminative target representations that are well aligned with the objective of down-stream visual tasks. In addition, qualitative illustrations are provided in Fig.4. It can be observed that our proposed HCL clearly outperforms UR and SFDA.
Generalization across learning setups: We study how HCL generalizes across learning setups by adapting it into two adaptation setups, i.e., partial-set adaptation and open-set adaptation. Experiments in Table 8 show that HCL achieves competitive performance consistently across both setups.

heading: 5 Conclusion
text: In this work, we studied historical contrastive learning, an innovative UMA technique that exploits historical source hypothesis to make up for the absence of source data in UMA. We achieve historical contrastive learning by novel designs of historical contrastive instance discrimination and historical contrastive category discrimination which learn discriminative representations for target data while preserving source hypothesis simultaneously. Extensive experiments over a variety of visual tasks and learning setups show that HCL outperforms state-of-the-art techniques consistently. Moving forward, we will explore memory-based learning in other transfer learning tasks.

heading: Acknowledgement
text: This research was conducted at Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU), which is a collaboration between Singapore Telecommunications Limited (Singtel) and Nanyang Technological University (NTU) that is supported by A*STAR under its Industry Alignment Fund (LOA Award number: I1701E0013).

Review summaries:
The authors address unsupervised model adaptation (UMA). While unsupervised domain adaptation (UDA) lets the model access the source domain data during an adaptation to a target domain, such access could raise multiple concerns such as privacy and portability. UMA does not allow the model to access the source domain data during adaptation. The proposed method, historical contrastive learning (HCL), utilizes the historical source hypothesis to remedy the lack of source data. The experimental results show that the proposed method achieves state-of-the-art performance for multiple visual recognition tasks.

In this paper, the authors propose to solve the problem of unsupervised model adaptation. In this task, the aim is that given a classifier trained on a source dataset and a set of unlabelled target examples, an unsupervised model adaptation is performed to improve the performance of the model on the target dataset without requiring access to source examples. The method is evaluated on standard domain adaptation datasets for classification, detection and segmentation. The approach is related to EM and some theoretical insights provided. 

This paper studies the unsupervised model adaptation problem, where only the trained model, but not the source data, is available to use for target domain adaptation. The authors propose historical contrastive learning (HCL) technique to help this source-free setting. HCL is composed of two components, HCID and HCCD. HCID learns from target instances by contrasting the features generated from the adapted model and historical models. HCCD explores self-training to pseudo label target instances to learn category discriminative representations. Experiments on three different vision applications are conducted. The paper is generally easy to follow, despite some unclear points. The major concern is the novelty.  

This paper proposes a generic method HCL to solve the source data-free domain adaptation problem across various computer vision tasks. Based on the memory mechanism, the method involves two designs to learn discriminative features while preserving the source hypothesis. The first design HCID leverages the contrastive instance discrimination where positive pairs are made of the feature from the current model and features from historical models including the original source model and model in previous epochs. The second design HCCD leverages the consistency between the current model and previous models to select confident pseudo labels for task-specific self-training. Experiments across various vision tasks and domain adaptation settings validate the effectiveness of both designs and the generic method.