Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning
Authors: Ziyi Zhang
Weikai Chen
Hui Cheng
Zhen Li
Siyuan Li
Liang Lin
Guanbin Li
Venue: NeurIPS 2022 Accept (2022)
Keywords: domain adaptation
source-free
contrastive learning

Abstract: We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source pretrained model is adapted to the target domain without access to the source data. Existing techniques mainly leverage self-supervised pseudo-labeling to achieve class-wise global alignment [1] or rely on local structure extraction that encourages the feature consistency among neighborhoods [2]. While impressive progress has been made, both lines of methods have their own drawbacks – the “global” approach is sensitive to noisy labels while the “local” counterpart suffers from the source bias. In this paper, we present Divide and Contrast (DaC), a new paradigm for SFUDA that strives to connect the good ends of both worlds while bypassing their limitations. Based on the prediction confidence of the source model, DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals under an adaptive contrastive learning framework. Specifically, the source-like samples are utilized for learning global class clustering thanks to their relatively clean labels. The more noisy target-specific data are harnessed at the instance level for learning the intrinsic local structures. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch. Extensive experiments on VisDA, Office-Home, and the more challenging DomainNet have verified the superior performance of DaC over current state-of-the-art approaches. The code is available at https://github.com/ZyeZhang/DaC.git.

Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning
Ziyi Zhang', Weikai Chen², Hui Cheng?, Zhen Li4.5, Siyuan Li Liang Lin?, Guanbin Li
National Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China Tencent America, 4 'The Chinese University Hong Kong, Shenzhen, China 5 Shenzhen Research Institute of Big Data, Shenzhen, China AI Lab, School of Engineering, Westlake University, Hangzhou, China zhangziyi 01 amda amda.nju www..ju..edeeee edu. cn, www..c..iguuubiinnm..... Liguanbindmail. sysu.edu.com edu.
Abstract
We investigate a practical domain adaptation task, called source-free unsupervised domain adaptation (SFUDA), where the source pretrained model is adapted the target domain without access to the source data. Existing techniques mainly leverage self supervised lebellla to achieve class- wise global alignment [1] or rely on local structure extraction that encourages the feature consistency among neighborhoods [2]. While impressive progress has been made, both lines methods have their own drawbacks the "global" approach is sensitive to noisy labels while the "local" counterpart suffers from the source bias. In this paper, we present Divide and Contrast (DaC), new paradigm for SFUDA that strives to connect the good ends of both worlds while bypassing their limitations. Based on the prediction confidence of the source model, DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals under an adaptive contrastive learning framework. Specifically, the source-like samples are utilized for learning global class clustering thanks to their relatively clean labels. The more noisy target-specific data are harnessed at the instance level for learning the intrinsic local structures. We further align the source- like domain with the farget-specific samples using memory-based maximum mean discrepancy (MMD) loss to reduce the distribution mismatch. Extensive experiments VisDA, Office-Home and the more challenging DomainNet have verified the superior performance of DaC over current saate-of-the-at approaches. The code is available at https: ///tthut.t///////////////g com wtt.//eeeZZaaggageeeeeeeaaa DDC. git.
Introduction
Deep neural networks have brought impressive advances to the cutting edges of vast machine learning tasks. However, the leap in performance often comes at the cost of large-scale labeled data. To ease the process of laborious data annotation, domain adaptation (DA) provides an attractive option that transfers the knowledge learned from the label-rich source domain to the unlabeled target data. Though most DA approaches require the availability of source data during adaptation, in real-world scenarios, one may only access a source- source-trained model instead of source data due to privacy issues. Hence, this work studies more practical task, coded source-free unsupervised domain adaptation (SFUDA), that seeks to adapt a source model a target domain without source data.
*Corresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

There are two mainstream strategies to tackle the SFUDA problem. One line of approaches focuses on class-wise global adaptation. The key idea to mitigate domain shift by pseudo-labeling the target data [1, 3] or generating images with target styles [4]. However, either the pseudo images or labels can noisy due to the domain discrepancy, which would compromise the training procedure and lead to erroneous classifications (Figure (a)). The other direction of research strives to exploit the intrinsic local structure [2. 5] by encouraging consistent predictions between neighboring samples. Nonetheless, the closeness of features may be biased by the source hypothesis, which could render false predictions. Further, as it fails to consider the global context, it may generate spurious local clusters that are detrimental to the discriminability of the trained model (Figure 1 (b)).
(a) Global adaptation method
(b) Local consistency method
(c) Divide and contrast (ours)
Figure I: UMAP [6] visualizations of target features trained for 60 epochs on the randomly selected 6 VisDA classes. The results are compared with two types of baselines, global adaption based CPGA [3] and neighborhood consistency based NRC [2]. CPGA can achieve clear global-scale clusters but suffer from false predictions inside each class. NRC can only maintain the intrinsic local consistency but fail to generate clear intra-class boundaries. In contrast, our method inherits the merits of both methods - clear global clusters and strong local consistency, while bypassing their limitations.
To address the above issues, we propose Divide and Contrast (DaC), a dedicated framework for SFUDA that aims to combine the merits of existing techniques, i.e., taking full advantage of both global and local structures, while sidestepping their limitations. Our key observation is that the "global" approach based on self-training can form clear clusters in the feature landscape (Figure I (a)). Its prediction artifacts are mainly caused by noisy labels with low prediction confidence. Hence, after self-training by pscudo labels, we propose divide the target data into source-like and target-specific samples, based on the prediction confidence of the source classifier [1]. The two groups of samples are then fully exploited with tailored learning strategies under a unified adaptive contrastive learning framework with a memory bank. Specifically, the memory bank consists of representations of all target samples, and momentum updated in the training stage [3, 7J. Thanks to the high prediction confidence, the memory bank generates robust class centroids as positive prototypes for source-like samples. This ensures DaC can obtain discriminative global structure which robust to noisy pseudo labels. In contrast, for the low-confidence target-specific samples, we ignore their noisy class-level supervisory signals and use the memory bank to generate the positive prototype via the local structure. This encourages the network to exploit the intrinsic local structures by contrastive learning.
To prevent the local clustering from forming spurious clusters, as shown in Figure 1 (b), we further transfer the cluster structure from the source-like domain to the target-specifer samples. In particular, we propose maximum mean discrepancy (MMD) based on memory bank for measuring the distribution mismatch between the source-like and target-specific data. Since the set of source-like and target-specific samples are dynamically updated in our framework, the introduction of memory bank can effectively alleviate the instability and batch bias caused by mini batches. As seen in Figure 1 (c), the proposed DaC framework can generate clearer and cleaner clusters than both the class-wise and instance discrimination methods. Finally, we provide a rigorous theoretical analysis of the upper bound of the task risk of the target model (Sec. 3) that justifies the soundness of our proposed framework. Extensive experiments demonstrate that our proposed DaC framework has set new state of the art number of challenging datasets, including VisDA, Office- Home, and NemainNet.
We summarize our contributions follows: 1) a novel divide-and-contraa paradigm for SFUDA that can fully exploit both the global and local structures of target data via data segmentation and customized learning strategies for data subsets; 2) unified adaptive contrastive learning framework

---

that achieves class-wise adaptation for source-like samples and local consistency for target-specific data; 3) a memory bank based MMD loss reduce the batch bias in batch training and an improved MMD loss with non-negative exponential form; 4) new state-of-the-art performance on VisDA, Office-Home and DomainNet.
2 Related Work
Unsupervised Domain Adaptation. Conventional UDA methods alleviate the domain discrepancy between the source domain and target domain by leveraging the Maximum Mean Discrepancy (MMD) [8] to match high-order moments of distributions [9, 10, 11] or by adversarial training to learn domain invariant features [12, 13). In addition, some methods [14, 15] utilize contrastive learning to enhance the global class-level structure by minimizing intra-class distance and maximizing inter-class distance, and others [16, 7] use memory bank to provide class-level information from the source domain and instance-level information from the target domain for contrastive learning. However, the absence of source data makes these methods cannot be applied directly.
Source-free Unsupervised Domain Adaptation. Source-free Unsupervised Domain Adaptation (SFUDA) aims adapt the well-trained source model to the target domain without the annotated source data. Some methods focus on generating target-style images [4] or reconstructing fake source distributions via source model [17]. Another stream of SFUDA methods exploiting the information provided by the source model. Some methods [1, 18, 19, 3] only leverage the class-level information from the source model and adapt the source model to the target domain by pseudo-labeling. while the other [2, 5] only exploit the neighborhood information and encourage the consistent predictions among samples with highly local affinity.
Contrastive Learning. Contrastive learning achieves the promising improvement on unsupervised visual representation learning [20, 21, 22, 23, 24] by learning instance discriminative representations. Although the instance-level contrastive learning has well generalization capability in downstream tasks, does not perform well on the source-free domain adaptation tasks, which demand correct measurement of inter-class affinities on the unsupervised target domain.
3 Preliminaries and Analysis
Our Divide and Contrast paradigm mainly divides the target data Dr into source-like samples Ds and target-specific outliers Do via the source classifier. We claim the consistency robustness (Claim 3.1) of the source-like samples, and further show in Theorem 3.2 an upper bound of task error on the target domain. In this part, we introduce some notations, assumptions, and our theoretical analysis.
Preliminary. Only with well-trained model source domain, the goal of the SFUDA task is to learn a model h which minimizes the task risk (D, in the target domain, i.e. CD, (h) = [h(x)) * h² (a:))], where Dr is our available n, unlabeled i.i.d samples from target domain, and h" is the ideal model in all model space H. Specifically, we consider a C-way classification task, where the source and target domain share the same label space. The model h consists of a feature extractor e and a classifier g, i.e. h((x) g g((())) which maps input space R' to prediction vector space C and h(x)) = arg max, h(z))1))- The source model is denoted as: h4 = 9s Φ.. The feature from the feature extractor is denoted as f. = db(;)). For the training stage, the batch data randomly sampled from Dy is denoted as Br, and 5 is the softmax function.
Assumptions. Following the subpopulation assumption in [25, 26, 27, 28], we also denote Dr. the conditional distribution of target data Dr given the ground-truth y = i, and further assume DT, n Dr, 0 for all i + j. To introduce the expansion assumption [27, 26], we first define that the suitable set of input transformations B(x) takes the general form B(x)) {x' 3A e A s.d.1l=' - A(x)] < r for a small radius r > and set of data augmentations A. Then, the neighborhood of a sample x E Dr. is defined as [x() == Dr O {a' ((())) n B(x')) & 0), as well as that of set 5 Dr as: (S)) == (xr). The consistency error of on domain Dr is defined as : RD, (h) = [ E B(x) s.t. h((x) + h(x)),, which indicates the model stability of local structure and input transformations. To this end, we introduce the expansion assumption to study the target domain.
3

---

Definition 3.1 ((g. 7)-cosstant expansion [27, 26))), We say Q satisfies (q. - expansion for some constant 4 E (0, 1), if for any set S Q with P Po[S] > q, we have P PolW(() \ min {y, Pq
Theoretical analysis. Our Divide and Contrast paradigm mainly divides the target data into source- like and target-specific samples via the source classifier. Specifically, by freezing the source classifier i.e. h 9e o I [I]. we select confident samples with prediction probability greater than a threshold Tes and regard them source-like samples: Ds {ai max, (h(xz)) Te, E Dr), 1 and the rest target data is target-specific samples Do Dr Ds. Denote by Ds, the conditional distribution of Dg where Ds, Ds n Dr.- The definition is similar for Do.. The following claim guarantees the existence T. and the consistency robustness of source-like samples:
Claim 3.1. Suppose h is La Lipsschht w.rt the L1 distance, there exists threshold Te (0, 1) such that the source-like set D is consistency robust, i.e. RDg (h) = 0. More specifically,
Lar 4
2
Remark 1. Claim 3.1 illustrates the source-like set with small consistency error, as long as the To is large enough. Moreover, we empirically claim that model predictions on Ds are more robust, i.e. eDg (h) S &Dx (l) [18]. The great properties of the source-like set, consistency, and robusiness, motivate us to transfer knowledge from source-like samples to target-specffic samples, by contrastive learning and distribution matching.
Assume we have a pseudo-labeler hyt based on the source model. The following theorem establishes the upper bound the target risks and states the key idea behind our method. The proofs of both the claim and theorem are provided Appendix A.
Theorem 3.2. Suppose the condition of Claim 3.1 holds and Dr, Ds satisfies (g. 7) conssannt expansion. Then the expected error of model h H. target domain Dr is bounded,
RD,(()) +7) (1) S (PD, & haa(=)]] EDS (hpp) + v) 7 min {9,7}
max (oxANdHHHHsss Do,)) +E(C)
where constant w.r.l the expansion constant and task risk of the optimal model.
Remark The Theorem 3.2 states that the target risk is bounded by the following three main parts, the fitting accuracy between model h and pseudo-labeler PDr [h(x) + hza(=)). the consistency regu- larization RD, (h). and the H-divergence between the source-like samples target-specift samples (HAH(Dss, Do.). To constrain the above three parts, the theoretical insight of our method lies in class-wise adaptation. consistency of local structure, and alignment between target-specific and source-like samples.
Based on the theoretical analysis, our method consists of three parts: 1) achieves preliminary class-wise adaptation by Leelf. which fits h to pscudo-labels; 2) leverages adaptive contrastive loss Loon to jointly achieve robust class-wise adaptation for source-like samples and local consistency regularization for target- sprget- samples; 3) minimizes the discrepancy between the source-like set and target-specific outliers by LEMMD. The entire loss is defined as:
== Leon + ofoeef + 3L
Divide and Contrast for Source-free Unsupervised Domain Adaptation
In this section, we introduce DaC, our proposed method for source-free unsupervised domain adaptation. As the overview shown in Figure 2, our method consists of three parts, self-training by pseudo-labels, an adaptive contrastive learning framework, and distribution alignment. The overall algorithm of DaC is summarized in Appendix C.
4.1 Data Segmentation and Self- feffTTaiinng
To achieve preliminary class-wise adaptation without target annotations, we generate pseudo-labels to supervise the transformation set of input B(x). In practice, we consider two different augmentation

---

Feature Extractor
Source Classifier
strongly augmented features
self
features
Divide
Target Set
C con
Memory Bank
Pseudo
Update
L EMMD
source-ikke target specitic
source- anchor
target-specific anchor
target-specific feature
source-llke anchor feature with pseudo-laelle
larget anchor
postive pair
K-neighbors
* cless centroids Local structure prototype
strongly sugmented
target-ppeificc
Figure 2: The illustration of the proposed DaC framework.
strategies in augmentation set: A {AA. A. where weak augmentation Aw refers to random cropping and flipping, and strong augmentation A, refers to the automatically learned augmentation strategy [29].
We apply the strategy proposed in [1] to update pseudo-labels in each epoch. (f:)) the prediction from source classifier. The initial centroid for each class Denote by p: (gg=(f "h2 f.p.[A]
Pseudo-labeling.
is attained by: Ca 2 p.[k] The centroids estimate the centers of different categories, and the
samples are labeled identically with its nearest initial centroid:
5 [ū.l = arg max cos (ő(z:), Ca), oz E Drh.
54 (jx -ojjk)h E (
where cos (-:-) is the cosine similarity. The k-th centroid further modified by: C₁
where 1 (-) is the indicator function, and pseudo-labels are further updated by the modified centroids Cj based on Eqn. 2 in each epoch.
Self-training for Consistency Regularization. During the training iteration, transfer each sample t; into two views: A, (1;;), (I;)). Network predictions on the two views are denoted as p and pf respectively. Denote by ú: onehot(ū.. the one-hot coding of pseudo-labels. Since the same semantic content of the two augmented views, we encourage consistent predictions between two types outputs by the following loss:
Laelf - Ex.EBr
Σ os log(p!(()) i log(vi(c)
Σ (L.( + (w)))))
where P. Eg [pt [c]] is regularized by uniform distribution encourage output diversity, HPp")) - Eg, |E, pda cc] log (p." [c]] is the Shannon entropy [30], which is used to encourage confident outputs and accelerate convergence, and w is the corresponding parameters.
Dataset Division. As the analysis before, we divide all samples in Dr into the source-like set and target-specific outliers. During the training process, samples are dynamically grouped into the source-like set by the threshold Te:
D = {æ. max(p""[c)) 2 Te, arg max(p = k, Dr}.
(4)
and the target-specific samples are the rest target data, i.e. Do = Dr Ds. where Ds = UC D..
More specifically, D = {2gl E Do. = k. j'. To this end, we design adaptive contrastive learning framework that jointly adapts source-like features to class-wise prototypes and optimizes sprget- features by local structures.

---

4.2 Adaptive Contrastive Learning
Similar [20, 16], we employ momentum updated memory bank to store all target features F = { The memory bank dynamically generates C' source-like class centroids {we)amn and n. target-specific features 1 where n, [Dol is the current number of target-specific samples, which dynamically changes in the training stage.
For general anchor feature during training time fo = (( 2t Ds U Do. we conduct prototype contrastive loss with similarity measured by inner product:
cxp(fi k- /T)
Leon B, log
(5)
exp (f: k+
/T) + E 1,= Ctno exp (f: k,)/))
where the sum is over one positive pair (fi, k+) and C n, 1 negative pairs {(fi, k }} the temperature is a hyper-parameter and is empirically set as 0.05.
4.2.1 Prototype Generation
The contrastive loss tries to classify anchor feature f as its positive prototype k* from all negative samples. Memory bank generates + Ti, features, including the class-level signals, i.e. C class centroids [w] as well as n. target-specific instance features {v). The source-like and target-specific samples are jointly optimized by generating corresponding positive prototypes.
For the source-like anchor, i.e. I; DS. since the network prediction k (in Eqn. 4) is relatively reliable [18], we encourage contrastive learning to achieve class-wise adaptation by designating the positive prototype as class centroid wh, i.c. k+ wi. The rest class centroids and n. target-specific features are used to form negative pairs.
For the target specific anchor, due to its noisy class-level supervisory signal, generate a positive prototype k= by introducing Local Structure, including neighborhood consistency and transformation stability. Specifically, we enhance the semantic consistency with the strongly augmented features f = ((A.())) and the K-nearest features. Nxx (fi), that can be easily found memory bank via cosine similarity: (f.))) = toppK (cos z;)) Vzy F]. To this end, the positive prototype k+ is generated ks: K+I J3 Σk=1 2k): where 2 (fi)).). Note that k+ depicts the local information of its corresponding feature Us and we use the rest no target-specific features {u,} and C class centroids form negative pairs.
Discussion. For the source- like anchor, the positive prototype k+ = we is the mean all source-like features with confident and consistent predictions, which makes the class-wise adaptation more robust to noisy pseudo-labels. For the i-th target-specific anchor, its corresponding memory bank feature 2; F can be found in Nxx (fi). The contrastive loss constrains the consistency error RD, in Thm 3.2. Unlike the previous contrastive method [3], Eqn. 5 jointly achieves class-wise adaptation and instance-wir wise adaptation. The source-like and target-specific samples dynamically updated, and adaptively benefit representation learning.
4.2.2 Memory Bank
One of the most important reasons to utilize the memory bank is conserve source information by momentum updating features. Therefore, the memory bank is initialized with the features by performing forward computation of source feature extractor Φ.: F = {2,1=, & (1r)), n, E "Dr). At each iteration, the memory bank features are updated via momentum strategy for the i-th input feature: 2; mz, + (1 mi, where m [0, is the momentum coefficient and is empirically set as 0.2.
The memory bank provides source-lite like centroids and target-specific features for contrastive learning. The target-specific features can be directly accessed in memory bank: {wy/e, 2,, E Do). We initialize the source-like set before initializing the source-like like centroids. Samples with confident predictions from the source classifier are termed source-like. To guarantee sufficient samples in source-like set in the early training stage, we initialize the source-like set by drawing samples with top 5% predictions in each class. The c-th source-like set is initialized as:
Σ P [c],
arg max & x.E

---

where N = 5%n, is the number of samples in each class, And Dú is updated in training stage by Eqn. After updating the source-like set Ds (= U Dī), the e-th class centroids we is generated by the mean of all source-like features:
1
We
2;.
DD
J,ED
4.3 Distribution Alignment
To obtain better adaptation performance, we further achieve alignment by minimize the H-divergence between source-like set and target-specific samples, i.e. dHAH! (Ds.: Do,), where the kernel-based
Max Mean Discrepancy (MMD) [8] is widely used. We Dis Di estimate Ds, Do,, the related optimization target mingeR (iMD((3 D)): where x is the kernel function. MMD depicts the domain discrepancy by embedding two distribution into reproducing kernel Hilbert space. Considering the MMD between two domain S, T with the linear kernel function in mini-batch (batch size B):
1
(SM (S,T)
8,
t
8/
(8)
f==1
where 8, t represent the features of two domain, m, n are their corresponding amounts in mini- batch (m BB. The previous UDA methods regard S, T as the source and the target domain, respectively. Two domains take an equal number of samples ( (.c. m = n = B/2). Since the absence of source data, samples, whose amounts m, n are uncertain in the random batch sampling. The vanilla MMD is not applicable in our setting because samples in mini-batch have estimation bias of the whole distribution, especially when the m or is small. To this end, we use features in the memory bank to estimate the whole distribution and reduce batch bias. Specifically, for S = DDg T = DE we replace m Σx= 8;' with Egg [z] = We: Ey== ty with Wig [z]. and rewrite Eqn. 8 as our linear memory bank-based MMD:
MMD measure the discrepancy between source-like and target-speeific
1
LLMMD EZ,EB, f( 9,")
where is the correlating prototype in the memory bank at the same domain. For example, if fi is from the source-like set, q; We, q.th Egg [z]. To avoid the negative term in Eq, 9, we improve it as non-negative form. By simply clipping max f0, fq fq+), ), we have:
max {0, fq - fq+} = max {fq* fq fq I log (exp(fq*) ) + exp(fq - fq
the inequality holds by the log-sum-exp bound. And the last term above can be more generally organized into our Exponential-MMM. loss as follows:
exp(f.q /T)
LEMAD EER,EB, log (xp(f.qt/7)) exppf/)))xxpp...1111 /T)
(10)
where is the temperature hyper-parameter, note that set the temperature the same as that in Equ. 5 (i.e. = 0.05). Thus the Exponential-MMD loss can be obtained by the calculated results in the memory bank without additional computing costs.
Experiments
5.1 Experimental Setup
Datasets and benchmarks. We conduct experiments on three benchmark datasets: Office-Home [31] contains 65 classes from four distinct domains (Real, Clipart, Art, Product) and total of 15,500 images. VisDA-2017 [32] is large-scale dataset, with 12-class synthetic-to-re
object recognition tasks. The dataset consists of 152k synthetic images from the source domain while 55k real object images from the target domain. DomainNet [33] is originally a large-scale multi- multi-source domain adaptation benchmark, which contains 6 domains with 345 classes. Similar to the setting of [34, 35],
7

---

we select four domains (Real, Clipart, Painting and Sketch) with 126 classes as the single-sourg unsupervised domain adaptation benchmark and construct seven single-souree adaptation scenarios from the selected four domains. All results are the average of three random runs.
Implementation Details. We adopt the Resnet-50 [36] as backbone for Office-Homa and ResNet-101 for VisDA, and conduct ResNet-34 for NomainNet similar to previous work [34, 35J. We use the same basic experimental setting in [1, 2] for fair comparison. For the network architecture, the feature extractor consists of the backbone and a full-connected layer with batch normalization, and the classification head is full-connected with weight normalization. Following SHOT, the source model is supervised by smooth labels, the source model initializes the network, and we only train the feature extractor. The learning rate for the backbone is 10 times greater than that of the additional layer. The learning rate for the backbone is set as 2c-2 on Office-Home, 5c-4 on VisDA, and le-2 on DomainNet. We train 30 epochs for Office-Home 60 epochs for VisDA, and 30 epochs for DomainNet. More training details are delineated in Appendix B.
Baselines. We compare DaC with multiple source-present and source-free domain adaptation baselines. Here we briefly introduce some of the most related state-of-the-art source- frecee-free methods: SHOT [1] and CPGA [3] exploit pseudo label prediction achieve class-wise adaptation, NRC [2] and G-SFDA [5] strengthen the local structure by neighborhood information from source model, and +HOT++ [37] is a two-stage extension of SHOT, which adds the rotation prediction auxiliary task [38] to SHOT [1] in the first stage and trains the second stage in semi-suuervee manner (MixMatch [39]). SF in tables is short for source-free.
5.2 Comparison with State-of-the-ard
We compare DaC with the state-of-the-art methods the Office-Home, VisDA, and DomainNet. As the results for VisDA shown in Table 2, our method surpasses all baselines in terms of average accuracy, including the re- cent source-present method BCDM and the most recent source-free method CPGA (87.3% v.8. 86.0%). For the NomainNet, Table 3 illustrates the proposed DaC has sig- nificantly outperformed the best source- free baseline by more than 3 percent (68.3 v.s. 65.1), and outperforms all source-present and source-free methods w.r.t average ac- curacy. Similar observation on the results of Office-Home can be found in Table 4. The reported results indicate the superiority of our method.
85
Accuracy
82
SIOT CPGA NRC
We further compare the effectiveness of our framework with the state-of-the-art SFUDA methods, in which CPGA [3] focuses on class-level supervision while NRC [2] focuses on neighborhood consistency. We show the UMAP visualization results in Fig. I, and the accuracy curve Fig. 3. Since the early accuracy of CPGA is low, Fig. 3 records the curve from 18 epochs. While CPGA suffers false prediction within each cluster, NRC source bias local clusters due to the loss of global context, causing target eгror amplification. Our method can achieve fast convergence speed and stable high classification performance thanks to the effectiveness of the proposed divide and contrast scheme.
40
60
Epochs
Figure 3: Average accuracy curve on VisDA over 60 epochs.
To validate the scalability of model, we extend DaC into a two-stage version called DaC++. The second training stage is the same as SHOT++ for a fair comparison. Table shows that DaC++ outperforms SHOT++ by more than 1 percent, and DaC++ is quite close to the target supervised learning performance even without any target annotations.
Table 1: Comparison of DaC++ and other state-of-the-ar methods VisDA.
VisDA
NRC [2]
CPGA [3]
SHOT++ [37]
DaC
DaC++
srgget-uupervssed
Avg. (%)
85.9
86.0
87.3
87.3
88.6
89.6
8

---

Table 2: Accuracy (%) on the VisDA dataset (ResNet-101),
Method
SF
plane
bicycle
bus
horse
knife
meyel
person
plant
sktbrd
train
truck
Avg.
ResNet-101 [36]
X
55.1
53.3
61.9
80.6
17.9
79.7
31.2
81.0
26.5
73.5
8.5
52.4
CDAN [40]
X
85.2
66.9
83.0
50.8
84.2
74.9
88.
74,5
83.4
76.0
81.9
38.0
73.9
SWD [41]
X
82.5
81.7
86.3
85.6
29.2
MCC [42]
X
88.7
80.3
80.5
71.5
90.1
93.2
85.0
71.6
89.4
73.8
85.0
36.9
78.8
STAR [43]
X
95.0
84.0
84.6
73.0
91.6
91.8
85.9
78.4
94.4
84.7
87.0
42.2
82.7
BCDM [44]
95.1
87.6
81.2
92.7
95.4
86,9
82.5
88.1
39.5
3C-GAN [4]
V
94.8
73.4
68.8
74.8
93.1
95.4
88.6
84.7
84.7
83.5
48.1
81.6
SHOT [1]
V
94.3
88.5
80.1
57.3
93.1
94.9
80.7
80.3
91.5
89.1
86.3
58.2
82.9
G-SFDA [5]
96.1
88.3
85.5
97.1
95.4
89.5
79.4
95.4
92.9
89.1
42.6
NRC [2]
96.8
91.3
82.4
62.4
96.2
95.9
86.1
94.8
94.1
90.4
59,7
85.9
CPGA [3]
95.6
89.0
75.4
64.9
91.7
97.5
89.7
83.8
93.9
93.4
87.7
69.0
86.0
DaC
96.6
86.8
86.4
78.4
96.4
96.2
93.6
83.8
96.8
95.1
89.6
50.0
87.3
Table Accuracy (%) on the DomainNet dataset (ResNet-34). The * baselines are implemented by the official codes.
Method
SF
Rw
CCI
Rw +t
Pt-CCI
CI-sSk
Sk~PPt
Rw->Sk
Pt-+Rw
Avg.
ResNet-34 [36]
X
58.4
62.5
56.0
50.1
41.9
48.2
70.1
56.7
MME [34]
X
70.0
67.7
69.0
56.3
64.8
61.0
76.1
66.4
CDAN [40]
X
65.0
64.9
63.7
53.1
63.4
54.5
73.2
62.5
VDA* [42]
x
63.5
65.7
62.6
52.7
53.6
62.0
74.9
62.1
GVB [45]
68.2
63.2
56.6
62.2
73.8
65.2
BAIT* [46]
64.7
65.4
62.1
57.1
61.8
56.7
73.2
63
SHOT* [1]
67.1
65.1
67.2
60.4
63
56.3
76.4
65.1
G-SFDA [5]
63.4
62.5
58.3
63.3
NRC* [2]
67.5
68.0
67.8
57.6
59.3
58.7
74.3
64.7
DaC
70.0
68.8
70.9
62.4
66.8
60.3
78.6
68.3
Table Accuracy (%) on the Office Horice dataset (ResNet-50),
Method
SF
Ar-+CI Ar-Pr
Ar-Rw CI-AAr
CI-PPr
Cl-Rw
Pr-tAr
Pr-+CI
Pr-Rw Rw
Ar Rw-iCI
Rw-PPr Avg.
ResNer-50 [36]
38.5
41.2
MCD [13]
X
48.9
68.3
74.6
61.3
67.6
68.8
57.0
47.1
75.1
69.1
52.2
79.6
64.1
[40]
50.7
70.6
77.3
70.9
BNM [47]
X
52.3
73.9
80.0
63.3
72.9
74.9
61.7
49.5
79.7
70.5
53.6
82.2
67.9
SHOT [1]
56.9
78.1
81.0
67.9
78.4
78.1
67.0
54.6
81.8
73.4
$8.1
84.5
71.6
G-SFDA [5]
57.9
78.6
66.7
77.2
77.2
82.2
72.0
57.8
71.3
CPGA [3]
59.3
78.1
79.8
65.4
75.5
76.4
65.7
58.0
81.0
72.0
64.4
83.3
71.6
[2]
DaC
V
59.1
79.5
81.2
69.3
78.9
79.2
67.4
56.4
82.4
74.0
61.4
84.4
72.8
5.3
Ablation Analysis
Role of Divide and Contrast paradigm. To study the effectiveness of proposed Divide and Contrast paradigm, we conduct contrastive learning without splitting target data. We strengthen the local class-level global structure with two contrastive learning schemes. Scheme-S focuses obtaining discriminative class-wise clusters using a contrastive learning method similar to CPGA [3]. Specifically, the positive prototype is the class centroid w.r.l, the anchor's pseudo label, while other centroids are the negative samples. Scheme-T only enhances the local structure by instance discrimination learning following [22, 20], the positive prototype is the anchor's corresponding feature in the memory bank, and the negative samples are the remaining features in the memory bank.
As shown in Table 5, both schemes improve the performance of self-training thanks to the contrastive framework. However, Scheme-S is vulnerable to the noisy pseudo labels while the Scheme-T can not further transfer the robust class-level classification into target outliers. Our method, in comparison, outperforms all the alternatives by a large margin.
Component-wiss Analysis. we conduct our ablation study by isolating each part of our method on VisDA. As the results shown in Table 6, each component of our methods helps enhance the performance, in which the adaptive contrastive learning framework makes the most contributions to

---

Table 5: Comparison with different contrastive learning schemes the best accuracy over 60 epochs on VisDA.
SHOT
Scheme-S
Scheme-T
DaC
Avg. (%)
82.9
84.1
84.4
87.3
the promotion of accuracy (over 3% points), Besides, both strong augmentation and local structure enhance the local neighborhood information for each sample, which benefit the theoretical consistency RD, (h) and improve the performance. Last but not least, removing the distribution alignment degrades the average accuracy to 86.5% which means both our linear and exponential memory-bank based MMD are helpful. The LeMmD outperforms LLMMD because the non-negative exponential form upper bound of the other, and is stable batch training.
Table 6: Ablation study of different losses (left) and different modules (right) on VisDA.
BackBone Leels Leon LLMMD LEMMO Avg.
Method
Acc
59.5
/ /
83.3
DaC
87.3
86.5
DaC w/o Local Structure
86.7
/
/
87.0
DaC w/o Strong Augmentation
85.6
/
/
87.3
Conclusion
In this paper, we have presented Divide and Contrast (DaC), a novel learning paradigm for the SFUDA problem that inherit the advantages of (global) class wise and (local) neighbors consistency approaches while sidestepping their limitations. The key idea is to divide the target data according to the prediction confidence of the source hypothesis (source-like v.s. target-specific) and apply customized learning strategies that can best fit the data property. We achieve this goal via a proposed adaptive contrastive learning where different groups of data samples learned under unified framework. We also proposed MMD loss based on a memory bank to transfer the knowledge from the source-like domain to the target-specific data. While promising performance has been achieved using our proposed approach, it would be an interesting future avenue to investigate how to extend the DaC framework to more DA tasks, e.g. supervisd SFUDA or source-free open-set DA.
Acknowledgments
This work was supported in part by the Guangdong Basic and Applied Basic Research Foundation N00.20200B1515020044 in part by National Key Research and Development Program of China (2020AAA0107200) and NSFC(61876077), in part by Open Research Projects of Zhejiang Lab (N0.2019KDOAD01/017) in part by the National Natural Science Foundation China (No.61976250, No.U181 1463), in part by the Guangzhou Science and technology project (No.202102020633)
References
[1] Jian Liang, Dapeng Hu, and Jiashi Feng. Do really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning (ICML), pages 6028 6028-6039, 2020.
[2] Shiqi Yang. Yaxing Wang, Joost van de weijer, Luis Herranz, and SHANGLING JUL. Exploiting the intrinsic neighborhood structure for source-free domain adaptation. In A. Beygelzimer, Dauphin, Liang, and Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
10

---

[3] Zhen Qiu, Yifan Zhang, Hongbin Lin, Shuaicheng Niu, Yanxia Liu, Qing Du, and Mingkui Tan. Source-free domain adaptation via avatar prototype generation and adaptation. International Joint Conference Artificial Intelligence, 2021.
[4] Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsuper- vised domain adaptation without source data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9641-9650, 2020.
[5] Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. ICCV, 2021.
[6] Leland Mclnnes, John Healy, Nathaniel Saul, and Lukas Grossherger. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018.
[7] Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, and hongsheng Li. learning with hybrid memory for domain adaptive object re-id. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 11309-11321. Curran Associates, Inc., 2020.
contrastive
[8] Karsten Borgwardt, Arthur Gretton, Malte J Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformaticx, 22 4): -57, 2006.
[9] Eric Tzeng. Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv: 142.3444, 2014.
[10] Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation alignment for unsupervised domain adaptation. In Domain Adaptation Computer Vision Applications, pages 153-171. Springer, 2017.
[11] Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger- Central moment discrepancy (emd) for domain-invariant representation learn- ing. arXiv preprint drXiv:1702.08811 2017.
[12] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180-1189. PMLR, 2015.
[13] Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 3723-3732, 2018.
[14] Shuyang Dai, Yu Cheng, et al. Contrastively smoothed class alignment for unsupervised domain adaptation. In ACCV, 2020.
(15) Guoliang Kang, Lu Jiang, et al. Contrastive adaptation network for unsupervised domain adaptation. CVPR, 2019.
[16] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, and Kate Saenko. Universal domain adaptation through self-supervision. In NeurIPS, 2020.
[17] Jiayi Tian, Jing Zhang, Wen Li, and Dong Xu. VDM-DA: virtual domain modeling for source data-free domain adaptation. CoRR, 2021.
[18] Youngeun Kim, Donghyeon Cho, et al. Progressive domain adaptation from a source pre-trained model. ArXiv, 2020.
[19] Weijie Chen, Luojun Lin, Shicai Yang, Di Xie, Shiliang Pu, Yueting Zhuang, and Wenqi Ren. Self-supervised noisy label learning for source-free unsupervised domain adaptation. CoRR, 2021.
[20] Aron van den Oord, Yazhe Li. and Oriol Vinyals. Representation learning with contrastive predictive coding. CoRR, abs//187.00374 2018.
[21] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. arXiv preprint 10X1v21111.05722, 2019.
11

---

[22] Ting Chen, Simon Korbblith, et al. A simple framework for contrastive learning of visual representations. In ICML, 2020.
[23] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. ECCV, 2020.
[24] Chengxu Zhuang, Alex Lin Zhai, and Daniel Yamins. Local aggregation for unsupervised learning of visual embeddings. CoRR, 2019.
[25] Hong Liu, Jianmin Wang, and Mingsheng Long. Cycle self-training for domain adaptation. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 22968-22981. Curran Associates, Inc., 2021.
[26] Colin Wei, Kendrick Shen, Yining Chen, and Tengyu Ma. Theoretical analysis of self-training with deep networks on unlabeled data. ICLR, 2021.
[27] Tianle Cai, Ruiqi Gao, Jason Lee, and Qi Lei. A theory of label propagation for subpopulation shift. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Con- ference on Machine Learning, Proceedings of Machine Learning Research. PMLR, 18-24 Jul 2021.
[28] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1)1151-175, 2010.
[29] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings the IEEE conference on computer vision and pattern recognition, pages 113-123, 2019.
[30] Claude Elwood Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27:379-423 1948.
[31] Hemanth Venkateswara, Jose Eusebio, et al. Deep hashing network for unsupervised domain adaptation. CVPR, 2017.
[32] Xingchao Peng, Ben Usman, et al. Visda: The visual domain adaptation challenge. ArXiv, 2017.
[33] Xingchao Peng. Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference Computer Vision, pages 1406-1415, 2019.
[34] Kumiaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi-supervised domain adaptation via minimax entropy. ICCV, 2019.
[35] Jichang Li, Guanbin Li, Yemin Shi, and Yizhou Yu. Cross-domain adaptive clustering for semi-supervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2505-2514 14,, June 2021.
[36] Kaiming He, Xiangyu Zhang, et al. Deep residual learning for image recognition. CVPR, 2016.
[37] Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021. In Press.
[38] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. ICLR, 2018
[39] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning. NeuralPS, 2019.
[40] Mingsheng Long, Zhangjie Cao, et al. Conditional adversarial domain adaptation. In NerulPS, 2018.
[41] Chen- Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel Ulbricht. Sliced wasserstein discrepancy for unsupervised domain adaptation. In CVPR, 2019.
12

---

[42] Ying Jin, Ximei Wang, et al. Minimum class confusion for versatile domain adaptation. In ECCV, 2020.
[43] Zhihe Lu, Yongxin Yang, Xiatian Zhu, Cong Liu, Yi-Zhe Song, and Tao Xiang, Stochastic classifiers for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2020.
[44] Shuang Li, Fangrui Lv, Binhui Xie, Chi Harold Liu, Jian Liang, and Chen Qin. Bi-classifier determinacy maximization for unsupervised domain adaptation. In Thirty-Fifth AAAI Conference on Artificial Intelligence (AAA1-21), 2021.
[45] Shuhao Cui, Shuhui Wang. Junbao Zhuo, Chi Su, Qingming Huang, and Tian Qi. Gradually vanishing bridge for adversarial domain adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[46] Shiqi Yang. Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Casting a bait for offline and online source-free domain adaptation. arXiv preprint MXii:20100124222 2020.
[47] Shuhao Cui, Shuhui Wang, et al. Towards discriminability and diversity: Batch nuclear-nore norm maximization under label insufficient situations. CVPR, 2020.
[48] Guanglei Yang, Haifeng Xia, al. Bi-directional generation for unsupervised domain adaptation. In AAAI, 2020.
[49] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoe Le. Unsupervised data aug- mentation for consistency training. H. Larochelle, M. Ranzato, Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6256 6268. Curran Associates, Inc., 2020.
[50] Alex Kurakin, Chun-Liang Li, Colin Raffel, David Berthelot, Ekin Dogus Cubuk, Han Zhang, Kihyuk Sohn, Nicholas Carlini, and Zizhao Zhang. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, 2020.
13


Reviews:
[{"review_id": "lPffnudFu1g", "reviewer": "Reviewer_4tiB", "summary": "This paper presents a source-free domain adaptation method. Previous methods either use self-supervised pseudo labeling to conduct class-wise global alignment or leverage local structure to enforce feature consistency. This work combines the idea of both. The proposed method divides target samples into source-like and target-specific ones. Source-like samples are used for global class clustering and target-specific samples are used for learning local structures. The two are further aligned using maximum mean discrepancy loss.", "questions": "See weaknesses above.", "limitations": "No", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths are as follows.\n\nThe idea is interesting. The target samples are divided based on confidence output of source classifier. Different groups are treated differently, either globally in class-level or locally in instance-level. Two different groups are aligned to encourage consistency that is also interesting. The presentation is generally good. Ablation study is conducted for each part.\n\nWeaknesses are as follows.\n\nThe major weakness is the performance compared to prior work. It shows that the results of the proposed method are just marginally good compared to [2] and [3] in 2 out of 3 datasets (table 1 and table 3). \n\nThe performance is good in table 2. Can the authors explain more about the baseline results? For example, how the official code was used and how the authors ensure the hyper parameters are reasonably well tuned? \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "b73UJPbAfGX", "reviewer": "Reviewer_xRX9", "summary": "The authors proposed a method for source-free unsupervised domain adaptation (SFUDA) task. The key idea is to combine the advantages of global alignment [1] and feature consistency [2]. The authors divided the target data into source-like and target-specific samples and \ntreat them by different learning methods. The authors demonstrate extensive experiments on three datasets and verify the performance of the proposed method", "questions": "1. When do the method update the source-like set and class centroids, after one batch or one epoch？", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1.The paper is well-organized and easy to follow.\n2. The \"divide and contrast\" strategy is simple but effective; it can exploit both the global and local structures of target data.\n3. The proposed Exponential-MMD loss is novel, it also makes sense to align the source-like and target-specific samples to reduce distribution mismatch.\n4. The experiment and ablations are sufficient to support the conclusion.\n\nWeaknesses\n1. The section of preliminaries and analysis is a bit unclear to introduce the task.\n2. A more recent work that outperforms proposed approach ([a]) is not compared against.\n[a] Liang, Jian, et al. \"Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer.\" IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).\n3. Eqn. 4 and Eqn. 5 share the same parameter \\tau but they are different from each other. The threshold \\tau in Eqn. 4 is set to 0.95 according to supplementary material following [47], however, I could not find the reference [47]. Besides, the choice of the threshold \\tau is not included in the ablation study, it would be necessary to see the ablation of choosing the threshold \\tau.\n4. As described in Discussion(L200-L207), unlike previous method [3], Eqn. 5 jointly achieves class-wise adaptation and instance-wise adaptation. It would be interesting if the authors could compare the proposed loss with separate class-wise adaptation and instance-wise adaptation losses.\n5. When do the method update the source-like set and class centroids, after one batch or one epoch.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Fq9YKma9Hgt", "reviewer": "Reviewer_shxH", "summary": "This paper is about unsupervised domain adaptation when the source data is unavailable at the time of adaptation. Authors term this problem as source-free unsupervised domain adaptation (SFUDA). Authors make the insightful  observation that existing SFUDA techniques use all pseudo labels on target data including noisy labels or enforce local feature consistency at the expense of being source-biased. Motivated by these observations, authors propose a new SFUDA approach that they call Divide and Contrast (DaC) which divides the target data into two disjoint groups: source-like samples and target-specific samples. An adaptive learning framework is presented that treats each of these target sample groups differently during the training process. Authors present both theoretical results and numerical results that illustrate the superiority of DaC over state of the art methods for SFUDA.", "questions": "1. Line 51: Provide a brief explanation for the \"memory bank\".\n2. Line 260: What is SHOT? Cite a reference for that.\n", "limitations": "Authors do not explicitly discuss the limitations of the proposed method, but mention semi-supervised SFUDA  and source-free open-set DA as possible research extension topics. If space permits, it may be interesting to provide a few more sentences about each of these topics and how DaC might fare in those cases.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\nAuthors illustrate well (e.g., Fig. 1) the tradeoffs involved in the global and local approaches to SFUDA.\n\nThe proposed DaC approach for SFUDA is a novel approach to address the limitations of prior methods to SFUDA.\n\nAuthors present theoretical derivations to justify the DaC approach.\n\nNumerical results shown on multiple datasets are pretty convincing of the superiority of DaC approach.\n\nWeaknesses\n\"Memory bank\" is invoked early on without sufficient explanation. It is explained in Section 4.2.2, but may make the paper more readable if this introduction could be provide earlier in the manuscript.\n\nNo error bars are provided for the numerical results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Cp36eEwy2HX", "reviewer": "Reviewer_rqR4", "summary": "This paper proposes a new source-free unsupervised domain adaptation method named DaC. The key idea is to leverage the advantages of existing “global” methods and “local” counterparts. Specifically, DaC uses the source model to split the target data into source-like and target-specific samples. After that, an adaptive contrastive learning strategy is used to achieve class-wise adaptation (global) and local consistency (local). Finally, MMD is used to minimize the distribution mismatch between source-like samples and target-specific samples. The proposed method achieves the best performance on widely used benchmarks.\n", "questions": "1. Line 262: change “trains” to “train”\n2. Any reason for missing the experiments on Digit datasets and the Office31 dataset?\n", "limitations": "Not applicable.\n", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "---\n\nOriginality: the proposed method is a combination of well-known techniques, but achieves good performance. Integrating contrastive learning into source-free domain adaptation is novel and brings new insights into this community.\n\n---\n\nQuality: \n\nStrengths: this work is technically sound with theoretical proof and empirical evaluation. The effectiveness of the proposed regularizations are evaluated on various downstream tasks. The work is complete.\n\nWeakness: (1) the major concern is the performance. As shown in Table 1 and Table 3, compared to existing SOTA methods (NRC and CPGA), the performance improvement is quite limited. Although it outperforms the baseline SHOT by a large margin, it is still hard to convince readers. (2) The proposed method is not evaluated on Digit datasets (MNIST, SVHN, and USPS) and the Office31 dataset which are two important benchmarks. Any reason behind this?\n\n---\n\nClarity: this paper is well written and well organized. It is easy to follow. Detailed implementation details are also provided in the supplementary.\n\n---\n\nSignificance: compared to the baseline SHOT, the result is important. Specifically, the proposed method brings a larger improvement than SHOT. However, it is not clear to me whether other methods can borrow the same idea and get performance improvement.\n\n---\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept