CLDA: Contrastive Learning for Semi-Supervised Domain Adaptation
Authors: Ankit Singh
Venue: NeurIPS 2021 Poster (2021)
Keywords: Semi Supervised Domain Adaption

Abstract: Unsupervised Domain Adaptation (UDA) aims to align the labeled source distribution with the unlabeled target distribution to obtain domain invariant predictive models. However, the application of well-known UDA approaches does not generalize well in Semi-Supervised Domain Adaptation (SSDA) scenarios where few labeled samples from the target domain are available.
This paper proposes a simple Contrastive Learning framework for semi-supervised Domain Adaptation (CLDA) that attempts to bridge the intra-domain gap between the labeled and unlabeled target distributions and the inter-domain gap between source and unlabeled target distribution in SSDA. We suggest employing class-wise contrastive learning to reduce the inter-domain gap and instance-level contrastive alignment between the original(input image) and strongly augmented unlabeled target images to minimize the intra-domain discrepancy. We have empirically shown that both of these modules complement each other to achieve superior performance. Experiments on three well-known domain adaptation benchmark datasets, namely DomainNet, Office-Home, and Office31, demonstrate the effectiveness of our approach. CLDA achieves state-of-the-art results on all the above datasets.

CLDA: Contrastive Learning for Semi-Supervised Domain Adaptation
Ankit Singh
Department of Computer Science Indian Institute of Technology, Madras singh ankit@cse, iitm..itm.... ac. in
Abstract
Unsupervised Domain Adaptation (UDA) aims to align the labeled source distri- bution with the unlabeled target distribution obtain domain invariant predictive models. However, the application of well-known UDA approaches does not gener- alize well in Semi-Supervised Domain Adaptation (SSDA) scenarios where few labeled samples from the target domain are available. This paper proposes a simple Contrastive Learning framework for cuiissuere Domain Adaptation (CLDA) that attempts to bridge the intra-domail gap between the labeled and unlabeled target distributions and the inter-domain gap between source and unlabeled target distribution in SSDA. We suggest employing class-wise contrastive learning to reduce the inter-domain gap and instance-level contrastive alignment between the (iiputal image) and strongly augmented unlabeled target images to mini- mize the intra-domain discrepancy. We have empirically shown that both of these modules complement each other to achieve superior performance. Experiments on three well-known domain adaptation benchmark datasets, namely DomainNet, Office-Home and Office31, demonstrate the effectiveness of our approach. CLDA achieves state-of of-the--rt results on all the above datasets.
Introduction
Deep Convolutional networks 30 52 have shown impressive performance in various computer
vision tasks, e.g., image classification 19 [2 and action recognition [48] 23 57 32 However, there is an inherent problem generalizability with deep-learning models, f.e., models trained on dataset(source domain) does perform well another domain. This loss of generalization is due to the presence domain shift [1I (55) across the dataset. Recent works (46] 29 have shown significantly boost the performance
that the presence of few labeled data from the target domain
of the convolutional neural network(CNN) based models. This observation led to the formulation of Semi-Supert Suni-Superviied Domain Adaption (SSDA), which is variant of Unsupervised Domain Adaptation where we have access to few labeled samples from the target domain.
Unsupervised domain adaptation methods label rich source domain to the unlabeled target domain. Many such existing domain adaptation approaches 42 (1] align the features of the source distribution with the target distribution without considering the category the samples. These aggostii methods fail to generate discriminative features when aligning global distributions. Recently, owing to the success of contrastive approaches
42
36
51 35 try to transfer knowledge from the
66 18 39), in self-representatioon learning, some recent works [26] 28 have turned to instance-basee
contrastive approaches to reduce discrepancies across domains.
446 reveals that the direct application of the well-known UDA approaches Semi- Suni-Supervised Domain Adaptation yields sub-optimal performance. (29] has shown that supervision from labeled
source and target samples can only ensure the partial cross domain feature alignment. This creates
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Labeled Source Samples
Labeled Target Samples
Source Class centroids
Unlabeled Target Samples
X
Target Class Centroids
Figure 1: Conceptual description of CLDA approach. (a) Intial distribution of samples from both domain (bb Instance Contrastive Alignment ensures unlaheled target samples move into the low entropy forming robust clusters (c) Inter-Domain Contrastive Alignment minimizes the distance between the clusters of same class from both domain (d) The clusters of both domain are well aligned and samples are far away from decision boundary.
aligned and unaligned sub- uub-distributions of the target domain, causing intra-domain discrepancy apart from inter-domain discrepancy in SSDA.
In this work, we propose CLDA, a simple single-stage novel contrastive learning framework to address the aforementioned problem. Our framework contains significant components to learn domain agnostic representation. First, Inter-Domain Contrastive Alignment reduces the discrepancy between centroids of the same class from the source and the target domain while increasing the distance between the class centroids of different classes from both source and target domain. This ensures clusters of the same class from both domains are near each other in latent space than the clusters of the other classes from both domains.
Second, inspired by the success of self-repee relfeeeprennnttiion learning in semi-supervised settings we propose to use Instance Contrastive A Alignment to reduce the intra-domain discrepancy. In this, we first generate the augmented views of the unlabeled target images using image augmentation methods. Alignment of the features of the original and augmented images of the unlabeled samples from the target domain ensures that they closer to each other in latent space. The alignment between two variants of the same image ensures that the classifier boundary lies in the low-density regions assuring that the feature representations of two variants of the unlabeled target images are similar, which helps to generate better clusters for the target domain.
In summary, our key contributions framework for supervised Domain Adaptation. 2eWe propose using alignment at class centroids and instance levels to reduce inter and intra domain discrepancies present in SSDA. 3)We evaluate the effectiveness of different augmentation approaches, for instance- three well-known Domain Adaptation datasets
follows. 1) We propose novel, simple single-stage training
contrastive alignment in
the SSDA setting. We evaluate our approach (DomainNet, Office-Home, and Office31) to gain insights. Our approach achieves the state of the art results across multiple datasets showing its effectiveness. We perform extensive ablation experiments highlighting the role of different components of our framework.
2 Related Works
2.1
Unsupervised Domain Adaptation
Unsupervised Domain Adaptation (UDA) [14) is well-studied problem, and most UDA algorithms
reduce the domain gap by matching the features the sources and target domain []6
36
51 27). Feature- alignment methods reduce the global divergence [16] 51 between source
41 based approaches have shown impressive performance in reducing the divergence between source and target domains. It involves training the model to generate features to deceive the domain classifier, invariantly making the generated features domain agnostic. Recently, Image translation methods [20] 21 38 have been explored in UDA where an image from the target domain is translated to the source domain to be
and target distribution. Adversarial learning [12 34 35 42
treated as image from the source domain to overcome the divergence present across domains.

---

Despite remarkable progress in UDA, [46] shows the UDA approaches do not perform well in the SSDA setting, which we consider in this work.
2.2 Semi-Supervised Learning
Semi-Supervised Learning(SSL) aims to leverage the vast amount of unlabeled data with limited labeled data to improve classifier performance. The main difference between SSL and SSDA is that SSL uses data sampled from the same distribution while SSDA deals with data sampled from two
domains with inherent domain discrepancy. The current line of work in SSL [50]
31 I0 follows consistency-basee approaches to reduce the intra- mntr--ommain gap. Mean teacher (53) uses two copies of
the same model (student model and teacher model) to ensure consistency across augmented views of the images. Weights of the teacher model are updated as the exponential moving average of the weights of the student model. Mix-Match (3) and ReMixMatch (2) use interpolation between labeled and unlabeled data to generate perturbed features. Recently introduced FixMatch [50] achieves impressive performance using the confident pscudo labels of the unlabeled samples and treating them as labels for the strongly perturbed samples. However, direct application of SSL in the SSDA setting yields sub-optimal performance as the presumption in the SSL is that distributions of labeled and unlabeled data are identical, which is not the case in SSDA.
2.3
Contrastive Learning
Contrastive Learning(CL.) has shown impressive performance in self-representation learning
1
18
54
39
Most contrastive learning methods align the representations of the positive pair (similar
images) be close to each other while making negative pairs apart. In semantic segmentation, (33
uses patch-wise contrastive learning reduce the domain divergence by aligning the similar patches across domains. In domain adaptation, contrastive learning [28] 26) has been applied for alignment at
the instance level to learn domain agnostic representations. 26 28 use samples from the same class as positive pairs, and samples from different classes are counted as negative pairs. (26) modifies Maximum Mean Discrepancy (MMD) [16) loss to be used as contrastive loss. In contrast to [28] 26). our work proposes to use contrastive learning in SSDA setting both at the class and instance level (across perturbed samples of the same image) to learn the semantic structure of the data better.
2.4 Semi-Supervised Domain Adaptation
Semi-Supervised Domain Adaptation (SSDA) aims to reduce the discrepancy between the source and target distribution in the presence of limited labeled target samples. (46) first proposed to align the
source and target distributions using adversarial training. (29) shows the presence of intra domain
discrepancy in the target distribution and introduces a framework to mitigate it. [25] uses consistency
alongside multiple adversarial strategies on top of MME 466] (9) introduced the meta-learning framework for Semi- Supervised Domain Adaptation. [58 breaks down the SSDA problem into
two subproblems, namely, SSL in the target domain and UDA problem across the source and target domains, and learn the optimal weights of the network using co-training- (37 proposed to
pretraining of the feature extractor and consistency across perturbed samples as a simple yet effective strategy for SSDA. (44) introduces a framework for SSDA consisting of shared feature extractor and two classifiers with opposite purposes, which are trained in an alternative fashion; where one classifier tries cluster the target samples while the other scatter the source samples, so that target features are well aligned with source domain features. Most of the above approaches are based on adversarial training, while our work proposes to use contrastive learning based feature alignment at the class level and the instance level to reduce discrepancy across domains.
3 Methodology
In this section, we present our novel Semi-Supervised Domain Adaptation approach to learn domain agnostic representation. We will first introduce the background and notations used in our work and then describe approach and its components in detail.
3

---

Source Samples
Intter-domain Contrastive Alignment
Weagts
Unlabeled Target Samples
staied Wegges
Instance Contrastive Alignment
Strong Augmented Unlabeled Target Samples
Figure 2: Outline of our CLDA Framework Our approach consists of aligning the outputs of the neural network at two levels. At the instance level, try maximize the similarity between features of unlabeled target images and strongly augmented unlabeled target images using Instance Contrastive Alignment. At the class level, we pass the images from both domains through the network, where we assign the labels to features of unlabeled target images and compute the centroids of each class of the target domain. Similarly, class labels. Finally, we maximize the similarity between centroids of the same class across domains by employing Inter-Domail Contrastive Alignment. We also used cross-entropy loss the labeled source and target images, apart from the above components in our framework.
compute the centroids for source domain features using their
3.1 Problem Formulation
In Semi-Supervised Domain Adaptation, have datasets sampled from two domains. The source dataset contains labeled images D, {( yf ) C Rd y sampled from some distribution (x, Y). Besides that, we have two sets of data sampled from target domain distribution Pr Y). We denote the labeled set of images sampled from the target domain as Die {x= v\" Tizi
The unlabeled set sampled from target domain D, {x contains large number of images (N, > Nit) without any corresponding labels associated with them. We also denote the labeled data from both domains Dj D, U Duz- Labels yf and yf of the samples from source and labeled target set correspond to one of the categories of the dataset having different classes/categories i.e Y = 21,2, K). Our goal is to learn task specific classifier using D., Di, and D, to accurately predict labels on test data from target domain.
3.2 Supervised Training
Labeled source and target samples are passed through the CNN-based feature extractor G(.) to obtain corresponding features, which are then passed through task-specific classifier F-)) to minimize the well-known cross-entropy loss on the labeled images from both source and target domains.
K
Csup = - "00") (y') log(F(G((z)))
(1)
k=1
3.3 Inter-Domain Contrastive Alignment
Our method is based on the observation that the samples from the same category across domains must cluster in the latent space. However, this is observed only for the source domain due to the availability the labels. Samples from the target domain do not align to form clusters due to domain shift between the target and the source distributions. This discrepancy between the cluster of the same category across domains is reduced by aligning the centroids of each class of source and target domain.
6 17 have shown that having separate projection space is beneficial for contrastive

---

training. Instead of using separate projection, we have used the outputs from the task-specific classifier as features align the clusters across the domain.
We represent the centroid of the images from the source domain belonging to class l as the mean of their features, which can be written
B
Σ (1z=x)
C
Σ (m)=
where B is the size of batch. We maintain memory bank (C* [Ct, Ck....Ckk) store the centroids of each class from source domain. We use exponential moving average to update these centroid values during the training
C = ===pCCk)stpp + (1 ()
where p is momentum term, and (Ct) and step-1 are the centroid values of class & at the current and previous step, respectively.
We also need to cluster the unlabeled target samples for Inter-Domain Contrastive Alignment. The pseudo labels obtained from the task specific classifier as shown in Eq (3) is used as the class labels for the corresponding unlabeled target samples.
J = argmar (FF()((((((()))))
(3)
Similar to the source domain, we also calculate the separate cluster centroid C for each of the class of the target samples present in the minibatch per the Eq (2) where unlabeled target images replace the images from the source domain with their corresponding pseudo label. The model then trained to maximize the similarity between the cluster representation of each class & from the source and the target domain. C₁ and C₁ form the positive pair while the remaining cluster centroids from both domains form the negative pairs. The remaining clusters from both domains pushed apart in the latent space. This is achieved through employing modified NT-Xent (normalized temperature-scaled cross-entropy) contrastive loss
39
49
33
for domain adaptation given by
M(C,.;))
Latu(C, Cf)
log
(4)
K
h hC,, Cf) + Σ (C,,
gE(..))
where h(u, v) = exp
measures the exponential of cosine similarity. 1 is an indicator
|
function and 7 is the temperature hyperparameter.
3.4 Instance Contrastive Alignment
Recent works contrastive learning 18 39
show encouraging results in single domain settings. [28] extends contrastive learning into mumliddomain settings. Inspired by such success, we employ Instance Contrastive Learning to form stable and correct cluster cores in the target domain.
To perform contrastive alignment the instance level, we first generate a strongly augmented version
of the unlabeled target image i. z
vh(x?) where vi(.) is the strong augmentation function [8] defined in Eq (5) ensure that these variants of the same image are closer to each other in the latent space while the rest of the images in minibatch
Next, employ the NT--Xent loss 6 39
of size B are pushed apart. This idea stems from the cluster assumption in an ideal classifier, which states the decision boundary should lie in the low-density region, ensuring consistent prediction for different augmented variants the same image.
Lins (ōf, 22)== log
(5)
Σ ((F(F(E))FF(CL)) + Σ (F
5

---

In SSDA, 29 has shown that target distribution gets divided into aligned and unaligned subdistri- bution in the presence of very few labeled target data. Thus, aligning the unaligned subdistribution can lead to improved performance, while perturbing the aligned sub-distribution result in negative transfer. Therefore, we only propagate the gradients for strongly augmented images to avoid perturbing the aligned sub-distribution in the target domain.
[6] shows stronger augmentation in contrastive learning leads to improved performance. Consistent prediction across the imput and strongly augmented unlabeled images in Instance Contrastive Align- ment forces the unaligned target sub-distribution to move away from the low-density region towards aligned distribution. This ensures better clustering in the unlabeled target distribution, which is validated by improved accuracy as shown in Table a employing Instance Contrastive Alignment with Inter- Domain Contrastive Alignment.
Both of the components of the CLDA framework are necessary for the improved performance, as shown in Table S Instance Contrastive Alignment ensures that unlabeled target samples are consistent and are the high-density region. However, it does not assure alignment between source and unlabeled target samples. DomainD Contrastive Alignment reduces the discrepancy between unlabeled target samples and source domain but unlabeled target samples closer to the decision boundary might get pushed towards the wrong classes resulting in negative transfer. Thus, combining both components results in a much better alignment of the unlabeled target samples towards the source domain, leading to improved performance of the framework.
3.5 Overall framework and training objective
The overall training objective employs supervised loss, Inter-Domain Contrastive Alignment and Instance Contrastive Alignment which can be formulated as follows:
Liot = Leup Letu * Cins
(6)
We train the model in our framework by employing overall training loss described as (6).
Experiments
4.1 Experimental Setup
We evaluate the effectiveness our approach on three different domain adaptation datasets: Domain- Net [43). Office-Home Home (56) and Office31 [45). DomainNet (43) is a large-scale domain adaptation dataset with 345 classes across 6 domains. Following MME [66. we use a subset of the dataset containing 126 categories across four domains: Real(R), Clipart Sketch(S), and Painting(P). The performance on DomainNet is evaluated using 7 different combinations out of possible 12 combina- tions. Office-HHor Home [56) is another widely used domain adaptation benchmark dataset with 65 classes across four domains: Ar), Product(Pr), Clipart(CI), Cl), and Real (RI). We perform experiments on all possible combinations of 4 domains. Office31 (45] is relatively smaller dataset containing just 31 categories of data across three domains- Amazon A), Dslr(D), Webcam(W) WW) Following prior work combinations for the office31 dataset.
46
29
evaluate approach
For the fair comparison, we use the data-splits (train, validation, and test splits) released by [46] on Github We use the same settings for the benchmark datasets in the prior work [46 29 including the number of labeled samples in the target domain, which are consistent across all experiments.
4.2 Implementation Details
Similar to the previous works on SSDA networks in our paper. We only used VGG for Offie311 due to its higher memory requirements. The feature generator model is initialized with ImageNet weights, and the classifier is randomly initialized and has the same architecture in [46] 29 9.. All our experiments are performed using Pytorch [40]. We use identical set of hyperparameters (a 4, 3 1 across all our experiments other
46
29
we use Resnet34 and Alexnet as the backbone

---

Ner
Method
RE-SCI
RI-1P -Pr
RI
>Ar
Pr. -R1
Pv -CI
Pr -AA
Ar-sPI
Ar -C1
Ar -RI
CI- :RI
CI Cl-sAr
CI -Pr
Mean
S-T
44.6
06.7
41.7
57.5
44.4
36.1
51.6
38.8
57.0
54.3
57.5
37.9
20.0
56.6
ADR
37.8
63.5
45.4
53.5
32.5
32.2
49.5
31.8
53.4
49.7
34.2
50.4
44.5
36.1
52.7
41.2
Alexnet
ENT
44,9
70.1
47.1
60.3
41.2
34.6
60.7
37.8
60.5
58.0
31,8
63.4
50,9
MME
51.2
73.0
50.3
61.6
47.2
40.7
63.9
43.8
61.4
59.9
44.7
64.7
55.2
Mcn-MME
50.3
44.5
BEAT
56.4
51.9
74.6
51.2
47.9
58.1
55.6
C1 (ours)
51.5
74.1
54.3
67.0
47.9
47.00
65.8
47.4
66.6
64.1
46.8
67.5
58.3
S+T
55.7
50.
67.8
73.1
53.8
74.2
68.3
57.6
66.2
DANN
57.3
75.5
65.2
69.2
51.8
$6.6
68.3
$1.7
73.8
67.1
55.1
67.5
63.5
79.9
79.1
71.9
Resnet34
MME
64.6
85.5
71.3
80.1
64.6
65.5
79.0
63.6
79.7
76.6
67.2
79.3
73.1
Mcta-MME
65.2
64.5
66.7
63.3
67.5
APE
66.4
86.
73.4
82.0
66.1
80.2
76.8
666.6
74.0
CLDDA (ours)
66.0
87.6
76.7
X2.2
63.9
72.4
81.4
63.4
81.3
80.1
70.5
80.9
75.5
Table I: Performance Comparison in Office-Home. Numbers show top-1 accuracy values for different domain adaptation scenarios under 3-shot setting using Alexnet and Resnet34 as backbone networks. We have highlighted the best method for each transfer task. CLDD surpasses all the baseline methods in most adaptation scenarios. Our Proposed framework achieves the best average performance among all compared methods.
than minibatch size. All the hyperparameters values are decided using validation performance on Product to Art experiments on the Office-Home dataset. We have set T = our experiments. Each minibatch of size B contains equal number of source and labeled target examples, while the number of unlabeled target samples is μ. x B. We study the effect of µ in section 4.5 Resnet34 experiments
performed with minibatch size, B = 32 and Alexnet models trained with B 24. We use μ = 4 for all our experiments. We use SGD optimizer with a momentum of 0.9 and an initial learning rate of 0.01 with cosine learning rate decay for all experiments. Weight decay is set to 0.0005 for all our models. Other details of the experiments are included in the supplementary.
4.3 Baselines
We compare our CLDA framework with previous state-of-the-art SSDA approaches MME (46).
APE [29]. BiAT [25] UODA [44) Meta-MME [9] and ENT [15] using the performance reported by these papers. papers. We also included the results from adversarial based baseline
methods: DANN [13], ADR (47) and CDAN [35] as reported in [46). We also provide the S+T
results where the model is trained using all the labeled samples across domains.
4.4 Results
Table ssow top-1 accuracies and mean accuracies for different combination of domain adaptation scenarios for all three datasets in comparison with baseline SSDA methods.
Office-Home. Table contains the results of the Office-Home dataset for 3-shot setting with Alexnet and Resnet34 as backbone networks. Results for the 1-shot adaptation scenarios are included in the supplementary. Our method consistently performs better than baseline approaches and achieves 58.3% and 75.5% mean accuracy with Alexnet and Resnet34, respectively. Our approach surpasses the state-of-the-art SSDA approaches in most the adaptation tasks. In some domain adaptation cases, such as CI RI, RI - Ar and Pr + Ar, exceeded APE by more than 3%.
NemainNet: Our CLDA approach surpasses the performance of existing SSDA baselines as shown in Table 2 Using Alexnet backbone, our method improves over BiAT by 5.2% and 4.9% in 1-shot and 3- shot settings, respectively. We obtain similarly improved performance when we switch the neural backbone from Alexnet to Resnet34. With Resnet34 as the backbone, we gain 4.3% and 3.6% over APE 1-shot and 3-shot settings, respectively. Similar to the Office-Home, our approach surpasses the well-known domain adaptation benchmarks methods in most domain adaptation tasks of the DomainNet dataset. Such consistent improved performance shows that our approach reduces both inter and intra domain discrepancy prevalent in SSDA.

---

Net
Method
I-shot
R-4C
3-shot
I-shet
R-PP
3-sbot
I-chre
P-SC
3-shot
I-chot
C-45
3-shot
I-sbet
S-IP
3-dhot
I-shet
R-4S
3-shot
I-shot
P-oR
3-shot
1-shot
Moan
3-shot
S-T
49.3
45.0
38.4
DANN
43.3
46.1
41.6
43.5
39.1
41.3
35.9
56.5
36.9
38.9
52.5
33.4
53.5
57.3
40.4
42.4
44.4
32.8
38.9
55.9
42.7
CDAN
46.3
46.8
45.7
45.0
38.3
42.3
27.5
29.5
20.2
35.7
28.8
31.3
56.7
58.7
29.1
41.0
Alcont
ENT
$7.0
45.5
35.6
42.6
26.8
40.4
18.9
51.1
15.1
29.6
18.0
29.6
52.2
60.0
29.1
39.8
49.0
Mcn-MME
36.4
50.2
51.9
39.6
43.7
38.7
60.7
48.8
BLAT
54.2
38.6
50.6
37.7
42.1
56.9
49.4
47.7
34.6
50.5
38.5
42.3
57.5
48.9
CLDA (ours)
56.3
59.0
56.0
57.2
50.8
54.6
42.5
47.3
46.8
$1.4
36.0
42.7
64.4
67.00
50.7
$4.3
S-T
55.6
60.0
60.5
62.2
36.8
59.4
50.5
55.0
$6.0
59.5
46.5
50.1
71.8
73.9
56.9
600
DANN
58.2
598
61.4
62.5
$6.
99.5
52.8
55.4
57.4
59.9
51.2
$4.9
70.3
77.2
58.4
60.7
ADR
57.1
61.9
51.0
59.9
72.0
60.4
CDAN
65.0
69.0
64.9
67.3
633.7
68.4
55.1
57.8
63.4
65.3
54.5
$9.0
75.2
78.5
60.5
66.5
Resnet34
ENT
68.2
71.0
69.2
65.4
54.6
60.0
59.7
62.1
61.1
75.0
67.6
MME
70.0
72.2
67.7
69.7
69.0
71.7
56.3
61.8
64.8
65.8
61.0
61.9
76.1
78.5
66.4
68.9
LODA
72.7
15.4
70.5
71.5
69.8
73.2
60.5
64,1
66.4
69.4
62.7
64.2
77.3
90.5
68.5
71.2
Mcl-MME
70.1
BLAT
73,0
74.9
68.0
68.5
71.6
74.5
57.9
61.5
63.9
67.5
55.5
62.1
77.0
78.6
67.1
69.7
APE
70.4
76.
70.8
72.1
72.9
76.7
56.7
63.1
64.5
65.1
63.0
67.8
76.6
79.4
67.6
71.7
Table 2: Performance Comparison in DomainNet. Numbers show Top-1 accuracy values for different domain adaptation scenarios under 1-shot and 3-shot settings using Alexnet and Resnet34 as backbone networks. CLDA achieves better performance than all the baseline methods most of the domain adaptation tasks. We have highlighted the best approach for each domain adaptation task. Our Proposed framework achieves the best average performance among all compared methods.
Alexnel
VGG
W >A
D-A
Mean
W-A
D D->A
Mean
Method
1-shot
3-shot
1-shot
3-shot
1-shot
3-shot
I-shot
3-shot
1-shot
3-shot
1-shot
3-shot
S+T
50.4
61.2
50.0
62,4
50.2
61.8
169.2
73.2
68.2
73.3
68.7
73.25
DANN
57.0
64.4
54.5
65.2
64.8
75.4
70.4
74.6
69.85
75.0
ADR
50.2
61.2
50.9
61.4
50.6
61.3
69.7
73.3
69.2
74.1
69.45
73.7
CDAN
50.4
60.3
48.5
61.4
49.5
60.8
65.9
74.4
64.4
71.4
65.15
72.9
ENT
50.7
64.0
50.0
66.2
50.4
65.1
69.1
75.4
72.1
75.1
70.6
75.25
MME
57.2
67.3
55.8
67.8
56.5
67.6
73.1
76.3
73.6
77.6
73.35
76.95
BiAT
57.9
68.2
54.6
68.55
56.3
68.4
APE
67.6
69.0
68.3
CLDA
64.6
70.5
62.7
72.5
63.6
71.5
76.2
78.6
75.1
76.7
75.6
77.6
Table 3: Performance Comparison in Office31. Numbers show Top-1 accuracy values for different domain adaptation scenarios under 1-shot and 3-shot settings using Alexnet and VGG as backbone networks. CI.DA outperforms all the baseline approaches in both scenarios. We have highlighted the superior method on each domain adaptation task. Our Proposed framework achieves the best mean accuracy among all baseline methods.
Office31: Similar to other datasets, our proposed method with Alexnet and VGG as neural backbone achieves the best performance in both domain adaption scenarios for office31 as shown in Table 3
Using Alexnet backbone, we beat the APE (29 by 3.2% in 3-shot and BiAT by 7.3% in 1-shot settings. We observe similar gains over all the baselines methods with VGG as the neural network backbone. This shows the efficacy of our proposed approach irrespective of the used backbone.
4.5 Ablation Studies
We perform extensive ablation experiments to analyze our CLDA framework and the effects of the different components and hyperparameters. We perform these experiments the 3-shot Pr Ar domain adaptation task the Office-Home dataset using Resnet34 unless specified otherwise.
Our CLDA framework is composed of two modules: Inter- Domain Contrastive Alignment and Instance Contrastive Alignment. We investigate the significance of each component of our framework by dropping the other during training. We observe that the test
Effectiveness of Individual Modules:
8

---

Augmentation
Test Accuracy( Pr + Ar)
Test Accuracy(RI - Ar)
Horizontal Flipping (Hflii)
68.1
73.4
Hîlip Color Jitter
67.6
74.9
Hflip+ Color Jitter + Grayscale
70.2
76.2
Rand Augment (RA)
18
71.1
74.6
RA Grayscale
72.4
76.7
Auto Augment
7
69.9
75.3
Table 4: Effect of Strong Augmentations Numbers show the test accuracy on 3-shot domain adaptation tasks of the Office- Home dataset with Resnet34 with different augmentation policies.
Asusaay Woigh Inulanoo Contasaion Abgeount
Accunncy
trtes Consin Contardre Algerrett
Accusecy vo
Valisaion Aceursoy
Acousiy
Wiliatiion Accunicy
Accinaty
valdea Accusccy
Text
75.0
20.0
65.0
Wo boaans Cortosabe AlquiTEnE
Weight
Rublo Uniidaded Labe CLDA of Lurpet Chenar
Figure 3: Effect of different hyperparameters on 3-shot Pr + Ar (Product to Art) data adap- tation scenario the Office-Home using Resnet34. (a) Effect of varying the weight of Instance Contrastive Alignment on validation and test Accuracy (b) Effect of varying weight of Inter-Domain Contrastive Alignment on validation and test Accuracy (c) Effect of μ., ratio of unlabeled target to labeled target data on validation and test accuracy.
accuracy drops from 72.4% to 68.3% when only Inter-Domain Contrastive Alignment is used, and it drops to 67.7% when Instance Contrastive Alignment is used alone as shown T)) Though individual modules do not yield high performance on their own but once combined, they surpass their individual performance by a margin of around 4%.
Effect of Different Hyperparameters: We analyze the importance of different hyperparameters used in our approach. We observe that the weight Instance Contrastive Alignment affects the performance of our approach as the test accuracy drops from 72.4% to 70.7% when we set a to 1 instead of its optimal value of as shown figure 3 We also notice that increasing led to a reduction of the validation and test performance. We also look into the effect of ju, which is the ratio of unlabeled to labeled data miníbatch. We observe that an increasing value of j increases the performance till j 4, after which it starts to drop, as shown in figure
Importance of Instance Contrastive Alignment: Instance Contrastive Alignment ensures similar representation across different variants of the unlabeled target images. This consistency is also ensured by other well-known SSL approaches like FixMatch [50). We perform an ablation experiment replacing Instance Contrastive Alignment with FixMatch. We also compare with LI and L2 loss to have fair analysis. As shown Table 5 (b) Instance Contrastive Alignment helps achieve superior performance in comparison with other consistency-base approaches.
Approach
Test Accuracy
Approach
Test Accuracy
Fix-Mater
70.8
CLDA w/o Instance Contrastive
68.3
Ll loss
69.4
CLDA w/o Inter-Domain Contrastive
67.7
L2 loss
69.3
CLDA (ours)
72.4
CLDA (ours)
72.4
(a) Ablation Study on the effectiveness of Individual components of the CLDA framework on Pr - Ar adaptation task of the Office- Home dataset using Resnet34.
(b) Ablation Study on other consis- tency based approaches on Pr -> Ar domain adaptation task of the Office- Home using Resnet34.
Table 5: Experiments to understand the significance of individual components of our framework.

---

Experiments
samples mislaheled
samples mislaheled 12%)
samples mislabeled (~~ 25%)
66.2
66.0
65.7
72.6
72.05
71.56
Table 6: Ablation study to understand the effect of outliers in target domain. Numbers show the test accuracy of 1-shot domain adaptation tasks of the Office-Hooee dataset with Resnet34.
Contrastive Alignment requires pseudo labels for the unlabeled target data for clustering. In this ablation experiment, we replace our approach of using the model's prediction as pseudo label with K-means clustering, which invoke after every 50 steps and use the generated centroids for the next 50 steps to obtain pseudo-class labels for unlabeled target data. We observe a drop in performance (from 72.4% 71.2%) when using K-means to obtain the pseudo label for unlabeled target images.
Effect of Other Clustering Techniques:
We look into different augmentation policies for the Instance Contrastive Alignment. As suggested in [6]. a stronger augmentation policy for contrastive learning increases the performance of the model. We find that RandAugment [8] with Grayscale augmentation
Effect of Augmentation Policy:
policy gives better results over other augmentation policies. The influence of the strong augmentation can be observed from ~ 4% improvement in the performance when the augmentation policy is switched from horizontal flipping to RandAugment with Grayscale. Table 4 contains the test accuracy of different augmentation policies 3-shot Pr Ar and RI - Pr domain adaption tasks of the Pffice-Homm dataset with Resnet34.
Effect of Noisy-Labeled Target Samples: In SSDA, we have few labeled samples from the target domain; however, the presence of labsellabeled target samples can have an adverse effect on the performance. To understand the effect of noisy-labeled target samples on the framework, we conducted experiments on the 1-shot Pr + and - Ar domain adaptation scenarios of Office-Home dataset with Resnet34, where mislabeled some previously labeled target samples as shown in Table (6] We observe a small decrease in performance of our framework ( ffrom 66. 2% to 65.7% for Pr - Ar and from 72.6% to71.56% for RI + Ar) when mislabeled target samples increase from 0% to ~ 25% in both domain adaptation scenarios showing the robustness of our framework.
Conclusion
In this work, we present a novel single-stage contrastive learning framework for semi-supervised domain adaptation. The framework consists of Inter-Domain Contrastive Alignment and Instance- Contrastive Alignment, where the former maximizes the similarity between centroids of the same class from both domains and later maximizes the similarity between augmented views of the unlabeled target images. We show that both of the components of the framework are necessary for improved performance. We demonstrate the effectiveness of our approach on three standard domain adaptation benchmark datasets, outperforming the well-known SSDA methods.
Acknowledgments and Disclosure of Funding
The work is supported by Half Time Research Assistantship (HTRA) grants from the Ministry of Education, India. We would also like to thank Saurav Chakraborty and Athira Nambiar for their valuable suggestions and feedback to improve the work.
References
[1] Philip Bachman, R Devon Hjelm, and William Buchwalter. "Learning Representations by Maximizing Mutual Information Across Views". In: Neural Information Processing Systems.
2019.
[2] David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. "Rthh: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring". In: International Conference on Learning Representations. 2020.
10

---

[3] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. "MixMatch: Holistic Approach to Semi-Supervised Learning". In: Neural Information Processing Systems. 2019.
[4] Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. "Domain Separation Networks". In: Neural Information Processing Systems. 2016.
[5] Chao Chen, Zhihong Chen, Boyuan Jiang, and Xinyu Jin. "Joint Domain Aligmment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation", In: AAAI Conference on Artificial Intelligence. Vol. 33, 01. 2019, pp. 3296-3303.
[6] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. "A Simple Framework for Contrastive Learning of Visual Representations" In: International Conference Machine Learning. PMLR. 2020, pp. 1597-1607.
[7] Ekin Dogus Cubuk, Barret Zoph, Dandelion Mané, Vijay Vasudevan, and Quoc V. Le. "Au- toAugment: Learning Augmentation Strategies From Data". In: IEEE Conference on Computer Vision and Pattern Recognition (2019), pp. 113-123.
[8] Ekin Dogus Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. "Randaugment Practical automated data augmentation with a reduced search space". In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2020), pp. 3008-3017.
[9]
Timothy Hospedales Da Li. "Online Meta-L for Multi-Source and Semi-Supervised Domain Adaptation".. In: European Conference on Computer Vision. 2020.
[10] Zihang Dai, Zhilin Yang, Fan Yang, William W. Cohen, and Ruslan Salakhutdinov. "Good Semi supervise Learning That Requires a Bad GAN". In: Neural Information Processing Systems 10s//705.10 00773099333..00 (2017).
[11] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. "DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition".
In: International Conference on Machine Learning. PMLR. 2014.
[12] Yaroslav Ganin and Victor Lempitsky. "Unsupervised Domain Adaptation by Backpropaga- tion". In: International Conference on Machine Learning. PMLR. 2015, pp. 1180-1189
[13] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. "Domain- Adversarial Training of Neural Networks". In: The Journal of Machine Learning Reseurch 17.1 (2016), pp. 2096-2030.
[14] Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. "Domain adaptation for object recognition: An unsupervised approach". In: IEEE International Conference on Computer Vision (2011), pp. 999-1006.
[15] Yves Grandvalet and Yoshua Bengio. "Semi-supervised learning by entropy minimization".
In: Neural Information Processing Systems. 2004, pp. 529-536.
[16] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. "A Kernel Two-Sample Test". In: The Journal of Machine Learning Research 13.1 (2012), pp. 723-773.
[17] Jean- -ant-Batienn Grill, Florian Strub, Florent Altch's e, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. "Bootstrap Your Own Latent: A New Approach to Self- Self-Suuer Leamnin".. In: ArXiv abs/2006.07733. (2020).
[18] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. "Momentum Contrast for Unsupervised Visual Representation Learning". In: IEEE Conference Computer Vision and Pattern Recognition (2020), pp. 9726-9735.
[19] Kaiming Heand, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep Residual Learning for Image Recognition". In: IEEE Conference on Computer Vision and Pattern Recognition. 2016, pp. 770-778.
[20] Judy Hoffman, Eric Tzeng, Taesung Park, Lun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, and Trevor Darrell. "CyCADA: Cycle-Consistent Adversarial Domain Adaptation", In: International Conference on Machine Leurning. PMLR. 2018.
[21] Lanqing Hu, Meina Kan, Shiguang Shan, and Xilin Chen. "Duplex Generative Adversarial Network for Unsupervised Domain Adaptation". In: IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 1498-1507.

---

[22] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. "Densely Connected Convolutional Networks". In: IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 2261-2269.
[23] Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu. "3D Convolutional Neural Networks for Human Action Recognition". In: IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (2013), pp. 221-231.
[24] Jian Shen and Yanru Qu and Weinan Zhang and Yong Yu. "Wasserstein Distance Guided Rep- resentation Learning for Domain Adaptation". In: AAAI Conference on Artificial Intelligence. 2018.
[25] Pin Jiang, Aming Wu, Yahong Han, Yunfeng Shao, Meiyu Qi. and Bingshuai Li. "Bidirec- tional Adversarial Training for Supervised Domain Adaptation" In: International Joint Conferences on Artificial Intelligence. 2020.
[26] Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. "Contrastive Adaptation Network for Unsupervised Domain Adaptation", In: IEEE Conference on Computer Vision and Pattern Recognition (2019), pp. 4888-4897,
[27] Guoliang Kang. Liang Zheng, Yan Yan, and Yi Yang. "Deep Adversarial Attention Alignment for Unsupervised Domain Adaptation: the Benefit of Target Expectation Maximization". In: European Conference Computer Vision. 2018.
(28) Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh, Bryan A. Plummer, Stan Sclaroff, and Kate Saenko. "Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels". In: IEEE International Conference on Computer Vision abs/2003.08264 (2021).
[29] Tae- fae-Kyung Kim and Changick Kim. "Attract, Perturb, and Explore: Learning Feature Alignment Network for Semi-supervised Domain Adaptation". In: European Conference on Computer Vision. 2020.
[30] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks". In: Neural Information Processing Systems 25 (2012),
pp. 1097-1105.
[31] Chongxuan Li, Kun Xu, Zhu Jun, and Zhang Bo. "Triple Generative Adversarial Nets". In:
Neural Information Processing Systems 703.02299 (2017).
[32] Ji Lin, Chuang Gan, and Song Han. "TSM: Temporal Shift Module for Efficient Video Understanding" In: IEEE International Conference Computer Vision (2019), pp. 7082- 7092.
[33] Weizhe Liu, David Ferstl, Samuel Schulter, Lukas Zebedin, Pascal Fua, and Christian Leistner. "Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learning". In: arXiv preprint arXiv:2104. I056 (2021).
[34] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. "Learning Transferable Features with Deep Adaptation Networks". In: International Conference on Machine Learning. PMLR. 2015, pp. 97-105.
[35] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. "Conditional Adver- sarial Domain Adaptation". In: Neural Information Processing Systems. 2018.
[36] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. "Unsupervised Domain Adaptation with Residual Transfer Networks". In: Neural Information Processing Systems. 2016.
[37] Samarth Mishra, Kate Saenko, and Venkatesh Saligrama. "Surprisingly Simple Semi- Supervised Domain Adaptation with Pretraining and Consistency". In: ArXiv abs/2101.12727 (2021).
[38] Zak Murez, Soheil Kolouri, David Kriegman, Ravi Ramamoorthi, and Kyungnam Kim. "Image to Image Translation for Domain Adaptation". In: IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 4500-4509.
[39] Aaron van den Oord, Yazhe Li, and Oriol Vinyals, "Representation Learning with Contrastive Predictive Coding". In: ArXiv /a0///07.07774 (2018).
[40] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. "PyTorch: An Imperative Style, High- Higf-Peeformance Deep Learning Library". In: Neural Information Processing Systems. 2019.
12

---

[41] Sujoy Paul. Yi-Hsuan Tsai, Samuel Schulter, Amit K Roy-Choor -hyoddhuy, and Manmohan Chandraker. "Domain adaptive semantic segmentation using weak labels". In: European Conference on Computer Vision. 2020, pp. 571-587.
[42] Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. IMMuli-Adversarri Domain Adaptation". In: AAAI Conference on Artificial Intelligence. 2018.
[43] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. "Mo- ment Matching for Multi-Source Domain Adaptation", In: IEEE International Conference on Computer Vision. 2019, pp. 1406-1415.
[44] Can Qin, Lichen Wang, Qianqian Ma, Yu Yin, Huan Wang, and Yun Fu, "Contradictory Structure Learning for Semi-supervised Domain Adaptation". In: Proceedings of the 2021
SIAM International Conference on Data Mining (SDM). SIAM. 2021, pp. 576-584.
[45] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. "Adapting Visual Category Models to New Domains". In: European Conference on Computer Vision. 2010.
[46] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. "Semi- Supervised Domain Adaptation via Minimax Entropy". In: IEEE International Conference on Computer Vision (2019), pp. 8049-8057.
[47] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. "Adversarial Dropout Regularization". In: International Conference on Learning Representations. 2018.
[48] Karen Simonyan and Andrew Zisserman. "Two-Stream Convolutional Networks for Action Recognition in Videos". In: Neural Information Processing Systems. 2014.
[49] Ankit Singh, Omprakash Chakraborty, Ashutosh Varshney, Rameswar panda, Rogerio Feris, Kate Saenko, and Abir Das. "Semi- Seemi-Superiied Action Recognition With Temporal Contrastive Learning" " In: IEEE Conference on Computer Vision and Pattern Recognition. June 2021, pp. 10389-10399.
[50] Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. "FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence". In: Neural Information Processing Systems (2020).
[51] Baochen Sun and Kate Saenko. "Deep CORAL: Correlation Alignment for Deep Domain Adaptation". In: European Conference on Computer Vision Workshops. 2016.
[52] Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander Amir Alemi, "Inception- v4, inception-Reseet and the Impact of Residual Connections Learning". In: AAAI Confer- ence on Artificial Intelligence. 2017.
[53] Antti Tarvainen and Harri Valpola. "Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results". In: Neural Information P Processing Systems. 2017.
[54] Yonglong Tian, Dilip Krishnan, and Phillip Isola. "Contrastive Multiview Coding". In: Euro- pean Conference on Computer Vision. 2020.
[55] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. "Adversarial Discriminative Domain Adaptation". In: IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 2962-2971.
[56] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. "Deep Hashing Network for Unsupervised Domain Adaptation". In: IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 5385-5394.
[57] Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. "Temporal Segment Networks for Action Recognition in Videos". In: IEEE Transactions on Pattern Analysis and Machine Intelligence 41 (2019), pp. 2740-2755.
[58] Luyu Yang. Yan Wang, Mingfei Gao, Abhinav Shrivastava, Kilian Q. Weinberger, Wei-Lun Chao, and Ser-Nam Lim. "Deep Co-Training with Task Decomposition for Semi-Supervised Domain Adaptation". In: IEEE International Conference on Computer Vision. 2021.
13


Reviews:
[{"review_id": "MREJxbZ3IZ", "reviewer": "Reviewer_bD47", "summary": "This paper proposes a contrastive learning method for semi-supervised domain adaptation (CLDA), where few labelled samples from target domain are available. The CLDA consists in inter-domain contrastive alignment, which decreases distance between same classes from source and target domains, and instance contrastive alignment based on strongly augmented unlabeled target images to reduce intra-domain discrepancy. The experiments are reported on three commonly used domain adaptation datasets. \n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n\n1. The proposed method is simple and makes sense. It is interesting to see that the contrastive learning is effectively applied to semi-supervised domain adaptation problem. \n\n2. The ablation studies are thorough and performance on three benchmarks are good. \n\nWeaknesses\n\n1. The inter-domain contrast alignment depends on the pseudo labels of the target samples. It is well known that some of the pseudo labels are inaccurate, which may be hurtful for the centroid estimation of target categories. Some recent studies [34,33, R1] have investigated different ways for mining high-confident pseudo labels and I am wondering how the proposed method will improve by considering this situation.  \n\n2. The instance contrastive alignment only makes use of target samples. It will  also be interesting to consider the contrastive loss for the source samples, particularly as a regularizing term for the supervised, cross-entropy loss. \n\n3. Some recent works on semi-supervised domain adaptation are missing, e.g., [R1-R3]. It is necessary for the authors to cite these advances and compare with them. In particular, connection with and difference from R3 should be discussed and clarified.\n\n \n[R1] Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation. CVPR, 2021.\n[R2] Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation. CVPR, 2021.\n[R3] Prototypical Cross-domain Self-supervised Learning for Few-shot\nUnsupervised Domain Adaptation. CVPR, 2021.\n\n----------------------------------------------------------------\nAfter rebuttal\nThe authors' responses basically addressed my concerns and I have also read the comments of other reviewers. I keep my initial rating and recommend acceptance of this paper. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitation of the proposed method regarding inaccuracy of pweduo labels is not favorably addressed. See Main Review for my suggestion. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "9KGaI2XHvFD", "reviewer": "Reviewer_aZTZ", "summary": "This work presents a contrastive learning framework for Semi-Supervised Domain Adaptation. A basic assumption of few labeled target samples is made according to the Semi-Supervised setting. The key idea is to apply an instance-level and inter-domain contrastive alignment, using labeled source samples, few labeled target samples, and pseudo-labeled target samples. For instance-level contrastive loss, strong augmentations are used as positive pairs. For inter-domain alignment, cluster centroids are aligned across the two domains. The proposed method shows consistent improvement over multiple datasets.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n=========\n\na. The approach is simple. It is interesting to see that the two contrastive learning losses collaboratively improve the performance.\n\nb. The results look promising.\n\nc. The manuscript is easy to read\n\n\nWeaknesses / Scope of improvement\n==============================\n\na. Comparison against prior arts: \n\n* The proposed contrastive losses are inspired by prior works [24, 26], however no comparison is made against these methods in the experiments (or did I miss it?).\n\n* Since the proposed method uses strong augmentation, which can be seen as additional data, it is expected that the performance would improve. However, the results reported for the baselines are taken from their papers (L210). Considering that the baseline papers use weak augmentations (e.g. MME [42] uses horizontal flipping and random cropping), in my opinion, this comparison is unfair. The baselines should be evaluated using strong augmentations as shown in Table 4 (e.g., consider RA + Grayscale augmentation for competing methods).\n\n\nb. Quality of pseudo-label / few-shot samples: According to Eq. 4, all the target samples are assigned a pseudo-label which is the model’s predicted output. This would also contain misclassified target samples. However, there is no discussion on how to mitigate the noise in pseudo labels. Moreover, there is no discussion on the performance when the one-shot labeled target samples are outliers.\n\nc. L283 (“Instance Contrastive Learning improves pseudo-label accuracy”): It is unclear whether this is truly the case, since the individual modules don’t yield high performance on their own (L242). Moreover, there is no discussion regarding how the combination of the two losses gives significant improvement. It would be interesting to study why this is the case.\n\nd. Supplementary L11-12 (“results in the main paper are achieved through one-time training”): Could the authors clarify if multi-run statistics are reported in the main paper? For instance, it seems that the result presented in Table 1 of the paper (Office-Home - Alexnet - CLDA - Mean = 57.4%), is around 1% less than that mentioned in the Supplementary Table 4 (Mean = 58.31%). Moreover, this discrepancy suggests that the results do not truly indicate the average-case performance of this method. The authors are suggested to present multi-run statistics for all experiments, since the performance depends upon the chosen one-shot / few-shot samples. \n\n\nAdditional clarifications\n===================\n\na. Algorithm 1 (Supplementary): The source centroids are updated using the exponential average as shown in Eq. 3. However, the target centroids are not updated likewise. Could the authors clarify the intuition behind this choice?\n\nb. How is the validation set constructed (Fig. 3) and used? Does this require labeled target samples beyond the few-shot labeled samples used for training? Is the validation set required for every experiment?\n\nc. It is a bit surprising that such a simple method is able to outperform other methods by a significant margin. Could the authors confirm if the test set and the few-shot labeled samples are identical across all the compared methods?\n\nd. L207 (“Complete details of all the experiments are included in the Appendix.”): Unfortunately, I could not find any relevant detail about the experiments in the Supplementary. Please update the supplementary to include details about the aforementioned points.\n\n\nMinor comments\n===============\n\na. Page 9: “Figure 4” -> “Table 4”\n\nb. L53: “Summary” -> “summary”\n\nc. Please maintain consistency in references (e.g. [1] is cited as NeurIPS while [4] cited as NIPS; [32] is published in ICML 2015, but cited as arXiv etc.).\n\nd. A convergence plot could be useful in determining the stability of the training.\n\n\nConsidering both the strengths and weaknesses, I would suggest a revision at this stage. I would be happy to increase the score if my major concerns are addressed (see Weaknesses and Additional Clarifications).\n\n--------\n\nPost Rebuttal Comments: My major concerns were addressed in the authors' responses. Therefore, I am increasing the score.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Adequately addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "97BYQpamXGQ", "reviewer": "Reviewer_MSYR", "summary": "The paper is about semi-supervised domain adaptation.  The inputs are labeled source samples, as well as labeled and unlabeled target domain samples.  The task is to train a model that performs well on the target domain.  The authors propose a framework with two main components to learn a domain-agnostic representation.\n\n- Inter-domain contrastive alignment: This module reduces the discrepancy between classes centroids of the same class from source and target domain, while increasing the distance between class centroids of different classes from both source and target domains. Class centroids are computed as an exponential moving average via equations (2) and (3).  A contrastive loss is used to pull and push the centroids.\n- Instance contrastive alignment: The goal of this module is to pull the classifier boundary into a low-density region, which is done by regularizing the classifier on the unlabeled target data.  Image augmentation is used to create perturbed images of unlabeled target data and a contrastive loss is employed to pull the representation of the same instance (perturbed and unperturbed) together while pushing it away from other samples.\n\nExperiments on Office-Home, DomainNet and Office31 show promising results.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Clarity\n\n- It seems from lines 141ff that class centers for unlabeled target samples are computed via pseudo labels, which are then pushed closer to the class centers of the labeled data. But in the paragraph around Equation 4, I did not find anything about the labeled target images?! Shouldn't those also contribute to the class center and pushed towards the source domain class centers?\n\n- Line 251ff: How is the instance contrastive alignment actually different to FixMatch?\n\n- In Section 3.3, how do you back-propagate through the exponential moving average of the class centers. Doesn't this create a computational graph that spans over all previous training iterations?\n\n\n### Experiments\n\n- In line 193, the experimental setup is defined as to use the same **number** of labeled samples in the target domain. But are the exact same samples used as in prior work for the labeled target set or just the same number? I would assume the final performance (strongly) depends on **which** samples are chose; not all subsets of size N from the target domain dataset are equal!\n\n- In Section 4.3, did you also compare with a sort of upper bound where all source and target data is labeled?\n\n- Line 217: Does \"3-shot setting\" mean that 3 samples are labeled? I'm only asking because this nomenclature is often used for few-shot learning but not for semi-supervised learning.\n\n\n### References\n\n- There is a related work that I think could be discussed as related (concurrent) work [A].\n\n- I think references to some relevant work on class-aware domain adaptation are missing, like [B,C]. These works use a class-specific discriminator in an adversarial domain adaptation scheme, where class information for unlabeled target data is also obtained via pseudo labels.\n\n- In line 133, I think there could be a reference to \"using the output space for adaptation\": [D], although this paper does use an adversarial approach for domain adaptation.\n\n- I think the sentence in lines 162ff (\"This idea ...\") should get a reference.\n\n#### References\n- [A] Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learning. Liu et al. arXiv'21\n- [B] Ssf- dan: Separated semantic feature based domain adaptation network for semantic segmentation. Du et al. ICCV'19\n- [C] Domain Adaptive Semantic Segmentation Using Weak Labels. Paul et al. ECCV'20\n- [D] Learning to Adapt Structured Output Space for Semantic Segmentation, Tsai et al, CVPR'18\n\n\n\n### Minor comments and typos\n\n- Type in caption Figure 1: \"... samples from both domains. (b) ...\"\n- Line 45: success ... success ?\n- Line 53: \"In summary, ...\" & \"... we propose ...\"\n- Line 90: \"[24,26] use ...\"\n- Line 97: \"... of limited labeled target ...\"\n- Line 118: The notation would be more consistent if the labeled source data is denoted as $\\mathcal{D}_{ls}$\n- Line 124: Remove the dot after $\\mathcal{F}(\\cdot)$\n- Line 132: \"[6,16] have ...\"\n- Line 159: \"... is the strong augmentation function.\" More details are required at this point, e.g, a definition, a citation or a reference to the implementation details of the paper or something.\n- Line 179: Use \"eqref\" to refer to equations.\n- Line 183: \"... Office-home [52] and Office31 [41].\"\n- Line 210: \"... Maeta-MME [30] and ENT [14] ...\"\n- Line 240: \"... accuracy drops from 72.4\\% ...\"\n- Line 245: \"... contrastive loss affects the ...\"\n\n### Post-rebuttal update\n\nAfter having read all other reviews and the authors' responses, I have decided to keep my initial rating and recommend accepting the submission. I appreciate the authors' effort in providing all additional explanations and experiments.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- Limitations are okay, albeit being rather generic.\n- Is there a possibility that the domain transfer/alignment method introduces additional biases, e.g., because the target data (although mostly unlabeled) is biased?", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "8NGO2rzbmFN", "reviewer": "Reviewer_nEH7", "summary": "The authors propose a method for semi-supervised domain adaptation. In their methodology, contrastive prototypical alignment is used for the source and target examples (pseudo labels are computed for unlabeled target examples) and contrastive loss is used for the unlabeled examples from the target domain. The hope is that by using the contrastive loss on the unlabeled examples, the target examples would form clusters and the classification boundaries would lie on the low density data regions. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n-------------\n- The paper is very well written. Easy to read and follow. \n- Everything including datasets and implementation details are well described\n- Experiments on multiple datasets and  ablation studies are also well conducted.\n- They achieve improved performance in most experiments over the baselines. \n\nWeakness:\n--------------\n- The novelty is very limited. The method proposed in 3.2 Inter domain contrastive alignment is very close to other protortypical alignment methods in unsupervised and semi-supervised domain adaptation. \n- Its is not well understood how contrastive learning on unlabeled examples help. The authors say it clusters the target examples better and point to improved overall performance as evidence for it. We know that clusters formed by self-supervised learning methods do not necessarily correspond to classes of interest.  I do believe in this case, a theoretical justification and an empirical evaluation of the type of formed clusters is needed. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are discussed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: 
 Overall, the reviewers found the proposed method simple but interesting, the paper well-written, and the experiments/ablations thorough. The reviewers raised a number of concerns including comparisons to related works (and concurrent work) and fairness of comparisons [aZTZ, bD47, MSYR], pseudo-labeling noise [aZTZ, bD47], as well as smaller clarifications and analysis. The authors provided a thorough rebuttal with significant additional experiments, including even comparisons to the concurrent CVPR works. The reviewers have been overall satisfied with the responses and some have decided to increase their scores. 

As a result, I recommend acceptance and strongly encourage the authors to incorporate the many experimental results and clarifications into the revised paper, including discussions on the experimental setup and noise resilience. 

Decision: Accept (Poster)