CLDA: Contrastive Learning for Semi-Supervised Domain Adaptation
Year: 2021

Abstract:
Unsupervised Domain Adaptation (UDA) aims to align the labeled source distribution with the unlabeled target distribution to obtain domain invariant predictive models. However, the application of well-known UDA approaches does not generalize well in Semi-Supervised Domain Adaptation (SSDA) scenarios where few labeled samples from the target domain are available. This paper proposes a simple Contrastive Learning framework for semi-supervised Domain Adaptation (CLDA) that attempts to bridge the intra-domain gap between the labeled and unlabeled target distributions and the inter-domain gap between source and unlabeled target distribution in SSDA. We suggest employing class-wise contrastive learning to reduce the inter-domain gap and instance-level contrastive alignment between the original(input image) and strongly augmented unlabeled target images to minimize the intra-domain discrepancy. We have empirically shown that both of these modules complement each other to achieve superior performance. Experiments on three well-known domain adaptation benchmark datasets, namely DomainNet, Office-Home, and Office31, demonstrate the effectiveness of our approach. CLDA achieves state-of-the-art results on all the above datasets.

heading: 1 Introduction
text: Deep Convolutional networks [30, 52] have shown impressive performance in various computer vision tasks, e.g., image classification [19, 22] and action recognition [48, 23, 57, 32]. However, there is an inherent problem of generalizability with deep-learning models, i.e., models trained on one dataset(source domain) does not perform well on another domain. This loss of generalization is due to the presence of domain shift [11, 55] across the dataset. Recent works [46, 29] have shown that the presence of few labeled data from the target domain can significantly boost the performance of the convolutional neural network(CNN) based models. This observation led to the formulation of Semi-Supervised Domain Adaption (SSDA), which is a variant of Unsupervised Domain Adaptation where we have access to a few labeled samples from the target domain.
Unsupervised domain adaptation methods [42, 12, 36, 51, 35] try to transfer knowledge from the label rich source domain to the unlabeled target domain. Many such existing domain adaptation approaches [42, 12, 51] align the features of the source distribution with the target distribution without considering the category of the samples. These class-agnostic methods fail to generate discriminative features when aligning global distributions. Recently, owing to the success of contrastive approaches [6, 18, 39], in self-representation learning, some recent works [26, 28] have turned to instance-based contrastive approaches to reduce discrepancies across domains.
[46] reveals that the direct application of the well-known UDA approaches in Semi-Supervised Domain Adaptation yields sub-optimal performance. [29] has shown that supervision from labeled source and target samples can only ensure the partial cross-domain feature alignment. This creates
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
aligned and unaligned sub-distributions of the target domain, causing intra-domain discrepancy apart from inter-domain discrepancy in SSDA.
In this work, we propose CLDA, a simple single-stage novel contrastive learning framework to address the aforementioned problem. Our framework contains two significant components to learn domain agnostic representation. First, Inter-Domain Contrastive Alignment reduces the discrepancy between centroids of the same class from the source and the target domain while increasing the distance between the class centroids of different classes from both source and target domain. This ensures clusters of the same class from both domains are near each other in latent space than the clusters of the other classes from both domains.
Second, inspired by the success of self-representation learning in semi-supervised settings [17, 6, 49], we propose to use Instance Contrastive Alignment to reduce the intra-domain discrepancy. In this, we first generate the augmented views of the unlabeled target images using image augmentation methods. Alignment of the features of the original and augmented images of the unlabeled samples from the target domain ensures that they are closer to each other in latent space. The alignment between two variants of the same image ensures that the classifier boundary lies in the low-density regions assuring that the feature representations of two variants of the unlabeled target images are similar, which helps to generate better clusters for the target domain.
In summary, our key contributions are as follows. 1) We propose a novel, simple single-stage training framework for Semi-supervised Domain Adaptation. 2)We propose using alignment at class centroids and instance levels to reduce inter and intra domain discrepancies present in SSDA. 3)We evaluate the effectiveness of different augmentation approaches, for instance-based contrastive alignment in the SSDA setting. 4)We evaluate our approach over three well-known Domain Adaptation datasets (DomainNet, Office-Home, and Office31) to gain insights. Our approach achieves the state of the art results across multiple datasets showing its effectiveness. We perform extensive ablation experiments highlighting the role of different components of our framework.

heading: 2 Related Works

heading: 2.1 Unsupervised Domain Adaptation
text: Unsupervised Domain Adaptation (UDA) [14] is a well-studied problem, and most UDA algorithms reduce the domain gap by matching the features of the sources and target domain [16, 4, 24, 36, 51, 27]. Feature-based alignment methods reduce the global divergence [16, 51] between source and target distribution. Adversarial learning [12, 5, 34, 35, 42, 41] based approaches have shown impressive performance in reducing the divergence between source and target domains. It involves training the model to generate features to deceive the domain classifier, invariantly making the generated features domain agnostic. Recently, Image translation methods [20, 21, 38] have been explored in UDA where an image from the target domain is translated to the source domain to be treated as an image from the source domain to overcome the divergence present across domains.
Despite remarkable progress in UDA, [46] shows the UDA approaches do not perform well in the SSDA setting, which we consider in this work.

heading: 2.2 Semi-Supervised Learning
text: Semi-Supervised Learning(SSL) aims to leverage the vast amount of unlabeled data with limited labeled data to improve classifier performance. The main difference between SSL and SSDA is that SSL uses data sampled from the same distribution while SSDA deals with data sampled from two domains with inherent domain discrepancy. The current line of work in SSL [50, 3, 31, 10] follows consistency-based approaches to reduce the intra-domain gap. Mean teacher [53] uses two copies of the same model (student model and teacher model) to ensure consistency across augmented views of the images. Weights of the teacher model are updated as the exponential moving average of the weights of the student model. Mix-Match [3] and ReMixMatch [2] use interpolation between labeled and unlabeled data to generate perturbed features. Recently introduced FixMatch [50] achieves impressive performance using the confident pseudo labels of the unlabeled samples and treating them as labels for the strongly perturbed samples. However, direct application of SSL in the SSDA setting yields sub-optimal performance as the presumption in the SSL is that distributions of labeled and unlabeled data are identical, which is not the case in SSDA.

heading: 2.3 Contrastive Learning
text: Contrastive Learning(CL) has shown impressive performance in self-representation learning [6, 1, 18, 54, 39]. Most contrastive learning methods align the representations of the positive pair (similar images) to be close to each other while making negative pairs apart. In semantic segmentation, [33] uses patch-wise contrastive learning to reduce the domain divergence by aligning the similar patches across domains. In domain adaptation, contrastive learning [28, 26] has been applied for alignment at the instance level to learn domain agnostic representations. [26, 28] use samples from the same class as positive pairs, and samples from different classes are counted as negative pairs. [26] modifies Maximum Mean Discrepancy (MMD) [16] loss to be used as a contrastive loss. In contrast to [28, 26], our work proposes to use contrastive learning in SSDA setting both at the class and instance level (across perturbed samples of the same image) to learn the semantic structure of the data better.

heading: 2.4 Semi-Supervised Domain Adaptation
text: Semi-Supervised Domain Adaptation (SSDA) aims to reduce the discrepancy between the source and target distribution in the presence of limited labeled target samples. [46] first proposed to align the source and target distributions using adversarial training. [29] shows the presence of intra domain discrepancy in the target distribution and introduces a framework to mitigate it. [25] uses consistency alongside multiple adversarial strategies on top of MME [46]. [9] introduced the meta-learning framework for Semi-Supervised Domain Adaptation. [58] breaks down the SSDA problem into two subproblems, namely, SSL in the target domain and UDA problem across the source and target domains, and learn the optimal weights of the network using co-training. [37] proposed to use pretraining of the feature extractor and consistency across perturbed samples as a simple yet effective strategy for SSDA. [44] introduces a framework for SSDA consisting of a shared feature extractor and two classifiers with opposite purposes, which are trained in an alternative fashion; where one classifier tries to cluster the target samples while the other scatter the source samples, so that target features are well aligned with source domain features. Most of the above approaches are based on adversarial training, while our work proposes to use contrastive learning-based feature alignment at the class level and the instance level to reduce discrepancy across domains.

heading: 3 Methodology
text: In this section, we present our novel Semi-Supervised Domain Adaptation approach to learn domain agnostic representation. We will first introduce the background and notations used in our work and then describe our approach and its components in detail.

heading: 3.1 Problem Formulation
text: In Semi-Supervised Domain Adaptation, we have datasets sampled from two domains. The source dataset contains labeled images Ds = {(xsi , ysi )} Ns i=1 ⊂ Rd × Y sampled from some distribution PS(X,Y ). Besides that, we have two sets of data sampled from target domain distribution PT (X,Y ). We denote the labeled set of images sampled from the target domain as Dlt = {(xlti , ylti )} Nlt i=1 . The unlabeled set sampled from target domain Dt = {(xti)} Nt i=1 contains large number of images (Nt Nlt) without any corresponding labels associated with them. We also denote the labeled data from both domains as Dl = Ds ∪ Dlt. Labels ysi and ylti of the samples from source and labeled target set correspond to one of the categories of the dataset having K different classes/categories i.e Y = {1, 2, ...K}. Our goal is to learn a task specific classifier using Ds, Dlt and Dt to accurately predict labels on test data from target domain.

heading: 3.2 Supervised Training
text: Labeled source and target samples are passed through the CNN-based feature extractor G(.) to obtain corresponding features, which are then passed through task-specific classifier F(.) to minimize the well-known cross-entropy loss on the labeled images from both source and target domains.
Lsup = − K∑
k=1
(yi)k log(F(G((xil))k (1)

heading: 3.3 Inter-Domain Contrastive Alignment
text: Our method is based on the observation that the samples from the same category across domains must cluster in the latent space. However, this is observed only for the source domain due to the availability of the labels. Samples from the target domain do not align to form clusters due to the domain shift between the target and the source distributions. This discrepancy between the cluster of the same category across domains is reduced by aligning the centroids of each class of source and target domain. [6, 17] have shown that having a separate projection space is beneficial for contrastive
training. Instead of using a separate projection, we have used the outputs from the task-specific classifier as features to align the clusters across the domain.
We represent the centroid of the images from the source domain belonging to class k as the mean of their features, which can be written as
Csk =
i=B∑ i=1 1{ysi=k}F(G(x s i ))
i=B∑ i=1 1{ysi=k}
(2)
where B is the size of batch. We maintain a memory bank ( Cs = [Cs1 , C s 2 , ....C s K ] )
to store the centroids of each class from source domain. We use exponential moving average to update these centroid values during the training
Csk = ρ(C s k)step + (1− ρ)(Csk)step−1
where ρ is a momentum term, and (Csk)step and (C s k)step−1 are the centroid values of class k at the current and previous step, respectively.
We also need to cluster the unlabeled target samples for Inter-Domain Contrastive Alignment. The pseudo labels obtained from the task specific classifier as shown in Eq (3) is used as the class labels for the corresponding unlabeled target samples.
ŷti = argmax((F(G(x t i))) (3)
Similar to the source domain , we also calculate the separate cluster centroid Ctk for each of the class k of the target samples present in the minibatch as per the Eq (2) where unlabeled target images replace the images from the source domain with their corresponding pseudo label. The model is then trained to maximize the similarity between the cluster representation of each class k from the source and the target domain. Csk and C t k form the positive pair while the remaining cluster centroids from both domains form the negative pairs. The remaining clusters from both domains are pushed apart in the latent space. This is achieved through employing a modified NT-Xent (normalized temperature-scaled cross-entropy) contrastive loss [6, 39, 49, 33] for domain adaptation given by
Lclu(Cti , Csi ) = − log h ( Cti , C s i ) h ( Cti , C s i ) + K∑ r=1
q∈{s,t}
1{r 6=i}h ( Cti , C q r ) (4) where h(u,v) = exp ( u>v ||u||2||v||2 /τ ) measures the exponential of cosine similarity , 1 is an indicator
function and τ is the temperature hyperparameter.

heading: 3.4 Instance Contrastive Alignment
text: Recent works on contrastive learning [18, 39, 6] show encouraging results in single domain settings. [28] extends contrastive learning into multi-domain settings. Inspired by such success, we employ Instance Contrastive Learning to form stable and correct cluster cores in the target domain.
To perform contrastive alignment at the instance level, we first generate a strongly augmented version of the unlabeled target image i.e x̃ti = ψ(x t i) where ψ(.) is the strong augmentation function [8]. Next, we employ the NT-Xent loss [6, 39] as defined in Eq (5) to ensure that these two variants of the same image are closer to each other in the latent space while the rest of the images in minibatch of size B are pushed apart. This idea stems from the cluster assumption in an ideal classifier, which states the decision boundary should lie in the low-density region, ensuring consistent prediction for different augmented variants of the same image.
Lins(x̃ti, xti)=−log h ( F(G(x̃ti),F(G(xti)) ) B∑
r=1
h ( F(G(x̃ti)),F(G(xtr)) ) + B∑ r=1 1{r 6=i}h ( F(G(x̃ti)),F(G(x̃tr)) ) (5)
In SSDA, [29] has shown that target distribution gets divided into aligned and unaligned subdistribution in the presence of very few labeled target data. Thus, aligning the unaligned subdistribution can lead to improved performance, while perturbing the aligned sub-distribution can result in a negative transfer. Therefore, we only propagate the gradients for strongly augmented images to avoid perturbing the aligned sub-distribution in the target domain.
[6] shows stronger augmentation in contrastive learning leads to improved performance. Consistent prediction across the input and strongly augmented unlabeled images in Instance Contrastive Alignment forces the unaligned target sub-distribution to move away from the low-density region towards aligned distribution. This ensures better clustering in the unlabeled target distribution, which is validated by improved accuracy as shown in Table 5 after employing Instance Contrastive Alignment with Inter-Domain Contrastive Alignment.
Both of the components of the CLDA framework are necessary for the improved performance, as shown in Table 5 . Instance Contrastive Alignment ensures that unlabeled target samples are consistent and are in the high-density region. However, it does not assure alignment between source and unlabeled target samples. Inter-Domain Contrastive Alignment reduces the discrepancy between unlabeled target samples and source domain but unlabeled target samples closer to the decision boundary might get pushed towards the wrong classes resulting in negative transfer. Thus, combining both components results in a much better alignment of the unlabeled target samples towards the source domain, leading to improved performance of the framework.

heading: 3.5 Overall framework and training objective
text: The overall training objective employs supervised loss, Inter-Domain Contrastive Alignment and Instance Contrastive Alignment which can be formulated as follows:
Ltot = Lsup + α ∗ Lclu + β ∗ Lins (6)
We train the model in our framework by employing overall training loss described as in (6).

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: We evaluate the effectiveness of our approach on three different domain adaptation datasets: DomainNet [43], Office-Home [56] and Office31 [45]. DomainNet [43] is a large-scale domain adaptation dataset with 345 classes across 6 domains. Following MME [46], we use a subset of the dataset containing 126 categories across four domains: Real(R), Clipart(C), Sketch(S), and Painting(P). The performance on DomainNet is evaluated using 7 different combinations out of possible 12 combinations. Office-Home [56] is another widely used domain adaptation benchmark dataset with 65 classes across four domains: Art(Ar), Product(Pr), Clipart(Cl), and Real (Rl). We perform experiments on all possible combinations of 4 domains. Office31 [45] is a relatively smaller dataset containing just 31 categories of data across three domains- Amazon(A), Dslr(D), Webcam(W). Following prior work [46, 29], we evaluate our approach on two combinations for the office31 dataset.
For the fair comparison, we use the data-splits (train, validation, and test splits) released by [46] on Github 1. We use the same settings for the benchmark datasets as in the prior work [46, 29], including the number of labeled samples in the target domain, which are consistent across all experiments.

heading: 4.2 Implementation Details
text: Similar to the previous works on SSDA [46, 29, 9], we use Resnet34 and Alexnet as the backbone networks in our paper. We only used VGG for Office31 due to its higher memory requirements. The feature generator model is initialized with ImageNet weights, and the classifier is randomly initialized and has the same architecture as in [46, 29, 9]. All our experiments are performed using Pytorch [40].We use an identical set of hyperparameters (α = 4, β = 1 ) across all our experiments other
1https://github.com/VisionLearningGroup/SSDA_MME
than minibatch size. All the hyperparameters values are decided using validation performance on Product to Art experiments on the Office-Home dataset. We have set τ = 5 in our experiments. Each minibatch of size B contains an equal number of source and labeled target examples, while the number of unlabeled target samples is µ×B. We study the effect of µ in section 4.5. Resnet34 experiments are performed with minibatch size, B = 32 and Alexnet models are trained withB = 24. We use µ = 4 for all our experiments. We use SGD optimizer with a momentum of 0.9 and an initial learning rate of 0.01 with cosine learning rate decay for all our experiments. Weight decay is set to 0.0005 for all our models. Other details of the experiments are included in the supplementary.

heading: 4.3 Baselines
text: We compare our CLDA framework with previous state-of-the-art SSDA approaches : MME [46], APE [29], BiAT [25] , UODA [44], Meta-MME [9] and ENT [15] using the performance reported by these papers. papers. We also included the results from adversarial based baseline methods: DANN [13], ADR [47] and CDAN [35] as reported in [46]. We also provide the S+T results where the model is trained using all the labeled samples across domains.

heading: 4.4 Results
text: Table 1- 3 show top-1 accuracies and mean accuracies for different combination of domain adaptation scenarios for all three datasets in comparison with baseline SSDA methods.
Office-Home. Table 1 contains the results of the Office-Home dataset for 3-shot setting with Alexnet and Resnet34 as backbone networks. Results for the 1-shot adaptation scenarios are included in the supplementary. Our method consistently performs better than the baseline approaches and achieves 58.3% and 75.5% mean accuracy with Alexnet and Resnet34, respectively. Our approach surpasses the state-of-the-art SSDA approaches in most of the adaptation tasks. In some domain adaptation cases, such as Cl→ Rl, Rl→ Ar and Pr→ Ar, we exceeded APE by more than 3%. DomainNet: Our CLDA approach surpasses the performance of existing SSDA baselines as shown in Table 2. Using Alexnet backbone, our method improves over BiAT by 5.2% and 4.9% in 1-shot and 3-shot settings, respectively. We obtain similarly improved performance when we switch the neural backbone from Alexnet to Resnet34. With Resnet34 as the backbone, we gain 4.3% and 3.6% over APE in 1-shot and 3-shot settings, respectively. Similar to the Office-Home, our approach surpasses the well-known domain adaptation benchmarks methods in most domain adaptation tasks of the DomainNet dataset. Such consistent improved performance shows that our approach reduces both inter and intra domain discrepancy prevalent in SSDA.
Office31: Similar to other datasets, our proposed method with Alexnet and VGG as neural backbone achieves the best performance in both domain adaption scenarios for office31 as shown in Table 3. Using Alexnet backbone, we beat the APE [29] by 3.2% in 3-shot and BiAT by 7.3% in 1-shot settings. We observe similar gains over all the baselines methods with VGG as the neural network backbone. This shows the efficacy of our proposed approach irrespective of the used backbone.

heading: 4.5 Ablation Studies
text: We perform extensive ablation experiments to analyze our CLDA framework and the effects of the different components and hyperparameters. We perform these experiments on the 3-shot Pr→ Ar domain adaptation task of the Office-Home dataset using Resnet34 unless specified otherwise.
Effectiveness of Individual Modules: Our CLDA framework is composed of two modules: InterDomain Contrastive Alignment and Instance Contrastive Alignment. We investigate the significance of each component of our framework by dropping the other during training. We observe that the test
accuracy drops from 72.4% to 68.3% when only Inter-Domain Contrastive Alignment is used, and it drops to 67.7% when Instance Contrastive Alignment is used alone as shown in Table 5(a). Though individual modules do not yield high performance on their own but once combined, they surpass their individual performance by a margin of around 4%.
Effect of Different Hyperparameters: We analyze the importance of different hyperparameters used in our approach. We observe that the weight of Instance Contrastive Alignment affects the performance of our approach as the test accuracy drops from 72.4% to 70.7% when we set α to 1 instead of its optimal value of 4 as shown in figure 3. We also notice that increasing β led to a reduction of the validation and test performance. We also look into the effect of µ, which is the ratio of unlabeled to labeled data in a minibatch. We observe that an increasing value of µ increases the performance till µ = 4, after which it starts to drop, as shown in figure 3.
Importance of Instance Contrastive Alignment: Instance Contrastive Alignment ensures similar representation across different variants of the unlabeled target images. This consistency is also ensured by other well-known SSL approaches like FixMatch [50]. We perform an ablation experiment replacing Instance Contrastive Alignment with FixMatch. We also compare with L1 and L2 loss to have a fair analysis. As shown in Table 5 (b) Instance Contrastive Alignment helps to achieve superior performance in comparison with other consistency-based approaches.
Effect of Other Clustering Techniques: Inter-Domain Contrastive Alignment requires pseudo labels for the unlabeled target data for clustering. In this ablation experiment, we replace our approach of using the model’s prediction as a pseudo label with K-means clustering, which we invoke after every 50 steps and use the generated centroids for the next 50 steps to obtain pseudo-class labels for unlabeled target data. We observe a drop in performance (from 72.4% to 71.2%) when using K-means to obtain the pseudo label for unlabeled target images.
Effect of Augmentation Policy: We look into different augmentation policies for the Instance Contrastive Alignment. As suggested in [6], a stronger augmentation policy for contrastive learning increases the performance of the model. We find that RandAugment [8] with Grayscale augmentation policy gives better results over other augmentation policies. The influence of the strong augmentation can be observed from ∼ 4% improvement in the performance when the augmentation policy is switched from horizontal flipping to RandAugment with Grayscale. Table 4 contains the test accuracy of different augmentation policies on 3-shot Pr→ Ar and Rl→ Pr domain adaption tasks of the Office-Home dataset with Resnet34.
Effect of Noisy-Labeled Target Samples: In SSDA, we have few labeled samples from the target domain; however, the presence of noisy-labeled target samples can have an adverse effect on the performance. To understand the effect of noisy-labeled target samples on the framework, we conducted experiments on the 1-shot Pr → Ar and Rl → Ar domain adaptation scenarios of the Office-Home dataset with Resnet34, where we mislabeled some previously labeled target samples as shown in Table 6. We observe a small decrease in performance of our framework ( from 66.2% to 65.7% for Pr→ Ar and from 72.6% to71.56% for Rl→ Ar) when mislabeled target samples increase from 0% to ∼ 25% in both domain adaptation scenarios showing the robustness of our framework.

heading: 5 Conclusion
text: In this work, we present a novel single-stage contrastive learning framework for semi-supervised domain adaptation. The framework consists of Inter-Domain Contrastive Alignment and InstanceContrastive Alignment, where the former maximizes the similarity between centroids of the same class from both domains and later maximizes the similarity between augmented views of the unlabeled target images. We show that both of the components of the framework are necessary for improved performance. We demonstrate the effectiveness of our approach on three standard domain adaptation benchmark datasets, outperforming the well-known SSDA methods.

heading: 6 Acknowledgments and Disclosure of Funding
text: The work is supported by Half-Time Research Assistantship (HTRA) grants from the Ministry of Education, India. We would also like to thank Saurav Chakraborty and Athira Nambiar for their valuable suggestions and feedback to improve the work.

Review summaries:
This paper proposes a contrastive learning method for semi-supervised domain adaptation (CLDA), where few labelled samples from target domain are available. The CLDA consists in inter-domain contrastive alignment, which decreases distance between same classes from source and target domains, and instance contrastive alignment based on strongly augmented unlabeled target images to reduce intra-domain discrepancy. The experiments are reported on three commonly used domain adaptation datasets.   

The paper is about semi-supervised domain adaptation.  The inputs are labeled source samples, as well as labeled and unlabeled target domain samples.  The task is to train a model that performs well on the target domain.  The authors propose a framework with two main components to learn a domain-agnostic representation.  - Inter-domain contrastive alignment: This module reduces the discrepancy between classes centroids of the same class from source and target domain, while increasing the distance between class centroids of different classes from both source and target domains. Class centroids are computed as an exponential moving average via equations (2) and (3).  A contrastive loss is used to pull and push the centroids. - Instance contrastive alignment: The goal of this module is to pull the classifier boundary into a low-density region, which is done by regularizing the classifier on the unlabeled target data.  Image augmentation is used to create perturbed images of unlabeled target data and a contrastive loss is employed to pull the representation of the same instance (perturbed and unperturbed) together while pushing it away from other samples.  Experiments on Office-Home, DomainNet and Office31 show promising results.

This work presents a contrastive learning framework for Semi-Supervised Domain Adaptation. A basic assumption of few labeled target samples is made according to the Semi-Supervised setting. The key idea is to apply an instance-level and inter-domain contrastive alignment, using labeled source samples, few labeled target samples, and pseudo-labeled target samples. For instance-level contrastive loss, strong augmentations are used as positive pairs. For inter-domain alignment, cluster centroids are aligned across the two domains. The proposed method shows consistent improvement over multiple datasets.

The authors propose a method for semi-supervised domain adaptation. In their methodology, contrastive prototypical alignment is used for the source and target examples (pseudo labels are computed for unlabeled target examples) and contrastive loss is used for the unlabeled examples from the target domain. The hope is that by using the contrastive loss on the unlabeled examples, the target examples would form clusters and the classification boundaries would lie on the low density data regions.