Venue: NAACL 2022

# Domain Confused Contrastive Learning for Unsupervised Domain Adaptation

Quanyu Long, Tianze Luo, Wenya Wang and Sinno Jialin Pan

Nanyang Technological University, Singapore {quanyu001, tianze001, wangwy, sinnopan}@ntu.edu.sg

###### Abstract

In this work, we study Unsupervised Domain Adaptation (UDA) in a challenging self-supervised approach. One of the difficulties is how to learn task discrimination in the absence of target labels. Unlike previous literature which directly aligns cross-domain distributions or leverages reverse gradient, we propose Domain Confused Contrastive Learning (DCCL) to bridge the source and the target domains via domain puzzles, and retain discriminative representations after adaptation. Technically, DCCL searches for a most domain-challenging direction and exquisively crts domain confused augmentations as positive pairs, then it contrastively encourages the model to pull representations towards the other domain, thus learning more stable and effective domain invariances. We also investigate whether contrastive learning necessarily helps with UDA when performing other data augmentations. Extensive experiments demonstrate that DCCL significantly outperforms baselines.

+
Footnote †: publication: pubid: July 10-15, 2022 ©2022 Association for Computational Linguistics

## 1 Introduction

Pre-trained language models Devlin et al. (2019); Liu et al. (2019); Yang et al. (2019) have yielded considerable improvements with datasets drawing from various sources. However, the lack of portability of language model to adapt to a new textual domain remains a central issue Gururangan et al. (2020), especially when the training set and testing set do not follow the same underlying distribution (changing of topic and genres) - usually referred to as domain shift. In this paper, we focus on studying Unsupervised Domain Adaptation (UDA). UDA aims at designing adaptation algorithms that attempt to generalize well on the target domain by learning from both labeled samples from the source domain and unlabeled samples from the target domain. Studying UDA fits real-world scenarios since labeled data in the target domain is usually absent. Moreover, advances in UDA will also help out-of-distribution generalizations Ramponi and Plank (2020); Krueger et al. (2021).

Extensive algorithms have been proposed to mitigate the domain shift problem, for example, domain adversarial neural network (DANN) Ganin et al. (2016) and distribution matching Zhuang et al. (2015). For DANN, the training process of joint optimization is unstable, requiring extensive effort to tune the hyperparameters Shah et al. (2018); Du et al. (2020); Karouzos et al. (2021). As for distribution matching, it is very difficult to preserve the discriminative power of the model on the target task while trying to perform instance level alignment Saito et al. (2017); Lee et al. (2019). To this end, it is essential to develop stable and effective solutions to learn domain invariance and instance-wise matching for UDA.

Recent advances in self-supervised learning (SSL), such as contrastive learning (CL), have been proven effective at instance level by leveraging raw data to define surrogate tasks that help learn repre

Figure 1: Domain puzzles that are domain-confused and overlook domain-related information, could be regarded as lying in an intermediate domain that aims to pull source and target samples closer to each other and bridge the two domains by learning domain invariant representations.



sentations (Chen et al., 2020; Khosla et al., 2020; He et al., 2020; Chen et al., 2020). CL benefits from treating instances as classes and conducting data augmentations to generate positive instance pairs. Regarding CL for UDA, previous works mark cross-domain images with the same labels (for example, real and cartoon dogs) as the positive pairs in contrastive loss (Wang et al., 2021; Park et al., 2020). However, such methods are not applicable to NLP tasks because of the massive semantic and syntactic shifts between two cross-domain sentences. Besides, from the domain adaptation perspective, constructing cross-domain positive samples and aligning domain-agnostic pairs have received less emphasis in related literature, since previous works focus on designing label preserving text transformations, such as back-translation, synonym, dropout and their combinations (Qu et al., 2021; Gao et al., 2021).

Confronting with limitations mentioned above, we propose the concept of domain puzzles which discard domain-related information to confuse the model, making it difficult to differentiate which domain these puzzles belong to. Instead of directly seeking matched sentences across the source and target domains which is infeasible, we propose to pull the source (target) data and its corresponding domain puzzles closer to reduce the domain discrepancy, as shown in Fig. 1. A simple idea to craft domain puzzles is to mask domain-specific tokens. However, token-level operations are too discrete and non-flexible to reflect the complex semantic change of natural languages. Hence, we aim to seek better domain puzzles that retain high-confidence predictions and task-discriminative power in the representation space for each training instance.

In this paper, we propose Domain Confused Contrastive Learning (DCCL) to encourage the model to learn similar representations for the original sentence and its curated domain-confused version with contrastive loss. More specifically, we synthesize these domain puzzles by utilizing adversarial examples Zhu et al. (2020); Jiang et al. (2020). The algorithm will search for an extreme direction that roughly points to the opposite domain and produces most domain-challenging puzzles. We encourage the model to encode original and domain-confused samples closer, gradually pulling examples to the domain decision boundary as training progresses via CL, thus learning the domain invariance. Furthermore, in order to investigate whether CL necessarily benefits UDA, we conduct experiments and find that constructing domain puzzles as paired positive samples is favorable for UDA, however, other data augmentation methods such as back translation (Sennrich et al., 2016; Edunov et al., 2018) do not have the same effect. The experiment results show that the proposed DCCL significantly outperforms all the baselines. We also conduct quantitative experiments to measure the domain discrepancy after adaptation demonstrating that DCCL can decrease the divergence between domains in a self-supervised way. Overall, the paper makes the following contributions:

* First, a new concept of domain puzzles is put forward. We propose to craft the domain puzzles via domain-confused adversarial attack;
* Second, we propose DCCL, which is able to pull source and target samples closer to the crafted domain puzzles. The DCCL is capable of reducing domain shift and learning domain invariance;
* Third, experiments demonstrate that the proposed DCCL surpasses baselines with a large margin. We also conduct analyzing experiments to verify the effectiveness.

## 2 Preliminaries

### Unsupervised Domain Adaptation

**Problem Setup** Suppose we have access to a source dataset with \(n\) labeled data points \({\cal D}^{S}=\{x_{i},y_{i}\}_{1,...,n}\) sampled i.i.d. from the source domain, and a target dataset with \(m\) unlabeled points \({\cal D}^{T}=\{x_{j}\}_{1,...,m}\) sampled i.i.d. from the target domain, where \(x_{i},x_{j}\) are sequences of tokens, \(y_{i}\) is the class label for \(x_{i}\). For in-domain training with labeled training instances, the model aims to learn a function \(f(x;\theta_{f},\theta_{g}):x\to C\). \(\theta_{f}\) is the parameter of the deep neural network encoder (e.g., pretrained language model), \(\theta_{y}\) denotes parameters that compute the network's class label predictions, and \(C\) is the label set. The model is learned with the following objective:

\[\min_{\theta_{f},\theta_{y}}\sum\nolimits_{(x,y)\sim{\cal D}^{S}}[{\cal L}(f( x;\theta_{f},\theta_{y}),y)]. \tag{1}\]

However, for Unsupervised Domain Adaptation (UDA), the goal of the adaptation algorithm is to learn a discriminative classifier from the source 

domain, which at the same time could generalize well on the target domain by leveraging unlabeled target data and learning a mapping between source and target domains. It is generally acknowledged that the discrepancy between two datasets (domain shift) can be reduced by aligning two distributions Ben-David et al. (2006); Ben-David et al. (2010). The methods that learn domain invariant features for domain alignment include KL divergence Zhuang et al. (2015), Maximum Mean Discrepancy (MMD) Gretton et al. (2012), and Domain Adversarial Neural Network (DANN) Ganin et al. (2016) (details of DANN can be found in Appendix. B). DANN suffers from a vanishing gradient problem Shen et al. (2018), and the training process is unstable Shah et al. (2018); Du et al. (2020). Hence, more efficient and stable algorithms are essential for UDA Wu et al. (2019b).

### Adversarial Training

Adversarial training with perturbations has been shown to significantly improve the performance of the state-of-the-art language models for many natural language understanding tasks Madry et al. (2018); Zhu et al. (2020); Jiang et al. (2020); Pereira et al. (2021). The algorithm generally considers adversarial attacks with perturbations to word embeddings and minimizes the resultant adversarial loss around the input samples. In a single domain, adversarial training Goodfellow et al. (2015) is an inner max, outer min adversarial problem with the objective:

\[\min_{\theta_{f},\theta_{y}}\sum\nolimits_{(x,y)\sim\mathcal{D}}[\max_{ \delta}\mathcal{L}(f(x+\delta;\theta_{f},\theta_{y}),y)]. \tag{2}\]

With (2), the standard adversarial training can also be regularized using virtual adversarial training Miyato et al. (2018), which encourages smoothness in the embedding space. The \(\alpha_{adv}\) controls the trade-off between the two losses, usually set to be \(1\).

\[\begin{split}&\min_{\theta_{f},\theta_{y}}\sum\nolimits_{(x,y) \sim\mathcal{D}}\big{[}\operatorname*{\mathcal{L}}(f(x;\theta_{f},\theta_{y}), y)+\alpha_{adv}\\ &\quad\max_{\delta}\mathcal{L}(f(x+\delta;\theta_{f},\theta_{y}), f(x;\theta_{f},\theta_{y}))\big{]}.\end{split} \tag{3}\]

For (2)(3), the inner maximization can be solved by Projected Gradient Decent (PGD) Madry et al. (2018) with an additional assumption that the loss function is locally linear. A following iteration can approximate the adversarial perturbation \(\delta\):

\[\delta_{t+1}=\Pi_{||\delta||_{F}\leq\epsilon}(\delta_{t}+\eta\frac{g^{ adv}_{y}(\delta_{t})}{\|g^{adv}_{y}(\delta_{t})\|}_{F}), \tag{4}\]

\[g^{adv}_{y}(\delta_{t})=\nabla_{\delta}\operatorname*{\mathcal{L}}(f(x+\delta _{t};\theta_{f},\theta_{y}),y), \tag{5}\]

where \(\Pi_{||\delta||_{F}\leq\epsilon}\) performs a projection onto the \(\epsilon\)-ball. The advantages of PGD lie in that it only relies on the model itself to produce diverse adversarial samples, enabling the model to generalize better to unseen data.

## 3 Method

In this section, we focus our discussions on the proposed Domain Confused Contrastive Learning (DCCL) under a sentiment classification scenario. The overall framework of our method is illustrated in Fig. 2. The model will take source labeled and target unlabeled sentences as input. It will then augment the input data with domain puzzles by fabricating adversarial perturbations. With the augmented data, the next step produces a hidden representation for each instance with an encoder which will be further used to produce three losses to train the entire model, namely sentiment classification loss, contrastive loss and consistency loss.

### Crafting domain puzzles

For UDA, Saito et al. (2017) mentions that simply matching the distributions cannot ensure high accuracy on the target domain without the target labels.

Figure 2: Overview of the framework of the proposed method DCCL.

 

Moreover, it may cause negative transfer, deteriorating knowledge transfer from source domain to the target domain Wang et al. (2019). Even if the matched sentences have the same label, due to huge syntactic and semantic shift, instance-based matching strategies that align examples from different domains will introduce noises for pre-trained language models, for example, aligning source domain and target domain sentences in Fig. 3.

Alternatively, we can locate and mask domain-specific tokens which are related to sentence topics and genres. Since sentences in the green box of Fig. 3 become domain-agnostic, we refer to those domain-confused sentences (one cannot tell which domain these sentences belong to) as domain puzzles. Matching distributions between the source domain and the domain puzzles, as well as the target domain and the domain puzzles, will also make language models produce domain invariant representations.

However, the domain-specific tokens are not always evident, due to the discrete nature of natural languages, it is challenging to decide correct tokens to mask without hurting the semantics especially when the sentences are complicated1. Hence, we seek domain puzzles in the representation space and introduce adversarial perturbations, because we can rely on the model itself to produce diverse but targeted domain puzzles. Note that the purpose of adversarial attack here is not to enhance the robustness, but to construct exquisitely produced perturbations for a better domain invariance in the representation space.

Footnote 1: Masking is not our focus in this paper. More detailed implementation can be found in section 4.4. We will investigate how to extract better domain tokens in our future work.

To generate domain-confused augmentations, we adopt adversarial attack with perturbations for domain classification. The loss for learning a domain classifier with adversarial attack can be specified as follows:

\[{\cal L}_{\rm domain}={\cal L}(f(x;\theta_{f},\theta_{d}),d)+\] \[\alpha_{adv}\,{\cal L}(f(x+\delta;\theta_{f},\theta_{d}),f(x; \theta_{f},\theta_{d})), \tag{6}\] \[\delta=\Pi_{|\delta||_{F}\leq\epsilon}(\delta_{0}+\eta\frac{g^{ adv}_{d}(\delta_{0})}{\left\lVert g^{adv}_{d}(\delta_{0})\right\rVert_{F}}), \tag{7}\]

where \(\delta_{0}\) is the initialized noise, \(\theta_{d}\) is the parameter corresponding to the computation of the domain classification, and \(d\) is the domain label. Due to additional overhead incurred during fine-tuning large pre-trained language models, the number of iterations for perturbation estimation is usually 1 Jiang et al. (2020); Pereira et al. (2021), as shown in Eq. 7. We synthesize the perturbation \(\delta\) by searching for an extreme direction that perplexes the domain classifier most in the embedding space, and \(f(x+\delta;\theta_{f})\) is the crafted domain puzzles encoded by the language model.

### Learning invariance with domain puzzles

After acquiring domain puzzles, simply applying distribution matching will sacrifice discriminative knowledge learned from the source domain Saito et al. (2017); Lee et al. (2019), and instance-based matching will also overlook global intra-domain information. To learn sentiment-wise discriminative

Figure 4: Proposed adaptation sampling method

Figure 3: Two sentences sampled from Book and Music reviews. Alternatively, we can match original sentences with its degraded masked versions.

 

representations in the absence of the target labels, we propose to learn domain invariance via contrastive learning (CL). In general, CL benefits from the definition of the augmented positive and negative pairs by treating instances as classes Chen et al. (2020); Khosla et al. (2020); He et al. (2020); Chen et al. (2020). Furthermore, the contrastive loss encourages the positive pairs to be close to each other and negative pairs to be far apart. Specifically, maximizing the similarities between positive pairs learns an invariant instance-based representation, and minimizing the similarities between negative pairs learns a uniformly distributed representation from a global view, making instances gathered near the task decision boundary away from each other Saunshi et al. (2019); Grill et al. (2020). This will help to enhance task discrimination of the learned model.

For positive pairs, intuitively, we hope that the model could encode the original sentence and most domain-challenging examples to be closer in the representation space, gradually pulling examples to the domain decision boundary as training progresses. For negative sampling, it widens the sentiment decision boundary and promotes better sentiment-wise discriminative features for both domains. However, for cross-domain negative sampling, the contrastive loss may push the negative samples in the target (source) domain away from the anchor in the source (target) domain (see Fig. 4 (b) left). This is contradictory to the objective of domain puzzles which try to pull different domains closer. To avoid the detriment of cross-domain repulsion, excluding samples with different domains from the negative set is of great importance. Therefore, we write the following contrastive infoNCE loss Chen et al. (2020) as follow:

\[{\mathcal{L}_{\rm contrast}=-\frac{1}{N}\sum_{i}^{N}\log\frac{\exp(s(z_{i},z_{ i}^{\prime})/\tau)}{\sum_{k}^{N}\mathbbm{1}_{k\neq i}\exp(s(z_{i},z_{k})/\tau)}}, \tag{8}\]

where \(N\) is the mini batch size with samples from the same domain, \(z_{i}=g(f(x_{i};\theta_{f}))\), and \(g(\cdot)\) is one hidden layer projection head. We denote \(x^{\prime}=x+\delta\) as the domain puzzle augmentation, \(s(\cdot)\) computes cosine similarity, \(\mathbbm{1}_{k\neq i}\) is the indicator function, and \(\tau\) is the temperature hyperparameter.

### Consistency Regularization

Given perturbed embedding \(x+\delta\), which is crafted based on domain classification, we also encourage the model to produce consistent sentiment predictions with that of the original instance \(f(x;\theta_{f},\theta_{y})\). For this, we minimize the symmetric KL divergence, which is formulated as:

\[{\mathcal{L}_{\rm consist}={\mathcal{L}}(f(x;\theta_{f},\theta_{ y}),f(x+\delta;\theta_{f},\theta_{y}))}. \tag{9}\]

```
0: For simplicity, \(\theta\) is the parameter of the whole model. \(T\): the total number of iterations, \((x,y)\sim{\mathcal{D}}^{S}\): source dataset with sentiment label \(y\), \((x,d)\sim{\mathcal{D}}^{S}\,{\mathcal{D}}^{D}\): source and target dataset with domain label \(d\), \(K\): the number of iterations for updating \(\delta\), \(\sigma^{2}\): the initialized variance, \(\epsilon\): perturbation bound, \(\eta\): the step size, \(\gamma\): global learning rate, \(N\): batch size, \(\tau\): temperature, \(g(\cdot)\):one hidden layer projection head. \(\alpha_{adv}\), \(\alpha\), \(\lambda\) and \(\beta\): weighting factor.
1:for\({\rm epoch}=1,..,T\)do
2:for minibatch \(N\)do
3:\(\delta\leftarrow{\mathcal{N}}(0,\sigma^{2}I)\)
4:for\(m=1,...,K\)do
5:\(g_{ad}^{adv}\leftarrow\nabla_{S}\,{\mathcal{L}}(f(x+\delta;\theta),d)\)
6:\(\delta\leftarrow\Pi_{l\notin S\|_{F}\leq\epsilon}(\delta+\eta g_{d}^{adv}/\|g_ {d}^{adv}\|_{F})\)
7:endfor
8:\({\mathcal{L}_{\rm domain}\leftarrow{\mathcal{L}}(f(x;\theta),d)}\)  \(+\alpha_{adv}\,{\mathcal{L}}(f(x+\delta;\theta),d)\)
9:\(z=g(f(x;\theta))\)
10:\(z^{\prime}=g(f(x+\delta;\theta))\)
11:for\(i=1,...,N\) and \(j=1,...,N\)do
12:\(s_{i}^{\prime}=z_{i}^{\top}z_{j}^{\prime}/\|z_{i}\|\|z_{j}^{\prime}\|\)
13:\(s_{i,j}=z_{i}^{\top}z_{j}/\|z_{i}\|\|z_{j}\|\)
14:endfor
15:\({\mathcal{L}_{\rm contrast}\leftarrow-\frac{1}{N}\sum_{i}^{N}\log\frac{\exp( s_{i}^{\prime}/\tau)}{\sum_{j=1}^{N}\mathbbm{1}_{j\neq i}\exp(s_{i,j}/\tau)}}\)
16:\({\mathcal{L}_{\rm consist}\leftarrow{\mathcal{L}}(f(x;\theta),f(x+\delta; \theta))}\)
17:\(g_{\phi}\leftarrow\nabla_{\theta}\,{\mathcal{L}}(f(x;\theta),y)+\alpha\nabla_ {\theta}\,{\mathcal{L}_{\rm domain}}\)  \(+\lambda\nabla_{\theta}\,{\mathcal{L}_{\rm contrast}+\beta\nabla_{\theta}\,{ \mathcal{L}_{\rm consist}}}\)
18:\(\theta\leftarrow\theta-\gamma g_{\theta}\)
19:endfor
20:endfor Output:\(\theta\)
```

**Algorithm 1** DCCL

For our simplicity, \(\theta\) is the parameter of the whole model. \(T\): the total number of iterations, \((x,y)\sim{\mathcal{D}}^{S}\): source dataset with sentiment label \(y\), \((x,d)\sim{\mathcal{D}}^{S}\,{\mathcal{D}}^{D}\): source and target dataset with domain label \(d\), \(K\): the number of iterations for updating \(\delta\), \(\sigma^{2}\): the initialized variance, \(\epsilon\): perturbation bound, \(\eta\): the step size, \(\gamma\): global learning rate, \(N\): batch size, \(\tau\): temperature, \(g(\cdot)\):one hidden layer projection head. \(\alpha_{adv}\), \(\alpha\), \(\lambda\) and \(\beta\): weighting factor.

\begin{tabular}{p{39.8pt} p{39.8pt}}
1:for\({\rm epoch}=1,..,T\)do
2:for minibatch \(N\)do
3:\(\delta\leftarrow{\mathcal{N}}(0,\sigma^{2}I)\)
4:for\(m=1,..,K\)do
5:\(g_{ad}^{adv}\leftarrow\nabla_{S}\,{\mathcal{L}}(f(x+\delta;\theta),d)\)
6:\(\delta\leftarrow\Pi_{l\in S\|_{F}\leq\epsilon}(\delta+\eta g_{d}^{adv}/\|g_ {d}^{adv}\|_{F})\)
7:endfor
8:\({\mathcal{L}_{\rm domain}\leftarrow{\mathcal{L}}(f(x;\theta),d)}\)  \(+\alpha_{adv}\,{\mathcal{L}}(f(x+\delta;\theta),d)\)
9:\(z=g(f(x;\theta))\)
10:\(z^{\prime}=g(f(x+\delta;\theta))\)
11:for\(i=1,...,N\) and \(j=1,...,N\)do
12:\(s_{i}^{\prime}=z_{i}^{\top}z_{j}^{\prime}/\|z_{i}\|\|z_{j}^{\prime}\|\)
13:\(s_{i,j}=z_{i}^{\top}z_{j}/\|z_{i}\|\|z_{j}\|\)
14:endif
15:\({\mathcal{L}_{\rm confust}\leftarrow-\frac{1}{N}\sum_{i}^{N}\log\frac{\exp(s_{i}^{ \prime}/\tau)}{\sum_{j=1}^{N}\mathbbm{1}_{j\neq i}\exp(s_{i,j}/\tau)}}\)
16:\({\mathcal{L}_{\rm consist}\leftarrow{\mathcal{L}}(f(x;\theta),f(x+\delta; \theta))}\)
17:\(g_{\phi}\leftarrow\nabla_{\theta}\,{\mathcal{L}}(f(x;\theta),y)+\alpha\nabla_ {\theta}\,{\mathcal{L}_{\rm domain}}\)\(+\lambda\nabla_{\theta}\,{\mathcal{L}_{\rm contrast}+\beta\nabla_{\theta}\,{ \mathcal{L}_{\rm consist}}}\)
18:\(\theta\leftarrow\theta-\gamma g_{\theta}\)
19:endfor
20:endfor
21:\(\theta\) ```

**Algorithm 2** DCCL

Details of proposed DCCL are summarized in Algorithm 1.

 

Experiments

### Datasets

**Amazon Benchmark**Blitzer et al. (2007)2. We conduct experiments on this dataset for completeness since most of previous works report results on it. The dataset contains four domains: Book (BK), DVD (D), Electronics (E) and Kitchen housewares (K). There are 2,000 balanced labeled data for each domain, we randomly select 20,000 unlabeled reviews for BK, D and E. For K, only 17,856 unlabeled reviews are available.

Footnote 2: [https://www.cs.jhu.edu/](https://www.cs.jhu.edu/) mdredze/datasets/sentiment/

**Amazon review dataset**He et al. (2018)3. This dataset considers neutral instances which may not bias the dataset and bring more challenges4. The dataset also contains four domains: Book (BK), Electronics (E), Beauty (BT), and Music (M). Following He et al. (2018), we treat set \(1\) as labeled dataset containing \(6,000\) instances, and treat set \(2\) as unlabeled dataset which also contains \(6,000\) instances. More details about two datasets can be found in Appendix A.

Footnote 3: [https://github.com/riudan/DAS](https://github.com/riudan/DAS)

Footnote 4: The original crawled reviews contain star ratings (1 to 5 stars). Comparing with Amazon Benchmark which discards the neutral class, this dataset labels them with rating \(<3,>3\), \(=3\) as negative, positive, and neutral respectively.

### Experiment Settings

For unsupervised adaptation setting, we should not have access to target labeled data at the training phase, so trained models with minimum classification error on the source validation set is saved for evaluation. At this point, we suppose a good model that generalizes well on the target domain is able to reach high performance on both validation and test set at the same time. We evaluate our model with \(5\) runs in all experiments, and we report the average score, standard deviation and paired t-test results.

### Implementation Details

For pre-trained language model, we use BERT base uncased Devlin et al. (2019) as the basis for all experiments. The max length is set to \(512\). For optimizer, we use AdamW Kingma and Ba (2015); Loshchilov and Hutter (2019) with weight decay 0.01 (for BERT baseline, we set 1e-4). We set the learning rate as 1e-5, and we use a linear scheduler with warm-up steps \(0.1\) of total training steps.

We set the number of adversarial iterations to be 1, adversarial weighting factor \(\alpha_{adv}\) = 1 and we use \(l_{2}\) norm to compute projections. We also follow Zhu et al. (2020) to set other adversarial hyperparameters, e.g., adversarial step size \(\eta\) = 5e-2, perturbation bound \(\epsilon\) = 5e-2. For weighting factors, we set \(\alpha\) = 1e-3, \(\lambda\) = 3e-2 and \(\beta\) = 5. We train the model with \(8\) epochs, temperature \(\tau\) and batch size \(N\) will be discussed later. Each adaptation requires half one hour on one A-100.

### Baselines

**BERT base**: Fine-tune BERT on the source, without using target unlabeled data, then directly evaluate on the target labeled data. **KL**: Use symmetric KL-divergence loss of embedded instances between the source and target domains Zhuang et al. (2015). **MMD**: Maximum Mean Discrepancy loss Gretton et al. (2012) measures the distance based on the notion of embedding probabilities in a reproducing kernel Hilbert space. We implement a gaussian kernel which is a common choice. **DANN**: The adaptation rate is \(\lambda=\frac{2}{1+\exp(-\gamma p)}-1\), \(p=\frac{t}{T}\), where \(t\) and \(T\) are the number of current training steps and total steps. \(\gamma\) requires careful tuning within \([0.05,0.1,0.15,0.2]\). **backtrans+CL**: To investigate the effectiveness of domain puzzles, we implement Back translation Sennrich et al. (2016); Edunov et al. (2018), which is one of the widely used data augmentation techniques. We utilize en-de translation model pretrained on WMT19 and released in fairseq Ott et al. (2019). The model is trained with contrastive loss on the source and target domain respectively. **mask+CL**: We mask domain specific tokens to make the augmentation become domain-agnostic. Since information extraction is not our focus, we identify domain specific tokens via a simple frequency-ratio method Li et al. (2018); Wu et al. (2019): \(s(u,d)=\frac{\mathrm{count}(u,\mathcal{D}^{d})+\lambda}{\sum_{d^{\prime}\in \mathcal{D},d^{\prime}\neq d}\mathrm{count}(u,\mathcal{D}^{d^{\prime}})+\lambda}\), where \(\mathrm{count}(u,\mathcal{D}^{d})\) represents the number of times a token \(u\) appears in domain \(d\). Smoothing \(\lambda\) is set to 1. When \(s(u,d)\) is larger than \(5\), we mark a token \(u\) as a domain specific token. Through counting, the number of masked tokens accounted for 0.06 of the total length. **mask**: To investigate the effectiveness of contrastive loss, after masking domain-specific tokens, we further let the model train with augmented data without contrastive loss. **R-PERL**Ben-David et al. (2020) is a pivot-based method, and **DAAT**Du et al. (2020) combines DANN and post-training. All the methods in Tabel 1 and Table 2 are implemented based on BERT model for fair comparisons 



### Results

**Will contrastive learning necessarily help the Unsupervised Domain Adaptation?** As discussed earlier, when performing contrastive learning on source labeled examples and target unlabeled examples respectively, it learns a uniformly distribute representation and helps promote better discriminative features for both domains. However, for some adaptation tasks in Table 1, for example, BT\(\rightarrow\)E, M\(\rightarrow\)E, and E\(\rightarrow\)M, back-trans+CL shows that contrastive learning only gains marginal benefit. When masking domain-specific tokens and pulling original sentence representations to those domain puzzles, the effect of contrastive learning becomes more apparent (mask+CL with average score 65.17 compared with back-trans+CL 63.79 and mask 61.40 in Table.1). This finding helps to explain that choices of positive examples are critical and domain confused augmentations will further benefit adaptation.

**DCCL outperforms baselines.** From Table 1 we can observe that the proposed DCCL outperforms all other methods with a large margin and \(p<0.05\) using paired t-test, and 5.94% improvement over BERT base. From Table 2, we can also observe 1.74% improvement over BERT base, and DCCL also surpasses state-of-the-art methods R-PERL [8] and DAAT [10]. We note that Amazon Benchmark dataset is quite easy, since it discards neutral instances, BERT base model has already achieved high scores on this dataset. Besides, we observe that the effect of distribution matching methods (KL and MMD) is limited on two datasets. The reason might be that pre-trained language models trained with massive and heterogeneous corpora already have strong generalization ability. Learning such cross-domain and instance-based matching will bring perplexity to language models and sacrifice task discrimination [11, 12]. On the contrary, the proposed DCCL retains such information in a self-supervised way. Furthermore, we notice that DANN is very unstable, besides adaptation rate \(\lambda\), the model is also sensitive to other hyperparameters such as learning rate and training epochs because the performance on the target domain will keep decreasing when training with longer steps. Hence, it is difficult for the model to achieve the lowest error rates on both the source and target domains simultaneously. Compared to DANN, DCCL is much more stable and has lower standard deviations on most adaptation tasks.

**Contrastive learning designs.** We explore different hyperparameter values for the

\begin{table}
\begin{tabular}{l|c c c c c c c c c c c c} \hline \hline
**Method** & E\(\rightarrow\)BK & BT\(\rightarrow\)BK & M\(\rightarrow\)BK & BK\(\rightarrow\)E & M\(\rightarrow\)BE & BK\(\rightarrow\)BT & C\(\rightarrow\)B & MT\(\rightarrow\)BT & BC\(\rightarrow\)M & BE\(\rightarrow\)M & **Nete.** \\ \hline BERT base & \(64.34^{*}_{1}\) & \(65.87^{*}_{19}\) & \(64.12^{*}_{1}\) & \(52.52^{*}_{1}\) & \(66.01^{*}_{1}\) & \(59.49^{*}_{1}\) & \(55.01^{*}_{1}\) & \(56.02^{*}_{1}\) & \(56.05^{*}_{1}\) & \(55.61^{*}_{1}\) & \(53.43^{*}_{6}\) & \(62.34^{*}_{1}\) & \(60.74^{*}_{1.5}\) \\ KL & \(64.77^{*}_{0.8}\) & \(67.03^{*}_{0.7}\) & \(65.13^{*}_{1}\) & \(56.56^{*}_{1}\) & \(56.63^{*}_{1}\) & \(56.03^{*}_{1}\) & \(58.47^{*}_{0.9}\) & \(66.22^{*}_{1}\) & \(58.30^{*}_{1}\) & \(57.53^{*}_{0.00}\) & \(60.21^{*}_{1}\) & \(64.41^{*}_{1}\) & \(62.09^{*}_{1}\) \\ MMD & \(65.41^{*}_{0.7}\) & \(68.54^{*}_{1}\) & \(61.47^{*}_{1}\) & \(53.84^{*}_{1}\) & \(66.39^{*}_{1}\) & \(59.91^{*}_{1}\) & \(60.75^{*}_{1}\) & \(66.05^{*}_{0}\) & \(57.42^{*}_{1}\) & \(59.37^{*}_{1}\) & \(59.47^{*}_{1}\) & \(65.05^{*}_{1}\) & \(62.65^{*}_{1}\) \\ DNN & \(66.43^{*}_{2}\) & \(67.74^{*}_{2}\) & \(65.64^{*}_{1}\) & \(54.31^{*}_{1}\) & \(65.93^{*}_{1}\) & \(59.91^{*}_{1}\) & \(58.11^{*}_{1}\) & \(56.719^{*}_{1}\) & \(56.00^{*}_{1}\) & \(54.02^{*}_{0}\) & \(60.36^{*}_{1}\) & \(60.92^{*}_{1}\) & \(61.56^{*}_{2}\) \\ back-trans+CL & \(67.16^{*}_{1}\) & \(68.31^{*}_{0.8}\) & \(67.78^{*}_{1}\) & \(59.55^{*}_{1}\) & \(66.51^{*}_{0}\) & \(60.13^{*}_{1}\) & \(60.92^{*}_{0}\) & \(68.54^{*}_{0}\) & \(68.04^{*}_{0}\) & \(60.21^{*}_{1}\) & \(61.73^{*}_{1}\) & \(64.29^{*}_{0}\) & \(67.39^{*}_{1}\) \\ mask & \(65.17^{*}_{1}\) & \(66.91^{*}_{1}\) & \(65.95^{*}_{0}\) & \(52.74^{*}_{1}\) & \(67.14^{*}_{1}\) & \(60.48^{*}_{0}\) & \(54.76^{*}_{1}\) & \(66.50^{*}_{1}\) & \(56.61^{*}_{1}\) & \(55.64^{*}_{1}\) & \(62.17^{*}_{0}\) & \(63.09^{*}_{1}\) & \(61.40_{1}\) \\ mask+CL & \(68.80^{*}_{1}\) & \(69.93^{*}_{7}\) & \(70.65^{*}_{0}\) & \(60.50^{*}_{1}\) & \(68.02^{*}_{0}\) & \(60.80^{*}_{1}\) & \(62.17^{*}_{0}\) & \(69.17^{*}_{0}\) & \(62.66^{*}_{1}\) & \(60.50^{*}_{1}\) & \(63.94^{*}_{0}\) & \(65.40^{*}_{0}\) & \(65.17^{*}_{0}\) \\ \hline DCCL & \multicolumn{1}{r}{**70.30\({}_{3.0}\)

proposed DCCL in E\(\rightarrow\)BK, as is shown in Fig. 5. We find that a temperature of 0.5 combined with the batch size 32 can achieve the best performance. We also notice that setting the temperature too high or too low can significantly affect adaptation performances, while a larger batch size usually brings a relatively smaller improvement.

### Ablation Studies

We conduct an ablation study on each component in Eq. 10 to inspect the contribution of contrastive learning, as is shown in Table 3. We can see that every single component can render a better adaptation performance. In particular, the effectiveness of \({\cal L}_{\rm contrast}\) is observable with 3-5 performance gain compared to baselines (row 4 vs. row 2, and row 5 vs. row 3). When training curated domain puzzles as simple augmentations without contrastive loss, we can observe only a slight improvement (row 2 vs. row 1). This result demonstrates that the performance gain brought by DCCL does not come from data augmentation, and learning robustness against adversarial attacks will not largely help adaptation.

## 5 Analysis

### Visualization

We perform visualizations for trained representations as illustrated in Fig. 6. When training with the source domain and then adapting to the target domain (BERT-base), we can observe a considerable domain shift for BERT encoder on this amazon review dataset. Moreover, as mentioned before, continuing training DANN with larger epochs will substantially drop the score (from the highest point (DANN-best) to the lowest point (DANN-worst)). However, we can also see that DCCL mitigates domain shift but remains good sentiment discrimination on the target domain.

### Quantitative Results

\({\cal A}\)-distance measures domain discrepancies (Ben-David et al., 2006; Ben-David et al., 2010), with the definition as \(d_{\mathcal{A}}=2(1-2\epsilon)\), where \(\epsilon\) is the domain classification error. To fairly compare with \({\cal A}\)-distance of the baselines, we use linear SVM to calculate \(\epsilon\) following previous work (Saito et al., 2017; Du et al., 2020). We randomly select 2,000 instances for both source and target domain and split them with 1:1 for train and test for the SVM model. From Fig. 7, we can observe that DCCL can learn a good balance between sentiment classification and domain discrepancy, compared to DANN-best and DANN-worst.

## 6 Related Work

**Unsupervised Domain Adaptation**

UDA in NLP has the following approaches: (1) Pivot-based methods use unlabeled data from both domains, trying to discover characteristics that are similar (Pan et al., 2010). Some recent works

Figure 5: Hyperparameters for contrastive learning

\begin{table}
\begin{tabular}{l l|c c c} \hline \hline \({\cal L}_{\rm domain}\) & \({\cal L}_{\rm consist}\) & \({\cal L}_{\rm contrast}\) & E\(\rightarrow\)BK & M\(\rightarrow\)BT \\ \hline  & & & 64.65 & 55.85 \\ ✓ & & & 67.23 & 58.9 \\ ✓ & ✓ & & 67.85 & 59.1 \\ ✓ & & ✓ & 70.12 & 64.73 \\ ✓ & ✓ & ✓ & **70.21** & **64.87** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Ablation studies of DCCL on each component 

extend pivots with autoencoders and contextualized embeddings (Ziser and Reichart, 2017; Miller, 2019; Ben-David et al., 2020). (2) Pseudo-labeling leverages a trained classifier to predict labels on unlabeled examples, which are subsequently considered as gold labels (Yarowsky, 1995; Zhou and Li, 2005; McClosky et al., 2006). Recent works also combine this technique with pre-trained language models (Lim et al., 2020; Ye et al., 2020). (3) Data selection methods adopt domain similarity metrics to find the best match for each data and use curriculum learning for large pre-trained models (Ma et al., 2019; Aharoni and Goldberg, 2020). A recent method adopt distance-bandit (Guo et al., 2020) for similarity metric. (4) Domain Adversarial Neural Networks (Ganin et al., 2016). Some approaches leverage Wasserstein distance to stabilize adversarial training (Shen et al., 2018; Shah et al., 2018), and combine it with post-training which can produce better adversarial results (Du et al., 2020). (5) Adaptive pre-training is a more straightforward but effective method (Gururangan et al., 2020; Han and Eisenstein, 2019; Karouzos et al., 2021) by leveraging the objective of masked language model (MLM). A wide range of pre-training methods for domain adaptation (multi-phase, multi-task) are put forward (Han and Eisenstein, 2019; Karouzos et al., 2021).

**Contrastive learning**

CL has recently gained popularity as a reliable approach for unsupervised representation learning. It is generally acknowledged that a good representation should distinguish itself from other instances while identifying similar instances. For Computer Vision, there are approaches obtaining augmented images using transformations including cropping, rotation, etc. (Chen et al., 2020; Khosla et al., 2020; He et al., 2020; Chen et al., 2020b). As for Natural Language Processing, many works study different label-preserving augmentations, such as back-translations, synonyms, adversaries, dropout, and their combinations (Qu et al., 2021; Gao et al., 2021). In addition, many pre-trained language models trained with contrastive loss are also released. DeCLUTR (Giorgi et al., 2021) and CLEAR (Wu et al., 2020) jointly train the model with a contrastive objective and a masked language model setting. ConSERT (Yan et al., 2021) overcomes the collapse problem of BERT-derived sentence representations and makes them more suitable for downstream applications by using unlabeled texts.

## 7 Conclusion

In this work, we put forward a new concept, domain puzzles, which can be crafted through domain-specific token mask and domain-confused adversarial attacks. And we provide a more stable and effective solution to learn domain invariance for unsupervised domain adaptation. The proposed method DCCL surpasses baselines with a large margin by mitigating domain shift without losing discriminative power on the target domain. Moreover, the proposed framework can also be extended to other NLP tasks demanding adaptations, and we leave this for future work.

## Acknowledgements

This work is supported by the 2020 Microsoft Research Asia collaborative research grant.

## References

* R. Aharoni and Y. Goldberg (2020)Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp. 7747-7763. External Links: Link Cited by: SS1, SS2.
* E. Ben-David, C. Rabinovitz, and R. Reichart (2020)PERL: pivot-based domain adaptation for pre-trained deep contextualized embedding models. Transactions of the Association for Computational Linguistics8, pp. 504-521. External Links: Document, ISSN 1073-642X, Link Cited by: SS1, SS2.
* S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Wortman Vaughan (2010)A theory of learning from different domains. Machine learning79 (1), pp. 151-175. External Links: ISSN 0037-642X, Link Cited by: SS1, SS2.
* S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira (2006)Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems 19, Twenttieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 4-7, 2006, pp. 137-144. External Links: ISBN 978-1-447-1195-5, Link Cited by: SS1, SS2.

 

John Blitzer, Mark Dredze, and Fernando Pereira. 2007: Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic. Association for Computational Linguistics.
* Chen et al. (2020a) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020a. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, Vol. 119 of _Proceedings of Machine Learning Research_, pages 1597-1607. PMLR.
* Chen et al. (2020b) Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020b. Improved baselines with momentum contrastive learning. ArXiv preprintabs/2003.04297.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota. Association for Computational Linguistics.
* Du et al. (2020) Chunning Du, Haifeng Sun, Jingyu Wang, Qi Qi, and Jianxin Liao. 2020. Adversarial and domain-aware BERT for cross-domain sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. Association for Computational Linguistics.
* Edunov et al. (2018) Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium. Association for Computational Linguistics.
* Ganin et al. (2016) Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The journal of machine learning research17 (1):2096-2030.
* Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Giorgi et al. (2021) John Giorgi, Osvald Nitski, Bo Wang, and Gary Bader. 2021. DeCLUTR: Deep contrastive learning for unsupervised textual representations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online. Association for Computational Linguistics.
* Goodfellow et al. (2015) Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
* Gretton et al. (2012) Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Scholkopf, and Alexander Smola. 2012. A kernel two-sample test. _The Journal of Machine Learning Research_, 13(1):723-773.
* A new approach to self-supervised learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
* Guo et al. (2020) Han Guo, Ramakanth Pasunuru, and Mohit Bansal. 2020. Multi-source domain adaptation for text classification via distancenet-bandits. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 7830-7838. AAAI Press.
* Gururangan et al. (2020) Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. Association for Computational Linguistics.
* Han and Eisenstein (2019) Xiaochuang Han and Jacob Eisenstein. 2019. Unsupervised domain adaptation of contextualized embeddings for sequence labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China. Association for Computational Linguistics.
* He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020_, pages 9726-9735. IEEE.
* He et al. (2018) Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2018. Adaptive semi-supervised learning 

for cross-domain sentiment classification. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 3467-3476, Brussels, Belgium. Association for Computational Linguistics.
* Jiang et al. (2020) Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. 2020. SMART: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 2177-2190, Online. Association for Computational Linguistics.
* Karouzos et al. (2021) Constantinos Karouzos, Georgios Paraskevopoulos, and Alexandros Potamianos. 2021. UDALM: Unsupervised domain adaptation through language modeling. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2579-2590, Online. Association for Computational Linguistics.
* Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
* Krueger et al. (2021) David Krueger, Ethan Caballero, Jorn-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron C. Courville. 2021. Out-of-distribution generalization via risk extrapolation (rex). In _Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event_, volume 139 of _Proceedings of Machine Learning Research_, pages 5815-5826. PMLR.
* November 2, 2019_, pages 91-100. IEEE.
* Li et al. (2018) Juncein Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 1865-1874, New Orleans, Louisiana. Association for Computational Linguistics.
* Lim et al. (2020) KyungTae Lim, Jay Yoon Lee, Jaime G. Carbonell, and Thierry Poibeau. 2020. Semi-supervised learning on meta structure: Multi-task tagging and parsing in low-resource scenarios. In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_, pages 8344-8351. AAAI Press.
* Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. _ArXiv_, abs/1907.11692.
* Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
* Ma et al. (2019) Xiaofiea Ma, Peng Xu, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang. 2019. Domain adaptation with BERT-based domain classification and data selection. In _Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)_, pages 76-83, Hong Kong, China. Association for Computational Linguistics.
* May 3, 2018, Conference Track Proceedings_. OpenReview.net.
* McAuley et al. (2015) Julian J. McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-based recommendations on styles and substitutes. In _Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, August 9-13, 2015_, pages 43-52. ACM.
* McClosky et al. (2006) David McClosky, Eugene Charniak, and Mark Johnson. 2006. Reranking and self-training for parser adaptation. In _Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics_, pages 337-344, Sydney, Australia. Association for Computational Linguistics.
* Miller (2019) Timothy Miller. 2019. Simplified neural unsupervised domain adaptation. In _Proceedings of the 2019 Conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 414-419, Minneapolis, Minnesota. Association for Computational Linguistics.
* Miyato et al. (2018) Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. 2018. Virtual adversarial training: 

a regularization method for supervised and semi-supervised learning. _IEEE transactions on Pattern Analysis and Machine Intelligence_, 41(8):1979-1993.
* Ott et al. [2019] Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_, pages 48-53, Minneapolis, Minnesota. Association for Computational Linguistics.
* Pan et al. [2010] Sinno Jialin Pan, Xiaochuan Ni, Jian-Tao Sun, Qiang Yang, and Zheng Chen. 2010. Cross-domain sentiment classification via spectral feature alignment. In _Proceedings of the 19th International Conference on World Wide Web, WWW 2010, Raleigh, North Carolina, USA, April 26-30, 2010_, pages 751-760. ACM.
* Park et al. [2020] Changhwa Park, Jonghyun Lee, Jaeyoon Yoo, Minhoe Hur, and Sungroh Yoon. 2020. Joint contrastive learning for unsupervised domain adaptation. _ArXiv preprint_, abs/2006.10297.
* Pereira et al. [2021] Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, and Ichiro Kobayashi. 2021. Targeted adversarial training for natural language understanding. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5385-5393, Online. Association for Computational Linguistics.
* Qu et al. [2021] Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Weizhu Chen, and Jiawei Han. 2021. Coda: Contrast-enhanced and diversity-promoting data augmentation for natural language understanding. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
* Ramponi and Plank [2020] Alan Ramponi and Barbara Plank. 2020. Neural unsupervised domain adaptation in NLP--A survey. In _Proceedings of the 28th International Conference on Computational Linguistics_, pages 6838-6855, Barcelona, Spain (Online). International Committee on Computational Linguistics.
* Saito et al. [2017] Kuniaki Saito, Yoshitaka Ushiku, and Tatsuya Harada. 2017. Asymmetric tri-training for unsupervised domain adaptation. In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017_, volume 70 of _Proceedings of Machine Learning Research_, pages 2988-2997. PMLR.
* Saunshi et al. [2019] Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. 2019. A theoretical analysis of contrastive unsupervised representation learning. In _Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA_, volume 97 of _Proceedings of Machine Learning Research_, pages 5628-5637. PMLR.
* Sennrich et al. [2016] Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 86-96, Berlin, Germany. Association for Computational Linguistics.
* Shah et al. [2018] Darsh Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, and Preslav Nakov. 2018. Adversarial domain adaptation for duplicate question detection. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 1056-1063, Brussels, Belgium. Association for Computational Linguistics.
* Shen et al. [2018] Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018. Wasserstein distance guided representation learning for domain adaptation. In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_, pages 4058-4065. AAAI Press.
* Wang et al. [2021] Rui Wang, Zuxuan Wu, Zejia Weng, Jingjing Chen, Guo-Jun Qi, and Yu-Gang Jiang. 2021. Cross-domain contrastive learning for unsupervised domain adaptation. _ArXiv preprint_, abs/2106.05528.
* Wang et al. [2019] Ziriu Wang, Zihang Dai, Barnabas Poczos, and Jaime G. Carbonell. 2019. Characterizing and avoiding negative transfer. In _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019_, pages 11293-11302. Computer Vision Foundation / IEEE.
* Wu et al. [2019a] Xing Wu, Tao Zhang, Liangjun Zeng, Jizhong Han, and Songlin Hu. 2019a. Mask and infill: Applying masked language model for sentiment transfer. In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019_, pages 5271-5277. ijcai.org.
* Wu et al. [2019b] Yifan Wu, Ezra Winston, Divyansh Kaushik, and Zachary C. Lipton. 2019b. Domain adaptation with asymmetrically-relaxed distribution alignment. In _Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA_, volume 97 of _Proceedings of Machine Learning Research_, pages 6872-6881. PMLR.
* Wu et al. [2020] Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. 2020. Clear: Contrastive learning for sentence representation. _ArXiv preprint_, abs/2012.15466 

Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021. ConSERT: A contrastive framework for self-supervised sentence representation transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online. Association for Computational Linguistics.
* Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 5754-5764.
* Yarowsky (1995) David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In _33rd Annual Meeting of the Association for Computational Linguistics_, pages 189-196, Cambridge, Massachusetts, USA. Association for Computational Linguistics.
* Ye et al. (2020) Hai Ye, Qingyu Tan, Ruidan He, Juntao Li, Hwee Tou Ng, and Lidong Bing. 2020. Feature adaptation of pre-trained language models across languages and domains with robust self-training. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Vol. , pp. 7386-7399, Online. Association for Computational Linguistics.
* Zhou and Li (2005) Zhi-Hua Zhou and Ming Li. 2005. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on knowledge and Data Engineering17 (11):1529-1541.
* Zhu et al. (2020) Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. 2020. Freelb: Enhanced adversarial training for natural language understanding. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
* Zhuang et al. (2015) Fuzhen Zhuang, Xiaohu Cheng, Ping Luo, Sinno Jialin Pan, and Qing He. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015, pages 4119-4125. AAAI Press.
* Ziser and Reichart (2017) Yftah Ziser and Roi Reichart. 2017. Neural structural correspondence learning for domain adaptation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 400-410, Vancouver, Canada. Association for Computational Linguistics.

 

Datasets

We obtain the Amazon review datasets from He et al. (2018). This dataset does not remove neutral labels and will not be problematic in UDA situation where the label information of the target domain is not available. In addition, reserving neutral labels also bring challenges for pre-trained language model, making it more favorable for self-supervised representation learning. Summary of this dataset is availble in Table 4.

Each domain contains two sets, set 1 contains 6000 instances with balanced class labels, and set 2 contains instances that are randomly sampled from the larger dataset McAuley et al. (2015), preserving authentic label distribution, examples in these two datasets do not overlap. Following He et al. (2018), we use set 1 from the source domain as the training set for all our experiments. Since label distribution in the target domain is unpredictable and out of control in real life, so it's more reasonable to use set 2 from the target domain as the unlabeled set, lastly the model will be evaluated in set 1 from target domain. For data split, we randomly sample \(1000\) instances from the source labeled dataset as validation set. When running UDA experiments, the model will train on \(5000\) source labeled examples and \(6000\) target unlabeled examples, then validate on \(1000\) source labeled examples.

For Amazon BenchmarkBlitzer et al. (2007), it also contains four domains: Book (BK), DVD (D), Electronics (E) and Kitchen housewares (K). There are 2000 balanced labeled data for each domain, we randomly select 20000 unlabeled reviews for BK, D and E. For K, only 17856 unlabeled reviews are available, statistics of Amazon Benchmark can be find in Table.5. For data split, 1600 balanced samples are randomly sampled from the source labeled dataset, and 400 for validation.

## Appendix B Damn

Domain Adversarial Neural Network Ganin et al. (2016) proposes Domain Adversarial Neural Network (DANN), which learns domain invariant and discriminative features simultaneously. This approach is motivated by the idea that an adaptation algorithm could learn good representations for cross-domain transfer if it cannot differentiate the domain of the input observations. The optimization objective is:

\[\min_{\theta_{f},\theta_{y}}\sum_{(x,y)\sim\mathcal{D}^{S}}[ \mathcal{L}(f(x;\theta_{f},\theta_{y}),y)+\lambda R_{\theta_{f}}], \tag{11}\] \[R_{\theta_{f}}=\max_{\theta_{d}}\sum_{(x,d)\sim\mathcal{D}^{S}, \mathcal{D}^{T}}[-\mathcal{L}(f(x;\theta_{f},\theta_{d}),d)], \tag{12}\]

where \(\theta_{d}\) is the parameter corresponding to the computation of the domain classification, \(d\) is the domain label, \(R_{\theta_{f}}\) is a regularizer weighted by \(\lambda\). Objective (11) learns task discrimination by minimizing task classification loss and tries to make features similar across domains by maximizing the domain classification loss.

Although the domain classifier with parameters \(\theta_{d}\) could perfectly classify different domains, the balance between two terms in (11) is hard to maintain. Hence, the training process becomes unstable, requiring an elaborate adaptation rate \(\lambda\) tuning Shen et al. (2018); Shah et al. (2018); Du et al. (2020). Furthermore, the encoder could learn trivial solutions Karouzos et al. (2021) which produce features with flipped domain predictions.

\begin{table}
\begin{tabular}{l|c c c c|c} \hline \hline Domain & \multicolumn{3}{c|}{\#Neg} & \#Neu & \#Pos & Total \\ \hline \multirow{2}{*}{Book} & Set 1 & 2000 & 2000 & 2000 & 6000 \\  & Set 2 & 513 & 663 & 4824 & 6000 \\ \hline \multirow{2}{*}{Electronics} & Set 1 & 2000 & 2000 & 2000 & 6000 \\  & Set 2 & 694 & 489 & 4817 & 6000 \\ \hline \multirow{2}{*}{Beauty} & Set 1 & 2000 & 2000 & 2000 & 6000 \\  & Set 2 & 616 & 675 & 4709 & 6000 \\ \hline Music & Set 1 & 2000 & 2000 & 2000 & 6000 \\  & Set 2 & 785 & 774 & 4441 & 6000 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Amazon review dataset

\begin{table}
\begin{tabular}{l|c c} \hline \hline Domains & Labeled & Unlabeled \\ \hline Book & 2000 & 973194 \\ \hline Dvd & 2000 & 122438 \\ \hline Electronics & 2000 & 21009 \\ \hline Kitchen \& housewares & 2000 & 17856 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Amazon Benchmark statistics