Year: 2021

Abstract:
Domain adaptation is an important problem and often needed for real-world applications. In this problem, instead of i.i.d. training and testing datapoints, we assume that the source (training) data and the target (testing) data have different distributions. With that setting, the empirical risk minimization training procedure often does not perform well, since it does not account for the change in the distribution. A common approach in the domain adaptation literature is to learn a representation of the input that has the same (marginal) distribution over the source and the target domain. However, these approaches often require additional networks and/or optimizing an adversarial (minimax) objective, which can be very expensive or unstable in practice. To improve upon these marginal alignment techniques, in this paper, we first derive a generalization bound for the target loss based on the training loss and the reverse Kullback-Leibler (KL) divergence between the source and the target representation distributions. Based on this bound, we derive an algorithm that minimizes the KL term to obtain a better generalization to the target domain. We show that with a probabilistic representation network, the KL term can be estimated efficiently via minibatch samples without any additional network or a minimax objective. This leads to a theoretically sound alignment method which is also very efficient and stable in practice. Experimental results also suggest that our method outperforms other representation-alignment approaches.

heading: 1 INTRODUCTION
text: With advances in neural network architectures (He et al., 2016; Vaswani et al., 2017), machine learning algorithms have achieved state-of-the-art performance in many tasks such as object classification, object detection and natural language processing. However, machine learning models have been focusing mostly on the case of independent and identically distributed (i.i.d.) datapoints; and such an assumption often does not hold in practice. When the i.i.d. assumption is violated and the target domain has a different distribution compared to the source domain, a typical learner trained on the source data via empirical risk minimization would not perform well at test time, since it does not account for the distribution shift. To tackle this problem, many methods have been proposed for domain adaptation (Zhao et al., 2019; Zhang et al., 2019; Combes et al., 2020; Tanwani, 2020) and domain generalization (Khosla et al., 2012; Muandet et al., 2013; Ghifary et al., 2015), the goal of which is to train a machine learning algorithm that can generalize well to the target domain.
A common approach to tackle these problems is to learn a representation such that its distribution does not change across domains. There are two types of distribution alignment: marginal alignment (aligning the marginal distribution of the representation) and conditional alignment (aligning the conditional distribution of the label given the representation) (Nguyen et al., 2021; Tanwani, 2020). For domain adaptation and domain generalization problems with multiple source domains, we can use the data and labels to align both the marginal and the conditional distributions across the source domains, aiming to generalize to the target domain. However, in a single-source domain adaptation problem, with only unlabeled data from the target domain, it is often only possible to align the marginal distribution of the representation. This marginal alignment should help the classifier avoid out-of-distribution data at test time.
This paper focuses on such a single-source domain adaption problem, which is also one of the most common settings in practice. Current marginal alignment techniques usually require additional computation (e.g., of an additional network) (Ganin et al., 2016; Li et al., 2018) and/or a minimax objective (Ganin et al., 2016; Shen et al., 2018), leading to an expensive and/or unstable
training procedure (Goodfellow, 2016; Kodali et al., 2017). For example, DANN (Ganin et al., 2016) employs an adversarial training procedure, with a domain discriminator that classifies the domain of the representation, and maximizes the adversarial loss of the discriminator. When the discriminator is completely fooled, the marginal distribution of the representation is aligned across domains. MMD (Gretton et al., 2012; Li et al., 2018) utilizes maximum mean discrepancy to align the representation distribution. This does not use a mimimax objective, thus leading to a more stable training; however, it does require additional computation of several Gaussian kernels. While more sophisticated (non-marginal-alignment) methods have been proposed recently to achieve better results in the domain adaptation problem, we argue that studying the family of plain marginal-alignment techniques is still an important task, since they are the backbones that most other domain adaptation (and domain generalization) methods are built upon.
To address the above issues of existing marginal-alignment techniques, we first derive a generalization bound on the loss of the target domain using the training loss and a reverse Kullback–Leibler (KL) divergence between the source and target distributions. There are existing bounds of the target loss in the literature (Ben-David et al., 2010), however, these analyses focus mostly on the case of binary classification and the bounds use a total variation distance or a H-divergence between the distributions, which are not easy to estimate in practice (for example, Ajakan et al. (2014) require an adversarial network to estimate the H-divergence). In this paper, we show that with a probabilistic representation network, we can estimate the KL divergence easily using samples, leading to an alignment method that requires virtually no additional computation nor a minimax objective. Therefore, our training procedure is simple and stable in practice. Moreover, the reverse KL has the zero-forcing effect (Minka et al., 2005), which is very effective to alleviate the out-of-distribution problem in practice. This can be explained as follows: the out-of-distribution problem arises when the classifier faces a new representation at test time that is in a (near) zero mass region of the source representation distribution (and thus it never faced before). The reverse KL tends to force the target representation distribution to have (near) zero mass wherever the source distribution has (near) zero mass (this is the zero-forcing property), which helps the classifier avoid out-of-distribution data. The reverse KL also has the mode-seeking effect (Minka et al., 2005) which allows for a more flexible alignment of the representation (to one or some of the modes of the source domain). For example, consider the classification problem of buildings (houses, hotels, etc.) where source images are collected from urban and remote areas of a country (two modes); while the target images are collected from urban areas but from a different country. Ideally, we want to match the representation distribution of the target domain to that of the first mode of the source domain since they are both from urban areas. The reverse KL allows this flexible alignment (as it results in a relatively small value of the reverse KL) due to its mode-seeking property. Meanwhile, other distance metrics/divergences aim to match the whole source and target representation distribution, which might collapse the two modes of the source domains.
Our contributions in this work are:
• We construct a generalization bound of the test loss in the domain adaptation problem using the reverse KL divergence.
• We propose to reduce the generalization bound by minimizing the above KL term. Furthermore, we show that with a probabilistic representation, the KL term can be estimated easily using minibatches, without any additional computation or a minimax objective as opposed to most existing works.
• We conduct extensive experiments and show that our method significantly outperforms relevant baselines, namely ERM (Bousquet et al., 2003), DANN (Ganin et al., 2016), MMD (Gretton et al., 2012; Li et al., 2018), CORAL (Sun and Saenko, 2016) and WD (Shen et al., 2018). We empirically show that the reverse KL divergence is very effective for representation alignment since it is very stable and efficient to compute in practice.

heading: 2 RELATED WORK
text: Generalization bound for the distribution shift problem There exist works studying bounds for the distribution shift problem in the literature (Ben-David et al., 2010; Mansour et al., 2009). However, their analyses of the classification problem are limited to the case of binary labels. Moreover, these bounds are only applicable or practical for deterministic labeling functions, which is not the
case for most datasets in practice. Therefore, their analyses cannot be generalized to the general case of supervised learning. The differences between our bound and theirs are as follows. First of all, our bound works for the general case of supervised learning: it works for both the classification (including multiclass classification) and regression problems, it makes no assumptions about the labeling mechanism (can be probabilistic or deterministic), and it works for virtually all predictive distributions commonly used in practice. Secondly, our bound uses a different divergence, namely the KL divergence, which is easier to estimate in practice compared to total variation or H-divergence. We provide a brief review of the above bounds and discuss their differences to ours in more detail in the appendix. Recently, Acuna et al. (2021) revise the previous domain adaptation bounds and generalize them to a multi-class classification setting, as well as to the class of f -divergence (including KL divergence). However, their setting is still very restricted: the loss function needs to satisfy the triangle inequality (which does not hold for many loss functions in practice), and they need to know the true labeling function (optimal Bayes classifier) for a probabilistic labeling mechanism (which is often not available). We also provide further analysis, that under some reasonable assumptions, the conditional misalignment in the representation space is bounded by the conditional misalignment in the input space, which allows for a sound marginal alignment method. Some specific cases of distribution shift have also been studied. For example, Cortes et al. (2010) and Johansson et al. (2019) study the generalization bound for the covariate shift problem, i.e., pT (x) 6= pS(x) but pT (y|x) = pS(y|x), where pS is the source distribution and pT is the target distribution. In contrast, Azizzadenesheli et al. (2019) provide a generalization bound for the label shift problem, i.e., pT (y) 6= pS(y) but pT (x|y) = pS(x|y).
Domain adaptation While the literature on the domain adaptation problem is vast, we cover the most closely related works to ours here. A common method for the domain adaptation problem is to align the marginal distribution of the representation between the source and target domains. DANN (Ganin et al., 2016) employs a domain discriminator to classify the domain of a representation and maximizes its adversarial loss (a minimax game). WD (Shen et al., 2018) uses a neural network function f (which is 1-Lipschitz continuous) to calculate the Wasserstein distance between two distributions and minimizes it. This is also a minimax game since the Wasserstein distance is the supremum over the search space of f . MMD (Gretton et al., 2012; Li et al., 2018) uses the maximum mean discrepancy to align the representation distribution. This method does not need a minimax objective; however, it requires additional computation of several Gaussian kernels. Finally, CORAL (Sun and Saenko, 2016) matches the first two moments of the distribution; and while being a simple method, it fails to align more complex distributions. We consider these marginal alignment techniques our main baselines since our method falls into this category, and investigate the effectiveness of the reverse KL divergence in aligning the distributions of representation. Recently, more sophisticated alignment methods (Kang et al., 2019; Xu et al., 2019; Zhu et al., 2020) have been proposed for the domain adaptation problem, which achieve state-of-the-art performance. Instead of simply aligning the marginal distribution of the representation, these methods minimize the intra-class distance of the representation across domains, and possibly maximize the inter-class distance between them, using the MMD or L2 distance. However, they require pseudo labels for the target domain (often obtained via clustering). Moreover, they are complimentary to our method, as we conjecture that our method can also be used in conjunction with these, leading to the same algorithms but with the KL distance instead of MMD or L2.

heading: 3 APPROACH

heading: 3.1 PROBLEM STATEMENT
text: In this paper, we consider one of the most common domain adaptation settings, which consists of a single-source domain S with the joint data distribution pS(x, y) and a target domain T with the data distribution pT (x, y), where x denotes the input sample and y is the label. We assume that these two domains have the same support sets X ,Y . Regarding the training process of the domain adaptation problem, we further denote a labeled dataset of size NS sampled from the source domain (x
(i) S , y (i) S ) NS i=1, where (x (i) S , y (i) S ) ∼ pS(x, y), and an unlabeled dataset of size NT from the target
domain (x(i)T ) NT i=1, where x (i) T ∼ pT (x).
The goal of a typical domain adaptation framework is to train a model with the labeled dataset of the source domain together with the unlabeled dataset from the target domain, so that the model will
perform decently in the target domain. Note that this is only effective if the labeling mechanism is not too different between the source and the target domains (Ben-David et al., 2010).
In the domain adaption problem, we expect the changes in the marginal distribution so that pS(x) 6= pT (x), or the conditional distribution so that pS(y|x) 6= pT (y|x), or both, which often render the typical empirical risk minimization training procedure ineffective. This motivates a line of approaches that learn a representation z of x whose marginal and conditional distributions are more aligned across the domains and use it for the prediction task, aiming at a better generalization performance to the target domain.
The general representation learning framework aims to learn a representation z from x with the mapping p(z|x), which can be deterministic or probabilistic. That latent representation z is expected to contain the label-related information; and is then used to predict the label y (by a classifier). Note that since the source and target domains have the same support set for x and share the representation mapping p(z|x), they also have the same support set for z, denoted by Z . Given the representation z, we learn a classifier to predict y throuh the predictive distribution p̂(y|z) that is an approximation of the ground truth conditional distribution pS(y|z). During training, the representation network p(z|x) and the classifier p̂(y|z) are trained jointly on the source domain and we “hope” that they can generalize to the target domain, meaning that both p(z|x) and p̂(y|z) are kept unchanged between training and testing. The graphical model of that representation learning process is represented in Figure 1. In this paper, we consider a probabilistic representation mapping; specifically, the representation network will output µ(x) and σ2(x) and p(z|x) = N (z;µ(x), diag(σ2(x))), where N denotes a Gaussian distribution. This can also be thought of as a generalization of a deterministic representation, as we recover the deterministic case if σ2(x)→ 0. Also note that our method is not limited to the choice of this representation distribution, i.e., the discussion in this section holds for virtually any other distribution.
The joint distributions of x, y, z for the source and target domains can be represented as follows
pS(x, y, z) = pS(x, y)p(z|x) , pT (x, y, z) = pT (x, y)p(z|x) . (1)
and we define the predictive distribution of y given x as
p̂(y|x) = Ep(z|x)[p̂(y|z)] . (2) Remark 1. On the inference complexity of a probabilistic representation.
Using a probabilistic representation, we need to sample multiple z from p(z|x) to estimate Eq. 2 during test time. However, this is not a big issue for the representation learning framework, since we only need to run the representation network p(z|x) (which is usually deep) once to get a distribution of z. After sampling multiple z from that distribution, we only need to rerun the classifier p̂(y|z), which is usually a small network (e.g., often contains one layer). Furthermore, we can also run p̂(y|z) (a small network) in parallel for multiple z to reduce inference time if necessary. During training, we usually sample a single z from p(z|x) for each x. The training objective is
ltrain = Ex,y∼pS(x,y),z∼p(z|x)[− log p̂(y|z)] (3) (this is also the upper bound of EpS(x,y)[− log p̂(y|x)] via Jensen Inequality)
= EpS(z,y)[− log p̂(y|z)] (4)
where − log p̂(y|z) is the loss of a “data point” (z, y). For common choices of the predictive distribution in the classification and regression problems, this is a non-negative quantity. For example,
for a classification problem with a categorical predictive distribution, this becomes the cross-entropy loss, while for a regression problem with a Gaussian predictive distribution (with a fixed variance), it becomes the squared error (with an additive constant).
Minimizing ltrain will enforce p̂(y|z) ≈ pS(y|z). We consider the below two assumptions of the representation z on the source domain: Assumption 1. IS(z, y) = IS(x, y), where IS(·, ·) is the mutual information term, calculated on the source domain. In particular:
IS(z, y) = EpS(z,y) [ log pS(z, y)
pS(z)pS(y)
] ; IS(x, y) = EpS(x,y) [ log pS(x, y)
pS(x)pS(y)
] (5)
This is often referred to as the “sufficiency assumption” since it indicates that the representation z has the same information about the label y as the original input x, and is sufficient for this prediction task (in the source domain). Note that the data processing inequality indicates that IS(z, y) ≤ IS(x, y), so here we assume that z contains maximum information about y. Remark 2. Assumption 1 is an optimization goal of the training process on the source domain.
In particular, ltrain (with an additive constant) is an upper bound of −IS(z, y), which is an upper bound of −IS(x, y). Thus, minimizing ltrain will enforce IS(z, y) to be equal to IS(x, y). For a more detailed discussion of this, please refer to, for example, Alemi et al. (2016). Assumption 2. pS(y|x) = Ep(z|x)[pS(y|z)] ∀x, y ∈ X ,Y When this assumption holds, the predictive distribution in Eq. 2 will approximate pS(y|x), as long as p̂(y|z) approximates pS(y|z). Remark 3. Assumption 2 is also an optimization goal of the training process on the source domain.
This is because ltrain is an upper bound of EpS(x,y)[− log p̂(y|x)], which is an upper bound of EpS(x,y)[− log pS(y|x)]. Thus, minimizing ltrain will enforce p̂(y|x) to be equal to pS(y|x). Therefore, pS(y|x) ≈ p̂(y|x) = Ep(z|x)[p̂(y|z)] ≈ Ep(z|x)[pS(y|z)]. These two assumptions ensure that our network has good performance on the source domain. Note also that we only make the above two assumptions about the source domain, where we can enforce them through the training process. We do not make these assumptions about the target domain, since we have no access to the full target distribution. These two assumptions will also be used to prove our later theoretical result (Proposition 2).

heading: 3.2 KL GUIDED DOMAIN ADAPTATION
text: Now we will consider the test loss in the domain adaptation problem, and how we can reduce it. The test loss (of the target domain) is:
ltest = EpT (x,y)[− log p̂(y|x)] = EpT (x,y)[− logEp(z|x)[p̂(y|z)]] (6) ≤ EpT (x,y)[Ep(z|x)[− log p̂(y|z)]] (Jensen Inequality) (7) = EpT (z,y)[− log p̂(y|z)] (8)
Note that if the representation z is invariant (both marginally and conditionally), then pT (z, y) = pS(z, y) and Eq. 8 becomes ltrain, and we have a perfect generalization between the source domain and the target domain. However, there is no way to guarantee the invariance, since we do not know the target domain and the target data distribution. In that case, we introduce the following proposition that ensures a generalization bound of the test loss based on the training loss and the KL divergence: Proposition 1. If the loss − log p̂(y|z) is bounded by M 1 ∀z ∈ Z, y ∈ Y , we have:
ltest ≤ ltrain + M√
2
√ KL[pT (y, z)|pS(y, z)] (9)
= ltrain + M√
2
√ KL[pT (z)|pS(z)] + EpT (z) [KL[pT (y|z)|pS(y|z)]] (10)
1In the classification problem, we can enforce this quite easily by augmenting the output softmax of the classifier so that each class probability is always at least exp (−M). For example, if we choose M = 3 ⇒ exp (−M) ≈ 0.05, and if the output softmax is (p1, p2, ..., pC), we can augment it into (p1 ·K + 0.05, p2 · K+0.05, ..., pC ·K+0.05), whereK = 1−0.05 ·C and C is the number of classes. This ensures the bound for the loss of a datapoint, while remaining the output prediction class.
Proof. provided in the appendix.
This bound is similar to other bounds in the literature (e.g., Ben-David et al. (2010)) in the sense that it also contains the training loss, a marginal misalignment term and a conditional misalignment term (KL[pT (z)|pS(z)] and EpT (z) [KL[pT (z)|pS(z)]] respectively in our case). However, BenDavid et al. (2010) consider a binary classification problem and their bounds are only practical for a deterministic labeling function (requires knowing the true labeling function, which is unknown for a probabilistic labeling mechanism, to compute the bound); while our bound works for the general case of supervised learning with any labeling mechanism. For a brief review of these bounds and a detailed discussion about their differences to ours, please refer to the appendix. Note that the bound in Proposition 1 is also true when applying to the input space directly (e.g., replacing z with x). However, we are more interested in the bound in the representation space, since we can reduce it by regularizing the KL term.
To reduce the generalization gap, we want pT (z, y) to be close to pS(z, y). Aligning the marginal distribution (i.e., pS(z) ≈ pT (z)) helps the classifier network p̂(y|z) avoid out-of-distribution data since the target representations it faces at test time belong to the source representation distribution which it was trained on; while aligning the conditional distribution (pS(y|z) ≈ pT (y|z)) makes sure the classifier gives more accurate predictions on the target domain since p̂(y|z) was trained to approximate pS(y|z). In the domain adaptation problem, since we only have the unlabeled data from the target domain, we often align the marginal distribution of z only. However, one problem is that the conditional misalignment also depends on the representation z, and when learning a representation z that aligns the marginal, we might accidentally increase EpT (z) [KL[pT (y|z)|pS(y|z)]] at the same time, leading to a net increase in the above generalization bound. For example, what if (and is it possible that) the conditional misalignment increases to infinity while we learn a representation z?
Therefore, it is crucial that we can bound the above conditional misalignment. The below proposition handles this problem.
Proposition 2. If Assumption 1 and 2 hold, and if pT (x,y)pS(x,y) < ∞ (i.e., there exists N, which can be arbitrarily large, such that pT (x,y)pS(x,y) < N ∀x ∈ X , y ∈ Y), we have:
EpT (z) [KL[pT (y|z)|pS(y|z)]] ≤ EpT (x) [KL[pT (y|x)|pS(y|x)]] (11)
Proof. provided in the appendix.
This shows that the conditional misalignment in the representation space is bounded by the conditional misalignment in the input space. This can also be viewed as an improvement over the analyses of Ben-David et al. (2010), where it is not clear if the conditional misalignment in the representation space is bounded or not. It then follows that:
ltest ≤ ltrain + M√
2
√ KL[pT (z)|pS(z)] + EpT (x) [KL[pT (y|x)|pS(y|x)]]. (12)
As mentioned earlier, in order for domain adaptation to be effective, we should expect that the labeling mechanism does not change too much (Ben-David et al., 2010). Thus, the conditional misalignment EpT (x) [KL[pT (y|x)|pS(y|x)]] is often small (and fixed – not dependent on the representation z). Therefore, to reduce the generalization bound, we can focus on minimizing KL[pT (z)|pS(z)], with the objective: ltrain + βKL[pT (z)|pS(z)] (13) where β is a hyper-parameter.
Discussion on the use of reverse KL: Our derivation leads to the reverse KL term KL[pT (z)|pS(z)] as a regularizer of the distance between the two domains representations. We argue that there are several reasons that make this a good choice as a divergence between the source and target representation distributions. (1) First of all, as mentioned earlier, the KL term can be computed easily without any additional network or a minimax objective (details in Subsection 3.3). This leads to an efficient and stable training procedure, which often results in improved performance. (2) Secondly, the reverse KL has the zero-forcing/mode-seeking effect (Minka et al., 2005) that helps to
alleviate the out-of-distribution problem. Specifically, the reverse KL forces the target representation distribution to have zero mass wherever the source distribution has zero mass (zero-forcing), thus preventing the out-of-distribution data at test time (Figure 2c). On the other hand, its mode-seeking nature allows flexible alignment of the representation. For example, consider the case where the source domain is a mixture of two components (Figure 2a, i.e., it has two modes), and the target distribution is close to one of the two components. Ideally, we want to learn a representation network that matches the representation of the target domain to that of the corresponding component on the source mixture (Figure 2b). This representation will still perform well at test time since we would not have the out-of-distribution problem (the classification network is already trained on this mode of the source distribution). This flexible alignment (to one or some of the modes) is accepted by the reverse KL since it leads to a relatively small reverse KL value. Meanwhile, other methods such as DANN, MMD, CORAL and WD aim to match the representation distribution of the target domain and that of the whole source domain together, which could compress the representation too much, negatively affecting its expressive power. For instance, in the above example, trying to match the whole distribution of source and target domains based on other distance metrics might force the two modes of the source domain to collapse. The flexible alignment of the reverse KL (while still being very effective to prevent out-of-distribution data) might be beneficial in some practical cases.
We also empirically found that adding an auxiliary term KL[pS(z)|pT (z)] (forward KL) with a small coefficient βaux to the objective can help to align the distribution faster, leading to the objective:
ltrain + βKL[pT (z)|pS(z)] + βauxKL[pS(z)|pT (z)] (14)
In practice, setting βaux to a (very) small value or zero often leads to the best results (using larger βaux can hurt the performance). Note that this does not invalidate our earlier discussion, as with such small values of βaux, the alignment behavior is still dominated by the reverse KL. Furthermore, the forward KL KL[pS(z)|pT (z)] is non-negative, so this is an upper bound of 13.

heading: 3.3 OPTIMIZATION
text: In practice, we estimate Eq. 14 using minibatches. In particular, given a labelled minibatch (x̃
(i) S , ỹ (i) S ) B i=1 of the source domain and an unlabelled one (x̃ (i) T ) B i=1 of the target domain, and a
single sampled representation for each x: (z̃(i)S ) B i=1 and (z̃ (i) T ) B i=1, we can get an unbiased estimator of the objective 14 as follows:
ltrain + βKL[pT (z)|pS(z)] + βauxKL[pS(z)|pT (z)] =EpS(z,y)[− log p̂(y|z)] + βEpT (z)[log pT (z)− log pS(z)] + βauxEpS(z)[log pS(z)− log pT (z)]
≈ 1 B B∑ i=1 − log p̂(ỹ(i)S |z̃ (i) S ) + β 1 B B∑ i=1 [ log pT (z̃ (i) T )− log pS(z̃ (i) T ) ]
+ βaux 1
B B∑ i=1 [ log pS(z̃ (i) S )− log pT (z̃ (i) S ) ]
(15)
However, it still requires knowing pS(z) and pT (z) to compute Eq. 15. We also use the minibatch to approximate these quantities:
pS(z) = EpS(x)[p(z|x)] ≈ 1
B B∑ i=1 p(z|x(i)S ); pT (z) = EpT (x)[p(z|x)] ≈ 1 B B∑ i=1 p(z|x(i)T ). (16)
Intuitively, we use a minibatch of data to construct a distribution of the representation z (which is a mixture of B components), and match that distribution for the two domains with the KL divergence. As mentioned earlier, we use a Gaussian distribution with a diagonal covariance matrix for p(z|x) in practice, and employ the reparameterization trick (Kingma and Welling, 2013) to sample z.
Although the estimator in Eq. 15 is unbiased, the approximations in Eq. 16 will introduce some bias into our estimator (however, the estimator is still consistent, i.e., it becomes exact when B → ∞). Therefore, the batch size might have an effect on the performance of the model. However, via an ablation study, we found that the effect of this bias estimator is not severe in practice, and our model achieve good performance even with a batch size of 64. For detailed results and discussion of this ablation study, please refer the appendix C.3.

heading: 4 EXPERIMENTS

heading: 4.1 DATASETS
text: RotatedMNIST consists of 70,000 MNIST (LeCun et al., 2010) images that are divided into six domains, each with 11,666 images. The images in each domain are rotated counterclockwise by 0◦, 15◦, 30◦, 45◦, 60◦ and 75◦ respectively. We denote the six domains as M0,M15,M30,M45,M60 andM75. We useM0 as the source domain, and perform five experiments, each withM15,M30,M45,M60 orM75 as the target domain. The task is classification of the ten digit labels.
DIGITS is a common domain adaptation dataset, with 3 digit classification sub-datasets, namely MNIST, USPS (Hull, 1994) and SVHN (Netzer et al., 2011). Three common adaptation experiments are MNIST→ USPS, USPS→MNIST and SVHN→MNIST. VisDA17 (Peng et al., 2017) is a challenging realworld classification dataset with a simulation-toreal adaptation task. This dataset contains over 280K images from 12 classes. The source domain contains rendering of 3D models, while the target domain contains real images.
Please refer to the appendix for results of datasets with more domains such as PACS (Li et al., 2017).

heading: 4.2 BASELINES
text: We consider all common marginal alignment methods for domain adaptation as our baselines, including DANN, MMD, CORAL and WD. We also consider ERM (empirical risk minimization) and its variant ERM (prob) (same as ERM but with the probabilistic representation network used in our model). For ERM, DANN, MMD and CORAL, we follow the implementation by Gulrajani and Lopez-Paz (2020); while for ERM (prob) and WD, we use our own implementation in Pytorch (Paszke et al., 2019). Note that we do not include methods that are not from the marginal-alignment family since they are out of the scope of this paper. For the full description of these baselines, please refer to our appendix.

heading: 4.3 EXPERIMENTAL SETTING
text: For the RotatedMNIST and DIGITS experiments, we use a simple convolutional neural network with four 3×3 convolutional layers (followed by an average pooling layer) as the representation network. For VisDA17, we use a Resnet50 as the representation network. Only the last layer of the representation network differs for a deterministic representation (ERM, DANN, CORAL, MMD, WD) and a probabilistic one (ERM (prob) and KL (ours)). For a representation of size dz , the last layer’s dimension of a deterministic representation network is dz , while that of a probabilistic network is 2 · dz (dz for µ and dz for σ2). Please refer to the appendix for the detailed experimental setting (including data split, hyper-parameter tuning for each model, evaluation protocol, etc.)

heading: 4.4 RESULTS
text: RotatedMNIST and DIGITS: Table 1 and Table 2 show the results for the RotatedMNIST and DIGITS experiments. It is clear that in these experiments, aligning the representation between domains does help improve the generalization performance. Among the baselines (DANN, MMD, CORAL, WD), MMD performs the best, which we attribute to the fact that it does not use a minimax objective, leading to more stable optimization. Meanwhile, CORAL performs the worst, since it only matches the first two moments of the distributions and might fail to align complex distributions. Our method, KL, largely outperforms the baselines, indicating its effectiveness.
VisDA17 (Table 2): In this challenging dataset, many marginal alignment techniques (CORAL, WD) fail the adaptation task (achieve similar accuracy as the ERM baselines). MMD and KL (ours) are again the best performers, confirming that a stable training objective is beneficial in practice. Our method outperforms all other marginal-alignment approaches significantly, suggesting the effectiveness of the KL divergence in representation alignment.

heading: 5 CONCLUSION
text: In conclusion, in this paper we derive a generalization bound of the target loss in the domain adaptation problem using the reverse KL divergence. We then show that with a probabilistic representation, the KL divergence can easily be estimated using Monte Carlo (minibatch) samples, without any additional computation or adversarial objective. By minimizing the KL divergence, we can reduce the generalization bound and have a better guarantee about the test loss. We also empirically show that our method outperforms relevant baselines with large margins, which we attribute to its simple and stable training procedure and the mode-seeking/zero-forcing nature of the reverse KL. We conclude that KL divergence is very effective as a tool for representation alignment. In general, a limitation of marginal alignment methods (ours included) is that when the conditional distribution changes significantly from the source domain to the target domain, aligning the marginal would not help the target domain’s performance. This is also reflected in our generalization bound. For future work, we would want to investigate the use of KL divergence in other types of alignment. For example, we can follow the algorithm in Kang et al. (2019) to minimize the intra-class distance of the representation across domains and maximize the inter-class distance between them, but using the KL divergence instead of MMD as the distance between representation distributions. Another direction would be using KL divergence to align the conditional distribution across domains in a multi-source setting.

heading: A PROOFS
text: For the following proofs, we treat the variables as continuous variables and always use the integral. If one or some of the variables are discrete, it is straight-forward to replace the corresponding integral(s) with summation sign(s) and the proofs still hold.

heading: A.1 PROPOSITION 1
text: Proof. We have:
ltest ≤ EpT (z,y)[− log p̂(y|z)] (17)
= ∫ − log p̂(y|z)pT (z, y)dzdy (18)
= ∫ − log p̂(y|z)pS(z, y)dzdy + ∫ − log p̂(y|z) [pT (z, y)− pS(z, y)] dzdy (19)
= ltrain + ∫ − log p̂(y|z) [pT (z, y)− pS(z, y)] dzdy (20)
LetA = {(z, y)|pT (z, y)−pS(z, y) ≥ 0} and B = {(z, y)|pT (z, y)−pS(z, y) < 0}, using the fact that − log p̂(y|z) ≥ 0 ∀z ∈ Z, y ∈ Y we have:∫
− log p̂(y|z) [pT (z, y)− pS(z, y)] dzdy (21)
= ∫ A − log p̂(y|z) [pT (z, y)− pS(z, y)] dzdy + ∫ B − log p̂(y|z) [pT (z, y)− pS(z, y)] dzdy
(22) ≤ ∫ A − log p̂(y|z) [pT (z, y)− pS(z, y)] dzdy (23)
= ∫ A − log p̂(y|z) |pT (z, y)− pS(z, y)| dzdy (24)
≤M ∫ A |pT (z, y)− pS(z, y)| dzdy (25)
(since− log p̂(y|z) ≤M) (26)
where |.| is the absolute value. Here, ∫ A |pT (z, y)− pS(z, y)| dzdy is also called the total variation of the two distributions pT (z, y) and pS(z, y). Note that: ∫ pT (z, y)− pS(z, y)dzdy = 0 (27)
⇔ ∫ A pT (z, y)− pS(z, y)dzdy + ∫ B pT (z, y)− pS(z, y)dzdy = 0 (28)
⇔ ∫ A pT (z, y)− pS(z, y)dzdy = ∫ B pS(z, y)− pT (z, y)dzdy (29)
⇔ ∫ A |pT (z, y)− pS(z, y)| dzdy = ∫ B |pT (z, y)− pS(z, y)| dzdy (30)
⇔ ∫ A |pT (z, y)− pS(z, y)| dzdy = 1 2 ∫ |pT (z, y)− pS(z, y)| dzdy (31)
Therefore:
ltest ≤ ltrain +M ∫ A |pT (z, y)− pS(z, y)| dzdy (32)
= ltrain + M
2
∫ |pT (z, y)− pS(z, y)| dzdy (33)
Using the Pinsker’s inequality, we have:(∫ |pT (z, y)− pS(z, y)| dzdy )2 ≤ 2 ∫ pT (z, y) log pT (z, y)
pS(z, y) dzdy (34)
Therefore, we finally have:
ltest ≤ ltrain + M
2
√ 2 ∫ pT (z, y) log pT (z, y)
pS(z, y) dzdy (35)
= ltrain + M√
2
√ KL[pT (z, y)|pS(z, y)] (36)
Which concludes our proof.
Also note that the KL divergence between pT (z, y) and pS(z, y) can further be decomposed into the marginal misalignment and conditional misalignment as follow:
KL[pT (z, y)|pS(z, y)] = EpT (z,y)[log pT (z, y)− log pS(z, y)] (37) = EpT (z,y)[log pT (z) + log pT (y|z)− log pS(z)− log pS(y|z)] (38) = EpT (z,y)[log pT (z)− log pS(z)] + EpT (z,y)[log pT (y|z)− log pS(y|z)]
(39) = EpT (z)[log pT (z)− log pS(z)]
+ EpT (z) [ EpT (y|z)[log pT (y|z)− log pS(y|z)] ] (40)
= KL[pT (z)|pS(z)] + EpT (z) [KL[pT (y|z)|pS(y|z)]] (41)

heading: A.2 PROPOSITION 2
text: Proof. According to Assumption 1, we have:
IS(z, y) = IS(x, y) (42) ⇔HS(y)−HS(y|z) = HS(y)−HS(y|x) (43) ⇔HS(y|z) = HS(y|x) (44) ⇔EpS(z,y)[log pS(y|z)] = EpS(x,y)[log pS(y|x)] (45) ⇔EpS(x,z,y)[log pS(y|z)] = EpS(x,y)[log pS(y|x)] (46) ⇔EpS(x,y) [ Ep(z|x)[log pS(y|z)] ] = EpS(x,y)[log pS(y|x)] (47)
⇔EpS(x,y) [ log pS(y|x)− Ep(z|x)[log pS(y|z)] ] = 0 (48)
According to Assumption 2, ∀x ∈ X , y ∈ Y we have:
pS(y|x) = Ep(z|x)[pS(y|z)] (49) ⇔ log pS(y|x) = logEp(z|x)[pS(y|z)] (50) ⇒ log pS(y|x) ≥ Ep(z|x)[log pS(y|z)] (51)
Since pT (x,y)pS(x,y) <∞, there exists N > 0 such that pT (x,y) pS(x,y) ≤ N ∀x ∈ X , y ∈ Y . Therefore:
EpT (x,y) [ log pS(y|x)− Ep(z|x)[log pS(y|z)] ] (52)
=EpS(x,y) [( log pS(y|x)− Ep(z|x)[log pS(y|z)] ) pT (x, y) pS(x, y) ] (53)
≤N.EpS(x,y) [ log pS(y|x)− Ep(z|x)[log pS(y|z)] ] (54)
=0 (55)
Therefore:
EpT (x,y) [ log pS(y|x)− Ep(z|x)[log pS(y|z)] ] = 0 (56)
⇔EpT (x,y) [log pS(y|x)] = EpT (x,y,z)[log pS(y|z)] (57) ⇔EpT (x,y) [log pS(y|x)] = EpT (z,y)[log pS(y|z)] (58)
We have:
EpT (z) [KL[pT (y|z)|pS(y|z)]] ≤ EpT (x) [KL[pT (y|x)|pS(y|x)]] (59) ⇔EpT (z,y) [log pT (y|z)− log pS(y|z)] ≤ EpT (x,y) [log pT (y|x)− log pS(y|x)] (60)
Using Eq 58, we now only need to prove that:
EpT (z,y) [log pT (y|z)] ≤ EpT (x,y) [log pT (y|x)] (61) ⇔−HT (y|z) ≤ −HT (y|x) (62) ⇔HT (y)−HT (y|z) ≤ HT (y)−HT (y|x) (63) ⇔IT (z, y) ≤ IT (x, y) (64)
(always true based on the Data Processing Inequality) (65)

heading: B REVIEW OF EXISTING GENERALIZATION BOUNDS
text: There have been several works studying the generalization bounds of the Domain Adaptation problem. We briefly review the most important and common ones here with a discussion about their differences to our proposed bound.
B.1 BEN-DAVID ET AL. (2010)
Ben-David et al. (2010) consider a binary classification problem. Let x be the input with the support setX and y be the binary label with the support setY = {0, 1}. Consider a source domain with a distribution P sX over the input x and the true labeling function f
s : X → {0, 1}; and similarly a target domain with a distribution P tX over the input x and the true labeling function f
t : X → {0, 1}. Note that the authors claim that this labeling function can be probabilistic; in that case, f : X → [0, 1] denoting the probability. However, we argue that this probabilistic setting is impractical since we would not know that true underlying function in order to calculate the bounds in practice). Therefore, we found that the bound is only practical for the case of a deterministic labeling mechanism.
The error of the classifier h, which is also a deterministc labeling function, on the source domain is:
s(h) = Ex∼P sX [|h(x)− f s(x)|], (66)
and similarly for the target domain:
t(h) = Ex∼P tX [|h(x)− f t(x)|]. (67)
Here |.| is the absolute value, which mean the loss of a data point is the L1 distance of the labels. Consider a hypothesis space H and let a classifier h be any function from that space. The first theorem in Ben-David et al. (2010) offers a bound of the target loss t(h) based on the source loss
s(h), and the total variation between P sX and P t X , and the difference between the two labeling
function fs and f t:
Theorem 1 (Ben-David et al. (2010))
t(h) ≤ s(h) + 2d1(P sX , P tX) + min PX∈{P sX ,P tX} Ex∼PX [|fs(x)− f t(x)|] (68)
where d1(P sX , P t X) is the total variational distance, i.e., d1(P s X , P t X) := supA∈X [P s X(A)−P tX(A)], and X is the sigma-field of X (set of all subsets of X ). In this theorem, the term 2d1(P sX , P t X) presents the marginal misalignment and minPX∈{P sX ,P tX} Ex∼PX [|f s(x)− f t(x)|] is the conditional misalignment.
Ben-David et al. (2010) also propose another bound based on a variant of the H-divergence, which is presented in the following theorem:
Theorem 2 (Ben-David et al. (2010))
t(h) ≤ s(h) + dH∆H(P sX , P tX) + λH (69)
where the H∆H-divergence dH∆H(P sX , P tX) := suph1,h2∈H |Prx∼P sX [h1(x) 6= h2(x)] − Prx∼P tX [h1(x) 6= h2(x)]| replaces the total variation to measure the marginal misalignment of the two domains. Meanwhile, λH = infh∈H[ s(h) + t(h)] measures the conditional misalignment of the two domains (if the two true labeling functions fs and f t are the same and belong to the hypothesis spaceH, this quantity is zero). The above bounds can also be applied to the representation space (similar to ours), leading to the same bounds where the input x is replaced by its representation z. However, as mentioned in the main text, it is not clear if the conditional misalignment in the representation space (e.g., minPZ∈{P sZ ,P tZ} Ez∼PZ [|g s(z)− gt(z)|]) is bounded or not.
Difference to our bound First of all, Ben-David et al. (2010) only consider a binary classification problem. Moreover, as discussed above, the bounds in Ben-David et al. (2010) are only practical with deterministic labeling mechanism for both domains. This assumption is hard to be true for most datasets since the labeling mechanism is usually probabilistic. This makes it not generalizable to the general case of supervised learning. Furthermore, the loss function is a L1 distance between the labeling function, which is also not a common choice in practice, which makes it challenging to generalize to the multiclass classification set-up (even if the there exists a deterministic labeling function for a multiclass dataset, using the L1 loss for the one-hot encoded labels would be unreasonable; the common loss function in practice for the multiclass classification problem is the cross entropy loss). Finally, the total variation and H-divergence might be hard to estimate in practice since it requires the computation of a supremum.
B.2 MANSOUR ET AL. (2009)
Mansour et al. (2009) consider a more flexible problem set-up than Ben-David et al. (2010). Specifically, instead of Y = {0, 1}, they consider the cases where Y = {0, 1} (for binary classification) or Y is a measurable subset of R (for regression). Note that their bound still cannot work for multiclass classfication. They also generalize the L1 loss function to a loss function L : Y × Y → R; however, this loss function must obey the triangle inequality. Although the L1 distance satisfies this inequality, it is not generally true for other common loss functions in practice (e.g., cross-entropy). They still consider deterministic labeling function fs and f t for the source and target domain.
Similar to Ben-David et al. (2010), with a hypothesis h from the hypothesis space H, the error of the source and target domain are:
s(h) = Ex∼P sX [L(h(x), f s(x))] (70) t(h) = Ex∼P tX [L(h(x), f t(x))] (71)
For convenience, denote also the error between two labeling function h and h′ in the source and target distribution as:
s(h, h′) = Ex∼P sX [L(h(x), h ′(x))] (72) t(h, h′) = Ex∼P tX [L(h(x), h ′(x))] (73)
(which means s(h) = s(h, fs) and t(h) = t(h, f t)).
Also, let h∗s and h∗t be the minimizer of s(h) and t(h) respectively. In particular:
h∗s = arg min h∈H s(h) = arg min h∈H Ex∼P sX [L(h(x), f s(x))] (74) h∗t = arg min h∈H t(h) = arg min h∈H Ex∼P tX [L(h(x), f t(x))] (75)
Mansour et al. (2009) introduce a generalization bound as follow:
Theorem 3 (Mansour et al. (2009)) Assume that the loss function L is symmetric and obeys the triangle inequality. Then, for any hypothesis h ∈ H , the following holds
t(h) ≤ t(h∗t) + s(h, h∗s) + disc(P sX , P tX) + t(h∗s, h∗t) (76)
where disc(P sX , P t X) := suph,h′∈H | s(h, h′) − t(h, h′)|, which is a generalized version of the H∆H-divergence. Here, the first term t(h∗t) is the ideal target loss (will be zero if the hypothesis spaceH contains ft), the second term s(h, h∗s) will be zero if we choose h = h∗s (which is the common practice, e.g., train the classifier h on the source domain), the third term disc(P sX , P t X) measures the marginal misalignment, and the final term t(h∗s, h∗t) is somewhat an indicator of the conditional misalignment (becomes zero if fs = f t ∈ H).
Difference to our bound The above bound is based on the ideal target loss, while our bound is based on the source loss. In practice, we have (an estimate) of the source loss calculated on the source domain’s training set; meanwhile, the ideal target loss is unknown. This makes the above bound less useful in practice compared to ours. Furthermore, the above bound has similar problems as the ones in Ben-David et al. (2010): it does not work for multiclass classification, it assumes a deterministic labeling mechanism (which does not holde in practice), it assumes the loss function obeys the triangle inequality (which generally is not true in practice), and it contains terms that are not easy to compute in practice (supremum and infimum).

heading: C ADDITIONAL EXPERIMENTS

heading: C.1 PACS
text: Table 3 presents the results for PACS, which is a challenging real-world dataset for domain adaptaion/generalization. In this dataset, our model outperforms the ERM baselines by roughly 9% on average, indicating the effectiveness of our representation-alignment technique. Our method is the best performer (with a large margin) on 8 out of 12 experiments, showing a clear benefit over other representation alignment techniques. Together with our method, MMD again performs the best among the representation-alignment baselines (DANN, MMD, CORAL and WD), confirming that a stable training procedure (with no minimax objectives in MMD and our model) is important and often leads to better results. It is also worth noting that our model still outperforms MMD despite being less computationally expensive (in this implementation, MMD needs to compute seven Gaussian kernels for each of three pairs of representation sets in each minibatch).
It is interesting that the ERM baselines perform the best in some experiments (e.g., S→ C, S→ P). This result also agrees with the one observed in Gulrajani and Lopez-Paz (2020) that domain generalization/adaptation techniques might have negative effects when applied unsuccessfully. It should be noted that the S (sketch) domain is undoubtedly the most different compared to others (only black sketch on a white background while other domains have colors), which might explain the difficulty when learning to transfer between domains.
We also provide additional results on the Office-31 dataset Saenko et al. (2010). Table 4 presents the performance of our model and two of the best performing baselines, namely MMD and DANN. Although our method is not state-of-the-art in this dataset (and understandingly so), it significantly outperforms the relevant baselines considered in this paper.

heading: C.3 ABLATION STUDY: EFFECT OF BATCH SIXE
text: In this subsection, we conduct an ablation study to investigate the effect of the batch size on our model’s performance. Table 5 shows the performance of our method on the RotatedMNIST dataset, withM0 as the source domain andM45 as the target domain and with various choices of the batch size. As expected, our model’s performance tends to benefit from a bigger batch size, since it would alleviate the bias of our objective estimator. We therefore recommend increasing the batchsize whenever possible. However, our model performs well even for a batch size as small as 64 (which is considered small in this era of deep learning).

heading: C.4 ABLATION STUDY: EFFECT OF THE AUXILIARY FORWARD KL
text: In this subsection, we investigate the contribution of the auxiliary forward KL term, by considering a variant of our method without this term (βaux = 0). Table 6 and Table 7 show the results for this ablation experiment. Clearly, most of the improvement comes from the reverse KL regularizer term. And even without the auxiliary forward KL term, our results are still significantly higher than the baselines. Note that the auxiliary term requires virtually no extra computation, so we believe adding the auxiliary term is reasonable.

heading: D DETAILED EXPERIMENTAL SETTINGS
text: In each experiment, we split both the source and the target data into two portions: 80% and 20%. We use 80% of the source domain data and 80% of the target domain data (without the labels) as the training data. We use the remaining 20% of the source data as the validation set, and the remaining 20% of the target domain data as the test set. Note that we do not use the labeled data from the target domain during training or validation. This evaluation protocol is recommended by Gulrajani and Lopez-Paz (2020).

heading: D.1 BASELINES
text: ERM Bousquet et al. (2003) is the typical empirical risk minimization training procedure, meaning that the model is trained normally in the training data and does not account for the distribution shift (domain adaptation).
ERM (prob): since we use a probabilistic network, we also include the probabilistic version of ERM. This is similar to ERM but uses a probabilistic representation network (same as ours).
DANN Ganin et al. (2016) utilizes a discriminator to distinguish the representation from the source and target domains. It uses an adversarial loss to enforce the distributions of the representation from source domain and target domain are the same.
MMD Li et al. (2018) uses the maximum mean discrepancy (MMD) to align the representation’s distributions.
CORAL Sun and Saenko (2016) aligns the representation distributions of the source and target domains by matching their first two moments.
WD Shen et al. (2018) uses the Wasserstein distance to match the distribution of the representation.

heading: D.2 REPRESENTATION NETWORK USED IN ROTATEDMNIST AND DIGITS
text: We use a simple CNN as the representation network in this experiment. This network is exactly the same as the one used in Gulrajani and Lopez-Paz (2020).
Our code is in PyTorch. The network is constructed by the following layers, where output dim=256 for a deterministic representation network and output dim=512 for a probabilistic representation network (256 for µ and 256 for σ2):
- Conv2d(in channels=1,out channels=64,kernel size=3,stride=1,padding=1) - ReLU() - GroupNorm(num groups=8,num channels=64) - Conv2d(in channels=64,out channels=128,kernel size=3,stride=2,padding=1) - ReLU() - GroupNorm(num groups=8,num channels=128) - Conv2d(in channels=128,out channels=128,kernel size=3,stride=1,padding=1) - ReLU() - GroupNorm(num groups=8,num channels=128) - Conv2d(in channels=128,out channels=output dim,kernel size=3,stride=1,padding=1) - ReLU()
- GroupNorm(num groups=8,num channels=output dim) - AdaptiveAvgPool2d(output size=(1,1))

heading: D.3 HYPER-PARAMETERS TUNING
text: We train each model for 100 epochs. To avoid hyperparameter bias, we tune the hyperparameters (learning rate, regularizer coefficients, weight decay, representation dimension and dropout rate) for each method and dataset independently. Following Gulrajani and Lopez-Paz (2020), we perform a random search (Bergstra and Bengio, 2012) to tune hyperparameters for the baselines. We use the Adam optimizer Kingma and Ba (2014) for all the models. We re-run each set of hyperparameters three times. We train all models on an NVIDIA Quadro RTX 6000 GPU.
The readers can also refer to our source code for the experiment setting.
Belows are the hyper-parameters considered by the random search in our experiments for each baseline. {.} means a set of hyper-parameters considered, while [., .] means a range of hyper-parameters considered.

heading: D.3.1 ROTATEDMNIST AND DIGITS
text: The representation’s dimension is 128 (details in Section D.2). For even more details about the below hyper-parameters, please refer to our provided code.
- ERM: learning rate: [10−4.5, 10−2.5], weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number of layers of the classifier (p̂(y|z)): 1 or 3. - ERM (prob): learning rate: [10−4.5, 10−2.5], weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number of layers of the classifier (p̂(y|z)): 1 or 3. - DANN Ganin et al. (2016): learning rate: [10−4.5, 10−2.5], weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number of layers of the classifier (p̂(y|z)): 1 or 3, adversarial loss coefficient: [10−2, 102], weight decay of discriminator: [10−6, 10−2], number of discriminator steps per generator steps: {1, 2, 4, 8}, grad penalty coefficient: [10−2, 101] - MMD Li et al. (2018): learning rate: [10−4.5, 10−2.5], weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number of layers of the classifier (p̂(y|z)): 1 or 3, MMD coefficient: [10−3, 10−1] - CORAL Sun and Saenko (2016): learning rate: [10−4.5, 10−2.5], weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number of layers of the classifier (p̂(y|z)): 1 or 3, CORAL loss coefficient: [10−3, 10−1]
- WD Shen et al. (2018): learning rate: [10−4.5, 10−2.5], weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number of layers of the classifier (p̂(y|z)): 1 or 3, wasserstein distance coefficient: [10−2, 102], weight decay of network f : [10−6, 10−2], number of f optimization steps per normal optimization steps: {1, 2, 4, 8}, grad penalty coefficient: [10−2, 101] - KL (ours): learning rate: [10−4.5, 10−2.5], weight decay: 0.0, dropout rate: 0.0, batch size: 256, number of layers of the classifier (p̂(y|z)): 1, β : 0.3, βaux : 0.1
D.3.2 VISDA17 AND PACS
The representation network for VisDA17 is a Resnet50, while the one for PACS is a Resnet18. For even more details about the below hyper-parameters, please refer to our provided code.
- ERM: learning rate: [10−5, 10−3.5], weight decay: [10−6, 10−2], dropout rate: {0.0, 0.1, 0.5}, batch size: [8, 45], representation dimension: {16, 128, 256, 512}, number of layers of the classifier (p̂(y|z)): 1 or 3. - ERM (prob): learning rate: [10−5, 10−3.5], weight decay: [10−6, 10−2], dropout rate: {0.0, 0.1, 0.5}, batch size: [8, 45], representation dimension: {16, 128, 256, 512}, number of layers of the classifier (p̂(y|z)): 1 or 3. - DANN Ganin et al. (2016): learning rate: [10−5, 10−3.5], weight decay: [10−6, 10−2], dropout rate: {0.0, 0.1, 0.5}, batch size: [8, 45], representation dimension: {16, 128, 256, 512}, number of layers of the classifier (p̂(y|z)): 1 or 3, adversarial loss coefficient: [10−2, 102], weight decay of
discriminator: [10−6, 10−2], number of discriminator steps per generator steps: {1, 2, 4, 8}, grad penalty coefficient: [10−2, 101]
- MMD Li et al. (2018): learning rate: [10−5, 10−3.5], weight decay: [10−6, 10−2], dropout rate: {0.0, 0.1, 0.5}, batch size: [8, 45], representation dimension: {16, 128, 256, 512}, number of layers of the classifier (p̂(y|z)): 1 or 3, MMD coefficient: [10−3, 10−1] - CORAL Sun and Saenko (2016): learning rate: [10−5, 10−3.5], weight decay: [10−6, 10−2], dropout rate: {0.0, 0.1, 0.5}, batch size: [8, 45], representation dimension: {16, 128, 256, 512}, number of layers of the classifier (p̂(y|z)): 1 or 3, CORAL loss coefficient: [10−3, 10−1] - WD Shen et al. (2018): learning rate: [10−5, 10−3.5], weight decay: [10−6, 10−2], dropout rate: {0.0, 0.1, 0.5}, batch size: [8, 45], representation dimension: {16, 128, 256, 512}, number of layers of the classifier (p̂(y|z)): 1 or 3, wasserstein distance coefficient: [10−2, 102], weight decay of network f : [10−6, 10−2], number of f optimization steps per normal optimization steps: {1, 2, 4, 8}, grad penalty coefficient: [10−2, 101]
- KL (ours): learning rate: 10−4, weight decay: [10−6, 10−2], dropout rate: 0.0, batch size: 256, representation dimension: 16, number of layers of the classifier (p̂(y|z)): 1, β : {0.1, 0.05, 0.001}, βaux : {0.1, 0.05, 0.01, 0.0}

Review summaries:
this paper proposed a method for domain adaption. The idea and method are quite simple. Especially, KL divergence on representation alignment is used. And the experimental result looks promising on the selected task.

The paper proposes a domain adaptation method that is guided by the reverse Kullback-Leibler (KL) divergence between the target and source domains in the representation space. To this end, it is assumed that the representation z is sufficient and that p_S(y|x) = E_p(z|x)[p_S(y|z)]. The authors derive a bound on the test loss based on the training loss and a quantity based on the divergence of the joint distribution (y,z) between the target and source domain (Proposition 1). Since this bound requires labels from both domains, Proposition 2 helps to use the divergence computation on p_S(y|x) instead of p_S(y|z). The proposed method and optimization procedure are evaluated on multiple data sets and compared with multiple baselines. 

 In this paper, the authors propose a generalization bound for domain adaptation where discrepancy between distributions (i.e. marginals and class conditionals) is estimated using the reverse KL divergence. Inspired by this result, the authors then proposed a learning algorithm and show  that this outperforms some existing methods.