KL Guided Domain Adaptation
Authors: A. Tuan Nguyen
Toan Tran
Yarin Gal
Philip Torr
Atilim Gunes Baydin
Venue: ICLR 2022 Poster (2022)
Keywords: domain adaptation
invariant representation

Abstract: Domain adaptation is an important problem and often needed for real-world applications. In this problem, instead of i.i.d. training and testing datapoints, we assume that the source (training) data and the target (testing) data have different distributions. With that setting, the empirical risk minimization training procedure often does not perform well, since it does not account for the change in the distribution. A common approach in the domain adaptation literature is to learn a representation of the input that has the same (marginal) distribution over the source and the target domain. However, these approaches often require additional networks and/or optimizing an adversarial (minimax) objective, which can be very expensive or unstable in practice. To improve upon these marginal alignment techniques, in this paper, we first derive a generalization bound for the target loss based on the training loss and the reverse Kullback-Leibler (KL) divergence between the source and the target representation distributions. Based on this bound, we derive an algorithm that minimizes the KL term to obtain a better generalization to the target domain. We show that with a probabilistic representation network, the KL term can be estimated efficiently via minibatch samples without any additional network or a minimax objective. This leads to a theoretically sound alignment method which is also very efficient and stable in practice. Experimental results also suggest that our method outperforms other representation-alignment approaches.

Published as a conference paper at ICLR 2022
KL GUIDED DOMAIN ADAPTATION
A. Tuan Nguyen*
Toan Tran
Yarin Gal f
Philip H. S. Torr
Atilm Güneş Baydin
ABSTRACT
Domain adaptation is an important problem and often needed for real-world ap- plications. In this problem, instead of i..d. training and testing datapoints, we assume that the source (training) data and the target (testing) data have different distributions. With that setting, the empirical risk minimization training proce- dure often does not perform well, since it does not account for the change the distribution. A common approach in the domain adaptation literature is to learn representation of the input that has the same (marginal) distribution over the source and the target domain. However, these approaches often require ad- ditional networks and/or optimizing an adversarial (minimax) objective, which can be very expensive unstable in practice. To improve upon these marginal alignment techniques, in this paper, we first derive a generalization bound for the target loss based the training loss and the reverse gence between the source and the target representation distributions. Based on this bound, we derive an algorithm that minimizes the KL term to obtain better gen- eralization to the target domain. We show that with a probabilistic representation network, the KL term can be estimated efficiently via minibatch samples without any additional network alignment method which also very efficient and stable practice. Experimental results also suggest that our method outperforms other representation-aliinnn allimmmmn approaches.
Betllbacck-eibble (KL) diver-
minimax objective. This leads to a theoretically sound
INTRODUCTION
With advances in neural network architectures (He et al., 2016; Vaswani et al., 2017), machine learning algorithms have achieved otate-of-the-art performance in many tasks such as object classi- fication, object detection and natural language processing. However, machine learning models have been focusing mostly on the case of independent and identically distributed (ii.d.) datapoints; and such an assumption often does not hold in practice. When the i.i.d. assumption violated and the target domain has a different distribution compared to the source domain, a typical learner trained on the source data via empirical risk minimization would not perform well at test time, since it does not account for the distribution shift. To tackle this problem, many methods have been proposed for domain adaptation (Zhao et al., 2019; Zhang et al., 2019; Combes et al., 2020; Tanwani, 2020) and domain generalization (Khosla al., 2012; Muandet et al., 2013; Ghifary et al., 2015), the goal of which is to train a machine learning algorithm that can generalize well to the target domain.
A common approach to tackle these problems is to learn representation such that its distribution does not change across domains. There are two types of distribution alignment: marginal alignment (aligning the marginal distribution of the representation) and conditional alignment (aligning the conditional distribution of the label given the representation) (Nguyen et al., 2021; Tanwani, 2020). For domain adaptation and domain generalization problems with multiple source domains, we can the data and labels to align both the marginal and the conditional distributions across the source domains, aiming to generalize to the target domain. However, in a single-source domain adaptation problem, with only unlabeled data from the target domain, it is often only possible to align the marginal distribution of the representation. This marginal alignment should help the classifier avoid data at test time.
Coorrsppndiigg author: A. Tuan Nguyen, tuan@robots ox. ac.uk I University of Oxford, Oxford, United Kingdom I VinAl Research, Hanoi, Vietnam

---

Published as a conference paper at ICLR 2022
This paper focuses such single-source domain adaption problem, which is also one of the most common settings in practice. Current marginal alignment techniques usually require addi- tional computation (e.g.. of an additional network) (Ganin et al., 2016; Li et al., 2018) and/or a minimax objective (Ganin al., 2016; Shen al., 2018), leading an expensive and/or unstable training procedure (Goodfellow, 2016; Kodali et al., 2017). For example. DANN (Ganin et al., 2016) employs an adversarial training procedure, with domain discriminator that classifies the domain of the representation, and maximizes the adversarial loss of the discriminator. When the discrimina- tor is completely fooled, the marginal distribution of the representation is aligned across domains. MMD (Gretton al., 2012; Li et al., 2018) utilizes maximum mean discrepancy align the repre- sentation distribution. This does not use a minimax objective, thus leading to a more stable training; however, it does require additional computation of several Gaussian kernels. While more sophis- ticated (non- margimal-alignnaannen
methods have been proposed recently to achieve better results in the domain adaptation problem, we argue that studying the family of plain marginal -aligmee-ll tech- niques is still an important task, since they are the backbones that most other domain adaptation (and domain generalization) methods are built upon.
To address the above issues of existing marginal-alignmen techniques, we first derive gemeraliza- tion bound on the loss of the target domain using the training loss and a reverse Bullback-Leibbee (KL) divergence between the source and target distributions. There are existing bounds of the target loss in the literature (Ben-David al., 2010), however, these analyses focus mostly on the case of binary classification and the bounds use a total variation distance or a H-divergence between the distributions, which are not easy to estimate in practice (for example, Ajakan et al. (2014) require an adversarial network to estimate the H--divergence. In this paper, we show that with a proba- bilistic representation network, we can estimate the KL divergence easily using samples, leading to alignment method that requires virtually additional computation nor a minimax objective. Therefore, our training procedure is simple and stable in practice. Moreover, the reverse KL has the zero-foreing effect (Minka et al., 2005), which is very effective to alleviate the -utoof-distribution problem in practice. This can be explained as follows: the out-of disttiiibbuiio problem arises when the classifier faces a new representation at test time that is in a (near) zero mass region of the source representation distribution (and thus it never faced before). The reverse KL tends to force the target representation distribution to have (near) zero mass wherever the source distribution has (near) zero mass (this is the zero- zero-forcing property), ). which helps the classifier avoid out-of-distribution data. The reverse KL also has the mode-seeking effect (Minka et al.. 2005) which allows for a more flexible alignment of the representation (to one or some of the modes of the source domain). For example, consider the classification problem of buildings (houses, hotels, etc.) where source images col- lected from urban and remote areas of a country (two modes); while the target images are collected from urban areas but from different country. Ideally, we want to match the representation distri- bution of the target domain to that of the first mode of the source domain since they are both from urban areas. The reverse KI. allows this flexible alignment (as it results in a relatively small value property. Meanwhile, other distance metrics/divergences
of the reverse KI.) due to its aim to match the whole source and target representation distribution, which might collapse the two modes of the source domains.
Our contributions in this work are:
We construct a generalization bound of the test loss in the domain adaptation problem using the reverse KL divergence.
We propose to reduce the generalization bound by minimizing the above KL term. Further- more, we show that with a probabilistic representation, the KI. term can be estimated easily using minibatches, without any additional computation or a minimax objective as opposed to most existing works.
We conduct extensive experiments and show that our method significantly outperforms relevant basclines, namely ERM (Bousquet et al., 2003), DANN (Ganin et al., 2016), MMD (Gretton et al., 2012; Li et al., 2018), CORAL (Sun and Saenko, 2016) and WD (Shen et al., 2018). We empirically show that the reverse KL divergence is very effective for representation alignment since it is very stable and efficient compute in practice.
2 RELATED WORK
Generalization bound for the distribution shift problem There exist works studying bounds for the distribution shift problem in the literature (Ben-David al., 2010; Mansour et al., 2009). How-
2

---

Published as a conference paper at ICLR 2022
ever, their analyses of the classification problem are limited to the case of binary labels. Moreover, these bounds are only applicable or practical for deterministic labeling functions, which is not the case for most datasets in practice. Therefore, their analyses cannot be generalized to the general case of supervised learning. The differences between our bound and theirs are as follows. First of all, our bound works for the general case of supervised learning: it works for both the classification (includ- ing multiclass classification) and regression problems, it makes mechanism (can be probabilistic or deterministic), and it works for virtually all predictive distribu- tions commonly used in practice. Secondly, our bound uses a different divergence, namely the KL divergence, which is easier estimate in practice compared to total variation H-divergence, We provide a brief review of the above bounds and discuss their differences to ours in more detail in the appendix. Recently, Acuna et al. (2021) revise the previous domain adaptation bounds and gener- alize them to a multi-class classification setting, as well as to the class of f f-divergence (including KL divergence). However, their setting is still very restricted: the loss function needs to satisfy the triangle inequality (which does not hold for many loss functions in practice), and they need to know the true labeling function (optimal Bayes classifier) for a probabillstic labeling mechanism (which is often not available). We also provide further analysis, that under some reasonable assumptions, the conditional misalignment in the representation space is bounded by the conditional misalignment in the input space, which allows for sound marginal alignment method. This can be seen as an improvement over prior works. Some specific cases of distribution shift have also been studied. For example, Cortes et al. (2010) and Johansson et al. (2019) study the generalization bound for the covariate shift problem, i.c., pr(x) + ps(x) but (y))) ps (y)xr) where ps is the source distribution and pr is the target distribution. In contrast, Azizzadenesheli et al. (2019) provide a generalization bound for the label shift problem, i.e., (y) # (y but (xll)) = (s(l)y)
assumptions about the labeling
Domain adaptation While the literature on the domain adaptation problem is vast, we cover the most closely related works to ours here. A common method for the domain adaptation problem to align the marginal distribution of the representation between the source and target domains. DANN (Ganin et al.. 2016) employs a domain discriminator to classify the domain of a represen- tation and maximizes its adversarial loss minimax game). WD (Shen et al., 2018) uses neural network function f (which is I-Lipschitz continuous) to calculate the Wasserstein distance between two distributions and minimizes it. This is also a minimax game since the Wasserstein distance is the supremum over the search space of f. MMD (Gretton et al., 2012; Li et al., 2018) uses the maximum mean discrepancy to align the representation distribution. This method does not need a minimax objective; however, it requires the additional computation of several Gaussian kernels. Fi- nally, CORAL (Sun and Saenko, 2016) matches the first two moments of the distribution; and while being& simple method, it fails to align more complex distributions. We consider these marginal alignment techniques our main baselines since our method falls into this category, and investigate the effectiveness of the reverse KL divergence in aligning the distributions of representation. Re- cently, more sophisticated alignment methods (Kang et al., 2019; Xu et al., 2019; Zhu et al., 2020) have been proposed for the domain adaptation problem, which achieve state-of-the-an performance. Instead of simply aligning the marginal distribution of the representation, these methods minimize the intra-class distance of the representation across domains, and possibly maximize the inter-class distance between them, using the MMD or L2 distance. However, they require pseudo labels for the target domain (often obtained via clustering). Moreover, they are complementary to our method, as we conjecture that our method can also be used in conjunction with these, leading to the same algorithms but with the KL distance instead of MMD or L2.
APPROACH
3.1 PROBLEM STATEMENT
In this paper. we consider one of the most common domain adaptation settings, which consists of single-source domain S with the joint data distribution ps(x, y) and a target domain I' with the data distribution pr(z, y). where x denotes the input sample and is the label. We assume that these two domains have the same support sets X, Y. Regarding the training process of the domain adaptation problem, we further denote a labeled dataset of size Ns sampled from the source domain (8) (f) Ns
(x')) Us Jizp where (a (t) (6) ps( y)). and an unlabeled dataset of size Ny from the target
..yss
domain (xy NT where an ~ pr()).
The goal of typical domain adaptation framework is to train a model with the labeled dataset of the source domain together with the unlabeled dataset from the target domain, so that the model will
3

---

Published as a conference paper at ICLR 2022
Source Distribution
Target Distribution
Joint: Ps y, 2) = Ps (x,
Joint: p,(x,,2) = Pr (x, (2,)((2))
Figure I: Graphical model. Note that the distribution p(z|z) (green edge), corresponding to our representa- tion network, shared between the source and target domains.
perform decently in the target domain. Note that this is only effective if the labeling mechanism is not too different between the source and the target domains (Ben-David et al., 2010).
In the domain adaption problem, we expect the changes in the marginal distribution so that (x) * (x)) or the conditional distribution so that pyly) * Pr (y) or both, which often render the typical empirical risk minimization training procedure ineffective. This motivates a line of approaches that learn representation of whose marginal and conditional distributions more aligned across the domains and use it for the prediction task, aiming at better generalization performance to the target domain.
The general representation learning framework aims learn a representation z from with the mapping p(z|z)) which can be deterministic or probabilistic. That latent representation = is expected to contain the label-related information; and is then used to predict the label y (by classifier). Note that since the source and target domains have the same support set for x and share the representation mapping p(2[zr). they also have the same support set for x. denoted by Z. Given the representation 2, learn classifier predict y through the predictive distribution p(y =) that is an approximation of the ground truth conditional distribution ps(u]z). During training, the representation network p(=[z) and the classifier p(y|z) 2)) are trained jointly on the source domain and we "hope" that they can generalize to the target domain, meaning that both p(=[2r and p(y|=) are kept unchanged between training and testing. The graphical model of that representation learning process is represented in Figure 1. In this paper, we consider probabilistic representation mapping: specifically, the representation network will output (u(z) and (²) and p(= ()) = N N(2; (((r), diag(o (xx)), where N denotes Gaussian distribution. This can also be thought of as generalization of a deterministic representation, as we recover the deterministic case (x:) - 0. Also, note that our method not limited to the choice of this representation distribution, i.c., the discussion in this section holds for virtually any other distribution.
The joint distributions of x. y, z for the source and target domains can be represented as follows ps(x, y, =) ps(x, ()((((z))
pr(,,y, pT (z)(x)(((())
and we define the predictive distribution of given x as p(ylx) = [[vl]))
(2)
Remark 1. On the inference complexity of probabilistic representation.
Using a probabilistic representation, need to sample multiple from p(=[ær) to estimate Eq. during test time. However, this is not a big issue for the representation learning framework, since we only need to run the representation network p(2[ær (which is usually deep) once to get a distribution of z. After sampling multiple z from that distribution, we only need to rerun the classifier piy) =), which is usually a small network (c.g., often contains one layer). Furthermore, we can also run =(y]=) (a small network) in parallel for multiple reduce inference time if necessary.
During training, we usually sample a single z from p(2(xr) for each æ. The training objective is
Irain (r.a
log ((yl
(3)
(this is also the upper bound of E (pp log p(y [x) via Jensen Inequality)
= Eps((t.)) log p[y[z)]
(4)
where (og"p(y) 2) the loss of a "data point" (2, v). For common choices of the predictive dis- tribution in the classification and regression problems, this is non-negative quantity. For example, for classification problem with categorical predictive distribution, this becomes the cross-entropy
4

---

Published as a conference paper at ICLR 2022
loss, while for regression problem with a Gaussian predictive distribution (with fixed variance), becomes the squared error (with an additive constant).
Minimizing Arain will enforce p(yl2) = ps(y)z).
We consider the below two assumptions of the representation & on the source domain:
Assumption 1. Is(2, y) = [z y). where Is(:s)) is the mutual information term, calculated on the source domain. In particular:
Ts(2, v) = Eps(s.)) log ps(2, )
ps(())s(y)] ]
ps
Ts v) E. ps( log
(5)
ps(z)ps(v)
This is often referred to as the "sufficiency assumption" since indicates that the representation has the same information about the label y as the original input and sufficient for this prediction task (in the source domain). Note that the data processing inequality indicates that (s y) Is y), so here we assume that z contains maximum information about y.
Remark 2. Assumption / is an optimization goal of the training process on the source domain.
In particular, Itrann (with an additive constant) is upper bound of (2s v), which is upper bound of (I, y). Thus, minimizing Irain will enforce [s(2, y) be equal to Is(2,8) y). For more detailed discussion of this, please refer to, for example, Alemi et al. (2016).
Assumption 2. ps())z) Bp(=|=) [y]]}] Vz,y y X,Y
When this assumption holds, the predictive distribution in Eq. 2 will approximate ps(y)z), as long p(ylz) approximates ps ps(l)))
Remark 3. Assumption 2 is also an optimization goal of the training process on the source domain.
This is because lerain is an upper bound of Bps(z.0) log p[y[z)]. which is an upper bound of E (x.)) I- log (s)(2)) Thus, minimizing 4erain will enforce p(y(z) to be equal to (y(a))) Therefore, ps == p(yl x) = Eppx)=) [[(y]=)] ] == Ep(a(z) (ps(==))]
These two assumptions ensure that our network has good performance on the source domain. Note also that we only make the above two assumptions about the source domain, where we can enforce them through the training process. We do not make these assumptions about the target domain, since have access to the full target distribution. These two assumptions will also be used to prove our later theoretical result (Proposition 2).
3.2 KL GUIDED DOMAIN ADAPTATION
Now we will consider the test loss in the domain adaptation problem, and how we can reduce it. The test loss (of the target domain) is:
lest (pr(an)) log p(y[=)) Epr()))) loll--loop log Ep(( [p(y)=)]]
(6)
(p [pp(= log
log ((y|=)]] (Jensen Inequality)
(7)
Bpr(..)) !- ((yl=)
(8)
Note that if the representation is invariant (both marginally and conditionally), then pr y) = ps(2,8) y) and Eq. 8 becomes Itrinn and we have perfect generalization between the source domain and the target domain. However, there no way to guarantee the invariance, since we do not know the target domain and the target data distribution. In that case, we introduce the following proposition that ensures generalization bound of the test loss based the training loss and the KI. divergence:
Proposition If the loss log p(y(z) bounded by M Vz E Z,y Y, we have:
M
lest S ltrain +
KL (pr (1)p(),,))))))) (p)(
(9)
2
M
ltrain
KLIpT())lus(2)
+ Epr(s) [KL.pr(V>=]lps</>>>
(10)
2
'In the classification problem, we can enforce this quite easily by augmenting the output sofimax of the classifier so that each class probability is always least exp (-M). For example, if we choose A + 0.05, and if the output softmax is (p1, P2..PP....PPC Pc).
exp (-M)
augment it into (p1 K 0.05, pa K + 0.05, .... Pc K + 0.05). where K = 1 0.05 C. and C is the number of classes. This ensures the bound for the loss of datapoint, while remaining the output prediction class.
5

---

Published as a conference paper at ICLR 2022
Proof. provided in the appendix.
This bound is similar to other bounds in the literature (e.g., Ben-David et al. (2010) in the sense that it also contains the training loss, marginal misalignment term and a conditional misalignment respectively in our case). However, Ben- David et al. (2010) consider a binary classification problem and their bounds are only practical for deterministic labeling function (requires knowing the true labeling function, which is unknown for a probabilistic labeling mechanism, to compute the bound); while our bound works for the general case of supervised learning with any labeling mechanism. For brief review of these bounds and detailed discussion about their differences ours, please refer to the appendix. Note that the bound in Proposition 1 is also true when applying to the input space directly (e.g., replacing = with æ). However, we are more interested in the bound in the representation space, since we can reduce by regularizing the KL term.
term (KI[[p(22) [p)]ppss)]] and E. Egr(s) (=)
To reduce the generalization gap, we want pr(2,y) to be close to ps y). Aligning the marginal distribution (i.e.. (z()) R (2())) helps the classifier network p(yl 2) avoid out-of cut-of-distribtion data since the target representations it faces at test time belong to the source representation distribution which it was trained on; while aligning the conditional distribution (ps [y]z) = pr (y)+)) makes sure the classifier gives more accurate predictions on the target domain since p(y 2)) was trained to ap- proximate ps [.] In the domain adaptation problem, since we only have the unlabeled data from the target domain, we often align the marginal distribution of only. However, one problem that the conditional misalignment also depends on the representation 2, and when learning a representa- tion that aligns the marginal, we might accidentally increase (x)(=) (y)]p((ll=))ps []]]]]] at the same time, leading to a net increase in the above generalization bound. For example, what if (and is it possible that) the conditional misalignment increases to infinity while we learn representation 2?
Therefore, it is crucial that we can bound the above conditional misalignment. The below proposi- tion handles this problem.
Proposition 2. If Assumption I and 2 hold, and if pr(a-)) < oo (i.e., there exists N, which can be ps(z.9)
arbitrarily large, such that pr(a.)) pa(a.y)
< < Vx E X, E J)), we have:
(x) [KLppr (vl=)ps(((=))1 S (x) (KL[p)([])=)pp>(()>>))
(11) O
Proof. provided in the appendix.
This shows that the conditional misalignment in the representation space is bounded by the condi- tional misalignment in the input space. This can also be viewed as an improvement over the analyses of Ben-David et al. (2010), where it is not clear if the conditional misalignment the representation space is bounded or not. It then follows that:
M V2 (L(pr()))p((2)) + Epr(a) [KL[pr (vlz))ps(vlz)) [)]]]
Test S Utrain
(12)
As mentioned earlier, in order for domain adaptation to be effective, we should expect that the label- ing mechanism does not change too much (Ben-Davine et al., 2010). Thus, the conditional misalign-
ment (x) [KLjpr(yla (y)r()))))) ()) ps [)]]]]]]] is often small (and fixed not dependent on the representation
2). Therefore, to reduce the generalization bound, we can focus on minimizing (L(pr())lps()))
with the objective:
ltrain + +KLlppr>)(ps(a)
(13)
where 3 is a
Discussion on the use of reverse KL: Our derivation leads to the reverse KL term (=)pss)))ss as a regularizer of the distance between the two domains representations. We argue that there are several reasons that make this good choice as divergence between the source and target representation distributions. (1) First of all, as mentioned earlier, the KI. term can be com- puted casily without any additional network or minimax objective (details in Subsection 3.3). This leads to an efficient and stable training procedure, which often results in improved performance. (2) Secondly, the reverse KL has the woo--ooorm/od--eeeriiii alleviate the out-of-dis -isiiitiitttion problem. Specifically, the reverse KL forces the target representation distribution have zero mass wherever the source distribution has zero mass (zero-forcing). thus
effect (Minka et al., 2005) that helps to

---

Published as a conference paper at ICLR 2022
(a)
(b)
Figure 2: Reverse KL allows a flexible alignment of the representation while still effectively preventing the out-of distribuuiiiinnonnn problem. (a) Source representation distribution (black). Consider the case where the data distribution (=( of the source domain has two modes, then the representation distribution ps(=) will likely also have two modes; and consider the case where the target distribution has only one mode. (b) An acceptable larget representation distribution (green) that helps the classifier avoid the out-of-distribtion problem. Reverse KL allows for this type of ficxible alignment (match to onc/some of the modes) due to its mode-secking nature. (c) A problematic target representation distribution (red), since the classification network will face out-of- distribution data at test time, in the between the modes. Reverse KI. will prevent this due its zero-forcing nature.
preventing the out-of-distribution data at test time (Figure 2c). On the other hand, its mode-seeking nature allows flexible alignment of the representation. For example, consider the case where the source domain is a mixture of two components (Figure 2a, i.e., it has two modes), and the target dis- want to learn representation network
tribution is close to one of the two components. Idcally, that matches the representation of the target domain to that of the corresponding component on the source mixture (Figure 2b). This representation will still perform well at test time since we would have the out-o--distriuution problem (the classification network already trained on this mode of the source distribution). This flexible alignment (to one or some of the modes) is accepted by the reverse KL since it leads relatively small reverse KL value. Meanwhile, other methods such as DANN, MMD, CORAL and WD aim to match the representation distribution the target domain and that of the whole source domain together, which could compress the representation too much, negatively affecting
expressive power. For instance, in the above example, trying to match the whole distribution of source and target domains based on other distance metrics might force the two modes of the source domain to collapse. The flexible alignment of the reverse KL (while still being very effective to prevent -uitt--distribution data) might be beneficial in some practical cases.
We also empirically found that adding an auxiliary term KL[ps(2) (2)]ss())prrr (z)) (forward KL) with a small coefficient Bouz to the objective can help align the distribution faster, leading to the objective:
Ltvain 3KL(pp + Bouz KL(ps())lpp(2))
(14)
In practice, setting Bouz to a (very) small value or zero often leads to the best results (using larger Baur can hurt the performance). Note that this does not invalidate our earlier discussion, with such small values of Bauz, the alignment behavior is still dominated by the reverse KL.
3.3 OPTIMIZATION
In practice, estimate Eq. 14 using minibatches. In particular, given labelled minibatch (a ( ) ) of the source domain and an unlabelled one (al ) of the target domain, and single sampled representation for each B E and (B)) of the objective 14 follows:
/== get unbiased estimator
Itrain + KL (z)lps + Bauz Klpps
= (xp log pbyl =)] + BPp Doggr pT(() log (ogps())) + Bour Epote) - (rg)r(22)
B
B
log ((7 = + B
log (2(()) log us (=(0)
B
B
log Po (z - log pl (=(0))
+ Baux
(15)
B

---

Published as a conference paper at ICLR 2022
However, it still requires knowing (2( and pz(2) compute Eq. 15. We also use the minibatch to approximate these quantities:
B
B Σ 1=1
ps(2) = R (os( [p(/
~
Σ B
1
pr(2) = Epr [p(([)) p(2
~
p( (16)
B
Intuitively, we use a minibatch of data to construct a distribution of the representation (which is a mixture of B components), and match that distribution for the two domains with the KL divergence. As mentioned earlier, we use a Gaussian distribution with diagonal covariance matrix for p(=(æ) in practice, and employ the reparameterization trick (Kingma and Welling, 2013) to sample 2.
Although the estimator in Eq. 15 is unbiased, the approximations in Eq. 16 will introduce some bias into our estimator (however, the estimator is still consistent, i.c., it becomes exact when B + oo). Therefore, the batch size might have an effect on the performance of the model. However, via an ablation study, we found that the effect of this bias estimator not severe in practice, and our model achieves good performance even with a batch size of 64. For detailed results and discussion of this ablation study, please refer to the appendix C.4.
4 EXPERIMENTS
4.1 DATASETS
RotatedMNIST consists of 70,000 MNIST (LeCun et al., 2010) images that are divided into six domains, each with 11,666 images. The images in each domain are rotated counter- clockwise by 09, 15", 30, 45", 60% and 75 respectively. M1. Mas, ments, each with M1s.MMu M30, Mao or M7s as the target domain. The task is classification of the ten digit labels.
We denote the six domains as and.sss We use My as the source domain, and perform five experi-
is a common domain adaptation dataset, with digit classification sub-datasets, namely MNIST, USPS (Hull, 1994) and SVHN (Netzer et al., 2011). Three common adaptation experiments are MNIST + USPS, USPS + MNIST and SVHN - MNIST.
DIGITS
VisDA17 (Peng et al., 2017) is a challenging real-world classification dataset with simulation-te real adaptation task. This dataset contains over 280K images from 12 classes. The source domain contains renderings of 3D models, while the target domain contains real images.
Please refer to the appendix for results of datasets with more domains such PACS (Li et al., 2017).
4.2 BASELINES
We consider all common marginal alignment methods for domain adaptation our baselines, in- cluding DANN, MMD, CORAL and WD. We also consider ERM (empirical risk minimization) and its variant ERM (prob) (same as ERM but with the probabilistic representation network used in our model). For ERM, DANN, MMD and CORAL, we follow the implementation by Gulrajani and Lopez-Paze (2020); while for ERM (prob) and WD, we use our own implementation in Pytorch (Paszke et al., 2019). Note that we do not include methods that are not from the family since they are out of the scope of this paper. For the full description of these baselines, please refer to our appendix and the official code at https://gtthub.cm httpt/u.o.oo/attaaggnnente attunnguynennKKL.
4.3 EXPERIMENTAL SETTING
For the RotatedMNIST and DIGITS experiments, we use simple convolutional neural network with four 3x3 convolutional layers (followed by an average pooling layer) as the representation network. For VisDA17, we representation network differs for deterministic representation (ERM, DAÑN, CORAL, MMD, WD) and probabilistic one (ERM (prob) and KI. (ours)). For representation of size d.. the last layer's dimension of a deterministic representation network is d., while that of probabilistic network is 2- d, (dz for μ and d, for o²). Please refer to the appendix for the detailed experimental setting (including data split, hyper-parameter tuning for each model, evaluation protocol, etc.)
Resnet50 as the representation network. Only the last layer of the
8

---

Published as a conference paper at ICLR 2022
Table Rotated MNIST experiments with Mo the domain.
Target Domain
Model
M15
M30
M45
Mao
M75
Average
ERM
97.5+0.2
84.1
53.9=0.7
34.2+0.4
22.3+0.5
58.4
ERM (prob)
96.8+0.3
83.2+1.6
51.3=0.9
31.4+1.1
20.7+0.7
56.7
DANN
97.3+0.4
90.6+1.1
68.7+4.2
30.8+0.6
19.040.6
61.3
MMD
97.5+0.1
95.3=0.4
73.6#2.1
44.2+1.8
32.1+2.1
68.6
CORAL
97.1+0.3
82.3+0.3
56.0=2,4
30,8+0.2
27.1+1.7
58.7
WD
96.7+0.3
93.1+1.2
64.143.3
41.4+7.6
27.6+2.0
64.6
KL (ours)
97.8+0.1
97.1+0.2
93.4
75.5+2.4
68.1+1.8
86.4
Table 2: DIGITS and VisDA17 experiments.
DIGITS
VisDA17
Model
M + U
U-+ M
S + M
Average
S -
ERM
73.144.2
54.826.2
65.9+1.4
64.6
39.140.5
ERM (prob)
70.3+3.2
59.0#8.3
67.6+1.3
65.6
37.2+2.2
DANN
90.7+0.4
91.2+0.8
71.1+0.5
84.3
57.7+1.3
MMD
91.8+0.3
94.420.5
82.8+0.3
89.7
62.8+1.1
CORAL
88.0+1.9
83.3+0.1
69.3+0.6
80.2
39.5+4.5
WD
88.2+0.6
60.2+1.8
68.442.5
72.3
38.924.8
KI. (ours)
98.2+0.2
97.3+0.5
92.5+0.9
70.6#0.5
4.4 RESULTS
RotATedMNIST and DIGITS: Table and Table show the results for the RotatedMNIST and DIGITS experiments. It is clear that in these experiments, aligning the representation between domains does help improve the generalization performance. Among the baselines (DANN, MMD, CORAL, WD), MMD performs the best, which we attribute to the fact that it does not use minimax objective, leading to more stable optimization. Meanwhile, CORAL performs the worst, since it only matches the first two moments of the distributions and might fail to align complex distributions. Our method, KI,, largely outperforms the baselines, indicating its effectiveness. Visualization of the representation space also shows that our method aligns the representation better than existing methods. This visualization can be found in the appendix.
VisDA17 (Table 2): In this challenging dataset, many marginal alignment techniques (CORAL, WD) fail the adaptation task (achieve similar accuracy as the ERM baselines). MMD and KI. (ours) are again the best performers, confirming that a stable training objective is beneficial in practice. Our method outperforms all other marginal-aligmment approaches significantly, suggesting the ef- fectiveness of the KI. divergence in representation alignment.
5 CONCLUSION
In conclusion, in this paper, we derive a generalization bound of the target loss in the domain adapta- tion problem using the reverse KL divergence. We then show that with a probabilistic representation, the KL divergence can easily be estimated using Monte Carlo (minibatch) samples, without any ad- ditional computation or adversarial objective. By minimizing the KL divergence, we can reduce the generalization bound and have a better guarantee about the test loss. We also empirically show that method outperforms relevant baselines with large margins, which we attribute its simple and stable training procedure and the mode-seeking/zerr waaiking/zeroforring nature of the reverse KL. We conclude that KL divergence is very effective as tool for representation alignment. In general, limitation of marginal alignment methods (ours included) is that when the conditional distribution changes significantly from the source domain to the target domain, aligning the marginal would not help the target domain's performance. This is also reflected in generalization bound. For future work, would want to investigate the use of KL divergence in other types alignment. For example, we can follow the algorithm in Kang et al. (2019) to minimize the intra-class distance of the representation across domains and maximize the inter-class distance between them, but using the KL divergence instead of MMD as the distance between representation distributions. Another direction would be using KI. divergence align the conditional distribution across domains in multi-source setting.

---

Published as a conference paper at ICLR 2022
Acknowledgments This work is supported by the UKRI grant: Turing AI Fellowship EP/W002981// 1/1 and EPSRC/MURI grant: EP/N019474/1. We would also like to thank the Royal Academy of Engineering and FiveAl.
REFERENCES
D. Acuna, G. Zhang, M. T. Law, and S. Fidler. f-domain-adversariall learning: Theory and algo- rithms. arXiv preprint arXiv:2106. /1344 2021.
H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, and M. Marchand. Domain-adversare neural networks. arXiv preprint arXiv: /4/2.4446 2014.
A. A. Alemi, I. Fischer, J. V. Dillon, and K. Murphy. Deep variational information bottleneck. arXiv preprint ww1.vv::622.004444 200010, 2016.
K. Azizzadenesheli, Liu, F. Yang, and Anandkumar. Regularized learning for domain adapta- tion under label shifts. arXiv preprint arXiv: 2019.
S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from different domains. Machine learning, 79(1)1151-175, 2010.
J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. Journal of machine learning research, 13(2), 2012.
O. Bousquet, Boucheron, and G. Lugosi. Introduction to statistical learning theory. Summer School on Machine Learning, pages 169-207. Springer, 2003.
R. T. d. Combes, H. Zhao, Y.-X. Wang, and G. Gordon. Domain adaptation with conditional distri- bution matching and generalized label shift. arXiv preprint 20Xiv:2003.0445. 2020.
C. Cortes, Y. Mansour, and M. Mohri. Learning bounds for importance weighting. In Nips, vol- ume 10, pages 442-450. Citeseer, 2010.
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096-200 2016.
M. Ghifary, W. B. Kleijn, M. Zhang, and D. Balduzzi. Domain generalization for object recognition with multi-task autoencoders. In Proceedings of the IEEE International Conference on Computer Vision, pages 2551-2559 2015.
I. Goodfellow. Nips 2016 tutorial: arXiv: 1701.001010 2016.
Generative adversarial networks
arXiv preprint
A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf. and A. Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773 2012.
I. Gulrajani and D. Lopez-Paz. In search of lost domain generalization. 20l,2200..01444.2200
arXiv preprint
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
J. J. Hull. A database for handwritten text recognition research. IEEE Transactions Pattern Analysis and Machine Intelligence, 16(5):550-555 5544, 1994. doi: 10. 00.1009/44.291444
D. Johansson, D. Sontag, and R. Ranganath. Support and invertibility in Aomain-invariant repre- sentations. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 527-536. PMLR, 2019.
G. Kang, L. Jiang, Y. Yang, and A. G. Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4893 2019.
A. Khosla, T. Zhou, T. Malisiewicz, A. A. Efros, and A. Torralba. Undoing the damage of dataset bias. In European Conference on Computer Vision, pages 158-171. Springer, 2012.
10

---

Published as a conference paper at ICLR 2022
D. P. Kingma and J. Ba. arXiv: 1412.6980, 2014.
Adam: A method for stochastic optimization. arXiv preprint
D. Kingma and M. Welling, Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
N. Kodali, Abemethy, J. Hays, and Z. Kira. On convergence and stability of gans. arXiv preprint vrXiv:/705.0722 2017.
Y. LeCun, C. Cortes, and C. Burges. Mnist handwritten digit database. ATT Labs [Online]. Avail- able: http://yann. www.ll wttp:/a..oo//eeb/mniiiss.st 2, 2010.
D. Li, Y. Yang, Y.-Z. Song, and T. Hospedales. Deeper, broader and artier domain generalization. In International Conference on Computer Vision, 2017.
H. Li, S. J. Pan, S. Wang, and A. C. Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5400- 5409, 2018
Y. Mansour, M. Mohri, and A. Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430 2009.
Minka et al. Divergence measures and message passing. Technical report, Citeseer, 2005.
K. Muandet, D. Balduzzi. and Schölkopf. Domain generalization via invariant feature represen- tation. In International Conference on Machine Learning, pages 10-18. PMLR, 2013.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. 2011.
A. T. Nguyen, T. Tran, Y. Gal, and A. G. Baydin. Domain invariant representation learning with domain density transformations. arXiv preprint wrXiv:2102.05082. 2021.
A. Paszke, S. Gross, Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang. Z. Vetito, M. Raison, A. Tejani, Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-performance deep learning library. H. Wallach, H. Larochelle, Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024-8035. 2019.
X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang, and K. Saenko. Visda: The visual domain adaptation challenge, 2017.
K. Saenko, B. Kulis, M. Fritz, and Darrell. Adapting visual category models to new domains. In European conference on computer vision, pages 213-226. Springer, 2010.
J. Shen, Y. Qu, W. Zhang, and Yu. Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32, 2018.
B. Sun and K. Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision, pages 443-450. Springer, 2016.
A. K. Tanwani. Domain-invariant representation learning for sim-to-real transfer. arXiv preprint arXiv:2011.07589, 2020.
L. Van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of machine learning research, 911), 2008
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polo- sukhin. Attention is all you need. arXiv preprint arXiv: 1el:::0066.0722 2017.
X. Xu, X. Zhou, R. Venkatesan, G. Swaminathan, and O. Majumder. d-sne: Domain adaptation using stochastic neighborhood embedding. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 2497-2506, 2019.
Y. Zhang, T. Liu, M. Long, and M. Jordan. Bridging theory and algorithm for domain adaptation. In International Conference on Machine Learning, pages 7404-7413. PMLR, 2019.

---

Published as a conference paper at ICLR 2022
H. Zhao, R. T. Des Combes, K. Zhang, and G. Gordon. On learning invariant representations for domain adaptation. International Conference on Machine Learning, pages 7523-7532, PMLR, 2019.
Y. Zhu, F. Zhuang, J. Wang. G. Ke, J. Chen, J. Bian, H. Xiong, and Q. He. Deep subdomain adaptation network for image classification. IEEE transactions neural networks and learning systems, 1712 2020.
A PROOFS
For the following proofs, we treat the variables as continuous variables and always use the inte- gral. If one some of the variables are discrete, it is ttrwigtt-orwwar to replace the corresponding integral(s) with summation sign(s) and the proofs still hold.
A.1 PROPOSITION
Proof. We have:
hest S (pp( -- log log =)]
(17)
/ log p 2 ())r( dzdy
(18)
f log p(ul =) (2 dzzdy + log p(ylz) (pr(2,)) pz(z, v)l dzdy
(19)
Irain / 1 log log (pr v) ps(2, v)) dzdy
(20)
Let A (2,,)) pr (2, -)sp)))sa)))) 0} and B ((2,)))pr(z,))) y) ps(2.8) 0}. using the fact that log((y)(z) 2 0 V2 Z,v E y we have:
/ log p(yl=) ()) (= v)] dzdy
(21)
/ logg(y)= (2. - ps (z, () dzdy + /
log (()l()))))))))))))))))))())))( (pr(2. psss, v)) dady
(22)
< - log p(yl)) [pT(2, u) - ps (z, y)l dzdy
(23)
A
[ log p(yl2( |pr(2.)) v) ps (z, 4) dody
(24)
< M / |PT(2, y) ps(z, v) dady
(25)
(since - log p()12) = M)
(26)
where |.| is the absolute value.
Here, LA (pr(2. v) ps(z, v)l dzdy is also called the total variation of the two distributions Pr (2. y)
and ps (2.
12

---

Published as a conference paper at ICLR 2022
Note that:
/
pr (2,1)) ps(2, yddady ydddy = 0
(27)
pr(2,8) ()(2,,)dzdy +
PT (2(22)) - fddddy 0
(28)
pr(2,y) 8) (s((, ps(z, 1) - pr pr y)dzdy
(29)
18
(pr(2,8) ps(2.v) dzdy
(2r(2,0) ps(z, v) dzdy
(30)
J
4>
1pT(z, ps(z, dzdy
2 |pT(2, v) - ppssp, v) dzdy
(31)
Therefore:
leest S lerain M / (pT(2,)) ps ps w)1 dzdy
(32)
A
= -rein +
2 (pr(2,0) ps (2(2, y) dzdy
(33)
Using the Pinsker's inequality, we have:
(Ierte.w) -ps(eew))-s((z.)))
ps(2, v)1 dzdy) $2 pr((, v) pr(2.y) dzdy
(34)
ps(2,y) y)
Therefore, we finally have:
M
/ pr(z, v) log
pr v) ps(z,y)
hest S terain +
12
2
dzdy
(35)
2
M
Itain
KL (2 (2)
(36)
2
Which concludes our proof.
Also note that the KL divergence between pr(2z y) and ps(p,, y) can further be decomposed into the marginal misalignment and conditional misalignment follow:
KL y) (ss y) (pr(*.) (r y) log
(37)
Epr(=.))
PT(2) pr + logpr(u|2) - log ps(2) log (s(]=))]
(38)
Epr(:.g) [og - log (2)] + (pp().)) flog pr(12)) - log ps [3|[2]
(39)
(pr(s)llogp)) (z) - log (=)]
+ Epr(=) (log log [x(=)
(40)
[L(pp()))ps(()]]] Epr(t) (ollpr 2)]]
(41)
A.2 PROPOSITION 2
Proof. According to Assumption 1, we have:
Is(2,y) = Is(
(42)
*Hs(y) Hs(y(2)) = Hs(v) Hs(y(æ)
(43)
*Hs(v/2) Hs(y(æ))
(44)
+ [log log = Bps(ta) [ogns(u(ar)
(45)
* pps( [logss()[=)] = ()) Dog [ogls()))))
(46)
+ * Eps(aa) Bo
(s]l=) (ogss
= Bp [oggs()|>))
(47)
+ Bpp((r))
[logps()(>) (oggs
(48)
13

---

Published as a conference paper at ICLR 2022
According to Assumption 2, Vo X, y y we have:
ps(b/a) = (p(tsa)pss((l=)) + log ps (v/a) = log Bogeecessppssll
(49)
(50)
(ogps(y)z) 2
(51)
Since p=(2.))
pr(r.v)
< co, there exists N such that pr(za)) ps(a,))
S
Vr N. E V. Therefore:
(x.a)
[logs((ll:)
(52)
Elsste) pr(2,))
(logps(vlz)
(53)
ps(x, v)
N N.E (s(aa))
[logs(((>>) (n
(54)
(55)
Therefore:
(p)(w)) [log Ps (y/x) Ep(aa) Ps [t|]]]]]]]]]]]]
(56)
pr.w)) [log (y x)] (lo ps
(57)
E Bpr(sw) [syps(y x:] Epr(s (oggs(()))
(58)
We have:
Epr(s) KL (p S (x)(s) [[ppp((l=)lps<<>>]]]]
(59)
+ B (pr())) [log pr (gl=) 1)) log ps (u(==) & (ar((a) (logr( log ps (y|=)]
(60)
Using Eq 58, we now only need to prove that:
E (pr(2 [log pr (y) =)] s Epr(z.) [[ogr](]z)] Hr(y)=) < (y)
(61)
(62)
Hv) - Hr((l=) Hr(y) - Hr((lz)
(63)
*Ir(z,)) = (x
(64)
(always true based on the Data Processing Incquality)
(65)
B REVIEW OF EXISTING GENERALIZATION BOUNDS
There have been several works studying the generalization bounds of the Domain Adaptation prob- lem. We briefly review the most important and common ones here with a discussion about their differences to our proposed bound.
B. BEN-DAVID ET AL. (2010)
Ben-David al. (2010) consider a binary classification problem. Let x be the input with the support set X and be the binary label with the support set y = 10, Consider a source domain with distribution Px over the input x and the true labeling function f* X + {0, 13: and similarly target domain with distribution P over the input and the true labeling function f' X (0. 1). Note that the authors claim that this labeling function can be probabilistie; in that case, f X' [0, I] denoting the probability. However, we argue that this probabilistic setting is impractical since we would not know that true underlying function in order calculate/estimate the bounds in practice). Therefore, we found that the bound is only practical for the case of a deterministic labeling mechanism.
The error of the classifier h, which is also deterministe labeling function, on the source domain is:
e (h) = Ex [lb(x) f*(x))],
(66)
14

---

Published as a conference paper at ICLR 2022
and similarly for the target domain:
<(h)= - F [
(67)
Here |-| is the absolute value, which means the loss of a data point is the Ll distance of the labels.
Consider a hypothesis space H. and let classifier h be any function from that space. The first theorem in Ben-David al. (2010) offers a bound of the target loss eh(h) based on the source loss ch(h), and the total variation between Px and PX. and the difference between the two labeling function f and f":
Theorem (Ben-David et al. (2010))
é' <(h) - e' (h) + 2d (PX. Px)+
E Px Px Pxe(Px
min
(68)
where di (P). Pt) is the total variational distance, i.e., di (Px, Pt)== = | (A) and 2 is the sigma-fiele feeldfiidddd of X (set of all subsets of X)).
In this theorem, the term 2d (Px. Pk)) presents the marginal misalignment and min minexx PS) BE-Py (1f*()) (=)]] is the conditional misalignment.
Ben-David et al. (2010) also propose another bound based on variant of the H-divergence, which is presented in the following theorem:
Theorem (Ben-David et al. (2010))
e'(h) S (h) + dHAx(PX + Xz
(69)
where the HLA--divergence CHax((Xx: P\x)
A [Pra~Py (hy (z) * h2(x))
Prz=py (z) # h2 (z)]] replaces the total variation to measure the marginal misalignment of the two domains. Meanwhile, Au inf /c" (h) + c(h)] measures the conditional misalignment of the two domains (if the two true labeling functions f and f³ are the same and belong to the hypothesis space H. this quantity is zero).
The above bounds can also be applied to the representation space (similar to ours), leading to the same bounds where the input r is replaced by its representation z. However, as mentioned in the main text, is not clear if the conditional misalignment in the representation space (e.g., minp (P2. Bz- (z) - (²)=)]) is bounded or not.
Difference to our bound First of all, Ben-David et al. (2010) only consider a binary classification problem. Moreover, discussed above, the bounds in Ben Devid al. (2010) only practical with deterministic labeling mechanisms for both domains. This assumption is hard to be true for most datasets since the labeling mechanism is usually probabilistic. This makes it not generalizable to the general case of supervised learning. Furthermore, the loss function a L1 distance between the labeling function, which is also not a common choice in practice, which makes it challenging to generalize to the multiclass classification set-up (even if there exists deterministic labeling function for a multiclass dataset, using the L, loss for the one-hot encoded labels would unreasonable; the common loss function in practice for the multiclass classification problem is the cross-entropy loss). Finally, the total variation and H-divergence might be hard estimate practice since it requires the computation of a supremum.
B.2 MANSOUR ET AL. (2009)
Mansour et al. (2009) consider more flexible problem set-up than Ben-David et al. (2010), Specif- ically, instead of y = (0, },, they consider the cases where y = {0. 1) (for binary classification) or )' is a measurable subset of R (for regression). Note that their bound still cannot work for mul- ticlass classfication. They also generalize the LI loss function however, this loss function must obey the triangle inequality. Although the L1 distance satisfies this inequality, it is not generally true for other common loss functions in practice (c.g., cross-entres entropy). They still consider deterministic labeling function f and f' for the source and target domain.
loss function L )
R:
Similar to Ben-Dave et al. (2010), with hypothesis h from the hypothesis space H. the error of the source and target domain are:
15

---

Published as a conference paper at ICLR 2022
(h) E (L(()
(70)
<(hh) = Px [L(M()
(71)
For convenience, denote also the error between two labeling function h and / in the source and target distribution as:
(h,, Er pg (())) (h, = P [[(((r),
(72)
(73)
(which means (h)h) (h, Also, let h and h be the minimizer of (* and (h) respectively. In particular:
(h, f*) and (h) = e² (h, )')
h arg min ch(h) arg min Ex- (())) kCH
(74)
NEW
ht arg min e' (h) = arg min E Px (() WER
(75)
hEH
Mansour et al. (2009) introduce a generalization bound follow:
Theorem 3 (Mansour et al. (2009)) Assume that the loss function is symmetric and obeys the triangle inequality. Then, for any hypothesis h E H. the following holds
é' (h) I ('(h") + + (h, h²*) + disc Px, Pk) (h""",
(76)
where disc(Px. PX) suPa,NeH (h" (h, which is generalized version of the HLAH-divergence.
Here, the first term ch(ht) is the ideal target loss (will be zero if the hypothesis space H contains fi), the second term ch(,, h²*) will be zero if we choose h h (which is the common practice, e.g., train the classifier l on the source domain), the third term disc(Px, PX) measures the marginal mis- alignment, and the final term e' (h* h²t) ) is somewhat an indicator of the conditional misalignment (becomes zero if f
H).
Difference to our bound The above bound is based on the ideal target loss, while our bound is based the source loss. In practice, have (an estimate) of the source loss calculated the source domain's 's training set; meanwhile, the ideal target loss unknown. This makes the above bound less useful in practice compared to ours. Furthermore, the above bound has similar problems the ones in Ben-David al. (2010): it does not work for multiclass classification, it assumes a deterministic labeling mechanism (which does not hold in practice), assumes the loss function obeys the triangle inequality (which generally is not true in practice), and it contains terms that are easy to compute practice (supremum and infimum).
C ADDITIONAL EXPERIMENTAL RESULTS
C.1 VISUALIZATION OF THE ROTATEDMNIST EXPERIMENTS
Figure shows the representation space (for the RotatedMNIST experiment with source Mo and target M45) of our method compared to the basclines, visualized using t-SNE (Van der Maaten and Hinton, 2008). We can clearly see that the color clusters (which correspond to the digit classes) of our method are much more aligned when compared to other baselines such as MMD, DANN and ERM. This illustrates the effectiveness of our method in aligning the representation in the domain adaptation problem.
C.2 PACS
Table 3 presents the results for PACS, which is a challenging real-world dataset for domain adap- tation/generalization.. In this dataset, model outperforms the ERM baselines by roughly 9%
16

---

Published as a conference paper at ICLR 2022
Visualizatien
TSNE dim 2
TSNE dim
TSNE am
SSNE SIm
TSNE
TSNC
TSNC dim
(a) KL
(b) MMD
Visualization
Visualization
TSKE dim
TSNE dim
TSKE dim
TSNE dim
TSME dim
TSNE
(c) DANN
(d) ERM
Figure 3: Visualization using t-SNE of the representation space of our method KL and the baselines MMD, DANN, ERM. For each method, the left subfigure corresponds the source domain Mo and the right one corresponds to the target domain M15. Each color represents digit class.
on average, indicating the effectiveness of our ietpieenttttionallmmmee technique. Our method is the best performer (with large margin) 8 out of 12 experiments, showing clear benefit over other representation alignment techniques. Together with our method, MMD again performs the best among the prresentaaiinnalignmeen
baselines (DANN, MMD, CORAL and WD), confirming that a stable training procedure (with no minimax objectives in MMD and our model) is important and often leads to better results. It is also worth noting that our model still outperforms MMD de- spite being less computationally expensive (in this implementation, MMD needs to compute seven Gaussian kernels for each of three pairs representation sets each minibatch).
It is interesting that the ERM baselines perform the best in some experiments (e.g.. S C, P). This result also agrees with the observed in Gulrajani and Lopez-Paz (2020) that domain gener- dlization/adaptatioon techniques might have negative effects when applied unsuccessfully. It should be noted that the S (sketch) domain is undoubtedly the most different compared to others (only black sketch on white background while other domains have colors), which might explain the difficulty when learning to transfer between domains.
C.3 OFFICE-31
We also provide additional results on the Office-31 dataset Saenko et al. (2010). Table 4 presents the performance of our model and two of the best performing baselines, namely MMD and DANN. Although our method is not taite--f-thh-att this dataset (and understandingly so), it significantly outperforms the relevant baselines considered in this paper.
C.4 ABLATION STUDY: EFFECT BATCH SIXE
In this subsection, we conduct an ablation study to investigate the effect of the batch size our model's performance. Table 5 shows the performance of our method the RotatedMNIST dataset, with My as the source domain and M45 as the target domain and with various choices the batch size. As expected, our model's 's performance tends to benefit from bigger batch size, since it would alleviate the bias of our objective estimator. We therefore recommend increasing the batchsize whenever possible. However, our model performs well even for a batch size as small as 64 (which considered small in this of deep learning).
17

---

Published as a conference paper at ICLR 2022
Table 3: PACS experiments.
Model
Experiments
ERM
ERM (prob)
DANN
MMD
CORAL
WD
KL (ours)
A
66.1+1.3
63.5+0.8
71.03.2
79.5=0,4
62.7=10.4
76.2+0.9
73.143.4
A P
94.340.6
93.5+1.3
94.540.5
94.5+1.1
86.326.8
92.4+1.3
9..4+1.2
A S
53.6+0.8
60.9+3.5
58.6+12.8
62.1+2.0
46.2+3.5
53.9+2.7
67.4+1.9
C-AA
69.7+1.1
70.8+2.3
76.4+1.7
79.543.0
75.9+0.9
69.042.1
83.3+1.11
C -
82.0+0.9
81.5+2.1
78.6+3.4
80.8+2.3
78.3+3.6
72.948.6
83.147.4
C + S
72.2+1.4
70.4=1.5
76.1+1.0
74.1=1.3
56.9#11.0
48.726.1
68.240.5
P + A
65.7+2.3
63.3#1.2
68.0+2.7
67.7+1.8
70.0=1.5
62.621.5
75.5+2.5
P C
29.1+1.9
27.2+3.3
50.7+5.0
47.420.8
47.5+8.6
56.1+1.4
67.7+1.2
P + S
38.0+1.0
35.9=2.3
29.3+9.8
59.7+4.8
15.8+5.3
22.3+15.0
64.5+2,1
S + A 41.346.5
40,943.9
39.243.5
40.043.3
39.144,8
36.1
48.242.4
S 66.7+1.0
67.9=1.4
64.3+2.0
65.7=2.3
59,9#1.5
60.5+2.0
63.5+0.4
S 49.3+3.3
46.024.7
44.3+4.0
45.1
37.4#2.7
38.5+5.6
39.1+3.4
Average
60.6
60.2
62.6
66.3
56.3
57.4
69.1
Table 4: Office-31 experiments
Model
A-+D
A-W
D-+
D-+
W-+A
W-+D
Average
DANN
79.7+0.4
82.0+0.4
68 08.2+0.4
96.9=0.2
67.4+0.5
19.1
82.2
MMD
75,5+0.6
73.4+0.4
60.8+1.0
97.4=0.5
61.5+1.1
99.5+0..2
78.0
KL
85.6#0.6
87.9+0.4
70.1
99.0=0.2
69.3+0.7
100.0=0.0
85.3
Table Ablation study: Effect of batch size. Rotated MNIST experiments with Ma source and Mas target.
Batch size
256
128
64
32
KL (ours)
93.4+0.8
93.4+1.2
93.3+0.3
89.5+0.9
Table 6: Ablation study: Effeet of auxiliary term. RotatedMNIST.
Model
M1s
Mao Mas
Mso
Mr5
Average
KL (Sauz 0) KL (reported in the paper)
97.8 97.8
96.6+0.4
92.0=0.4 93.4
68,8
62.3+4.2
83.5
97. 170.2
75.5+2.4
68.1+1.8
86.4
Table 7: Ablation study: Effect of auxiliary term. DIGITS and VisDA17.
DIGITS
VisDA17
Model
M + U
U + M
S S-M
Average
S + R
KL (Baur KL (reported in the paper)
98.2+0.2
97.140.4 97.3+0.52
90.0+1.1 92.5+0.9
95.1 96.0
67.8 70.5
98.2+0.2
C.5 ABLATION STUDY: EFFECT OF THE AUXILIARY FORWARD KL
In this subsection, we investigate the contribution the auxiliary forward KL term, by considering a variant of our method without this term (Baux 0). Table 6 and Table 7 show the results for this ablation experiment. Clearly, most of the improvement comes from the reverse KI. regularizer term. And even without the auxiliary forward KL term, our results are still significantly higher than the baselines. Note that the auxiliary term requires virtually no extra computation, so we believe adding the auxiliary term is reasonable.
D DETAILED EXPERIMENTAL SETTINGS
In each experiment, split both the source and the target data into two portions: 80% and 20%. We use 80% the source domain data and 80% of the target domain data (without the labels) as the training data. We use the remaining 20% of the source data as the validation set, and the remaining 20% of the target domain data as the test set. Note that do not use the labeled data from the target domain during training or validation. This evaluation protocol is recommended by Gulrajani and Lopez-Paz (2020).
18

---

Published as a conference paper at ICLR 2022
D.1 BASELINES
ERM Bousquet al. (2003) is the typical empirical risk minimization training procedure, meaning that the model is trained normally in the training data and does not account for the distribution shift (domain adaptation).
ERM (prob): since we use probabilistic network, we also include the probabilistic version of ERM. This similar to ERM but uses a probabilistic representation network (same as ours).
DANN Ganin et al. (2016) utilizes discriminator to distinguish the representation from the source and target domains. It uses an adversarial loss to enforce that the distributions of the representation from the source domain and the target domain are the same.
MMD Li et al. (2018) uses the maximum mean discrepancy (MMD) align the representation's distributions.
CORAL Sun and Saenko (2016) aligns the representation distributions of the source and target domains by matching their first two moments.
WD Shen et al. (2018) uses the Wasserstein distance to match the distribution of the representation.
D.2 REPRESENTATION NETWORK USED IN ROTATEDMNIST AND DIGITS
We use simple CNN as the representation network in this experiment. This network exactly the same as the one used Gulrajani and Lopez-Paz (2020).
Our code is in PyTorch. The network is constructed by the following layers, where nutput..imm22 for deterministic representation network and output.din=511 for prob- abilistic representation network (256 for μ and 256 for a²):
Conv2d (ttr2ddinnncee (in.channels=1, oat.a//mee.........cccmmemee.s. hrt.aa/meeeeoek.enoo.eeeeessssed
ReLU ()
GroupNorm (mwm.groups=5 http:/upsssnnuucchhemem htt.:pppsssspuuuccccaaeeees--s
Conv22 htto:///aaccaaeee...ooooooooaana http://ccccccoaaeeeeeeeeeeeeeeee
ReeUU ()
GroupNorm
Conv22 2ate http://eee_eeeeekkeeeeeeeeeeeee.
kernel_size=:, stre:lsss-i paddingg-))
ReLU
GroupNorm (nun.groups=-, nww.channels=128)
Conv22 win.channeeee 2tt.:///otttccceeeeessoor dim, kernellsizeee stride=1, (arridgg],
ReLU
GroupNorm (num.groups=-3 wwm.channels=outpptee.
Adapt veAvg 2d (output size= (1, 1))
D.3 HYPER-PARAMETER TUNING
We train each model for 100 epochs. To avoid hyperparameter bias, we tune the hyperparameters (learning rate, regularizer coefficients, weight decay, representation dimension and dropout rate) for each method and dataset independently. Following Guirajani and Lopez-Paz (2020), we perform random search (Bergstra and Bengio, 2012) to tune hyperparameters for the baselines. We use the Adam optimizer Kingma and Ba (2014) for all the models. We re-run each of hyperparameters three times. We train all models on an NVIDIA Quadro RTX 6000 GPU.
The readers can also refer to our source code for the experiment setting.
Below the haper-parameters considered by the random search in our experiments for each base- line. {-} means a set of hyper-paramenn considered, while [., means a range of hyper-parameters considered.
D.3.1 ROTATEDMNIST AND DIGITS
The representation's 's dimension is 128 (details in Section D.2). For even more details about the below hyper-parameters, please refer to provided code.
19

---

Published as a conference paper at ICLR 2022
ERM: learning rate: [10-4.5, 10 10-2.61 weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number layers the classifier (f(u|z)): or
ERM (prob):
learning rate: [10-4.5 10-2.5], weight decay: 0.0, dropout rate: 0.0, batch size:
[8, 512], number of layers of the classifier (p(y(z)): or 3.
DANN Ganin et al. (2016): learning rate: [10 10-2.5), weight decay: 0.0, dropout rate: 0.0,
batch size: [8, 512), number of layers of the classifier (f(ylz)): or 3, adversarial loss coefficient: [10- 2 10 weight decay of discriminator: [10 6 1000 21. number of discriminator steps per gener-
ator steps: {1, 2, 4, 8), grad penalty coefficient:
-MDD Li et al. (2018): learning rate: [10-4.5, 10110-- 1000-..1000..0 ......., weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512], number layers the classifier (p(y) [=):: or 3, MMD coefficient: [10-3, 101=
CORAL Sun and Saenko (2016): learning rate: [10-1. 10-2.5]. weight decay: 0.0, dropout rate: 0.0, batch size: [8, 512). number of layers of the classifier (f(y)=)): or 3, CORAL loss coefficient:
[10-3 10
WD Shen et al. (2018): learning rate: [10 10010-20 10-2.3,, weight decay: 0.0, dropout rate: 0.0, batch
size: [8,512) number layers of classifier (p(yl=)): or 3, wasserstein distance coefficient:
[10-2 _10), weight decay of network [10-6 10- number of f optimization steps per normal
optimization steps: (1,2,4, 4, 8), grad penalty coefficient: [10-2,10"
KL (ours): learning rate: [10 [10-4.5
4.5 10 2.51, weight decay: 0.0, dropout rate: 0.0, batch size: 256,
10-2.51,
number of layers of the classifier (f(v)z)): 1, 3 0.3, Bouz 0.1
D.3.2 VISDAI7 AND PACS
The representation network for VisDA17 is a Resnet50, while the one for PACS is Resnet18. For even more details about the below parae-tas,meters, please refer to our provided code.
ERM: learning rate: [10-3 5, 1351,, weight decay: [10-5 6 1002), dropout rate: {0.0, 0.1, 0.5), batch size: [8, 45), representation dimension: (16, 128, 256, 512), number of layers of the classifier
(p(y22)) or 3.
ERM (prob): learning rate: [10-$,10-34 10-3.5, weight decay: [10-6 10-2), dropout rate: {0.0, 0.1, 0.5), batch size: [8, 45]. representation dimension: {16, 128, 256, 512), number of lay- ers the classifier (p(vlz)): or
DANN Ganin et al. (2016): learning rate: [10-5,10-3.5] weight decay: [10-6 10-2), dropout rate: {0.0, 0.1, 0.5}. batch size: [8, 45), representation dimension: {16, 128, 256, 512], number of layers of the classifier (p(gl))):
3, adversarial loss coefficient: [10-2, 102). weight decay of
discriminator: [10-6, 10-2), number of discriminator steps per generator steps: {1, 4, 8), grad
penalty coefficient:
MMD Li et al. (2018): learning rate: [10-5, 5, 10 1.5..,5, weight decay: [10- 6 10 1002). dropout rate: (0.0, 0.1, 0.5). batch size: [8, 45), representation dimension: [16, 128, 256, 512}. number of layers of the classifier (f(vl=)): 1 or 3, MMD coefficient: [10-3, 10-1]
CORAL Sun and Saenko (2016): learning rate: [10-5, 5 1103..5 weight decay: [10-6, 10 10-2], dropout rate: {0.0, 0.1,0.5), batch size: [8, 45). representation dimension: {16, 128, 256, 512).
number layers of the classifier (f(ylz)): 1 or 3, CORAL loss coefficient: [10-
(10-3,10-) 11
WD Shen et al. (2018): learning rate: [10-5, 10 3051. weight decay: [10-6 1002%, dropout rate:
(0.0, 0.1, 0.5), batch size: |8, 45), representation dimension: {16, 128, 256, 512). number layers
of the classifier (p()|2)):
wasserstein distance coefficient: [10-2 2 10 weight decay of
network f: [10-6 10 31, number of optimization steps per normal optimization steps: {1, 2, 4, 8), grad penalty coefficient: [10-2, 10a]
KL (ours): learning rate: 10-4, weight decay: [10-5, 10-2), dropout rate: 0.0, batch size: 256, representation dimension: 16, number of layers of the classifier (p(g12)): 1, B (0.1, 0.05, 0.001), Baux {0.1, 0.05, 0.01,
20


Reviews:
[{"review_id": "pf7ie1msI4D", "reviewer": "Reviewer_g5v8", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The paper is well written and easy to follow. The problem is well-motivated, and the derived bound and optimization procedure seem to be reasonable.\n- What I don't see convincing is the jump and oversimplification in Eq(15) where the last line replaces the expectation with the simple sum assuming that each of p_S(z) and p_T(z) to be uniformly distributed. This issue is neither mentioned nor discussed.\n- Similar to the previous comment, Eq(16) assumes that $p_S(x)$ and p_T(x) to be uniform!\n- I am wondering why the authors show selectivity when it comes to the results on the digits data sets. Why don't you show the different combinations? U-->S, S-->U, M U-->S.\n-Finally, the results in Table 1 show a clear failure of all the baselines on M75 and M60 *(achieving half of what KL achieves). I find it difficult to accept these results.\n", "summary_of_the_paper": "The paper proposes a domain adaptation method that is guided by the reverse Kullback-Leibler (KL) divergence between the target and source domains in the representation space. To this end, it is assumed that the representation z is sufficient and that p_S(y|x) = E_p(z|x)[p_S(y|z)].\nThe authors derive a bound on the test loss based on the training loss and a quantity based on the divergence of the joint distribution (y,z) between the target and source domain (Proposition 1). Since this bound requires labels from both domains, Proposition 2 helps to use the divergence computation on p_S(y|x) instead of p_S(y|z). The proposed method and optimization procedure are evaluated on multiple data sets and compared with multiple baselines.\n", "summary_of_the_review": "All-in-all, I find the paper of a great potential contribution to the field. I only have two issues (i) the sudden strange simplifications in equations 15 and 16, and (ii) selective experiments besides the weird results in Table 1.\n\nUpdate:\n\nI read the other reviews and the authors' responses, I'm fine with the authors' reply. So I am keeping my score.\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "iIQfH8UCKJ-", "reviewer": "Reviewer_V8Y3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n1. the idea is simple and the paper is easy to follow.\n2. the experimental result looks good.\n\nweakness:\n1. My most concerning part is the change from equation (12) to equation (13). Although KL(p_T(y|x) | p_S(y|x)) is fixed over the whole dataset. But in training, we sample a small mini-batch for training, and this term could be changed and unstable.\n2. Experiments are weak since all datasets except VisDA-17 are not popular domain adaptation benchmarks. Why not perform experiments on Office-31 and Office-Home datasets? Also, the results in this paper on VisDA-18 are not consistent with existing work such as [Zhu'20]. In addition, [Zhu'20] is based on marginal MMD and you need to compare it.\n3. the novelty. I am not familiar with domain adaption, but the proposed idea in this paper is very simple and straightforward. So I actually don’t believe there are no previous works that also tried KL divergence for representation alignment (for example, I searched “KL divergence domain adaption” and got many results).  So what is the new in this paper? The KL divergence over probabilistic representation? \n\n[Zhu'20] Zhu Y, Zhuang F, Wang J, et al. Deep subdomain adaptation network for image classification[J]. IEEE transactions on neural networks and learning systems, 2020, 32(4): 1713-1722.", "summary_of_the_paper": "this paper proposed a method for domain adaption.\nThe idea and method are quite simple. Especially, KL divergence on representation alignment is used. And the experimental result looks promising on the selected task.", "summary_of_the_review": "The paper is easy to follow, and the idea is easy and effective. However, some assumptions seem too strong and some datasets are missed in the experiment part.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "aJmZHZLFXjq", "reviewer": "Reviewer_i12Z", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n1. Marginal distribution alignment is an importance backbone for domain adaptation while the popular adversarial one requires minimax optimization and can be unstable. This paper proposes an reverse KL which can be optimized together with the source classification loss directly. \n\n2. This paper points out a deficiency of Ben David's bound on domain adaption: it fails to consider the problistic labeling mechanism. When the true labeling function is non-deterministic, the bound by David may be inaccurate. \n\n\nCons:\n\n1. My major concern is that : the implementation is inconsistent with the derived generalization bound.  I highly appreciate authors' honesty in writing down that they also use the forward KL term as additional penalty, but when $\\beta_{aux}$ is large, the objective becomes another one. For instance, in the digits, $\\beta=0.3, \\beta_{aux}=0.1$, the forward KL term also contributes a lot to the final objective. A very important point is that I didn't find the ablation studies about this term. It would be better if authors could present baseline method (e.g., $\\lambda_{aux}=0$) results and show the reverse KL is effective. \n\n2. It can be too strong to assume that $I(z,y)=I(x,y)$ when z has much lower dimension compared to a high dimension input $x$.\n\nSide question:\n\nThis paper computes the density by the Gaussian density function and then compute the KL term. I'm wondering how close are the learned representations to the multivariate gaussian distributions, especially for the high-dimensional images, i.e., the Visda datasets. \n\nUpdate:\n\nI read other reviews and the author reponses, I'm good with authors' reply, especially the KL term experiments. So I raise my score to 6.", "summary_of_the_paper": "This paper proposes a generalization bound and a reverse KL term to match the marginal distribution.", "summary_of_the_review": "This paper proposes an reverse KL term to match the marginal distributions in domain adaptation. The results are promising. The main concern is the implementation is inconsistent with the new generalization bound.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RlysZeKBfi", "reviewer": "Reviewer_FBaS", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths.**\n\n- The idea of a probabilitisc interpretation of DA is interesting.\n- The resulting algorithm seems  simple to implement in practice.\n\n**Weakness:**\n\n- **Major Concern #1** Novelty of the theoretical result.\n\nThe paper mentioned several times (e.g. first paragraph of related work) that existing theory of domain adaptation e.g. [1] is limited to the case of binary labels. While the results shown  in [1] are indeed for the binary classification setting,  I do not necesarily agree this  limitation exists, in [1] this was done for simplicity as it is typical, the extension to the multiclass setting just follows. Moreover, the multi-class setting using a novel discrepancy measure that extends [1] was analyzed in [2].  In [3] the authors  further extend the results from [1] to general f-divergences and also derived rigorous generalization bounds. The family of f-divergences include KL and r KL as particular cases.  A lot of work has also be done since [1] (2010) for example [1,2,3,4] and comparison, improvement wrt to them is not discussed .\n\nMoreover, if the goal is to show a bound where the discrepancy between marginals is estimated using the KL divergence rather than L1 (as in [1])  that could be  obtained by directly applying Pinsker's inequality on top of [1] Theorem 1.  How would that compare to the current setting?\n\n- **Major Concern #2** Significance of the derived bound.\n\nOne of the major motivation for the H-divergence introduced in [1] vs L1 and then several authors using hypot-based divergences such as [2,3,4]. It is that (quoting [1]) \"L1 is an overly strict measure that unnecessarily inflates the bound\". Pinsker's inequality shows KL  upper bounds  L1. Wouldnt the direct use of the KL then inflates more the bound? .  How does writing a generalization bound using the KL divergence which is further ubounded solves this problem? \n \n\n- **Major Concern #3** Comparison of the algorithm with modern baselines. The latest baseline WD is from 2018. Why there is no comparison to recent work in domain-adaptation? I would also recommend the authors to do experiments in the office-31 dataset as it has become standard practice. This will also make the comparison vs existing work easier.\n\n- **Major Concern #4** No experimental analysis is provided.\n\n[1] A theory of learning from different domains. Springer 2010\n\n[2] Bridging theory and algorithm for domain adaptation. ICML 2019\n\n[3] f-Domain Adversarial Learning: Theory and Algorithms. ICML 2021\n\n[4] On learning invariant representations for domain adaptation,ICML 2020 \n\n[5] Office-31 Dataset: [https://www.cc.gatech.edu/~judy/domainadapt/](https://www.cc.gatech.edu/~judy/domainadapt/)", "summary_of_the_paper": " In this paper, the authors propose a generalization bound for domain adaptation where discrepancy between distributions (i.e. marginals and class conditionals) is estimated using the reverse KL divergence. Inspired by this result, the authors then proposed a learning algorithm and show  that this outperforms some existing methods.", "summary_of_the_review": "Overall, I consider this work to have some interesting ideas however I consider it needs a major rewriting and clarity on the contribution of the theoretical results. In my opinion, they look more as a way to motivate an algorithm which could be done following the results from either [1,2,3,4] rather than a new generalization bound as claimed. The writing should also be more careful when comparing and criticizing previous work. I also believe comparison to recent DA algorithms/methods is need it, Similarly, more experimental analysis in order to validate the effectivity of the proposed algorithm is also need it.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper derives a generalization bound on target loss based on training loss and reverse KL divergence between source and target representation distributions. Then, proposes an algorithm for DA using inverse KL on representations. they show that inverse KL term can be estimated efficiently without the need for additional networks and minimax objective. The experiments show the efficiency of the proposed algorithm in terms of improving target accuracy. The paper touches an important problem and the proposed idea is simple and effective. 

There were several concerns regarding the paper that were addressed during rebuttal period, such as strength of assumptions, experiments with different values of beta, experiments on office31 dataset, novelty of theoretical results, significance of derived bounds and comparison to [3]. The remaining concern is on comparing the proposed method to recent work in domain adaptation.

I ask the authors to add the following to the camera ready (1) visualizations they have promised that depicts their method leading to better alignment and (2) add the points raised in defending the novelty of theoretical results.
Decision: Accept (Poster)