KL Guided Domain Adaptation

1 INTRODUCTION . With advances in neural network architectures ( He et al. , 2016 ; Vaswani et al. , 2017 ) , machine learning algorithms have achieved state-of-the-art performance in many tasks such as object classification , object detection and natural language processing . However , machine learning models have been focusing mostly on the case of independent and identically distributed ( i.i.d . ) datapoints ; and such an assumption often does not hold in practice . When the i.i.d . assumption is violated and the target domain has a different distribution compared to the source domain , a typical learner trained on the source data via empirical risk minimization would not perform well at test time , since it does not account for the distribution shift . To tackle this problem , many methods have been proposed for domain adaptation ( Zhao et al. , 2019 ; Zhang et al. , 2019 ; Combes et al. , 2020 ; Tanwani , 2020 ) and domain generalization ( Khosla et al. , 2012 ; Muandet et al. , 2013 ; Ghifary et al. , 2015 ) , the goal of which is to train a machine learning algorithm that can generalize well to the target domain . A common approach to tackle these problems is to learn a representation such that its distribution does not change across domains . There are two types of distribution alignment : marginal alignment ( aligning the marginal distribution of the representation ) and conditional alignment ( aligning the conditional distribution of the label given the representation ) ( Nguyen et al. , 2021 ; Tanwani , 2020 ) . For domain adaptation and domain generalization problems with multiple source domains , we can use the data and labels to align both the marginal and the conditional distributions across the source domains , aiming to generalize to the target domain . However , in a single-source domain adaptation problem , with only unlabeled data from the target domain , it is often only possible to align the marginal distribution of the representation . This marginal alignment should help the classifier avoid out-of-distribution data at test time . This paper focuses on such a single-source domain adaption problem , which is also one of the most common settings in practice . Current marginal alignment techniques usually require additional computation ( e.g. , of an additional network ) ( Ganin et al. , 2016 ; Li et al. , 2018 ) and/or a minimax objective ( Ganin et al. , 2016 ; Shen et al. , 2018 ) , leading to an expensive and/or unstable training procedure ( Goodfellow , 2016 ; Kodali et al. , 2017 ) . For example , DANN ( Ganin et al. , 2016 ) employs an adversarial training procedure , with a domain discriminator that classifies the domain of the representation , and maximizes the adversarial loss of the discriminator . When the discriminator is completely fooled , the marginal distribution of the representation is aligned across domains . MMD ( Gretton et al. , 2012 ; Li et al. , 2018 ) utilizes maximum mean discrepancy to align the representation distribution . This does not use a mimimax objective , thus leading to a more stable training ; however , it does require additional computation of several Gaussian kernels . While more sophisticated ( non-marginal-alignment ) methods have been proposed recently to achieve better results in the domain adaptation problem , we argue that studying the family of plain marginal-alignment techniques is still an important task , since they are the backbones that most other domain adaptation ( and domain generalization ) methods are built upon . To address the above issues of existing marginal-alignment techniques , we first derive a generalization bound on the loss of the target domain using the training loss and a reverse Kullback–Leibler ( KL ) divergence between the source and target distributions . There are existing bounds of the target loss in the literature ( Ben-David et al. , 2010 ) , however , these analyses focus mostly on the case of binary classification and the bounds use a total variation distance or a H-divergence between the distributions , which are not easy to estimate in practice ( for example , Ajakan et al . ( 2014 ) require an adversarial network to estimate the H-divergence ) . In this paper , we show that with a probabilistic representation network , we can estimate the KL divergence easily using samples , leading to an alignment method that requires virtually no additional computation nor a minimax objective . Therefore , our training procedure is simple and stable in practice . Moreover , the reverse KL has the zero-forcing effect ( Minka et al. , 2005 ) , which is very effective to alleviate the out-of-distribution problem in practice . This can be explained as follows : the out-of-distribution problem arises when the classifier faces a new representation at test time that is in a ( near ) zero mass region of the source representation distribution ( and thus it never faced before ) . The reverse KL tends to force the target representation distribution to have ( near ) zero mass wherever the source distribution has ( near ) zero mass ( this is the zero-forcing property ) , which helps the classifier avoid out-of-distribution data . The reverse KL also has the mode-seeking effect ( Minka et al. , 2005 ) which allows for a more flexible alignment of the representation ( to one or some of the modes of the source domain ) . For example , consider the classification problem of buildings ( houses , hotels , etc . ) where source images are collected from urban and remote areas of a country ( two modes ) ; while the target images are collected from urban areas but from a different country . Ideally , we want to match the representation distribution of the target domain to that of the first mode of the source domain since they are both from urban areas . The reverse KL allows this flexible alignment ( as it results in a relatively small value of the reverse KL ) due to its mode-seeking property . Meanwhile , other distance metrics/divergences aim to match the whole source and target representation distribution , which might collapse the two modes of the source domains . Our contributions in this work are : • We construct a generalization bound of the test loss in the domain adaptation problem using the reverse KL divergence . • We propose to reduce the generalization bound by minimizing the above KL term . Furthermore , we show that with a probabilistic representation , the KL term can be estimated easily using minibatches , without any additional computation or a minimax objective as opposed to most existing works . • We conduct extensive experiments and show that our method significantly outperforms relevant baselines , namely ERM ( Bousquet et al. , 2003 ) , DANN ( Ganin et al. , 2016 ) , MMD ( Gretton et al. , 2012 ; Li et al. , 2018 ) , CORAL ( Sun and Saenko , 2016 ) and WD ( Shen et al. , 2018 ) . We empirically show that the reverse KL divergence is very effective for representation alignment since it is very stable and efficient to compute in practice . 2 RELATED WORK . Generalization bound for the distribution shift problem There exist works studying bounds for the distribution shift problem in the literature ( Ben-David et al. , 2010 ; Mansour et al. , 2009 ) . However , their analyses of the classification problem are limited to the case of binary labels . Moreover , these bounds are only applicable or practical for deterministic labeling functions , which is not the case for most datasets in practice . Therefore , their analyses can not be generalized to the general case of supervised learning . The differences between our bound and theirs are as follows . First of all , our bound works for the general case of supervised learning : it works for both the classification ( including multiclass classification ) and regression problems , it makes no assumptions about the labeling mechanism ( can be probabilistic or deterministic ) , and it works for virtually all predictive distributions commonly used in practice . Secondly , our bound uses a different divergence , namely the KL divergence , which is easier to estimate in practice compared to total variation or H-divergence . We provide a brief review of the above bounds and discuss their differences to ours in more detail in the appendix . Recently , Acuna et al . ( 2021 ) revise the previous domain adaptation bounds and generalize them to a multi-class classification setting , as well as to the class of f -divergence ( including KL divergence ) . However , their setting is still very restricted : the loss function needs to satisfy the triangle inequality ( which does not hold for many loss functions in practice ) , and they need to know the true labeling function ( optimal Bayes classifier ) for a probabilistic labeling mechanism ( which is often not available ) . We also provide further analysis , that under some reasonable assumptions , the conditional misalignment in the representation space is bounded by the conditional misalignment in the input space , which allows for a sound marginal alignment method . Some specific cases of distribution shift have also been studied . For example , Cortes et al . ( 2010 ) and Johansson et al . ( 2019 ) study the generalization bound for the covariate shift problem , i.e. , pT ( x ) 6= pS ( x ) but pT ( y|x ) = pS ( y|x ) , where pS is the source distribution and pT is the target distribution . In contrast , Azizzadenesheli et al . ( 2019 ) provide a generalization bound for the label shift problem , i.e. , pT ( y ) 6= pS ( y ) but pT ( x|y ) = pS ( x|y ) . Domain adaptation While the literature on the domain adaptation problem is vast , we cover the most closely related works to ours here . A common method for the domain adaptation problem is to align the marginal distribution of the representation between the source and target domains . DANN ( Ganin et al. , 2016 ) employs a domain discriminator to classify the domain of a representation and maximizes its adversarial loss ( a minimax game ) . WD ( Shen et al. , 2018 ) uses a neural network function f ( which is 1-Lipschitz continuous ) to calculate the Wasserstein distance between two distributions and minimizes it . This is also a minimax game since the Wasserstein distance is the supremum over the search space of f . MMD ( Gretton et al. , 2012 ; Li et al. , 2018 ) uses the maximum mean discrepancy to align the representation distribution . This method does not need a minimax objective ; however , it requires additional computation of several Gaussian kernels . Finally , CORAL ( Sun and Saenko , 2016 ) matches the first two moments of the distribution ; and while being a simple method , it fails to align more complex distributions . We consider these marginal alignment techniques our main baselines since our method falls into this category , and investigate the effectiveness of the reverse KL divergence in aligning the distributions of representation . Recently , more sophisticated alignment methods ( Kang et al. , 2019 ; Xu et al. , 2019 ; Zhu et al. , 2020 ) have been proposed for the domain adaptation problem , which achieve state-of-the-art performance . Instead of simply aligning the marginal distribution of the representation , these methods minimize the intra-class distance of the representation across domains , and possibly maximize the inter-class distance between them , using the MMD or L2 distance . However , they require pseudo labels for the target domain ( often obtained via clustering ) . Moreover , they are complimentary to our method , as we conjecture that our method can also be used in conjunction with these , leading to the same algorithms but with the KL distance instead of MMD or L2 .

Summary: The paper proposes a domain adaptation method that is guided by the reverse Kullback-Leibler (KL) divergence between the target and source domains in the representation space. To this end, it is assumed that the representation z is sufficient and that p_S(y|x) = E_p(z|x)[p_S(y|z)]. The authors derive a bound on the test loss based on the training loss and a quantity based on the divergence of the joint distribution (y,z) between the target and source domain (Proposition 1). Since this bound requires labels from both domains, Proposition 2 helps to use the divergence computation on p_S(y|x) instead of p_S(y|z). The proposed method and optimization procedure are evaluated on multiple data sets and compared with multiple baselines.