f-Domain-Adversarial Learning: Theory and Algorithms for Unsupervised Domain Adaptation with Neural Networks
Authors: David Acuna
Guojun Zhang
Marc T Law
Sanja Fidler
Venue:  (2021)
Keywords: Domain Adaptation
Adversarial Learning
Transfer Learning
Neural Networks
Deep Learning

Abstract: The problem of unsupervised domain adaptation arises in a variety of practical applications where the distribution of the training samples differs from those used at test time. The existing theory of domain adaptation derived generalization bounds based on divergence measures that are hard to optimize in practice. This has led to a large disconnect between theory and state-of-the-art methods. In this paper, we propose a novel domain-adversarial framework that introduces new theory for domain adaptation and leads to practical learning algorithms with neural networks. In particular, we derive a novel generalization bound that utilizes a new measure of discrepancy between distributions based on a variational characterization of f-divergences. We show that our bound recovers the theoretical results from Ben-David et al. (2010a) as a special case with a particular choice of divergence, and also supports divergences typically used in practice. We derive a general algorithm for domain-adversarial learning for the complete family of f-divergences. We provide empirical results for several f-divergences and show that some, not considered previously in domain-adversarial learning, achieve state-of-the-art results in practice. We provide empirical insights into how choosing a particular divergence affects the transfer performance on real-world datasets. By further recognizing the optimization problem as a Stackelberg game, we utilize the latest optimizers from the game optimization literature, achieving additional performance boosts in our training algorithm. We show that our f-domain adversarial framework achieves state-of-the-art results on the challenging Office-31 and Office-Home datasets without extra hyperparameters.

Under review as a conference paper at ICLR 2021
GROMAIN-ADVERSARRE LEARNING: THEORY AND ALGORITHMS FOR UNSUPERVISED DOMAIN ADAPTA- TION WITH NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
The problem of unsupervised domain adaptation arises in variety of practical ap- plications where the distribution of the training samples differs from those used test time. The existing theory of domain adaptation derived generalization bounds based on divergence measures that are hard to optimize in practice. This has led to a large disconnect between theory and state-of-the-arr methods. In this pa- per, we propose a novel -adaainadversaaiill framework that introduces new theory for domain adaptation and leads to practical learning algorithms with neural net- works. In particular, we derive a novel generalization bound that utilizes a new measure of discrepancy between distributions based on variational characteri- zation of f-divergences. We show that our bound recovers the theoretical results from Ben-David et al. (2010a) as special case with particular choice of di- vergence, and also supports divergences typically used in practice. We derive a general algorithm for domain adversaral learning for the complete family of f- divergences. We provide empirical results for several f-divergences and show that some, not considered previously domain-adversarial learning, achieve state-of- the-art results in practice. We provide empirical insights into how choosing a particular divergence affects the transfer performance on real-world datasets. By further recognizing the optimization problem a Stackelberg game, utilize the latest optimizers from the game optimization literature, achieving additional performance boosts in training algorithm. We show that our f-domain adver- sarial framework achieves state-of-the-at results on the challenging Office-31 and Office-Homee datasets without extra hyperparameters.
INTRODUCTION
The ability to learn new concepts and skills from general-purpose data and transfer them to similar scenarios is critical in many mod- applications. For example, it is often the case that the leamer has access to only a small (unlabeled) subset of data on its domain of interest, but has access task) in a domain that is similar to the target domain. If the gap between these two domains not considerable, to train model by using the labeled and unlabeled data, and to generalize well to the target dataset. This scenario is called unsu- pervised domain adaptation, and it is the focus of this paper.
Be Mael agthe
larger labeled dataset (for the same
may expect
Figure 1: Domain Adaptation. A
learner is trained on abundant la- heled data and expected per- form well the target domain (marked as +). Decision bound- aries correspond a 2-layers neu- ral net trained using f-DAL.
The paramount importance of domain adaptation (DA) has led to remarkable advances in the field. From a theoretical point of view, the seminal works Ben-David et al. (2007; 201 2010a;b); Mansour et al. (2009) provided generalization bounds for unsupervised DA based on discrepancy measures that are reduction of the Total Variation (TV). More recently. Zhang et al. (2019) took step further and proposed the Margin Disparity Discrepancy (MDD) with the aim of closing the gap between theory and algorithms. Their notion of discrepancy is tailored to margin losses and builds the observation of only taking single supremum over the class set

---

Under review as a conference paper at ICLR 2021
to make optimization easier. Moreover, theories based weighted combination of hypotheses for multiple source DA have also been developed (Hoffman et al., 2018a).
From an algorithmic perspective, specifically in the context of neural networks, Ganin & Lempit- sky (2015); Ganin et al. (2016) proposed the idea of learning domain-invarian representations as two-player zero-sum game. This approach led to a plethora of methods including state-of-the approaches such as Shu et al. (2018); Long et al. (2018); Hoffman et al. (2018b); Zhang et al. (2019), 9). While these methods were explained with insights from the theory of Ben-David et al. (2010a), and more recently through MDD (Zhang et al., 2019), in deep neural networks, both the HAH diver- gence (Ben-David et al., 2010a) and MDD are hard optimize, and ad-hoc objectives have been introduced to minimize the divergence between source and target distributions in representation space. This has led to a disconnect between theory and the current SoTA methods. Specifically, approaches that follow Ganin et al. (2016) minimize a Jensen Shanon (JS) divergence, while the practical objective of MDD can be interpreted as minimizing a 7-weighted JS divergence. From the optimization perspective, the game-optimization nature of the problem has been ignored, and these min-max objectives are usually optimized using Gradient Descent Ascent (GDA) (referred to as Gradient Descent (GD) with the Gradient Reversal Layer (GRL)). Paradoxically, the last iterate of GDA is known to not converge to a Nash equilibrium even in simple bilinear games (Nemirovsky & Yudin, 1983).
The aim of this paper is to provide a novel perspective on the dowain-adversarial problem by de- riving theory that generalizes previous seminal works and translates into general framework that supports the complete family of f-divergences and is practical for modern neural networks. In particular, we introduce a novel measure of discrepancy between distributions and derive its corre- sponding learning bounds. Our notion of discrepancy is based on a variational characterization of f-divergences and includes both previous theoretical results (i.e. based on reductions of the TV) and practical results (i.e. based on JS). We empirically show that any f-divergence can be used to learn invariant representations. Most importantly, we show that several divergences that were not considered previously in an optimization point of view, we observe that under mild conditions, the optimal solution of our framework is a Stackelberg equilibrium. This allows us to plug-and-play the latest optimizers from the recent min-max optimization literature within our framework.
learning achieve SoTA results in practice. From
We also discuss practical considerations in deep networks, and compare how learning invariant representations for different choices of divergence affects the transfer performance on real-world datasets. We further discuss the practical gains (for popular f f-divergences) that can be achieved by introducing more advanced optimizers. We will release code upon acceptance.
2 PRELIMINARIES
In this paper, we focus on the unsupervised domain adaptation scenario. During training, we assume that the learner has access a source dataset of n., labeled examples S [(x] y "= and a target dataset of n, unlabeled examples T = {(x{)]² "" where the source inputs x are sampled i.i.d. from a distribution P3 (source distribution) over the input space X and the target inputs x; are sampled i..dd from a distribution P. (target distribution) over X Usually, in the case of binary classification, we have y = {0, }} and in the multiclass classification scenario, y [1,...:), k). When X' or y cannot be inferred from the context or other assumptions are required, we will mention it explicitly.
We denote a labeling function by f x -> y, and the source and target labeling functions by f. and f,, respectively. The task of unsupervised domain adaptation is to find a hypothesis func- tion : x' + y that generalizes to the target dataset T (i.e., to make as few errors as possi- ble by comparing with the ground truth label fc(z()). The risk of hypothesis h w.r.t. the la- beling function f, using a loss function é : y x y - R4 under distribution D is defined as: (h, f) : Eg> f())). We also assume that l satisfies the triangle inequality. For sim- plicity of notation,
define Rts and T refer to the source and target domains, respectively. We additionally use Rs, Ry to refer to the empirical risks over the source dataset S and the target dataset T.
R2, (h, fa) and R4) := R (h, fi) where the indices S
2

---

Under review as a conference paper at ICLR 2021
Divergence
((x)
x (t)
(1(1)
g(x)
Kullback-Leibler (KT.
x: log m
exp(t - 1) -1 log(
Reverse KI. (KI-rev)
log
exp
Jensen-Shanmen (IS)
(x + 1) log 1+z + log
log ( e²)
log T+exp(-n)
Pearson x2
(x 1 1)
t / 1
x
Total Variation (TV)
z/2 1
1 1-1/25151/2
1-1/2, 1/2]
tanh x
Table I: Popular f-divergences, their conjugate functions and choices of g. We take i(a, b) = g (argmnn)
2.1 COMPARING SOURCE AND TARGET DOMAINS WITH f /DIVVERGENCES
key component of domain adaptation is the study of the discrepancy between the source and target distributions. This differentiates transductive approaches and more generally transfer learning from traditional supervised learning methods. our work, we derive generalization bounds that capture the entire family of /-divergences. We define new discrepancies between source and target distributions, based on the variational characterization of popular choices of f-divergences. These new discrepancies play fundamental role in work.
Definition ( (f-divergence, Csiszár (1967); Ali & Silvey (1966)). Let P, and P. two distributions functions with densities ps and pr, respectively. Let ps and pt be absolute continuous with respect to base measure dx. Let R + R be convex, lower semi-continuous function that satisfies ((1) = 0. The f-divergence D is defined as:
((B()))))(((()))) Pr((r)
D&(P.1PP)) (1(()) ø
dr.
(2.1)
(r(x:)
Variational characterization of f-divergences: Nguyen et al. (2010) derive general variational method that estimates f-divergences from samples by turning the estimation problem into variational optimization. They show that any f-divergencc can be written as (see details in Appendix A.2):
(o(P)) 2 sup E [T(x)] - E 1 (T(x))
(2.2)
TET
where 6² is the (Fenchel) conjugate function of Φ R4 R defined as (3(y) SuPRER, {xy ((x)), and T' X' dom d² The equality holds if T is the set of all measurable functions. Many popular divergences that are heavily used in machine learning and information theory are special cases of f-divergences. We summarize them and their conjugate function in Table For simplicity, we assume in the following that X' C R= and each density (i.e and pt) is absolutely continuous.
3 DOMAIN ADAPTATION THEORY
Domain adaptation approaches generally build upon the idea of bounding the gap between the source and target domains' error functions terms of the discrepancy between their probability distribu- tions. Measuring the similarity between the distributions P. and P. is thus critical in the derivation of generalization bounds and/or the design of algorithms. We remind the reader of the seminal work of Ben-David al. (2010a) that bounds the risk of any binary classifier in the hypothesis class H with the following theorem:
Theorem 1. If f(. y) [h(x) yl and H is a class of functions, then for any h E H. we have:
(h) (h) + Drv + min (inn fa(x)).. Ex f+) (3.1)
Here Drv Drv(P. == suPTET Em [T) Ez [z)] is the TV and T is the set of measur- able functions. TV is an f-divergence such that (xx) = |x - 1| in Definition 1. For any function ((x) > - - 1/. one can replace Drv(P.)) in eq. 3.1 with Dp(P..)PPP. Theorem I thus bounds a classifiers target error in terms of the source error, the divergence between the two domains, and the dissimilarity of the labeling functions. Unfortunately. ( Pt) cannot be estimated from finite samples arbitrary distributions (Kifer et al., 2004). It also a very loose upper bound as involves the supremum over all measurable functions and does not account for the hypothesis class.
3

---

Under review as a conference paper at ICLR 2021
3.1 MEASURING DISCREPANCY WITH f- D-IVERGENNESS
This section introduces a new discrepancy that aims to solve the two aforementioned problems. namely (1) estimation of the divergence from finite samples of arbitrary distributions (Lemma 2) and (2) restriction of the discrepancy to the set including the hypothesis class H. (Defs. and 3). We show in Sec. 3.2 how these allows extend the bounds studied in Ben-David et al. (2010a).
Definition (De discrepaney). Let o² be the Fenchel conjugate of a convex, lower semi-continuous function 6 that satisfies 6(1) = O, and let T be a set of measurable functions such that T = {E(h(x), h'())) h. H}. We define the discrepancy between P. and P. as:
(P.II)) sup Fe=e))((((())) I.VEH
(3.2)
The D discrepancy can be interpreted divergences (Lemma 1). Therefore, for any hypothesis class H and choice of Φ, D2 is never larger than its corresponding f-divergence. We show in Lemma 2 that its computation can be bounded in terms of finite examples. Finally, we recover the HAH divergence (Ben-Davia et al., 2010a) if we consider (t) t and ((h(z), (z)) [h(x)) + h'(z),, which corresponds to the TV.
lower bound estimator of general class of f-
Definition (D hM discrepancy). Suppose the same conditions as above, the discrepancy between two distributions P, and P, is defined by:
( (Ps == sup |E (l =o ((0 MEH
(3.3)
Taking the supremum of Di, over h E H. we obtain D and thus D D (P)))P) S (P.IP)) This bound will be useful when deriving practical algorithms.
Lemma (lower bound). For any two functions h,h' in H. we have:
(h, h²) R of (h, S ( S (P.II)) S (P.IIP)).
(3.4)
Lemma 1 is fundamental in the derivation of divergence- based generalization bounds for DA. Specif- ically, it bounds the gap between the source and target domains' error functions in terms of the discrepancy between their distributions using f-divergences. We now show that the D h.H can be estimated from finite samples.
Lemma 2. Suppose f y x x y 10.10 x- L-Lipschitz. and [o. 1] C dom 6². Let S and be two empirical distributions corresponding to datasets containing n datapoints sampled ii.d from Ps and P,, respectively. Let us note the Rademacher complexity of a given class of functions, and lo H = fx E(h(x)), hi (x(x): h, h1 E H). v5 E (0, 1), we have with probability of at least 1 ō;
1 - (S < 2R, (lo H) 21 (e H) 2 log 6)/() (3.5)
In Lemma 2, we show that the empirical D converges to the true D discrepancy. It can then be estimated using a set of finite samples from the two distributions. The gap is bounded by the complexity of the hypothesis class and the number of examples (n). This result will be also important in the derivation of Theorem 3.
3.2 DOMAIN ADAPTATION: GENERALIZATION BOUNDS
We now provide a novel generalization bound to estimate the error of classifier in the target domain using the proposed D /, divergence and results from the previous section. We also provide a gener- alization Rademacher complexity bound for a binary classifier based on the estimation of the D from finite samples. We show that our bound generalizes previous existing results in Appendix D.
Theorem 2 (generalization bound). Suppose é :: y y + [0, 1] C dom o and that l(a, b) s E(a c) + É(c, b) for any a, b, c E Y. Denote X := (h)(h))) hypothesis. We have:
) + 1if (h"", and let ht be the ideal joint
(h) & R((()) Dr P.) x
(3.6)
4

---

Under review as a conference paper at ICLR 2021
The three terms in this upper bound share similarity with the bounds proposed by Ben-David et al. (2010a) and more recently by Zhang et al. (2019). The main difference lies in the discrepancy being used to compare the two marginal distributions. In the case of Ben- -avid et al. (2010a), they use the HAH divergence (a reduction of the TV), and in Zhang et al. (2019), they use the MDD. In our case, we use reduction of lower bound estimator of variational characterization of the general f-divergences. This generalizes the TV (and thus Ben-David et al. (2010a)) and also includes popular divergences typically used in practice (see Appendix D). Intuitively, the first term in the bound accounts for the source error, the second corresponds to the discrepancy between the marginal distributions, and the third measures the ideal joint hypothesis (A*). If H is expressive enough and the labeling functions are similar, this last term could be reduced to a small value. The ideal joint hypothesis incorporates the notion of adaptability: when the optimal hypothesis performs poorly in either domain, we cannot expect successful adaptation.
Theorem 3 (generalization bound with Rademacher complexity). Let E: y: x y + [0, 1] and o be L-Lipschitz. Let S and T be two empirical distributions (i.e. datasets containing n data points sampled Li.d. from P3 and P, respectively). Denote X2- == Rkg())) + RG, (ht). 45 E (0, 1), we have with probability of at least 1 - 6: (h) < (h) + = + XØ
+ 6e H) + 2(1 + L) (4 H. + +5
(- log (2n)
(3.7)
In Theorem 3, we show the computation of our generalization bound for a binary classifier in terms of the Rademacher complexity of the class H. We see that under the assumption of an ideal joint hypothesis, Az the generalization error can be reduced by jointly minimizing the risk in the source domain, the discrepancy between the two distributions, and regularizing the model to limit the com- plexity of the hypothesis class. We take all these into account when deriving practical algorithms in the next sections.
4 f J-DOMAIN ADVERSARIAL LEARNING (f-DAL)
We now use the theory presented in the previous sections to derive a novel generalized domain adversarial learning framework. The key idea of domain-ad the source error and align the two distributions in a representation space Z. Specifically, we let hypothesis h be the composition of h = hog (i.e let H := {hog: h E H. E 9) with H. another fune- tion class) where g : X + 2. This can be interpreted as a mapping that pushes forward the two densities ps and pt to a representation space Z where classifier h E H. operates. Consequently, we refer to 1² == 9#P, and pe == h as the push-forwards of the source and target domain densities, respectively. Figure 2 illustrates the f-DAL framework.
training is to simultaneously minimize
Z
i'
Figure 2: f-DAL. framework. We interpret h X + as the composition of two nets h h g. where g: X' + Z and h is classifier that operates in rep- resentation space 2. Inspired hy the theory, let H be another net of the same topology than h. This is intuitively interpreted as a per-category domain classi- fier. Our framework is different from works that follows from (Ganin et al., 2016) since they use a global domain-classifier or dis- criminator.
Clearly from Theorem 2, for adaptation be possible in the repre- sentation space Z, there has to be an h e H. such that the ideal joint risk X is negligible. This condition is necessary even if p pt' In other words, we need the difference between 12 and y to be small, and the ideal joint risk >" to be negligible. These are both sufficient and necessary conditions. We refer the reader to ben-David et al. (2010b) for details on the impossibility theorems for DA. Conse- quently, we state the following:
frame-
Assumption There a g E 9 and h-- E H. such that the ideal joint risk (X*) is negligible. We also assume that the class- conditional distributions between source and target are similar.
While these assumptions may seem restrictive, they are ubiquitous in modern DA methods, including SoTA methods i. Ganin al. (2016); Long et al. (2018); Hoffman et al. (2018b); Zhang et al. (2019)
Also referred to as the coovariate-shift assumption (Shimodaira, 2000). The source and target domains only differ their marginals according to the input space.
5

---

Under review as a conference paper at ICLR 2021
(sometimes not explicitly mentioned). Moreover, neural networks generally known to be able to learn rich and powerful representations, and in practical scenarios, g and h are both neural networks.
From Theorem and Assumption 1, the target risk (1) can be optimized by jointly minimizing the error in the source domain and the discrepancy between the two distributions. Letting y S. (2),, an optimization objective can be clearly written as:
min Esngy (00m::g((((())))))))))))))))) y)1 | + D h. (pEllPF). hER
(4.1)
Here, é is a surrogate loss function used to minimize the empirical risk in the source domain. Nonetheless, it does not have to be the binary classification loss (i.e it can be the cross-entropy loss). Under some assumptions (Proposition 1) and the use of Lemma I, the minimization problem in equation 4.1 can be upper bounded (hence replaced) by the following min-max objective":
min max Ez (( + Ez (((P h(=))] - Ez~ri l(o* (x), h(=)) RER ME
(4.2)
da,t
where we refer to the difference between the last two terms as dg.t- We now formalize this result.
Proposition 1. Suppose da, takes the form shown in equation 4.2 with ((h((z), h(z)) + dom ø*
forany and that for any h E H. there exists h' E H s.I. (e))))())))) = 2 for any 2 E supp(ri(=)),
with o' the derivative of é. The optimal da,t is (P(P;))P (i.e max dett = (P:
If we let the feature extractor E G be the one that minimizes both the source error and the discrep- ancy term, equation 4.2 can be rewritten as:
min max Eewn, [e(h o v)l + En 1&( o g. h e g)] - -Ermilé' o ( o g, h g)l. (4.3) kERgeG MEH
The choice of i is "somewhat arbitrary" as stated for GANs in Nowozin al. (2016). For the multiclass scenario, we let ((a g (Pargmax .)), where argmax is the index of the largest element of vector a. For the binary case, we define b(-,b)- = g(b). This implies that we choose the domain of ê to be RK x R& with k categories for the multi-class scenario and IR for binary classification. Intuitively, h'' is an auxiliary per-category domain classifier. Note that this is different from Ganin et al. (2016) where there a unique domain classifier or discriminator: For the choice of g. we follow Nowozin et al. (2016) and choose it to be monotonically increasing function when possible. We summarize our choices of for different f-divergences in Table 1. We also show other choices of l including generalizations of previous methods in Appendix D.
w-weighted JS divergence. If we relax the need for ((1) = 0 in Proposition 1, the new objective only shifts by constant, c.g, m ds,: = D3 (P: + o(1) with ō(x) == Φ(z) 6(1). By Lemma 4 (Appendix D), we can then rescale Φ², and & will change accordingly. This allows to include the practical objective from Zhang al. (2019) as part of our framework (i.e. w-weighted JS, see Appendix D). While these can be done for the general family of divergences, we do not pursue this line of research in practice as requires additional hyperparameter tuning 7
4.1 OPTIMALITY IN f-DAL
The main objective of our framework (i.e equation 4.3) is a minimax optimization problem and our desired (optimal) solution under mild assumptions a Stackelberg equilibrium. This key obser- vation allows us to incorporate in our framework the latest optimizers from the ootmmoopiiiiationn literature. We now formalize and prove this concept. Based on this, we propose use the extra- gradient algorithm and its aggressive version within our framework. We show the effectiveness through a toy example (Figure 3, Appendix E) and also empirically in our large scale experiments.
Stackelberg equilibria in f-DAL. We show the existence of Stackelberg equilibria in f f-Domain Adversarial Learning (f-DAL). Let G and H be class of functions defined by fixed parametric functional (i.e neural networks with fixed architecture). and define w1 such that is a vector composed
Indeed, under these assumptions, d can be seen as upper bound for the D discrepancy.

---

Under review as a conference paper at ICLR 2021
of the parameters of the feature extractor g and the source classifier h. Similarly, let w2 be the pa- rameters of the auxiliary classifier i', and 2i and f2 denote their separate domains. Equation (4.2) can be rewritten as:
min max (ww), we wi 2l2
(4.4)
In general, V is nonconvex in Wi and nonconcave in w2, and for the min-max game in equation 4.4, Nash equilibria may not exist as in (Farnia & Ozdaglar, 2020). A Stackelberg equilibrium is more general than Nash equilibrium (see definition in Definition 5) and reflects the sequential nature of our zero-sum game equation 4.4. We now show that the optimal solution of f-DAL. is a Stackelberg equilibrium. Such an equilibrium is stationary point under the assumption that V (wi, -) is (locally) strongly concave in W2 (Evtushenko, 1974), and we can then use gradient algorithms to search for such a desirable solution. In the following theorem, we use the explicit form of push-forward to emphasize the dependence on the feature extractor 9. rather than p'.
Theorem 4 (Stackelberg equilibrium, informal). Suppose da, takes the form shown in equa- tion 4.2, and assume that (a) There exists an optimal g* 9 that maps both the source and the (b) There exists an optimal classifier that yields the ground truth in a neighborhood. (c) For any g E G and h E H. there exists h' that achieves Then the objective of f-adversarial learning has
target distribution to the same distribution.
((). h(=)) = ((ooottp)))))(ptp)))))) Stackelberg equilibrium at (h'g'". hi
When E(h, =((h') (e.g., in a binary classification scenario or in Ganin et al. (2016)), the Stackelberg equilibrium can be shown to be a Nash equilibrium (see Theorem 6).
Extra-gradient algorithms. We have shown that the optimal solution in f-DAL is a Stackelberg equilibrium which is more general than Nash equilibrium. For convergence to a Nash equilibrium, the sim- plest method is GDA. However, the last iterate of GDA does not con- verge in the bilinear case (e.g. Nemirovsky & Yudin, 1983). To accel- erate and stabilize the convergence, the extra-gradient (EG) method was proposed in (Korpelevich, 1976). It was recently shown (Zhang al., 2020; Hsieh et al., 2020) that having is even more stable than vanilla EG, and is more suitable for conver- gence Stackelberg equilibria (Zhang et al., 2020). With the aim of quantifying whether exploiting Theorem 4 leads to practical gains, we follow those works, and therefore let the extrapolation step to be larger. We refer to this algorithm as Aggressive ExtraGGradient (AExG). We illustrate this in simple example (Appendix E), whose convergence/urajectoriss are shown in Figure 3 and we will explore AExG further in the experimental section.
Figure 3: Comparison of GDA vs AExG in a toy task with JS as divergence. G is the class of quadratic functions and H. linear. AExG can accelerate the convergence to the optimal solution. (Appendix E)
aggressive extra-step
5 EXPERIMENTAL RESULTS
We present experimental results of our framework in practical scenarios. In these scenarios, the learner is a neural network and the input domain is the set of natural images. Specifically, we aim to answer the following questions: (1) How does choosing particular divergence affect the do- main adaptation performance among different datasets? (2) Is there a better universal notion of f-divergence that achieves significant performance gains across different datasets and thus helps generalization? (3) Are there considerable practical gains by exploiting the fact that the optimal solution of f-DAL a Stackelberg Equilibrium? (4) How does our theoretical framework compare in practice to existing SoTA methods? We also compare in Figure 4 the difference in interpreta- tion of the auxiliary classifier of f-DAL vs Ganin al. (2016). The comparison shows significant performance boost for the same divergence i. DANN vs f-DAL (JS)
Datasets. In our experiments we use two main datasets. We use (1) the Office-31 dataset (Saenko et al., 2010) which contains 4,652 images and 31 categories, collected from three distinct domains: Amazon (A), Webcam (W) and DSLR (D). We also use (2) the Office- Home dataset ( (Venkateswara et al., 2017). This is more complex dataset containing 15,500 images from four different domains: Artistic images, Clip Art, Product images, and Real-world images. In each of our experiments, we report the average over 3 different seeds.
7

---

Under review as a conference paper at ICLR 2021
Method
A W
D W
W
A
D- A
W A
Avg
ResNet-50 (He et al., 2016)
68.400.2
96.7+0.1
99.3+0.1
68.9=0,2
62.5+0.3
60.7+0,3
76.1
DANN (Gamin et al, 2016)
82.010.4
96.940.2
99.110.11
79.7:0..4
68.24044
6..410.5
82.2
ADDA (Tzeng et al. 2017)
86.2+00.5
96.240.3
98.4+0.3
77.8+0.3
69.5+044
68.9+0.5
82.9
JAN (Long et al., 2017)
85.4
97.410.2
09.8+0.2
84.7=0.3
68.640.3
70.0=
84.3
GTA (Sankaranarayanan
al., 2018)
89.5++0.5
97.940.3
99,8+004
87.740.5
7288+0.3
71440.4
86.5
MCD (Saito al., 2018)
88.6+:0.2
98,5+0011
100,0=200
92.2=0.2
69.5+0.11
69,7+0,3
86.5
CDAN Long et al. (2018)
94.1++0.1
98.6++..1
100.0+00
92.9+0.2
71.0+-0.3
69.3+0.3
87.7
MDD (Zhung et 2019)
94.5++0.3
98, 98,4+0.1
100.0::0
93.5=0.2
74.6
72.240.1
88.9
Ours (Pearson x², SGD)
95.4 +-0.7
98.4 1..4+00.2
1000+-0
93.8 40.4
+3.5
74.2 +0.5
89.2
Ours (Pearson x². AExG)
95.3 1:0.1
98.8 + 0.3
100.0:10
94.2 0066
73.3
1.0.3
75.3
#0.2
89,5
Table 2: Comparison vs previous unsupervised domain adaptation approaches on the Office-31 benchmark. Accuracy represented in (%) with average and standard deviation. Impressively, our approach achieves SoTA results without the need of additional techniques (i.e CDAN) or additional hyperparameters (i.e MDD/
Implementation Details:
We implement our algorithm in PyTorch. We use ResNet-50 (He et al., 2016) pretrained on ImageNet (Deng et al., 2009) as the feature extractor. The main classifier (h) and auxiliary classifier (h') are both 2 layers neural nets with Leaky-Relu activation functions. We use spectral normalization (SN) in Miyato et al. (2018) only for these two (i.e h and h' ). We did not see any transfer improvement by using it, neither by using Leaky-Relu activation functions instead of Relu. The reason for this was to avoid gradient issues and instabilities during training for some divergences (i.e KL, TV) in the first epochs. For simplicity, and fair comparison with previous work, we perform simultancous updates using the GRL. We also use the GRL warm-up strategy. This is standard in most DA frameworks and follows from Eq. (14) in Ganin & Lempitsky (2015). For optimization, we use 1) mini-bated (32) SGD (or GDA) with the Nesterov momentum (0.9). 2) For experiments using AExG, we take inspiration from Gidel et al. (2019) and implement version of the ExtraGradient with momentum (0.9). For the aggressive step, we use a multiplier [10, 1] with polynomial decay rate with power= 0.5 for the first 10K iterations. In all cases, the learning rate of the classifiers is set 10 times larger than the one of the feature extractor (0.01) whose value is adjusted according to Ganin et al. (2016), which is standard practice. We will release source code.
Comparing fddivergences. We first compare the performance of f- divergences on Office-31. Specifi- cally, we evaluate the performance of the model on the six combinations of transfer tasks with different diver- gences. The optimizer is SGD with Nesterov Momentum. All hyperpa- rameters are kept constant for all di- vergences. As shown in Figure 4, the Pearson x² achieves the best over- all result among all the transfer tasks on this benchmark. This divergence was never used before to learn in- variant representations in the context of DA. Interestingly, a similar trend was observed for GANs in Nowozin et al. (2016). This observation is also reminiscent of histogram-based (visual) bag of words representations a per-category domain classifier vs a discriminator.
BatalinePas-022
DANN
KL
KL-m
TV 15 Peanson
Figure 4: Transfer performance of model trained using f-DAL for different choices of divergences and different transfer tasks on the Office-31 benchmark. Baseline is ResNet-50 w/o f-DAL. We additionally show the performance of DANN (Table 2). When compared with S-DAL (JS), boost. This line with our theory which suggests the use of
significant performance
that were shown to work better with x² distances than with E and l distances for image and text classification tasks (i.e. (Li et al., 2013) and references therein). The KI. divergence performs well in some transfer tasks but significantly worse than the rest others (i.e - A). The reason might be that, unlike the JS, TV and Pearson x² divergences which are lower and upper bounded by finite values, the KL divergence can grow exponentially and tend to too even when the densities ps and Pi are nonzero (Nielsen & Nock, 2013). The lack of upper bound of the KL divergence might lead to numerical instability of the optimizer and explain inconsistency of performance.
What do we get by the "extra" gradient? We now compare the use of the AExG vs GDA method with Nesterov Momentum. The main idea is to evaluate whether the characterization of the optimal solution of f-DAL Stackelberg Equilibrium leads to practical gains by exploiting more suitable
8

---

Under review as a conference paper at ICLR 2021
1.5
0.1
0.-
5CDNeraro
CasdColggrsie
3.30 02-45
W--6 AV20
Phenzan
Figure 5: left) Relative Improvement AExG vs GDA(SGD) for different choices of divergences and transfer tasks the Office-31 benchmark. Overall, observe gains in performance among all divergences. right) Transfer curves for Pearson x² on the task A-W on Office-31 benchmark (# Iter vs Acc). We can see AExG converges faster and also obtains slightly better results, This is inline with the insights obtained from the thcorctical results presented in Sec 4.1 and Appendix 5.
Ar-+C)
Ar-Pr
Ar-RRw
Cl-Ar
CI-Pr
Ci-Rw
Pr-AA
Pr-+CI
Ru-+CI
Rw-FF
Ang
ResNet- (lle
DANN (Ganin et 2016)
701
51.8
JAN (Long al. 2017)
43.9
61.2
68.9
59.7
61.0
45.8
43.4
70.3
63.9
52.4
58.
CDAN [Long et al. 2018)
50.2
70.6
76.0
57.6
70.0
70.0
$7.4
50.9
77.3
70.9
56.7
81.6
65.8
MDD (Zhang et al. 2019)
54.9
73.7
778
60.0
71.4
71.8
61.2
53.6
76.1
72.5
60.2
82.3
68.1
Ours (Peanon v2, SGD)
$4.7
69.4
77.8
61.0
72.6
12.2
00.8
53.4
80.0
72.3
60.6
8388
68.3
Ours (Peanson v AlxG)
55.2
70.2
7866
60.9
732
61.3
80.5
73.7
61.0
83.6
68.7
unsupervised DA. Impressively, our approach achieves SoTA with- out additional techniques (i.e. CDAN), or additional hyperparameters (i.e. MDD).
Table Accuracy (%) on Office-Hone
optimizers (Section 4.1). In both cases, we use momentum coefficient (0.9). The experiments are performed on Office-31 1 and hyperparameters are kept constant for all divergences. In Figure we observe that using AExG significantly improves the performance in some transfer tasks for some di- vergences (i.e IS in A - W). Overall, we also observe gains in performance among all divergences. Figure 5 illustrates the transfer curves of the AExG vs GDA with Nesterov Momentum for the task A- W. For this divergence and pair of datasets, AExG converges faster and also obtains slightly better accuracy. This is in line with the insights obtained from the theoretical results presented in Section 4.1 and Appendix 5. If computation not an issue we encourage the use of AExG. That said, fDL achieves comparable performance with GDA in terms of accuracy (see Tables 2 and 3).
look to state-of-the-art arena. We also compare our best method (i.e Pearson x² + AEXG and Pearson x² + GDA) with current SoTA unsupervised DA methods. We use this in a leader-board like fashion. For fair comparison, we use the same network architecture (i.e Resnet-50), training strategy and of hyprraraaeeeess from Long et al. (2018); Zhang et al. (2019) from where we took the baselines results. Our approach achieves SoTA results on Office-31 and Office-Home Datasets.
We compare with SoTAs that rely on adversarial training since this is the focus of our work. Therefore, methods such as Kang et al. (2019) are not included as they rely on additional techniques that neither our method nor the proposed baselines use and could be added to improve the performance further. Our goal is to propose unifying framework that connects the theory used explain DANN (Ganin al., 2016) (and similar SoTA algorithms), and the algorithms themselves. The new theory results in new adversarial framework (Sec 4), which impressively outperforms previous SoTA (Tables 2 and 3). This follows from connecting theory and algorithms and a proper interpretation of the former. Our results can be further improved with additional tuning or techniques (i.e CDAN) since most SoTAs either follow from Ganin et al. (2016) or are part of our framework (i.e MDD= 7-JS). This problem is deferred to future work.
Remark SoTA Comparison:
CONCLUSIONS
We have provided a novel perspective on the and learning algorithms that support the complete family of f-divergences, and that are practical for modern neural networks. We further recognize the learning objective of our framework as Stackelberg game, borrowing the latest optimizers from the same-optimization literature, achieving additional performance boosts. We show through large-scale experiments that any f-divergence can be used to minimize the discrepancy between source and target domains in a representation space. We also show that some divergences, not considered previously in domain-adversan learning, achieve SoTA results in practice, reducing the need for additional techniques and hyperparameter tuning required by previous methods.
problem by deriving new theory

---

Under review as a conference paper at ICLR 2021
REFERENCES
Syed Mumtaz Ali and Samuel Silvey. A general class of coefficients of divergence of one distri- bution from another. Journal of the Royal Statistical Society: Series (Methadological), 28(1): 131-142, 1966.
Shai Ben- Den-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In Advances in neural information processing systems, pp. 137-144, 2007.
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory of learning from different domains. Machine learning, 19(1-2)1511-15,,
2010a.
Shai Ben-David, Tyler Lu, Teresa Luu, and Dávid Pál. Impossibility theorems for domain adap- tation. In Proceedings of the Thirteenth International Conference Artificial Intelligence and Statistics, pp. 129-136, 2010b.
Patrick Billingsley. Probability and measure. John Wiley & Sons, 2008
Stephen Boyd and Lieven Vandenberghe. Corvex optimization. Cambridge university press, 2004.
Imre Csiszár. Information-type measures of difference of probability distributions and indirect ob- servation. studia scientiarum Mathematicrrum Hungarica, 2:229-318, 1967.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, IEEE, 2009.
Yurii Gavrilovich Evtushenko. Some local properties of minimax problems. Zhurnal Vychislitel noi Matematiki Matematicheskoi Fiziki, ((33)669-679, 1974.
Farzan Farnia and Asuman Ozdaglar. GANs may have no Nash equilibria. In International Confer- ence on Machine Learning, 2020.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference machine learning, pp. 1180-1189, PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural net- works. The Journal of Machine Learning Research, 17(1):209-2200 2030, 2016.
Gauthier Gidel, Hugo Berard, Gaëtan Vignoud. Pascal Vincent, and Simon Lacoste-Julien. A varia- tional inequality perspective on generative adversarial networks. In ICLR, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Judy Hoffman, Mehryar Mohri, and Ningshan Zhang. Algorithms and theory for multiple-source adaptation. In Advances in Neural Information Processing Systems, pp. 8246-8255 2018a.
Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Sycle-consistent adversarial domain adaptation. In International conference on machine learning, pp. 1989-1998. PMLR, 2018b.
Yu-Guan Hsieh, Franck lutzeler, Jérôme Malick, and Panayotis Mertikopoulos. Explore aggres- sively, update conservatively: Stochastic extragradient methods with variable stepsize scaling. arXiv preprint arXiv:2003. 10162, 2020.
Ferene Huszár. How (not) to train your generative model: Scheduled sampling, likelihood, adver- sary? arXiv preprint arXiv: 1511 151.001011.000111111101 2015.
Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4893-4902, 2019.
10

---

Under review as a conference paper at ICLR 2021
Daniel Kifer, Shai Ben-David, and Johannes Gehrke. Detecting change in data streams. In VLDB, volume 4. pp. 180-191. Toronto, Canada, 2004.
GM Korpelevich. The extragradient method for finding saddle points and other problems. Matecon, 12:747-756, 1976.
Ping Li, Gennady Samorodnitsk, and John Hoperoft. Sign eauchy projections and chi-square kernel. In Advances in Neural Information Processing Systems, pp. 2571-2579, 2013.
Mingsheng Long. Han Zhu, Jianmin Wang, and Michael Jordan. Deep transfer learning with joint adaptation networks, 2017.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. In Advances Neural Information Processing Systems, pp. 1640-1650, 2018.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In Proceedings of The 22nd Annual Conference on Learning The- ory (COLT 2009), Montréal, Canada, 2009. URL http: http:://wwww../ www.cc..... www.nuu....u..eee...e postscript htapst/nnaappppp
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In International Conference on Learning Representations, 2018.
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.
Arkadii Semenovich
Nemirovsky and David Borisovich Yudin. Problem complexity and method efficiency in optimization. Wiley-Interscience series in discrete mathematics, 1983.
XuanLong Nguyen, Martin J Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 2010.
56((1):58477
Frank Nielsen and Richard Nock. On the chi square and higher-order chi distances for approximating f-divergences. arXiv preprint 1309. 2013.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.). Advances in Neural Information Processing Systems 29, pp. 271-279, Cur- ran Associates, Inc., 2016.
levgen Redko, Emilie Morvant, Amaury Habrard, Mare Sebban, and Younès Bennani. Advances in Domain Adaptation Theory. Elsevier, 2019.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In European conference on computer vision, pp. 213-226. Springer, 2010.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier dis- crepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pp. 3723-3732. 2018
Swami Sankaranarayanan. Yogesh Balaji, Carlos D Castillo, and Rama Chellappa. Generate to adapt: Aligning domains using generative adversarial networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 8503-8512, 2018.
Igal Sason and Sergio Verdú. f-divergence inequalities. IEEE Transactions on Information Theory,
62(11):5973-606, 2016.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of statistical planning and inference, 90(2):227-244, 2000.
Rui Shu, Hung Bui, Hirokazu Narui, and Stefano Enmon. A dint-t approach to unsupervised domain adaptation. In International Conference Learning Representations, 2018.

---

Under review as a conference paper at ICLR 2021
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation, 2017.
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In (IEEE) Conference on Computer Vision and Puttern Recognition (CVPR), 2017.
Mei Wang and Weihong Deng. Deep visual domain adaptation: A survey. Neurocomputing, 312: 135-153, 2018.
Guojun Zhang, Pascal Poupart, and Yaoliang Yu. Optimality and stability non-convex smooth games. In arxiv: 2002. / 1875, 2020.
Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 7404-7413, Long Beach, California, USA, 09-15 Jun 2019. PMLR. URL http:/ /ttp://poocceenssseeeee.e mlr press v /aazzaaanggg 91. html.
12

---

Under review as a conference paper at ICLR 2021
A RELATED WORK
A.1 DOMAIN ADAPTATION
We briefty but clearly positioned approach w.r.t. related work mentioned in the paper. We refer the reader to Redko et al. (2019) and Wang & Deng (2018) for comprehensive survey.
A.2 DIVERGENCES BETWEEN PROBABILITY MEASURES
As explained above, the difference term between source and target domains is important in bounding the target loss. We now provide more details about the -iAA--iivegeece and f-divergences that are used compare both domains.
HAH-divergence The H-divergence is a restriction of total variation. For binary classification, define (h) == {x X h(x) = 1}. then the H-diivegennce between two measures μ and given the hypothesis class H is Den-David et al., 2010a):
dz (1,, v) = 2 sup (m(I(h)) - v(T())))-
(A.1)
NEH
Define HAH == {h a W h h H} (e: XOR), then dH v) can be used to bound the difference between the source and target errors. HAH divergence has been extended to general loss functions (Mansour et al., 2009) and marginal disparity discrepancy (Zhang et al., 2019).
f-divergence Given two measures l and v with μ < v (μs absolute continuous wrt v), the f- divergence (o(uilp) is defined as (Csiszár, 1967; Ali & Silvey, 1966):
1) = f
du
Do(u [I
dv,
(A.2)
dv
where dp/dv is known as the Radon-Nikodym derivative (e.g. Billingsley, 2008). Assume Φ is convex and lower semi-continuous, then from the Fenchel-Moreau theorem, 6 = o, with Ø² known as the Fenchel conjugate of Φ: 8" (y) sup (x,y) - -xx))
(A.3)
xEdom
which is convex since a supremum of an affine function. In order for x to take the supremum, it is necessary and sufficient that y E 0Ø(x) using the stationarity condition. Therefore, with equation A.2 and equation A.3, D [4] || 1)) can be written as:
Do(u v) sup Ex~x[T(X)) TET
Ez 10 (T(Z))),
(A.4)
where T = {T': : is a measurable function and T X + dom }. In practice we restrict T to a subset in Definition 2. For different choices of & see Table 4.
Nguyen et al. (2010) derive a general variational method to estimate f f-divergences given only sam- ples. Nowozin et al. (2016) extend their method from merely estimating a divergence for fixed model to estimating model parameters. While our method builds on this variational formulation, we in the context of domain adaptation.
B PROOFS
In this section, we provide the proofs for the different theorems and lemmas: Theorem 1. If ff v) [h(x)) yl and H is a class of functions, then for any h H we have: (h)( < (h) +
+ min(Es=..lf((()) -
- (3.1)
Proof. Rewriting the target loss we have:
(h) (h) R/, f() R2 (h. ft) - R²((h) Rh),
< (h) + (h) R$ f()l [b) - (h)
13

---

Under review as a conference paper at ICLR 2021
Divergence
((x)
s'(t)
6(1)
g(x)
MDD
log If7z
7 log 1+70
log(1 - - e¹)/T
log -
log
Kullback-Leibee (KL)
log ar
exp(1 1)
Reverse (KL-revv
log
-1 log(
exp=
Jensen-Shannon (JS)
(ar + +1) log 1ta + 2 log
log(2 -d)
log
2
[+oxp(-x)
Pearson x
(x 1)a
1/4+1
Squared Hellinger (SH)
(Vā 1)2
exp
7-weighted Pearson x²
(ya 1)2/7
(P 1/44t///
Neymman x
(1-x)²
2 2-221-2
exp tanh x tanh
y-weighted total variation
h 11
(t/7)1-1/9s1s//2
1/2 1/2]
3
Total Variation (TV)
1--
1-1/2/151/2
[ 1/2, 1/2)
Table 4: Popular f- d-iieeggences, their conjugate functions and choices of g. We take i(a, b) g(Durgann
where:
(1) - (h, fe)l = (h,
(h, Js) - St) [h(x)) fa(lll
Ez [1((x)
< Ex (lf((z)) fa(a)]
and:
(1) - (b, [4)1 (h, (h) / [m(x)) - -x(()) (M() - sx(x)/dr
/ ( (1(x) Px(x)
1)p((z)ldr = Do(P311P)
with 6(x) = =Ix which represents the total divergence.
Lemma (lower bound). For any two functions h,h' in H. have:
(g, Rq." (h, < (P.IP)) &
(3.4)
Proof.
(P.( = sup D
h.H (P.IP) 2 D (P.(PP)
(B.1)
NEW
sup [e<p:(((((z). (( (x(x)) WEW
Ex ((((()) (((M())
(B.2)
2 FE (((h()), (1(x()))) - E E (((M()
(B.3)
= IR& (h, h²) - R2 (h, b-)l-
(B.4)
For the rightmost inequality in equation 3.4, it is well-known that f-divergence Do is nonnegative (e.g. Sason & Verdú, 2016), and thus
D& (P( = sup E T(x) (o
(B.5)
TET
Restricting T to T as in Definition 2 we obtain (o((I)))) (P..IP))
Lemma 2. Suppose é y x y + 1]. 8" L-Lipschitz. and [0, 1] C dom 6*. Let and be two empirical distributions corresponding to datasets containing datapoints sampled i.i.d. from P. and P,, respectively. Let us note the Rademacher complexity of a given class of functions, and (o H. = {I E(h(x)) h' (x)) h h' H}. 45 (0, 1), we have with probability of at least 1 - 5:
DD (P.l(PP) D (SIIT) & 2R, (E H) 21.9 n (1 HH) 2 log 0)/())).
(3.5)
14

---

Under review as a conference paper at ICLR 2021
Proof. For reference, refer the reader to Chapter 3 of Mohri et al. (2018). Using the notations of R and R that represent the true and empirical risks, we have:
(P) - D (S|IT) = sup [1Rg (h, h²) R4 of (h, I)D
(B.6)
WEH
sup (1/gs (h, 11') o (h, N'I
MEH
< sup [[ h1) - B8G of (hh R')l - (h, R1) - RRG of (h, 1()l
WEW
S sup (R' h/) n of (b, h') -R, h/) + Rrg of (h, h')l
NEH
sup [Rgg(,, 1/) - (h, 1()1 + [eq of (h, R2g (h, h()l
NEW
log 3/4
log
S p.p (e H) +
2P (0 000 H) + V
2n
2n
where: (R' R²) )- (h, n²I S 2Rp, (eo H- + V
log
TThorem 3.3 of Mohri et al. (2018))
2m
Similarly, by Talagrand's lemma (Lemma 5.7 and Definition 3.2 of Mohri et al. (2018)) we have: Ro o o H) S pR (L o H), with o* ofo H := {z + (((((()), (x)))) h, E H}.
Theorem 2 (generalization bound). Suppose E y x y - 10. 1] C dom 6" and that l(a, b) S ((a,e) + E(c, b) for any a, a,b, V. Denote X = (1r( (h"), and let h² be the ideal joint hypothesis. We have;
(h) S (b) + D (P. +x-
(3.6)
Proof. We first introduce the following lemma for our proof:
Lemma 3. For any function & that satisfies Φ(1) = 0 we have (²())= 2 t where o² is the Fenchel conjugate of ø.
Proof. From the definition of Fenchel conjugate, (*(t)= = -SUP lot - x(x)) tt- =
With the triangle inequality of e, we can write:
RCt (h, ft) < (h, (h, (h,
(B.7)
(h, + (er(",,,, - (h, kr)
(B.8)
S Ry (h, h1)) - Rl) (hgh, R4 (h", ft)
(Lemma 3)
(B.9)
< |e of (h, 11" - (ts(h,1') (h, + R2 (h", St)
(B.10)
Di (Pal + RK( + R2 Jt)
(Lemma 1)
(B.11)
Dr Dz (P.l + (2,
(h" fa) + +R fc).
(B.12)
Theorem 3 (generalization bound with Rademacher complexity). Let E: y y x (0, 1] and o² be L-Lipschitz. Let S and T be two empirical distributions (i.e. datasets containing n data points sampled ii.dd from P, and P,, respectively). Denote Xх. == (2'g(h)) + RG (h*). 48 E (0, 1), we have with probability of at least 1 - 6:
R2 (h) (2)(() (S][TT + A
69 H) + 2(1 1.)97 (1 H) + 5 (ov-(llog log (2)
(3.7)
15

---

Under review as a conference paper at ICLR 2021
Proof. We show in the following that:
(h) - It's (h) + D (S|IT) + X2
(B.13)
+ 6e o H)) + 2(1 + () + + (og5l//olg)//// (2mm)
(B.14)
This follows from Theorem 2 where: (h) 2 (h) + D (P.I) + (hg())) + (h").. We
+
also have: (1) R2, (1)) S (e H) +
log 2n
(Theorem 3.3 Mohri et al. (2018)). From
Lemma 2, D H (P)) & 2Rp (l o H) + (e H) + 2V gives the desired results.
log 1 2n
Plugging in and rearranging
Proposition 1. Suppose da, takes the form shown in equation 4.2 with ((h((z), h(=)) + dom o* and that for any h E H. there exists h E H s.I. {(h'(s), = = with o' the derivative of é. The optimal da is (P(P))PP)) (i.e ma de,e = (PPl:()))
k+ m
for any 2 E supp(rf(=)), forany
Proof. We first rewrite from the definition of e in equation 4.2:
da = E Ezmp ((((())))h()))) - E. E. ())(
(B.15)
J ((())(((())) h(=)) (x=) ()),
(B.16)
(x(=)) (()) (0 ()), M())
vE(2)
(z),
de.
(B.17)
(T(2)
Maximizing w.r. h and assuming H is unconstrained we have: any supp(rt). From the definition of Fenchel conjugate have:
E (E() for गo
x Z 000((t)) + <(x) + + at = d'(z) =t
Plugging x = (=) and = (x)' ((),)()))=)))))))))))))))))))( we obtain ((h(()), (())) = d(()())\/((t)) Hence, from the definition of f-divergences (Definition 1) and its variational characterization (cq. 2.2), we write:
max da,t = Pp(P..)PP)) WeR
(B.18)
C THE EXISTENCE OF THE STACKELBERG/NASH EQUILIBRIUM IN f-DAL
In this appendix, we formally define the Nash/Stackelberg equilibrium and show that they exist in our f-DAL framework under mild assumptions.
Definition 4 (Stackelberg equilibrium), A Stackelberg equilibrium (wf, w2) E Qi x Oi of the min-max game sutisfies V(w1, E Qi 92, V V(wi, w2) S V(Wī, w/) S maxwges, (w). Definition 5 (Nash equilibrium). A Nash equilibrium (WT,W)) E O x Q2 of the min-max game equation 4.4 is defined such that V(w),w2) Ωi x D2, V(wi,w2) S V(Wi,Ws S V(w),w)).
Theorem 5 (Stackelberg equilibrium). Suppose da,t takes the form shown in equation 4.2, and assume that (a) There an optimal feature extractor g" G that maps both the source and the target distribution to the same distribution, i.e. g" #p, fier s.t. h o 9" fo is the ground truth, and h o fs for any (h.9) in neighborhood of ("",g") (c) For any g E G and h H. there exists h' s.t. for any z supp(g one has (())) = o'
#p. (b) There is an optimal classi-
Then the objective of f-udversarial learning has Stackelberg equilibrium at (ht, g"' He where Vz E supplg" #ps), (()((2)) (z)) = o()).
Proof. At g" and hr", we have
(h" .g" W0 Fizng #r. (z), (3
(C.1)
16

---

Under review as a conference paper at ICLR 2021
Maximizing over (x)) (z))) yields:
((h' hi (z)) (1), Vz supp(g' #ps).
(C.2)
In other words, dh (ichh""giii hh) & daalhi, ,g", ,h') for any h H. Now us prove that e de.lhhgghh) 2 dealh'.g', and Lemma I, maxweR da,lh, 1, = Da 20. O.
o(1) O. This is because from our assumptions
So far, have shown that
de,(h", .². hi) s dh, g", "",h'' max da (h, h')
(C.3)
for any h,h' H. and g E g. Also, (h', 9¹) is an optimal pair for the source loss, namely:
R,(h" gg) S R, (h,
(C.4)
for any H and g E G. Combining equation C.3 and equation C.8, we claim that (h',g, gg" hi") is Stackelberg equilibrium.
Theorem 6 (Nash equilibrium). With the same assumptions in Theorem 5 and assume also that l only depends the second argument, i.e., ((h,hh) E(h)). Then the objective of f-adversarial learning has a Nash equilibrium at (h",g", , where
V2 E U supplo#p.), E(1e" (x)) = (1).
(C.5)
gEg
Proof. The proof is in parallel to the proof of Theorem 5 except that at i/*
de.olghh*) h()) = E (x)) - Es (2))
o'(1) o² (d'(1))
(C.6)
is constant in terms of g, and thus have:
da fi't) 2 d him)) 2 daolg", ,h),
(C.7)
for any g E G and h e H. Combining with equation C.8 we conclude that (h*, g N') Nash equilibrium of da,z. Also, (h', .g") is an optimal pair for the source loss, namely:
(h g") S (2.
(C.8)
for any i E H and E G. Combining equation C.7, we claim that (h ."""""". Fe"' is Nash equilibrium of the objective f-DAL.
D CONNECTION TO PREVIOUS FRAMEWORKS
In this appendix we show that F-DAL encompasses previous frameworks domain adaptation, including HAA--divegeenc,, DANN (Ganin et al., 2016) and MDD (Zhang et al., 2019).
D. HAR-DIVERGENCE
We now show that Theorem 2 generalizes the bound proposed in Ben-David al. (2010a). Let the pair (ø(x), (1)) & x - 11,4) for [o, 1), such that D = DJY, and SリрдЕи DTV = DIV EdHAH. with dHAN defined in Ben-David et al. (2010a) (see also equation A.1). Theorem gives that (h) < R(g()) (h) + HUHPH X², recovering Theorem 2 of Ben-David et al. (2010a).
17

---

Under review as a conference paper at ICLR 2021
D.2 DANN FORMCLATION AND JS DIVERGENCE
The DANN formulation by Ganin & Lempitsky (2015) can also be incorporated in our framework if one takes E(a, b) log b and (t) = log(1 e²). Effectively, this formulation ignores the contribution of the source classifier and experimentally we saw had inferior performance compared to using É(a, b) g (Vaggmax
The original idea of domain adversarial training was introduced by Ganin et al. (2016) where the au- thors defined the following surrogate function to measure the discrepancy between the two domains:
de,t -p, [log h' (g(x,))) Exeop, [log(1 (g(xx)))
(D.1)
In this context, h' was defined to be a domain classifier. that is h 2 - {0.1} with 0 and corresponding to the source and target domain pseudo-labels. The following proposition shows that under the assumption an optimal domain classifier R', da,t achieves JS-divergence (up to constant shift,, which upper bounds the DR,
Proposition 2. Suppose follows form of eq. D.I and is the optimal domain classifier which unconstrained, then m ds,t (ss(S) log 2.
Proof. From the definition, we have:
(h' = log ()) (2) h (z))dz.
(D.2)
By taking derivatives and finding the optimal (z). we get (z)
+()))
By plugging h (z) into equation D.I, rearranging, and using the definition of the Jensen-Shanon (JS) divergence, we get the desired result.
is worth noting that the additional negative constant -2log 2 does not affect the optimization.
D.3 MDD FORMULATION AND 7-WEIGHTED JS DIVERGENCE
Now let us demonstrate how our f-DAL framework incorporates MDD naturally. Suppose ('())
- - log(1 - e') and ((h(2), (z)) = log h'(z), argmax (z) h(x) We retrieve the following result as in Zhang et al. (2019):
Proposition 3 (Zhang et al. (2019)). Suppose da,t takes the form of MDD, i.e,
yds = Alern log R arggga lu(s)) Eany h(()) log(1 - h1(()) argmax h(2)))
(D.3)
With unconstrained function class H. the optimal da,t satisfies:
max nax, ( ( + log - + log(y +1),
(D.4)
h
where JS, (pillp)) is y-weighted Jewsen--Saannnee divergence (Huszár, 2015; Nowozin et al., 2016);
Js (pill))
7 KL
TI + Vē
1
KL
(pil
Ti + vē
(D.5)
7 +
1+1
++1
++1
We remark that when ة 1, (p(ll>) is the original Jensen Shanennn-hanonn divergence. One should
also note the the additional negative constant 7 log 7 - (y + 1) log(y + 1). which attributes to the negativity of MDD, does not affect the optimization.
(t( = , log(1 e²) can be considered by rescaling the o" for the usual JS divergence (see Table 4). In general we can rescale 6* for any bf f-divergence with the following lemma:
Lemma 4 (Boyd & Vandenberghe (2004)). For any X O, the Fenchel conjugate of Xo is (1))*()) = X (1/X), with dom()x)* Xdom o
18

---

Under review as a conference paper at ICLR 2021
E A TOY EXAMPLE FOR THE TRAINING DYNAMICS
Suppose that the source dataset and target dataset ' contain one sample each and are formulated: = {0.55, 1)} and T {0.55). Let the feature extractor be quadratic fiunctions, and we choose linear predictors, i.e.:
(xx) w1xx + s. h(x) w2z, h' R(xx) a (wyar).
(E.1)
Let us consider the regression task with the JS divergence used to compare S and T.
min maxEx~P (f)( h(g(x)))² EA log h (g(x))+
wh wr, wyEk
+ Ez log(1 (g())))
If consider that g(0.5) (00.5)) h(((0.5)) 1 and w3 0, then the optimal solution (wf, w², w3) satisfies the assumption in Theorem 5. We plot the trajectories of GDA and AExG in Figure and show that AExG can accelerate the convergence the optimal solution.
19


Reviews:
[{"review_id": "sxTRjdqjV51", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "###############\n\nSummary: This paper proposes a new generalization bound for domain adaptation based on f-divergences. Accordingly, a new algorithmic framework is also derived.\n\n###############\n\nPros:\n\nSome theoretical results are new – to the best of my knowledge, there are no existing works establishing learning bounds for DA with f-divergences.\nOne benefit of f-divergence is that it can offer a very general framework that accommodates various discrepancy measures for DA. The authors empirically evaluate different choices of divergences over several benchmarks.\nThe paper is well-written and easy to follow.\n\nCons:\n\nMy main concern is the theoretical contribution and motivation of this work. Given a large variety of divergence measures (e.g., H-divergence, JS-divergence, Wasserstein distance, MDD…), this paper does not give me any new theoretical insights compared to previous results. From an algorithmic perspective, extending JS divergence to more general f-divergences has already studied in GAN training [1].\n\nIn addition, most of the theoretical analysis follows standard steps of existing works. For example, the high-level idea of Definition 3 follows the notion of Disparity Discrepancy in [2] and is also similar to the notion of source-guided discrepancy in [3]. The proofs of Lemma 1, Lemma 2, Theorem 2, and Theorem 3 are either straightforward or follow standard techniques in DA (e.g., [2,4,5]).\n\nThe empirical results in Table 3 indicate that the improvement of f-DAL is marginal compared to other algorithms (e.g., MDD). Furthermore, the improvement may even come from AExG, rather than f-DAL itself. I would speculate that f-DAL can even be outperformed by other baseline algorithms without AExG.\n\n################\n\nOverall: While I appreciate the f-divergences generalization bounds derived in this paper, I didn’t get any new perspective after reading the paper. Every point of the theoretical results in this paper seems familiar to me.\n\n[1] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pp. 271–279. Curran Associates, Inc., 2016.\n\n[2] Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 7404–7413, Long Beach, California, USA, 09–15 Jun 2019.\n\n[3] Kuroki, S., Charonenphakdee, N., Bao, H., Honda, J., Sato, I., and Sugiyama, M. Unsupervised domain adaptation based on source-guided discrepancy. In AAAI Conference on Artificial Intelligence (AAAI), 2019.\n\n[4] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1-2):151–175, 2010.\n\n[5] Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In International Conference on Machine Learning, pp. 7523-7532, 2019.\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "mwVV_NAEUs6", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "--------After rebuttal (updated)---------------\n\nThanks for your detailed response. I would like to apologize for my late feedback.\nSince your rebuttal is long without proper organization (5 pages without properly using markdown), I may miss several points. I directly commented my feedback on my original review, `by the marked text`\n\nSeveral of my concerns have been surely fixed and the understanding of the proposed approach is much more clear. However, it is still difficult to change my score because of the following reasons:\n1. *Current paper still requires careful polishing for facilitating reading.*  e.g. The authors claimed several times they will update the paper with pseudo-code, it is still missing after discussion. The paper and the rebuttal are still dense, which make the reader\ndifficult to understand.\n\n2. *The diverse datasets (such as NLP, digits experiment in the original DANN paper) and additional in-depth empirical analysis (not only accuracy) are indeed necessary.* \nI think you do not need to claim a significantly better performance with SOTA. The detailed empirical analysis is more important. \n\nI hope my additional comments and feedback can help you improve your paper. \n\n ------------------------------------------ \n\n\n\nSummary: \n\nThis paper proposed a hypothesis based f-divergence domain adaptation theory and algorithm. They found previous popular divergences such as H-divergence and MDD can be viewed as the special cases of f-divergence. Finally, they validated their practical benefits on office-31 and office-home dataset.\n\n------------------------------------------------------\n\nOverall review \n\nPros:\n\n[1] New DA theory on f-divergence.\n\n[2] The proof is technically sound.\n\nCons:\n\n[1] The significance of the paper (theoretical and practical) is rather unclear.\n\n[2] The experimental results are weak. \n\n[3] Some details need better justifications and discussions.\n\nBased on these, I recommend a rejection but encourage a major revision for resubmission.\n\n-----------------------------------------------------------\n\nDetailed explanations\n\n[A] Significance. (theoretical and practical aspects)\n\nTheoretical aspects\n\n[1] Using f-divergence in DA is not new. For instance, [1][2][3][4] have already discussed DA by using $\\beta$ (Rényi)-divergence, $\\chi$ divergence. I do not understand why such a **unified** f-divergence does not include these divergences. I guess because the theory requires Lipschitz f-divergence, but this is too limited. A theory on general f-divergence (not only Lipschitz $\\phi^{\\star}$) is highly expected.\n\n`Partially fixed. Missing point: it is possible to prove f-divergence only with f(1)=0 and f is convex? (the original definition of f-divergence)`\n\n\n[2] The theoretical assumptions are too restrictive for practice. I just list some of them:\n\nf-divergence is Lipschitz, \n$\\ell\\in[0,1]$ and strong triangle inequality,\ndeterministic label function setting.\n\nIn contrast, previous work [2,3] has proposed strong theoretical guarantees for the cross-entropy loss and **stochastic data generation process** $P(y|x)$ instead of $f(x)$. Given this paper is in the same nature as f-divergence. More general settings are **highly** expected.\n\n`Not fixed. My concern is that the previous approach has proved stochastic settings in some f-divergence. This should be properly discussed and addressed.`\n\n\n[3] The contribution of the optimization part is unclear. If this part is the theoretical contribution, an optimization convergence bound should be provided.\n\n`Partially fixed. The term *rigorous* is confusing. You did not provide a bound how to define it is rigorous.`\n\n\n[4] The theoretical assumptions on representation learning are strong. I am not sure how these are realistic in the office-31 and office-home dataset. The conditional distribution is not clearly defined. More discussions on this part are highly expected.\n\n`Not fixed. I am still not sure how these are realistic. The conditional distribution is referred as a labeling function on $x$ f(x)? or $z$ f(z), or probability distribution P(y|x) or P(y|z). This is quite important in the representation learning approach.`\n\n\n[5] Theorem 3 seems too coarse in deep learning. (Since this paper claims they have a strong theoretical contribution in deep learning. I think this ought to be addressed).\n\n`Fixed`\n\n\n[6] The KL-divergence is **not tight** if we use dual terms of f-divergence. A Donsker-Varadhan Theorem based theoretical analysis on KL divergence is expected.\n\n\n  `Partially fixed. The Wasserstein distance makes me more confused about introducing f-divergence. Since JS, TV has such problems, why not Wasserstein distance. It is always better.`\n\n\nPractical aspects.\n\n[7] From a practical aspect, f-divergence adversarial training is not new.\n\nBased on the f-gans paper, we can practically easily replace JS divergence with any-other f-divergence without any technical difficulty. For example, [5] derives some DA algorithms on general f-divergence. From this perspective, the new empirical insights are rather limited.\n\n`Fixed. Thanks for your additional figures. I understand your new practice.`\n\n\n[8]  The extra gradient approach presented in the experiments is rather unclear and inspired by existing optimization papers. I think a clear and complete discussion is expected. The current version seems like a plug-in approach.\n\n`Not fixed. Maybe the rebuttal is too dense. I can not get your point`\n\n\n[9] The whole empirical parts are presented in a dense mode, I suggest some parts can be safely moved to the appendix. \n\n`Partially fixed. It is better but the dense wrap figure makes it still hard to read.`\n\n\n[10] It is Ok not to provide the code, but a detailed algorithm description or protocol ought to be provided. This is particularly important when this paper aims at proposing several new ideas.\n\n`Not fixed. This is really important. Author claimed they will write a description but i have not seen even in the appendix`\n\n\n[B] Empirical results and analysis\n\n[1] From all the results. The empirical gain is too limited. Besides, the compared baselines are limited or not recent SOTA. (The newest results only come from ICML 2019). \n\n`Not fixed. If you claim your approach is SOTA, at least a statistical test should be added to show it is indeed significant.`\n\n\n[2] In office-home. I do not know why the std values are not reported in this case. The other 7 tasks are missing in the paper and appendix. The number of baselines is significantly fewer than office-31. \n\n`Partially fixed. STD still not reported and still fewer baselines. The author claimed previous papers did not report these..But do you think it is a good thing not reporting variance? Particularly in office-31 you reported std, which is really odd.`\n\n\n[3] The standard digits datasets are not evaluated. Since the Digits dataset is trained from scratch and different from the pre-training approach. Testing on these is expected.\n\n`It seems to be ignored..I think this is essential to validate the theory.`\n\n\n[4] When testing KL-divergence, the dual term of KL-divergence is not tight. I think a Donsker-Varadhan Theorem based practice should be tested. \n\n`Fixed.`\n\n\n[C] Other details\n\nThe only analysis of this paper is the numerical accuracy of office-home/31 and toy data. A deeper analysis of why the proposed f-divergence is better than the previous is quite lacking. I can not feel the strong motivation of why preferring f-divergence. \nI suggest to put additional analysis such as T-SNE, the optimal value on **real-data** (such as p_t(z)/p_s(z)), the convergence behavior, the evolution of f-divergence, ablation study.\n\n`Not fixed. The only indicator in this paper is numerical accuracy. I still do not know what makes f-divergence better.`\n\n\n--------------------------------------------\nSuggestions \n\nI suggest a major revision on the improved theory and empirical analysis (not simply accuracy) on the benefits of f-divergence.\n\n\nRef:\n\n[1] Multiple source adaptation and the Rényi divergence. UAI 2009\n\n[2] A new PAC-Bayesian perspective on domain adaptation. ICML 2016\n\n[3] Algorithms and theory for multiple-source adaptation. NeurIPS 2018\n\n[4] Revisiting (\\epsilon, \\gamma, \\tau)-similarity learning for domain adaptation. NeurIPS 2018\n\n[5] Domain adaptation with asymmetrically-relaxed distribution alignment. ICML 2019\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "mTcI67OsE_S", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Further comments after the rebuttal\n\nFirst, I would thank the authors for the great efforts in trying to address my comments. I should say that many of my previous concerns have been clarified. Again, I like the theoretical part. Nonetheless, I agreed with some other reviewers that the experiments seemed not to fully convince me.  Particularly to myself, the authors may want to show some analysis on how their method could truly stand out by even a toy example, which may further enhance the paper. I tend to keep my original rating  after reading all the rebuttal messages in the whole thread.\n\n====================\nThis paper proposed a f-domain adversarial learning framework (f-DAL) using the complete family of f-divergences as domain discrepancy measurements. The proposed method extended the seminal works of Ben-David et al. (2007; 2010a;b); Mansour et al. (2009) that provided generalization bounds for UDA based on a special type of f-divergence. To enable the complete family of f-divergences to measure the domain distribution discrepancy, the variational characterization of f-divergences is leveraged to estimates f-divergences from samples by turning the estimation problem into variational optimization. Furthermore, a new type of discrepancy was used to compare two marginal distribution and its corresponding generalization bound was tailored for a general class of f-divergence, which can mitigate two limitations of the seminal works. In addition, the optimal solution of f-DAL is a Stackelberg equilibrium, which allows f-DAL to incorporate the latest optimizers from the game-optimization literature, such as Aggressive Extra-Gradient (AExG).\n\nThe key strength of this paper is, it enables a generalized version of f-divergences that can be used for adversarial domain adaptation. This is valuable for UDA algorithms in many application areas. As verified in the experiments, the proposed method achieved comparable results in Office-31 and Office Home dataset.\n\nThe paper was well organized and written. However, there are some major concerns as follows:\n(1)\tAlthough the authors provided theoretical insight for their method, the performance improvements are not very significant. \n\n(2)\tThe ablation study needs to be further conducted. It is necessary to eliminate the Aggressive Extra-Gradient (AExG) and other learning strategies, such as spectral normalization (SN) and GRL warm-up strategy when compared with state-of-the-art methods. More specifically, the authors employed an integrated model that makes the comparison experiments seemingly unfair especially in the case of using MDD as the baseline. Although achieving the Stackelberg equilibria is a good characteristic of f-DAL, AExG may also increase the performance. Furthermore, the spectral normalization (SN) and GRL warm-up strategy may also potentially improve the performance. However, these techniques were seemingly not applied in other comparison methods in Table 2 and Table 3.\n\n(3)\tThe experiments may not answer the general question ‘(2) Is there a better universal notion of f-divergence that achieves significant performance gains across different datasets?’. In Figure 3, it is hard to say that there is a consistent increment tendency for a specific f-divergence across different transfer tasks. More importantly, it is better to demonstrate if choosing different f-divergence can lead to consistent performance increase on more diverse datasets or applications. \n\n(4)\tFor a typical UDA theory method, it is better to show results on the VisDA-2017 dataset.\n\n(5)\tThe authors offered some hints to generalize the proposed method to a multi-class scenario. It is better to provide theoretical insight/details.\n\n(6)\tIt may enhance the paper if more illustrative or even toy examples can be conducted to further show clearly the advantages of the proposed method. Again, the improvement may not be very obvious as observed in the experiments though it is theoretically interesting. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "XqpmtgiABr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update after reading authors' response.\n\nThe authors didn't address my question on the statistical significance of their results compared with baselines.\nThe authors can address this question by \"perform a double-sided student-t test to verify whether the performance difference between your method and the baselines are statistically significant.\"\n\nI suggest the authors to perform experiments to compare with SOTA baselines in the entire field of domain adaptation, instead of just comparing with a subset of algorithms that are based on adversarial training. After all, you want to see how your work stands in the entire field, rather than limited to a sub-field. \n\n------------------------\n\nThis paper studies unsupervised domain adaptation. The authors derive a generalization bound that utilizes a new measure of discrepancy between distributions based on a variational characterization of f-divergences. They develop an algorithm for domain-adversarial learning for the family of f-divergences. \n\nOverall, I recommend to reject this paper, due to the following major concerns: 1) writing is not self-contained; 2) important baselines are missing; 3) insufficient justification of the advantage of the proposed method; 4) experimental results are not strong. \n\nMy major concerns of this paper include:\n1. The writing needs to be significantly improved, especially the experiment section. For example,  it is very difficult to find out what those baselines in Table 2 refer to. The paper needs to be self-contained. Directing the readers to a third paper for important details such as experimental settings and results is not proper.\n\n2. Some state-of-the-art baselines are not compared with. For example, Contrastive Adaptation Network for Unsupervised Domain Adaptation. Guoliang Kang, Lu Jiang, Yi Yang, Alexander G Hauptmann. CVPR 2019.\n\n3. In the generalization bound in Theorem 2, the authors utilize a new discrepancy to compare the two marginal distributions. The authors didn't articulate the advantage of this new discrepancy over existing ones. Why is it significant so that we need to study it?\n\n4. In table 5, are the results significantly different? It would be nice to perform a double-sided student-t test to verify whether the performance difference between your method and the baselines are statistically significant.\n\n5. Values of hyperparameters are missing, making this paper difficult to reproduce. If space is an issue, the authors can put the hyperparameters in the appendix.\n\n6. The related works section in the appendix needs to significantly improved. There are a lot of works in the space of adversarial domain adaptation and in the broader space of general domain adaptation methods. The authors need to give a more comprehensive review. Again, the review needs to be self-contained in this paper instead of directing the readers to read other review papers. And the review needs to summarize the high-level key ideas, contributions, limitations of those papers instead of focusing on detailed math.\n\nHowever, this paper does have a few strong points.\n\n1. The theoretical analysis in Section 3 is sound. I read the proofs, which appear to be correct.\n\n2. Section 3 and 4 are well-organized and easy to follow.\n\nOther comments\n1. What does ADAA in Table 2 refer to?\n\n2. It's better to use an algorithm box to outline the optimization algorithm in Section 4.1.\n\n3. The paragraph title \"Experimental results\" in Section 5 is confusing. This paragraph is more like an introduction of this section instead of one presenting results. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a novel theoretical analysis for unsupervised domain adaptation based on f-divergences. The reviews unanimously pointed out the interest and the quality of the theoretical part. However, some limitations in the experiments, presentation and the significance of the result have been raised. The authors provided a rebuttal that addresses some concerns.
However, the reviewers agree that the experimental part still requires some extension to fully support the claim of the paper, as well as some writing improvement.
The paper was evaluated to be not ready for ICLR, thus I recommend rejection.

Decision: Reject