Title: DOMAIN AGGREGATION NETWORKS FOR MULTI-SOURCE DOMAIN ADAPTATION Anonymous

Abstract: In many real-world applications, we want to exploit multiple source datasets of similar tasks to learn a model for a different but related target dataset -e.g., recognizing characters of a new font using a set of different fonts. While most recent research has considered ad-hoc combination rules to address this problem, we extend previous work on domain discrepancy minimization to develop a finite-sample generalization bound, and accordingly propose a theoretically justified optimization procedure. The algorithm we develop, Domain AggRegation Network (DARN), is able to effectively adjust the weight of each source domain during training to ensure relevant domains are given more importance for adaptation. We evaluate the proposed method on real-world sentiment analysis, digit recognition and object recognition datasets and show that DARN can significantly outperform the state-of-the-art alternatives.

1. INTRODUCTION:

Many machine learning algorithms assume the learned predictor will be tested on the data from the same distribution as the training data. This assumption, although reasonable, is not necessarily true for many real-world applications. For example, patients from one hospital may have a different distribution of gender, height and weight from another hospital. Consequently, a diagnostic model constructed at one location may not be directly applicable to another location without proper adjustment. The situation becomes even more challenging when we want to use data from multiple source domains to build a model for a target domain, as this requires deciding, e.g., how to rank the source domains and how to effectively aggregate these domains when training complex models like deep neural networks. Including irrelevant or worse, adversarial, data from certain source domains can severely reduce the performance on the target domain, leading to undesired consequences.

To address this problem, researchers have been exploring methods of transfer learning (Pan & Yang, 2009) or domain adaptation (Mansour et al., 2009a; Cortes et al., 2019) , where a model is trained based on labelled source data and unlabelled target data. Most existing works have focused on single-source to single-target ("one-to-one") domain adaptation, using different assumptions such as covariate shift (Shimodaira, 2000; Gretton et al., 2009; Sugiyama & Kawanabe, 2012) or concept drift (Jiang & Zhai, 2007; Gama et al., 2014) . When dealing with multiple source domains, one may attempt to directly use these approaches by combining all source data into a large joint dataset and then apply one-to-one adaptation. This naïve aggregation method will often fail as not all source domains are equally important when transferring to a specific target domain. There are some works on multi-source to single-target adaptation. Although many of them are theoretically motivated with cross-domain generalization bounds, they either use ad-hoc aggregation rules when developing actual algorithms (Zhao et al., 2018; Li et al., 2018) or lack finite-sample analysis (Mansour et al., 2009b; c; Hoffman et al., 2018a) . This leaves a gap between the theory for multi-source adaptation and theoretically sound algorithm for domain aggregation. This research has three contributions: First, we extend prior work on one-to-one adaptation using discrepancy (Cortes et al., 2019) to develop a finite-sample cross-domain generalization bound for multi-source adaptation. We show that in order to improve performance on the specific target domain, there is a trade-off between utilizing all source domains to increase effective sample size, versus removing source domains that are underperforming or not similar to the target domain. Second, motivated by our theory and domain adversarial method (Ganin & Lempitsky, 2015; Ganin et al., 2016) , we propose Domain AggRegation Network (DARN), that can effectively aggregate multiple source domains dynamically during the course of training. Unlike previous works, our aggregation scheme (Eq. ( 6)), which itself is of independent interest in some other contexts, is a direct optimization of our generalization upper bound (Eq. ( 3)) without resorting to heuristics. Third, our experiments on sentiment analysis, digit recognition and object recognition show that DARN can significantly outperform state-of-the-art methods.

Section 2 introduces necessary background on one-to-one adaptation based on discrepancy. Then Section 3 elaborates our theoretical analysis and the corresponding algorithm deployment. Section 4 discusses about related approaches in more detail, highlighting the key differences to the approach developed here. Section 5 empirically compares the performance of the proposed method to other alternatives. Finally, Section 6 concludes this work.

2. BACKGROUND ON CROSS-DOMAIN GENERALIZATION:

This section provides necessary background from previous work on one-to-one domain adaptation (Cortes et al., 2019) . Let X be the input space, Y ⊆ R be output space and H ⊆ {h : X → Y} be a hypothesis set. A loss function

EQUATION

The discrepancy (Mansour et al., 2009a) between two distributions P, Q over X is defined as

disc(P, Q) = max h,h ∈H |L P (h, h ) -L Q (h, h )| where L P (h, h ) = E x∼P [L(h(x), h (x))].

This quantity can be computed or approximated empirically given samples from both distributions.

For classification problems with 0-1 loss, it reduces to the well-known d A -distance (Kifer et al., 2004; Blitzer et al., 2008; Ben-David et al., 2010) and can be approximated using a domain-classifier loss w.r.t. H (Zhao et al., 2018; Ben-David et al., 2007, Sec.4) , while for regression problems with L 2 loss, it reduces to a maximum eigenvalue (Cortes & Mohri, 2011, Sec.5) ; see Section 3.2 for more details. For two domains (P, f P ), (Q, f Q ) where f P , f Q : X → Y are the corresponding labeling functions, we have the following cross-domain generalization bound: 

EQUATION

be a constant measuring how well H can fit the true models where supp( P ) means the support of the empirical distribution P (using the µ from Eq. (1)), and

M Q = sup x∈X ,h∈H L( h(x), f Q (x) )

be the upper bound on loss for Q. Given Q with m points sampled iid from Q labelled according to f Q , for δ ∈ (0, 1), ∀h ∈ H, w.p. at least 1 -δ,

L P (h, f P ) ≤ L Q (h, f Q ) + disc(P, Q) + 2R m (H Q ) + η H + M Q log(1/δ) 2m .

This theorem provides a way to generalize across domains when we have sample Q labelled according to f Q and an unlabelled sample P . The first term is the usual loss function for the sample Q, while the second term disc(P, Q) can be estimated based on the unlabelled data Q, P . η H measures how well the model family H can fit the example from the datasets, and it is not controllable once H is given. The final term, as a function of sample size m, determines the convergence speed.

3. DOMAIN AGGREGATION NETWORKS:

3.1 THEORY Suppose we are given k source domains {(S i , f Si ) : i ∈ [k] def = {1, 2, . . . , k}} and a target domain (T, f T ) where S i , T are distributions over X and f Si , f T : X → Y are their respective labelling functions. For simplicity, assume that each sample S i has m points, drawn iid from S i and labelled according to f Si . We are also given m unlabelled points T drawn iid from T . We want to leverage all source domains' information to learn a model h ∈ H minimizing L T (h, f T ).

One naïve approach could be to combine all the source domains into a large joint dataset and conduct one-to-one adaptation to the target domain using Theorem 1. However, including data from irrelevant or even adversarial domains is likely to jeopardize the performance on the target domain, which is sometimes referred to as negative transfer (Pan & Yang, 2009) . Moreover, as certain source domains may be more similar or relevant to the target domain than the others, it makes more sense to adjust their importance according to their utilities. We propose to find domain weight α i ≥ 0 such that k i=1 α i = 1 to achieve this. Our main theorem below sheds some light on how we should combine the source domains (the proof is provided in Appendix A):

Theorem 2 Given k source domains datasets {(x

(i) j , y (i) j ) : i ∈ [k], j ∈ [m]} with m iid examples each where S i = {x (i) j } and y (i) j = f Si (x (i) j ), for any α ∈ ∆ = {α : α i ≥ 0, i α i = 1}, δ ∈ (0, 1), and ∀h ∈ H, w.p. at least 1 -δ L T (h, f T ) ≤ i α i L Si (h, f Si ) + disc(T, S i ) + 2R m (H Si ) + η H,i + α 2 M S log(1/δ) 2m ,

(3) where H Si = {x → L(h(x), f Si (x)) : h ∈ H} is the set of functions mapping x to the corresponding loss, η H,i is a constant similar to Eq. (2) with Q = S i , P = T and M S = sup i∈[k],x∈X ,h∈H L(h(x), f Si (x)) is the upper bound on loss on the source domains.

There are several observations. (1) In the last term of the bound, m/ α 2 2 serves as the effective sample size. If α is uniform (i.e., [1/k, . . . , 1/k] ), then the effective sample size is km; if α is onehot, then we effectively only have m points from exactly one domain. (2) R m (H Si ) determines how expressive the model family H is w.r.t. the source data S i . It can be estimated from samples (Bartlett & Mendelson, 2002, Theorem 11) , but the computation is non-trivial for a model family like deep neural networks. The η H,i is uncontrollable once the hypothesis class H (e.g., neural network architecture) is given. Using a richer H is not always beneficial. Richer H can reduce the η H,i and also help us find a better function h with smaller source losses L Si , but it will increase the R m (H Si ). As removing these two terms does not seem to be empirically significant, we ignore them below for simplicity. (3) Let g h,i def = L Si (h, f Si ) + disc(T, S i ). Small g h,i indicates that we can achieve small loss on domain S i , and it is similar to the target domain (i.e., small disc(T, S i ), estimated from T , S i ). We may want to emphasize on S i by setting α i close to 1, but this will reduce the effective sample size. Therefore, we have to trade-off between the terms in this bound by choosing a proper α. (4) When S i and T are only partially overlapped, it might be difficult to find a suitable α i . In such cases, we can artificially split the given source domains into smaller datasets (e.g., by using clustering) then apply our method on the finer scale. This strategy requires that the learning algorithm has low computational complexities w.r.t. k, the number of source domains. As we will see later in Section 3.3, this is indeed the case for our algorithm.

Before we proceed to develop an algorithm based on the theorem, let us compare Eq. (3) to existing finite-sample bounds. The bound of Blitzer et al. (2008, Theorem 3) is informative when we have access to a small set of labelled target examples. In such cases, we can improve our bound by using this small labelled target set as an additional source domain S k+1 . We can also perform better model selection using such labelled set. The bound of Zhao et al. (2018, Theorem 2 ) is based on the d A -distance, which is a special case of disc in our bound. Moreover, our bound (Eq. ( 3)) use samplebased Rademacher complexity, which is generally tighter than other complexity measures such as VC-dimension (Bartlett & Mendelson, 2002; Koltchinskii et al., 2002; Bousquet et al., 2003) .

3.2. ALGORITHM:

In this section, we illustrate how to develop a practical algorithm based on Theorem 2. Ignoring the constants, we would like to minimize the upper bound of Eq. ( 3):

min h∈H min α∈∆ U h (α) = g h , α + τ α 2 (4)

where g h = [g h,1 , . . . , g h,k ] and τ > 0 is a hyper-parameter. If we can solve the inner minimization exactly given h, then we can treat the optimal α * (h) as a function of h and solve the outer minimization over h effectively. In the following, we show how to achieve this.

Given g h , the inner minimization can be reformulated as a second-order cone programming problem, but it has no closed-form solution due to the τ α 2 term. Consider the following problem (z = -g h /τ recovers the inner minimization):

min α∈∆ -z, α + α 2 (5)

The Lagrangian for its dual problem is

EQUATION

Taking the derivative w.r.t. α and setting it to zero gives

∂Λ ∂α = -z + α/ α 2 -λ + ν1 = 0 =⇒ α * / α * 2 = z -ν1 + λ.

Notice that α = 0 so we have the constraint z -ν1 We would like to decrease ν as much as possible, and the best we can attain is the

+ λ 2 = 1. Using this α * in Λ gives the following dual problem max ν,λ -ν s.t. z -ν1 + λ 2 = 1 and λ 0

ν * sat- isfying [z -ν * 1] + 2 = 1 where [v] + = max(0, v).

In this case, the optimal λ * can be attained as

λ * i = 0 if z i -ν * > 0, other- wise λ * i = ν * -z i ; see Fig. 1.

Although we do not have a closed-form expression for the optimal ν * , we can use binary search to find it, starting from the interval [z min -1, z max ] where z min , z max are the minimum and maximum of z respectively. Then we can recover the primal solution as

α * = [z -ν * 1] + / [z -ν * 1] + 1 . (6)

Eq. ( 6) gives rise to a new way to project any vector z to the probability simplex, which is of independent interest and may be used in some other contexts.2 It resembles the standard projection onto the simplex based on squared Euclidean distance (Duchi et al., 2008, Eq.( 3)). One subtle but crucial difference is that our Eq. ( 5) uses α 2 instead of α 2 2 . Recall that z = -g h /τ . Here τ can be interpreted as a temperature parameter. On one hand, if τ 0, all z will have similar values and thus the optimal ν * will be close to z max and α * will be close to uniform. On the other hand, as τ → 0, z max will stand out from the rest z i and eventually ν * = z max -1. This means the g h,i corresponding to the z max is small enough so we focus solely on this domain and ignore all other domains (even though this will reduce effective sample size as discussed in Section 3.1).

From a different perspective, if we define F * ∆ (α) = α 2 , then Eq. ( 5) equals -F ∆ (z), where F ∆ , F * ∆ are convex conjugates of each other, where we use ∆ to emphasize their dependency on the simplex domain. Then our objective Eq. ( 4) can be expressed as

min h∈H -F ∆ (z) = -F ∆ (-g h /τ )

Figure 2 : Model architecture with two source domains (best viewed in color). Mini-batches of x Si and x T are fed to the network. x Si will go through the classification/regression path on the upper h y box, while all x will go through to the corresponding discrepancies (in the lower h d box). The gradients from the discrepancies will be reverted during backpropagation.

We can optimize our objective and train a neural network h using gradient-based optimizer. The optimal α * (h) is merely a function of h and we can backprop through it. To facilitate the gradient computation and show that we can efficiently backprop through the projection of Eq. ( 6), we derive the Jacobian J = ∂α/∂z in Appendix B. This ensures effective end-to-end training. Now we elaborate more on how to compute g h,i . The L Si (h, f Si ) is the task loss. The disc(T, S i ) depends on whether the task is classification or regression. For classification, disc coincides with the d A -distance, so we use the domain classification loss (Zhao et al., 2018; Ben-David et al., 2007) :

disc( T , S i ) = 2 1 -min h d T,Si (h d ) T,Si (h d ) = 1 2m 2m i=1 h d (x) -δ x∈ T

where T,Si (h d ) is the sample domain classification loss of a domain classifier h d : X → {0, 1}. This minimization over h d will become maximization once we move it outside of the disc due to the minus sign. Then our objective consists of min h and max h d , which resembles adversarial training (Goodfellow et al., 2014) : learning a task classifier h to minimize loss and a domain classifier h d to maximize domain confusion. More specifically, if we decompose the neural network h into a feature extractor h fea and a label predictor h y (i.e., h(x) = h y (h fea (x))), we can learn a domainclassifier h d on top of h fea to classify h fea (x) between S i and T as a binary classification problem, where the domain itself is the label (see Fig. 2 ). To achieve this, we use the logistic loss to approximate T,Si (h d ) and apply the gradient reversal layer (Ganin & Lempitsky, 2015; Ganin et al., 2016) when optimizing disc through backpropagation.

If we are solving regression problems with L 2 loss, then disc( (Mansour et al., 2009a; Cortes & Mohri, 2011, Sec.5 ), which can be conveniently approximated using mini-batches and a few steps of power iteration.

T , S i ) = M T -M Si 2 is the largest eigenvalue (in magnitude) of the difference of two matrices M T = 1 m j h fea (x (T ) j )h fea (x (T ) j ) and M Si = 1 m j h fea (x (i) j )h fea (x (i) j )

3.3. COMPLEXITY:

Here we analyze the time and space complexities of the algorithm in each iteration. Similar to MDAN (Zhao et al., 2018) , in each gradient step, we need to compute the task loss L Si (h, f Si ) and the disc(T, S i ) (or d A -distance) using mini-batches from each source domain i ∈ [k]. The question is whether one can maintain the O(k) complexity given that we need to compute the weights using Eq. ( 6) and backprop through it. For the forward computation of α * , in order to compute the threshold ν * to the > 0 relative precision, the binary search will cost O(k log(1/ )). As for the backward pass of gradient computation, according to our calculation in Appendix B, the Jacobian J = ∂α/∂z has a concise form, meaning that it is possible to compute the matrix-vector product Jv for a given vector v in O(k) time and space. Therefore, our space complexity is the same as MDAN but our time complexity is slightly slower by a factor of log(1/ ). In comparison, the time complexity for MDMN (Li et al., 2018) is O(k 2 ) because it requires computing the pairwise weights within the k source domains. When we have a lot of source domains, MDMN will be noticeably slower than MDAN and our DARN.

4. RELATED WORK:

The idea of utilizing data from the source domain (S, f S ) to train a model for a different but related target domain (T, f T ) has been explored extensively for the last decade using different assumptions (Pan & Yang, 2009; Zhang et al., 2015) . For instance, the covariate shift scenario (Shimodaira, 2000; Gretton et al., 2009; Sugiyama & Kawanabe, 2012; Wen et al., 2014) assumes S = T but f S = f T , while concept drift (Jiang & Zhai, 2007; Gama et al., 2014) assumes S = T but f S = f T . More specifically, Theorem 1 (Cortes et al., 2019) shows that both covariate shift (as measured by the discrepancy disc) and model misspecification (as controlled by η H ) contribute to the adaptation performance (Wen et al., 2014) .

Finding a domain-invariant feature space by minimizing a distance measure is common practice in domain adaptation, especially for training neural networks. Tzeng et al. (2017) provided a comprehensive framework that subsumes several prior efforts on learning shared representations across domains (Tzeng et al., 2015; Ganin et al., 2016) . DARN uses adversarial domain classifier and the gradient reversal trick from Ganin et al. (2016) . Instead of proposing a new loss for each pair of the source and target domains, our main contribution is the aggregation technique of computing the mixing coefficients α, which is derived from theoretical guarantees. When dealing with multiple source domains, our aggregation method can certainly be applied to other forms of discrepancies such as MMD (Gretton et al., 2012; Long et al., 2015; 2016) , and other model architectures such as Domain Separation Network (Bousmalis et al., 2016) , cycle-consistent model (Hoffman et al., 2018b) , class-dependent adversarial domain classifier (Pei et al., 2018) and Known Unknown Discrimination (Schoenauer-Sebag et al., 2019) .

Our work focuses on multi-source to single-target adaptation, which has been investigated in the literature. Sun et al. (2011) developed a generalization bound but resorted to heuristic algorithms to adjust distribution shifts. Zhao et al. (2018) proposed a certain ad-hoc scheme for the combination coefficients α, which, unlike ours, are not theoretically justified. Multiple Domain Matching Network (MDMN) (Li et al., 2018) computes domain similarities not only between the source and target domains but also within the source domain themselves based on Wasserstein-like measure. Calculating such pairwise weights can be computationally demanding when we have a lot of source domains. Their bound requires additional smooth assumptions on the labelling functions f Si , f T , and is not a finite-sample bound, as opposed to ours. As for the actual algorithm, they also use adhoc coefficients α without theoretical justification. Mansour et al. (2009b; c) consider multi-source adaptation where T = i β i S i is a convex mixture of source distributions with some weights β i . Our analysis does not require this assumption. Hoffman et al. (2018a) provides similar guarantees with different assumptions, but unlike ours, their bounds are not finite-sample bounds.

5. EXPERIMENTS:

In this section, we compare DARN with several other baselines and state-of-the-art methods for the popular tasks: sentiment analysis, digit recognition and object recognition. The following methods are compared. (1) The SRC (for source) method uses only labelled source data to train the model. It merges all available source examples to form a large dataset to perform training without adaptation.

(2) TAR (for target) is another baseline that uses only m labelled target data instances. It serves as upper bound of the best we can achieve if we had access to the true label of the target data.

(3) Domain Adversarial Neural Network (DANN) (Ganin et al., 2016) is similar to our method in that we both use adversarial training objectives. It is not obvious how to adapt DANN to the multisource setting. Here we follow previous protocol (Zhao et al., 2018) and merge all source data to form a large joint source dataset of km instances for DANN to perform adaptation. (4) Moment Matching for Multi-Source Domain Adaptation (M3SDA) (Peng et al., 2019 ) is a recent state-ofthe-art method that combines moment matching and maximizing classifier discrepancy (Saito et al., 2018) . We use their public code with a few necessary adjustments (change classification head based on the number of classes etc.) (5) Multisource Domain Adversarial Network (MDAN) (Zhao et al., 2018) resembles our method in that we both dynamically assign each source domain an importance weight during training. However, unlike ours, their weights are not theoretically justified. We use the soft version of MDAN since they reported that it performs better than the hard version, and we use their code. (6) Multiple Domain Matching Network (MDMN) (Li et al., 2018) computes weights not only between source and target domains but also within source domain themselves. We use their code of computing weights in our implementation. All the methods are applied to the same neural network structure to ensure fair comparison. Our PyTorch implementation will be available online after acceptance.

5.1. SENTIMENT ANALYSIS:

Setup. We use the Amazon review dataset (Blitzer et al., 2007; Chen et al., 2012 ) that consists of positive and negative product reviews from four domains (Books, DVD, Electronics and Kitchen). Each of them is used in turn as the target domain and the other three are used as source domains. Their sample sizes are 6465, 5586, 7681, 7945 respectively. We follow the common protocol (Chen et al., 2012; Zhao et al., 2018) of using the top-5000 frequent unigrams/bigrams of all reviews as bag-of-words features and train a fully connected model (MLP) with [1000, 500, 100] hidden units for classifying positive versus negative reviews. The dropout drop rate is 0.7 for the input and hidden layers. In each run, we randomly sample 2000 reviews from each domain as labelled source or unlabelled target training examples, while the remaining instances are used as test examples for evaluation. The hyper-parameters are chosen based on cross-validation. The model is trained for 50 epochs and the mini-batch size is 20 per domain. The optimizer is Adadelta with a learning rate of 1.0. The soft version of MDAN has an additional parameter γ = 1/τ which is the inverse of our temperature τ . The chosen parameters are γ = 10.0 for MDAN and γ = 0.9 for our DARN, which are selected from a wide range of candidate values.

Results and Analysis. Table 1 summarizes the classification accuracies. The last column is the average accuracy of four domains, and the standard errors are calculated based on 20 runs. (1) Most of the time, DANN with joint source data performs worse than SRC which has no adaptation. This may be because it does not adjust for the each source domain, and as a result, it fails to ignore irrelevant data to avoid negative transfer. (2) Some domains are harder to adapt to than the others. For example, the accuracies of SRC and TAR on the Electronics domain are very close to each other, indicating that this requires little to no adaptation. Yet, our DARN is the closest to the TAR performance here. The Books domain is more challenging as the improvement over the SRC method is small, even though there exists a large gap between SRC and TAR. (3) DARN is always within the best performing methods and significantly outperforms others in the Electronics domain. Note that MDMN additionally computes similarities within source domains in each iteration, which can be computationally expensive (O(k 2 ) per iteration) if the number of source domains is large. Instead, our method focuses on the discrepancy between source and target domains (O(k) per iteration) and can achieve similar performance on this problem. 

5.2. DIGIT RECOGNITION:

Setup. Following the setting from previous work (Ganin et al., 2016; Zhao et al., 2018) , we use the four digit recognition datasets in this experiment (MNIST, MNIST-M, SVHN and Synth). MNIST is a well-known gray-scale images for digit recognition, and MINST-M (Ganin & Lempitsky, 2015) is a variant where the black and white pixels are masked with color patches. Street View House Number (SVHN) (Netzer et al., 2011) is a standard digit dataset taken from house numbers in Google Street View images. Synthetic Digits (Synth) (Ganin & Lempitsky, 2015) is a synthetic dataset that mimic SVHN using various transformations. One of the four datasets is chosen as unlabelled target domain in turn and the other three are used as labelled source domains. Results and Analysis. Table 2 shows the classification accuracy of each target dataset over 20 runs.

(1) All methods except DANN can consistently outperform SRC. Again, without proper adjustment for each source domain, DANN with joint source data can perform worse than SRC. This suggests the importance of ignoring irrelevant data to avoid negative transfer.

(2) DARN significantly outperforms MDAN and MDMN across all four domains, especially on the MNIST-M and SVHN domains. Notice that even though MDAN and MDMN have generalization guarantees, they both resort to ad-hoc aggregation rules to combine the source domains during training. Instead, our aggregation (Eq. ( 6)) is a direct optimization of the upper bound (Theorem 2) thus is theoretically justified and empirically superior for this problem. 

5.3. OBJECT RECOGNITION: OFFICE-HOME:

To showcase the applicability of our method to more complicated real-world tasks, we use the challenging Office-Home dataset (Venkateswara et al., 2017) . It contains images of 65 everyday objects such as spoon, sink, mug and pen from four different domains: Art, Clipart, Product and Real-World. One of the four datasets is chosen as unlabelled target domain in turn and the other three are used as labelled source domains. Table 3 shows the classification accuracy of each target dataset over 20 runs. Most existing works in the literature focused on single-source to single-target adaptation on this problem (e.g., see (Long et al., 2018) ). Compared to them, using multi-source methods can significantly boost performance, revealing the importance of using multiple source domains when possible. Even though this is a significantly more challenging problem with more classes and much fewer images compared to the digits datasets, our DARN achieves state-of-the-art performance in this setting, excelling existing methods by a noticeable margin. To demonstrate how DARN can effectively aggregate multiple source domains, we visualize the source domain weights (i.e., α in DARN) for the Amazon dataset. We also compared to the weights produced by MDMN, using the original authors' code. 

6. CONCLUSION AND FUTURE WORK:

Our work uses the discrepancy (Mansour et al., 2009a; Cortes et al., 2019) to derive a finite-sample generalization bound for multi-source to single-target adaptation. Our theorem shows that, in order to achieve the best possible generalization upper bound for a target domain, we need to trade-off between including all source domains to increase effective sample size and removing source domains that are underperforming or not similar to the target domain. Based on this observation, we derive an algorithm, Domain AggRegation Network (DARN), that can dynamically adjust the weight of each source domain during end-to-end training. Experiments on sentiment analysis, digits recognition and object recognition show that DARN outperforms state-of-the-art alternatives. Recent analysis (Zhao et al., 2019; Johansson et al., 2019) show that solely focusing on learning domain invariant features can be problematic when the marginal label distributions on Y between source and target domains are significantly different. Thus it makes sense to take η H into consideration when a small amount of labelled data is available for the target domain, which we will explore in the future.

A PROOF OF THEOREM 2:

Theorem 2 Given k source domains datasets {(x

(i) j , y (i) j ) : i ∈ [k], j ∈ [m]} with m iid examples each where S i = {x (i) j } and y (i) j = f Si (x (i) j ), for any α ∈ ∆ = {α : α i ≥ 0, i α i = 1}, δ ∈ (0, 1), and ∀h ∈ H, w.p. at least 1 -δ L T (h, f T ) ≤ i α i L Si (h, f Si ) + disc(T, S i ) + 2R m (H Si ) + η H,i + α 2 M S log(1/δ) 2m ,

where H Si = {x → L(h(x), f Si (x)) : h ∈ H} is the set of functions mapping x to the corresponding loss, η H,i is a constant similar to Eq. (2) with Q = S i , P = T and

M S = sup i∈[k],x∈X ,h∈H L(h(x), f Si (x)

) is the upper bound on loss on the source domains.

Proof: The proof is similar to Cortes et al. (2019) ; Zhao et al. (2018) . Given α ∈ ∆, the mixture S = i α i S i can be considered as the joint source data with km points, where a point

x (i) from S i has weight α i /m. Define Φ = sup h∈H L T (h, f T ) -i α i L Si (h, f Si ). Changing a point x (i) from S i will change Φ at most M S αi m . Using the McDiarmid's inequality, we have Pr(Φ -E[Φ] > ) ≤ exp -2 2 m M 2 S α 2 2

. As a result, for δ ∈ (0, 1), w.p. at least 1 -δ, the following holds for any h ∈ H

L T (h, f T ) ≤ i α i L Si (h, f Si ) + E[Φ] + α 2 M S log(1/δ) 2m . Now we bound E[Φ]. Let H Si = {x → L(h(x), f Si (x)) : h ∈ H}. E[Φ] = E S sup h∈H L T (h, f T ) - i α i L Si (h, f Si ) ≤ E S sup h∈H i α i L Si (h, f Si ) - i α i L Si (h, f Si ) + sup h∈H L T (h, f T ) - i α i L Si (h, f Si ) ≤ E S i α i sup h∈H L Si (h, f Si ) -L Si (h, f Si ) + i α i sup h∈H (L T (h, f T ) -L Si (h, f Si )) = i α i E Si sup h∈H L Si (h, f Si ) -L Si (h, f Si ) + i α i sup h∈H (L T (h, f T ) -L Si (h, f Si )) ≤ 2 i α i R m (H Si ) + i α i sup h∈H (L T (h, f T ) -L Si (h, f Si )) ≤ i α i (2R m (H Si ) + disc(T, S i ) + η H,i ) .

where first and second inequalities are using the subadditivity of sup, followed by the equality using the independence between the domains { S i }, the second last inequality is due to the standard "ghost sample" argument in terms of the Rademacher complexity and the last inequality is due to Cortes et al. (2019, Proposition 8) for each individual S i .

B JACOBIAN:

Here we calculate the Jacobian J ij = ∂α i /∂z j for Eq. ( 6) in the main text:

α * = [z -ν * 1] + / [z -ν * 1] + 1 .

In the following, we write α = α * , ν = ν * to simplify notations. Let S = {i : z i -ν > 0} be the support of the probability vector α. J ij = 0 if i / ∈ S or j / ∈ S since α i = 0 in the former case while z j does not contribute to the α in the latter case. Now consider the case i, j ∈ S. Let

EQUATION

where δ i=j is the indicator or delta function. Now we compute ∂ν ∂zj and ∂K ∂zj . By the definition of ν, we know that 

The first right-arrow is due to the quadratic formula and realizing that j∈S z j /|S| is the mean of the supported z j so ν must be smaller than it (i.e., we takein the ± of the quadratic formula, otherwise some of the z j will not be in the support anymore). And

EQUATION

Plugging Eq. ( 8) and Eq. ( 9) in Eq. ( 7) gives

∂α i ∂z j = 1 K δ i=j - 1 |S| + B j √ A • 1 |S| -α i Note that B j K = j ∈S z j -|S|z j j ∈S (z j -ν) = j ∈S (z j -ν) + |S|(ν -z j ) j ∈S (z j -ν) = 1 -|S|α j .

Then

J ij = ∂α i ∂z j = 1 K δ i=j - 1 |S| + |S| √ A 1 |S| -α i 1 |S| -α j .

In matrix form,

J = 1 K Diag(s) - ss |S| + |S| √ A s |S| -α • s s |S| -α • s ,

where s = [s 1 , . . . , s k ] is a vector indicating the support s i = δ i∈S and • is element-wise multiplication. More often, we need to compute its multiplication with a vector v

Jv = s K • v - s v |S| + |S| √ A s |S| -α • s s |S| -α • s v.

Note that all quantities except A have been computed during the forward pass of calculating Eq. ( 6).

A can be computed in O(|S|) time so the overall computation is still O(k) since |S| ≤ k. C REGRESSION ON SYNTHETIC DATA Due to the lack of publicly available multi-source regression datasets, we demonstrate our method on a synthetic regression problem here.

Setup. We use eight source domains, where the ith (i = {0, 1, . . . , 7}) domain data is generated by x i ∼ N ( π 4 i -7π 8 , 0.2 2 ) and the output is y = sin(x) + where ∼ N (0, 0.05 2 ) is random noise. These eight domains evenly cover the sin function on [-π, π] (see Fig. 5a ). Next we construct four target domains, where the jth (j = {0, 1, 2, 3}) domain is generated by x j ∼ N ( π 2 j -3π 4 , 0.4 2 ). Similar to the source domains, these four target domains evenly cover [-π, π] (see Fig. 5b ). Each source/target domain has 100 data points.

We use labelled source data and unlabelled target data for learning. We take on one target domain at a time, and learn a linear model from all eight source domains with MSE loss. The goal is to see whether our method can focus on the relevant source domains and learn a linear model that can perform well on the target domain.

Results and Analysis. First, Fig. 5b shows the learned models. The learned linear models can fit the target data very well. This shows that our DARN can learn a meaningful model for a specific target domain, using only labelled source data and unlabelled target data. Second, Fig. 5c shows the source domain weights (the α) for each target domain after training. The weight colors correspond to the colors in Fig. 5a . It is noticeable that DARN can focus well on the respective relevant source domains for each target domain and ignore the rest. Note that these values are automatically learned during the training of the model.

:

For example, the common Lp losses are µ-admissible(Cortes et al.,

2019, Lemma 23).

For example, if we consider z to be the logits of the final classification layer of a neural network, this projection provides another way to produce class probabilities similar to the softmax transformation.

Figures and Tables:

Figure 1: Optimal solution to Eq. (5) (best viewed in color). λ * compensates what is below the ν * , while ν * is chosen such that the vector of the green bars has L 2 norm of 1.

MNIST images are resized to 32 × 32 and represented as 3-channel color images in order to match the shape of the other three datasets. Each domain has its own given training and test sets when downloaded. Their respective training sample sizes are 60000, 59001, 73257, 479400, and the respective test sample sizes are 10000, 9001, 26032, 9553. In each run, 20000 images are randomly sampled from each domain's training set as actual labelled source or unlabelled target training examples, and 9000 images are randomly sampled from each domain's test set as actual test examples for evaluation. The model structure is shown in Appendix D. There is no dropout and the hyperparameters are chosen based on cross-validation. It is trained for 50 epochs and the mini-batch size is 128 per domain. The optimizer is Adadelta with a learning rate of 1.0. Validation selected γ = 0.5 for MDAN and γ = 0.1 for DARN.

The respective sample sizes are2427, 4365, 4439, 4357.  In each run, 2000 images are randomly sampled from each domain as labelled source or unlabelled target training examples, and the rest images are used as test images for evaluation. We use the ResNet50He et al. (2016) pretrained from the ImageNet in PyTorch as the base network for feature learning and put an MLP with [1000, 500, 100, 65] units on top for classification. It is trained for 50 epochs and the mini-batch size is 32 per domain. The optimizer is Adadelta with a learning rate of 1.0. MDAN uses γ = 1.0 while DARN uses γ = 0.5.

Figure 3: Domain weights of DARN for the Amazon data.

Figure 4: Domain weights of MDMN for the Amazon data.

Fig. 3 and Fig. 4 compare the evolution of source domain weights during training. In each subfigure, every row corresponds to the weights of the source domains when learning for one target domain. The white stripe indicates there is no target domain weight and darker stripe means less weight. They are evaluated at the end of each epoch over 50 epochs.To avoid noisy values due to small mini-batch size, the values are exponential moving averages with a decay rate of 0.95. (1) For DARN, as Electronics and Kitchen are more related to each other than Books and DVD, their respective weights remain higher during training. This is reasonable since we have overlapping products (e.g., blenders) in both domains. The α is changing dynamically during training, showing the flexibility of our method to adjust domain importance when needed. (2) In comparison, the domain weights produced by MDMN are not very stable. We take a closer look at the MDMN weights and notice that they change drastically, especially towards the end of the training. It can produce alternating one-hot vectors α, changing from one domain to a different domain and ignoring the rest. This instability makes their domain weights hard to interpret.

j∈S (z j -ν) 2 = |S|ν 2 -2ν j∈S

Figure 5: Regression experiment (best viewed in color).

Classification accuracy (%) of the target sentiment datasets. Mean and standard error over 20 runs. The best method(s) (excluding TAR) based on one-sided Wilcoxon signed-rank test at the 5% significance level is(are) shown in bold for each domain. ± 0.39 80.38 ± 0.30 85.48 ± 0.10 85.46 ± 0.34 82.62 ± 0.20 DANN 79.13 ± 0.29 80.60 ± 0.29 85.27 ± 0.14 85.56 ± 0.28 82.64 ± 0.14 M3SDA 79.42 ± 0.17 80.82 ± 0.35 85.52 ± 0.19 86.45 ± 0.43 83.05 ± 0.14 MDAN 79.99 ± 0.20 81.66 ± 0.19 84.76 ± 0.17 86.82 ± 0.13 83.31 ± 0.08 MDMN 80.13 ± 0.20 81.58 ± 0.21 85.61 ± 0.13 87.13 ± 0.11 83.61 ± 0.07 DARN 79.93 ± 0.19 81.57 ± 0.16 85.75 ± 0.16 87.15 ± 0.14 83.60 ± 0.08 TAR 84.10 ± 0.13 83.68 ± 0.12 86.11 ± 0.32 88.72 ± 0.14 85.65 ± 0.09

Table content: <table><tr><td>Method</td><td>Books</td><td>DVD</td><td>Electronics</td><td>Kitchen</td><td>Avg.</td></tr><tr><td>SRC</td><td>79.15</td><td/><td/><td/><td/></tr></table>

Classification accuracy (%) of the target digit datasets. Mean and standard error over 20 runs. The best method (excluding TAR) based on one-sided Wilcoxon signed-rank test at the 5% significance level is shown in bold for each domain. ± 0.13 60.10 ± 0.27 70.19 ± 1.30 83.83 ± 0.25 77.63 ± 0.35 M3SDA 96.95 ± 0.06 65.03 ± 0.80 71.66 ± 1.16 80.12 ± 0.56 78.44 ± 0.36 MDAN 97.10 ± 0.10 64.09 ± 0.31 77.72 ± 0.60 85.52 ± 0.19 81.11 ± 0.21 MDMN 97.15 ± 0.09 64.34 ± 0.27 76.43 ± 0.48 85.80 ± 0.21 80.93 ± 0.16 DARN 98.09 ± 0.03 67.06 ± 0.14 81.58 ± 0.14 86.79 ± 0.09 83.38 ± 0.06 TAR 99.02 ± 0.02 94.66 ± 0.10 87.40 ± 0.17 96.90 ± 0.09 94.49 ± 0.07

Table content: <table><tr><td>Method</td><td>MNIST</td><td>MNIST-M</td><td>SVHN</td><td>Synth</td><td>Avg.</td></tr><tr><td>SRC</td><td colspan="5">96.78 ± 0.08 60.80 ± 0.21 68.99 ± 0.69 84.09 ± 0.27 77.66 ± 0.14</td></tr><tr><td>DANN</td><td>96.41</td><td/><td/><td/><td/></tr></table>

Classification accuracy (%) of the Office-Home datasets. Mean and standard error over 20 runs. The best method (excluding TAR) based on one-sided Wilcoxon signed-rank test at the 5% significance level is shown in bold for each domain. ± 0.47 57.29 ± 0.30 74.26 ± 0.22 77.98 ± 0.25 66.89 ± 0.16 DANN 57.39 ± 0.69 57.35 ± 0.35 73.78 ± 0.27 78.12 ± 0.21 66.66 ± 0.19 M3SDA 64.05 ± 0.61 62.79 ± 0.37 76.21 ± 0.30 78.63 ± 0.22 70.42 ± 0.18 MDAN 68.14 ± 0.58 67.04 ± 0.21 81.03 ± 0.22 82.79 ± 0.15 74.75 ± 0.18 MDMN 68.67 ± 0.55 67.75 ± 0.20 81.37 ± 0.18 83.32 ± 0.14 75.28 ± 0.15 DARN 70.00 ± 0.38 68.42 ± 0.14 82.75 ± 0.21 83.88 ± 0.16 76.26 ± 0.13 TAR 71.19 ± 0.38 79.16 ± 0.16 90.66 ± 0.15 85.60 ± 0.14 81.65 ± 0.12 5.4 VISUALIZING DOMAIN IMPORTANCE

Table content: <table><tr><td>Method</td><td>Art</td><td>Clipart</td><td>Product</td><td>Real-World</td><td>Avg.</td></tr><tr><td>SRC</td><td>58.02</td><td/><td/><td/><td/></tr></table>

Peer review (iclr2020):
###Summary### This paper tackles the multi-source domain adaptation by aggregate multiple source domains dynamically during the training phase. The observation is that in many real-world applications, we want to exploit multiple source datasets of similar tasks to learn a model for a different but related target datasets. Firstly, the paper derives a multiple-source domain adaptation upper-bound from single-to-single domain adaptation generalization bound, based on the theoretical work from Cortes et al (2019). The idea is similar to Zhao et al (2019), which introduces a weighted parameter alpha to combine the source domains together. Secondly, based on the theoretical result, the paper proposes an algorithm to minimize the upper bound of the theoretical result. The upper bound can be simplified as the quartic form (Eq. 4) and can be optimized with the Lagrangian form. Since no closed-form expression for the optimal v can be derived, the authors propose to use binary search to find it. Based on the theoretical results and the algorithm, the paper introduces Domain AggRegation Network (DARN), which contains a base network for feature extraction, h_y to minimize the task loss and h_d to evaluate the discrepancy between each source domain and target domain. The loss is aggregation with the parameter alpha. Finally, the paper conduct experiments on sentimental analysis benchmark, Amazon Review and digit datasets. The paper selects MDAN, DANN, MDMN as the baselines. On the amazon review dataset, the performance of the proposed DARN model is comparable with the MDMN baseline. On the digit dataset, the model can outperform the baselines. ### Novelty ### The theoretical results in this paper are extended from Cortes et al (2019) and Zhao et al (2018). Thus, the theoretical contribution of this paper is limited. The algorithm proposed in this paper is interesting. However, the motivation of the proposed method is to minimize the upper bound, not the loss itself, i.e. L_T(h, f_T). Intuitively, when the upper bound of the loss is minimized, it will be beneficial to minimize the loss itself. But it*s not guaranteed as the upper bound contains other variables, such as the number of training samples and model complexity. If the training samples and model complexity (think about the parameters in the deep models) are significantly large, the upper bound of the loss might be also very large. As for the experimental results, the paper only provides results on the sentimental analysis results and digit datasets, which are small benchmarks. The selected baselines are not sufficient. The improvement from the baselines is also limited. ###Clarity### Overall, the paper is well organized and logically clear. The images are well-presented and well-explained by the captions and the text. The derivation of the algorithm in Sec 3.2 is logically clear and easy to follow. ###Pros### 1) The paper proposes a new theoretical upper-bound based on the prior works, the upper-bound and its derivation are interesting and heuristic to the domain adaptation research community. 2) The paper is applicable to many practical scenarios since the data from the real-world application is typically collected from multiple sources. 3) The paper is overall well-organized and well-written. The claims of the paper are verified by the experimental results. ###Cons### 1) The critical issue of this paper is that the algorithm is designed to minimize the upper bound. The idea is intuitive when the upper bound is small. However, the proposed upper bound in the paper involves other parameters, such as the model complexity and the number of training samples. It*s an intuitive idea to weight different source domains in multi-source domain adaptation. The paper derives the weight by the Lagrangian form to minimize the upper bound. While another trivial trick is to evaluate alpha by the domain closeness between each source domain with the target domain. 2) The experimental results provided in this paper are weak. In the abstract and introduction, the paper motivates the multi-source domain adaptation (MSDA) problem by arguing that the MSDA has a lot of real applications. But the paper only provides empirical results on sentimental analysis and digit recognition. Besides, the results on the sentimental analysis are comparable with the compared baselines. It will be also interesting to see how does the proposed method perform on large-scale datasets such as DomainNet and Office-Home dataset: DomainNet: Moment Matching for Multi-Source Domain Adaptation, ICCV 2019. http://ai.bu.edu/DomainNet/ Office-Home: Deep Hashing Network for Unsupervised Domain Adaptation, CVPR 2017. http://hemanthdv.org/OfficeHome-Dataset/ 3) The novelty of this paper is incremental as the theoretical results are extended from Cortes et al (2019) and Zhao et al (2018). Based on the summary, cons, and pros, the current rating I am giving now is weak reject. I would like to discuss the final rating with other reviewers, ACs. To improve the rating, the author should explain the following questions: 1). Why minimizing the upper bound in this scenario would help to minimize the loss on the target domain, when the upper bound can be substantially huge? How about just evaluate alpha by the closeness of the source domain with the target domain? 2). In the introduction, the paper motivates the multi-source domain adaptation (MSDA) problem by arguing that the MSDA has a lot of real applications. While the experiments are only performed on sentimental analysis and digit recognition. How about evaluating the proposed methods on real image recognition such as DomainNet or Office-Home?