Multi-Step Decentralized Domain Adaptation
Authors: Akhil Mathur
Shaoduo Gan
Anton Isopoussu
Fahim Kawsar
Nadia Berthouze
Nicholas D. Lane
Venue:  (2020)
Keywords: domain adaptation
decentralization

Abstract: Despite the recent breakthroughs in unsupervised domain adaptation (uDA), no prior work has studied the challenges of applying these methods in practical machine learning scenarios. In this paper, we highlight two significant bottlenecks for uDA, namely excessive centralization and poor support for distributed domain datasets. Our proposed framework, MDDA, is powered by a novel collaborator selection algorithm and an effective distributed adversarial training method, and allows for uDA methods to work in a decentralized and privacy-preserving way.  


Under review as a conference paper at ICLR 2020
MULTI-STEP DECENTRALIZED DOMAIN ADAPTATION
Anonymous authors
Paper under double-blind review
ABSTRACT
Despite the recent breakthroughs in unsupervised domain adaptation (uDA), no prior work has studied the challenges of applying these methods in practical ma- chine learning scenarios. In this paper, we highlight two significant bottlenecks for uDA, namely excessive centralization and poor support for distributed domain datasets. Our proposed framework, MDDA, is powered by novel collaborator selection algorithm and an effective distributed adversarial training method, and allows for uDA methods to work in a decentralized and privacy-preserving way.
Introduction
In practical machine learning systems, test samples are often drawn from different data distribu- tion than the training samples, due to variations in data acquisition processes between the training and test sets caused by for example, different illumination conditions and cameras in the con- text of visual tasks. This shift in data distributions, known as domain shift, is a core reason which hinders the generalizability of predictive models to new domains. As manual labeling of data in each test domain is prohibitively expensive. unsupervised domain adaptation (uDA) has emerged as a promising solution to transfer the knowledge from a labeled source domain to unlabeled target domains (Long et al. (2015 2017); Ganin al. (2016); Tzeng et al. (2017); (offman et al. (008)) Shen et al. [2018] Sankaranarayanan et al. (2018): Hoffman et al. (2018)).
While uDA methods indeed effective, little attention has been paid on how they would be in- corporated real-world machine learning systems. In this paper, we study and propose solutions for practical problems that arise while applying uDA techniques in ML systems, namely the chal- lenges of distributed domain datasets and the overly centralized nature of existing uDA approaches. As a motivating example, consider scenario wherein model is trained for the task of fetal head detection from labeled ultrasound images collected in a hospital in Finland (source domain Sna). Subsequently, this pre-trained model has to be deployed in three target hospitals one in the US (Tus) and two in China (Ten1 and T..)). Due to variations in sonogram machines and medical train- ing of sonographers, a domain shift is likely to occur in the test samples; hence we need to apply uDA to adapt the source classifier in the target domains.
Existing uDA methods are centralized by design, that they assume that each target domain would always adapt from the labeled source domain (Smn). This raises two issues: firstly, if the machine hosting the labeled source dataset is unavailable (c.g., it is undergoing maintenance or has connec- tivity issues), then clearly adaptation is not possible. More importantly, always adapting from a labeled source is not optimal from an adaptation perspective, because the domain discrepancy between the labeled source and potential target domain could be high in some cases. Our work seeks to explore an interesting proposition: in addition to adapting from the labeled source, can we also perform uDA with other target domains, which themselves may have undergone domain adaptation in the past.
argue that this choice of
Further, existing uDA methods do not support distributed domain datasets and assume that source and target data are available on the same machine. Clearly, this raises privacy and legal concerns since either the source domain or the target domains need to send their sensitive data (i.e., sono- grams) to each other to perform uDA. Moreover, such transfer of potentially large datasets also incurs severe communication costs.
In summary, this paper makes the following contributions:
We formulate and study a brand-new problem focusing on the challenges of uDA in practical machine learning systems.
We propose a multi-step uDA framework, wherein target domains can adapt not only from the la- beled source, but also from other target domains. Powering this framework is novel collaborator selector algorithm that chooses the optimal adaptation collaborator for each target domain, before initiating the adaptation.

---

Under review as a conference paper at ICLR 2020
We propose an effective technique for allowing uDA algorithms and adversarial training to work across distributed datasets. Through extensive experiments on five image and speech datasets, we demonstrate the efficacy of our proposed solution.
2 Problem Formulation and Related Work
Consider a practical scenario of deploying a uDA method in a real-world ML system. Assume that data collection exercise yields a labeled dataset upon which model is trained using supervised learning. Thereafter, this model needs to be deployed to a population of users (or targets) whose data is unlabeled and divergent from the training dataset. Mathematically, single source domain S = (IR", (s, y)), with input data Xs and labels Ys. There are multiple target domains {T
are presented with
with data X drawn from target distribution pty (z, y). without labeled observations. Using supervised learning. we train a representation mapping, Ms, and a classifier, Cs for the source domain. However, for the target domains, due to the absence of labeled observations, supervised learning is not possible and hence we would like to do adaptation with the source domain. We assume that target domains are introduced sequentially, one at time.
Under this problem formulation, we highlight two unexplored research challenges:
Collaborator Selection. For each target domain that joins the system, how do we select an optimal collaborator for domain adaptation that will lead to highest post-adaptation accuracy for the target domain? Existing uDA methods (e.g.. Hoffman et al.] (018);; Tzeng ct al.) (2017); Ganin et al. (2016) always use the labeled source domain S as the adaptation collaborator for each target domain, however argue that it is not optimal to always adapt for the labeled source domain for two reasons: i) if the domain shift between the target domain and the labeled source is high, we may not achieve good adaptation performance (Wulfmeier et al. (2017); (ii) from practical perspective, it makes the entire system centralized and prone to failure if the device hosting the source dataset is unavailable for adaptation (e.g.. due to connectivity issues). Instead, we propose multi-step decentralized domain adaptation approach built upon the idea that a new target domain can adapt not only from the labeled source domain, but also from other target domains in the system which have already undergone domain adaptation.
More concretely, we define a collaborator set the set of domains that are available to collaborate with a target domain on an adaptation task. At step 7 Tu. only the source domain is present in the system, hence C. = {S}. At step r Ti. the first target domain T'l joins the system at this moment, only the source domain S has a learned representation Mss. Thereafter, 7 performs uDA with the source 5 and learns representation MyT. Subsequently, we have C₁ (S, T*), ), i.e. future target domains now have two candidates with whom they can collaborate to perform domain adaptation. In general, at step r = TK3 CK = {S} U [TT == LK]
In $.3 propose algorithm to select optimal collaborator E Cx for each target domain, that is likely to yield the best accuracy post-adaptation. Once the optimal collaborator is chosen, any existing uDA method could be employed to perform the pairwise adaptation between the chosen collaborator and the target domain. In $ we show the effectiveness of this decentralized and multi- step adaptation approach against conventional baselines and across a number of uDA techniques.
Distributed and Private Data. Existing uDA methods assume that the datasets from the source domain S and given target domain T are available on the same compute unit (e.g., on a server). However, this assumption is violated in practical scenarios, as the datasets are often private, and users or companies may not be allowed to share them with other parties due legal reasons such as the data privacy law (GDPR) in Europe. In addition to the privacy issue, exchanging potentially large datasets incur high communication costs, making it undesirable in practical settings. This raises the question: can we make domain adaptation methods to work in distributed and crivacy-preserving manner such that the collaborators in the domain adaptation process can keep their data private and still receive the benefits of adaptation?
These challenges about making uDA algorithms work in a decentralized and distributed way are closely related, and addressing them is critical for practical usage of uDA methods. We first describe (in 3 our idea of supporting uDA with distributed domain datasets, by enabling knowledge ex- change between adaptation collaborators using the gradients of Discriminator, which allows for the raw data and features of each domain to remain private. Next, in $3.3, we propose an algorithm for selecting an optimal collaborator for each target domain.
2

---

Under review as a conference paper at ICLR 2020
2.1 Related Work
Bobu et al. (2018) presented the problem of continuous unsupervised adaptation wherein the target domain smoothly varying temporally. They proposed an iterative uDA method with replay loss to prevent the model from forgetting knowledge from past domains. Although our solution of multi-step domain adaptation also iterative, we do not assume any smooth ordering between target domains, hence warranting the need for collaboration selection. [Zhao et al. (2018) proposed MDAN where a target domain can adapt from multiple labeled source domains. As there only one labeled source domain in our setup, this method cannot be directly applied. However, we can still combine the adversarial losses from multiple domains as proposed in MDAN as such, we implement variant of MDAN as baseline for our method.
In our paper. the training data from different domains is distributed in separate nodes, which re- lates problem to distributed training. The motivation of most existing works in this field is to accelerate the training process by partitioning the training data into multiple computational nodes and calculating the gradients in parallel. If the results from all nodes are aggregated globally by a central node (c.g., parameter server (Li ct al. (2014); Dai et al. (2015))) or by message passing process (c.g., All-Reduce (Sergeev & Del Balso (2018)), then called centralized distributed training. By contrast, if the system does not require to synchronize globally (Lian et al. (2017a); Tang et al. (008bb), it is called decentralized training. Besides, decentralized training can be further accelerated by its conjunction with low-pre low-precision training (Tang et al. (2018a), or asynchronous training (Lian et al. (2017b)) In addition to acceleration, federated learning (Bonawitz et al. (2019); Konečný et al. (2016)) 6); Yang et al. (2019) Liu et al. (2018) considers training setting where the training data is stored over a large number of separate devices and cannot be shared due to privacy reasons. Our work builds upon the distributed training literature and show, for the first time, how distributed training can be used in the context of uDA. Further, it is important to clarify that use 'decentralization" in this paper in the context of domain adaptation, i.e., target domains can choose to adapt from any one of the available domains, instead of single source domain (akin to a central node). This makes our techniques and their subsequent evaluation different from traditional decentralized learning papers.
Finally, an important goal of our paper is support uDA while keeping the domain datasets private. There has been active research recently at the intersection of privacy and ML. Melis et al. (2019) showed that collaborative learning is vulnerable to membership inference attacks and leakage of dataset properties, and suggested employing dimensionality reduction and Dropout as ways to mit- igate these attacks. [Aono al.) (2017) demonstrated that the gradients of classifier exchanged in distributed training can be used to reconstruct the raw training data and proposed using additively homomorphic encryption to fix the problem. Geyer et al. (2017) proposed a differential privacy approach to hide single client's data contribution during federated learning. Finally, Nasr et al. (2018) did an in-depth analysis of passive and active white-box attacks on deep learning models. To the best of our knowledge, no prior work has shown that sharing gradients of discriminator in adversarial learning which is the key component of our approach- - can leak raw training data. As such, approach provides clear privacy benefits over conventional uDA techniques. That said, we do not discount the possibility that privacy attacks could be developed on our technique in the future, and leave open topic of further research.
3
Our Approach
3.1 Primer on Adversarial Domain Adaptation
Before describing our approach, we provide primer adversarial domain adaptation which is widely used approach for uDA and serves as the basic foundation of our solution. The core idea here to use adversarial learning to align the feature representations of the source and target domains, thereby allowing a source classifier to be used in the target domain. Initially, a source extractor Es and source classifier C's are trained using supervised learning by solving the optimization problem:
K
min Lds EEE -E(r....)><Xss... Es
1 k== (og.:lloo(ss((ssa
3

---

Under review as a conference paper at ICLR 2020
The goal of the adaptation process is to learn target extractor Ey using adversarial learning. To this end, the source extractor Es is used to initialize the target extractor Er. The weights of the source model are fixed during adversarial training. As in standard adversarial learning, two losses are optimized in the training process, a discriminator loss Lader and a mapping loss Ladug- Different uDA methods compute these two losses in their own way, e.g., ADDA (Tzeng et al. (2017) uses the following loss formulations:
min Lodvo NEs [Dg((((E( Ex [log(1 D(Er(z)))
min Laduus = R (log (D(((((()))) Er
(2)
where D represents a domain discriminator which aims to distinguish source domain data and target domain data. Details about other uDA methods are provided in $A.4 in the Appendix.
Our work in general applies to any feature-alignment based uDA technique, which allows for source and target encoders to be trained independently. We note that there are indeed uDA approaches which optimize the source classification loss and adversarial loss in the same batch during training, but they are not compatible with our problem formulation wherein a pre-trained source model is to be deployed on set of target domains. Further, uDA approaches based on generative modeling (Hoffman et al. 2018); Hosseini-Asi et al. (2018) are also out of scope of our work.
3.2 Distributed Domain Adaptation
Prior adversarial uDA methods assume that source and target data is available on the same compute unit to perform adversarial learning. However as discussed in 2 data from different domains is often distributed and private in realistic applications, and sharing it could be legally prohibited. We present an approach of performing domain adaptation with distributed domain datasets. In this vein, two key questions are: a) how to distribute the adversarial network architecture across nodes?, b) how to exchange information between the source and target nodes during training?
We propose to split each constituent neural network in the adversarial training framework across source and target nodes. Consequently, the extractor, domain discriminator and task classifier have source components (Es, Ds. Cs) and target components (ET, DT, CT). Source data and target data can therefore be fed separately into their own model components to prevent any exchange of raw data or feature representations across nodes, Given this network architecture, we now present our training strategy (also summarized in Algorithm 1). The first step is pre-training and initialization. We assume that the extractor and classifier (Ess and Cs) of the source domain have been trained either using supervised learning or through prior domain adaptation, and the source discriminator Ds is initialized randomly. Thereafter, the target domain components, Er, CT and Dr, are initialized with the respective source components. Figure 3 in Appendix A illustrates our distributed architecture and in
A.4 we provide the equations for various adversarial losses.
During adversarial training, for each step, source and target domains sample a training batch from their own domain data and calculate stochastic gradients for Ds on source domain data and for Er, Dr on target domain data accordingly. However, as shown in Eq [ the discriminator needs to be trained on data from both the source and target domains, therefore we need mechanism transfer knowledge across between source and target domains. As opposed to transferring the raw data or extracted features across nodes, our approach performs knowledge exchange through the gradients of the discriminators. This in turn does afford certain privacy benefits, as we do not transmit raw data or the extracted features or even the gradients of the feature extractor which can potentially leak raw data (Aono et al. (2017)) Although to the best of our knowledge, no prior work has shown that discriminator gradients can leak training data, we do not discount the possibility that such privacy attacks can be developed in the future. We leave their study and mitigation
future work.
simple strategy for knowledge transfer between nodes would be to exchange and average the gradients of the two discriminators after each step, and then update Ds and Dr with the averaged gradients. While this will ensure that the discriminators are synchronized, it incurs a significant communication cost for each step. Instead, we propose a simple but effective method, called Lazy Synchronization, to reduce the communication cost of the algorithm. The basic idea is to synchronize the source and target discriminators every p training steps. We refer to the training steps which the synchronization takes place as the sync-up steps while the other steps during which both nodes computing the gradients locally are called local steps. For discriminators Dg and Dr, their local
4

---

Under review as a conference paper at ICLR 2020
gradients accumulated during local steps, and during the sync-up steps the accumulated gradients are averaged and applied to Ds and Dr. Through this, we can ensure there is no divergence between able to decrease the communication cost to 1/p. Note
the discriminators, and at the same time that as we only exchange discriminator gradients, our communication costs are independent of the feature extractors, which can be potentially very deep and large in size.
Algorithm 2: COLAB-SELECC
Algorithm 1: DISTRIBUTED.UDA
Wasserstein-Distance Guided Collab- orator Selection
Lazy-Synchronized Distributed Domain Adaptation
Input: Candidate set C {(p1, E1, C1), (p2, E2, C2)... Ex. Cx)), Target
Result: Ey and Cr
Input: Pre-trained Eg and Cs; Randomly
Dataset pT;
Initialize Ds: Initialize Er = Eis.
for (pi. Es C2) in C do
Dr=Ds, and Cr = Cs: Syne up step p;
W.
number of total steps NN; for n = 1, ........V do Source node and target node sample
pr);
Compute source e, using small
batch of data respectively, (!() and <!n):
labeled test set.;
Compute the Lipschitz Constant K for the source extractor E; and classifier C.
Calculate gradients Vg(Ds, (
source node, gradients Vg(ET, m())) and (( on target node; if iTargetNNode then Apply Vg(( ()( Er end
Toee = + 2.K.W;
end Optimal Collaborator O = arg min i=1...N
(ct...)): );
Return Eo and Co;
Add (g( ((") to source gradients buffer Gs. add Vg( Dr, (r,
Algorithm 3: MDDA: Multi-step Decen-
tralized Domain Selection
target gradients buffer Gr; if n' == 0 then
Input: Initial Candidate set C = {(Ess Cs)}. Ordering of K target domains T Tí. ....TT.
10
Exchange gradients buffer and calculate averaged gradients Gs+Gr.
for T, in do
Gavg 28
Obtain the optimal collaborator O E C using COLAB SELECT: Run DISTRIBUTED DA between O and T; obtain Fr, and Cr. ;
11
Apply Javg to Ds and Dr Clear Gs and Gr
13
end
14 end
C+ C (ET, Cr,)):
end
The target extractor Er is independent of the source data, therefore it is updated for each batch using the adversarial formulation proposed by the underlying adaptation algorithm. For instance, Equation 9 shows the loss function update Er as proposed in ADDA. On the other hand, GRL (Ganin et al. (2016)) uses gradient reversal technique which results in Ladv -Lady Shen al. (2018) uses the Wasserstein Distance as metric for adversarial training. Details about the adversarial loss functions of different algorithms used in this paper are provided in $A.4 Finally, the target classifier Or is initialized with Cs and not updated in the training process. The intuition that if the feature space of the target domain be successfully aligned with the source domain, then Cs can directly be used the target domain without any adaptation.
In 2 empirically show that method can obtain comparable accuracy non-distributed training algorithms, while at the same time, preserving private user data and minimizing the com- munication costs in the training process.
3.3
Wasserstein distance guided collaborator selection
We discuss how to select optimal adaptation collaborator Cg for each target domain. Indeed, we should select a collaborator that likely to yield the lowest target error post adapta- tion, but choose the optimal collaborator before performing domain adaptation? In
5

---

Under review as a conference paper at ICLR 2020
seminal theoretical work, Bavid et al. (2010) showed that the target error is bounded by the sum of source error and the divergence between source and target distributions, Redko et al. (2017) and [Shen et al. (2018) did similar analysis using the Wasserstein distance, which, together with Lipschitz functions, forms the basis for our collaborator selection method.
A function/ f X - R is d-Lipschitz if it satisfies the inequality [[fx)) - x| for some 0 E R4. The smallest such is called the Lipschitz constant Lfp(f) of f. Further, the 1- I-Wasserstein distance (Villani (0088) between two distributions pl and p2 - using the duality formula is
W1 (p1. P2) = sup Ep, [f] - Bp2
(3)
f-Lip(()s
Shen et al. 20188) computed the following robustness bound
Epg h') S <pi (h, h¹) 20W, (p1,P2) for any two 0-Lipschitz hypotheses and hi, where we denoted
(4)
(h, h²) Ep [|A-- I']l]-
(5)
Our method is motivated by the observation that the above bound is good approximation of the error (p2 (h) = 'pp htuee, provided that both the true hypothesis htue and the learned hypotheses h E. C' are i-Lipschiizz, and the push-forwant measures E.p are close together in the Wasser- stein distance, for a sufficiently small 0, These conditions are motivated by the fact that the domains should be well aligned for unsupervised domain adaptation to be effective. Using Lipschitz contin- uous extractor allows us to perform adversarial alignment of higher order features, while maintaining Lipschitz continuity guarantee the true hypothesis function of the encoded features E(x), where x ~ p2-
Now assume we trying to use domain adaptation to find model for domain D (TR" p) and
we have a set of candidate domains D3 = =R"" pp),, k = 1,..., ., K each with a pre-trained model Mr (Es, Ck), obtained by using either supervised learning or prior domain adaptation. We use the estimate provided by Equation to select the optimal collaborator domain
D opt argmin Epp. (Ex C2)) - 20W (PWi(pk.P)
k=
and thereafter perform adaptation from Dapt to D.
Enforcing Lipschitz continuity and computing the Wasserstein distance. Training neural net- works with minimal Lipschitz constants can be done by directly regularizing the spectral norm of the linear part of each layer Mises & Pollaczek-Genie Geiringgr (1929) Gouk et al. (2018), or a gradient penalty Gulrajani et al. (2017), or indirectly by using L1 or L.2 weight decay. Enforcing hard con- straint on can be done by rescaling the linear part of each layer or, indirectly, by weight clipping. employ Spectral Norm regularization
For the feature extractor and classifier neural networks, proposed by Gouk et a (2018). Further, to compute the Wasserstein distance between distributions, train Wasserstein critic with gradient penalty proposed by Gulrajani et al. (2017). This critic can also be trained a distributed manner following our lazy synchronization strategy ($ 3.2).
3.4 Multi-Step Decentralized Domain Adaptation (MDDA)
Using the two core idcas discussed $3.2 and $ 3..3 we now summarize our multi-step DA algorithm MDDA (Algorithm Bp. Assume
given source domain with pre-trained model and bitrary ordering of K unlabeled target domains. For every target domain that joins the system, we collaborator selection with the candidates available in the system. Upon selecting an optimal collaborator, we run the distributed DA algorithm to enable uDA in a privacy-preserving manner. The adaptation process results in a feature extractor ET and classifier CT for the target domain. Finally, the recently adapted target domain is added to the candidate set, and may serve as a potential collaborator for future domains.
4
Evaluation
4.1
Implementation
Datasets. We conduct experiments five image and audio datasets: Rotated MNIST, Digits, and Office-Caltcch, DomainNet and Mic2Mic. Rotated MNIST is a variant of MNIST with numbers rotated from to at increments of 30". Each rotation is considered separate domain. The

---

Under review as a conference paper at ICLR 2020
Digits adaptation task has five domains: MNIST (M), USPS (U). SVHN (S) (Ntzerr et al. 2011
MNIST-M and SynNumbers (SYN) (Ganin & Lempitsky (2014)) Each domain consists of 10 digit classes ranging from 0-9 in different styles. The Office- Caltech dataset contains object images from 10 classes obtained from Amazon (A), DSLR camera (D), Web camera (W), and Caltech- 256 (C). DomainNet (Peng et al.] (2018)) is new dataset from which use four labeled image domains containing 345 classes each: Real (R), QuickDraw (Q), Infograph (I), and Sketch (S). Finally, Mic2Mic (Mathur et al. (2019)) is a speech-based keyword detection dataset wherein the keywords are recorded with four different microphones: Matrix Creator (C), Matrix Voice (V), ReSpeaker (R) and USB (U). Each microphone represents a domain.
Baselines. To evaluate our collaborator selection strategy, we compare it against the following:
Labeled Source. This represents the most commonly used approach in uDA works, wherein every target domain performs a pairwise adaptation with the labeled source domain S.
Random Collaborator. Here, we randomly choose a collaborator с Ck from the collaborator set and perform adaptation with the target domain.
This approach is based on MDAN proposed by Zhao et al. (2018), wherein all available candidate domains are used for adaptation, however the contribution each domain dynamically re- weighted in the adversarial training process. However, note that MDAN assumes that all the candidate domains are labeled and jointly optimize the classification loss with the adversarial training. In scenario, only candidate domain has labeled data (S) while others are unlabeled. As such, we implement a modified version of MDAN which only optimizes their proposed adversarial loss in line with our problem formulation in 33.1
Further, we compare our proposed distributed adversarial learning architecture against (i) non- distributed uDA baseline wherein the source and target data reside on the same node, and (ii) D- PSGD proposed by Lian et al. (2017a) for distributed training of neural networks.
Experiment Setup. We follow the same evaluation protocol as earlier uDA works (Hoffman et al. (2018): TTenn al. (2017)) wherein the unlabeled training instances from source and target domains are used for adversarial training, and the adapted model is evaluated on a held-out test dataset from target domain. In addition, we small subset of the training instances (10% of the total instances) for doing collaborator selection. Further, as we explained earlier, we assume that target domains join the system sequentially in an order, one at a time. Therefore, for each dataset, we randomly select different orderings of source and target domains and present average results across them. Our system implemented with Tensorflow 2.0 and trained on Nvidia V100 GPUs. For more details about network architectures and training hyper-parameters, please see Appendix A.
4.2 Results
We now present our results of applying MDDA on five datasets. Overall, our results show that MDDA selects the right collaborator for adaptation in 82% of the cases, which result in the highest mean target domain accuracy when compared other baselines.
Performance of Collaborator Selection. For each dataset, we choose three random orderings of source and target domains, e.g., for Office-Caltech, we choose O,=W.A,D.C: O₂=D.C.A,VV W: and Oz=A,D,WC W.C. Here the first domain in the order (in bold) represents the labeled source domain, while the others are unlabeled target domains in the order in which they join the system. Please refer to Appendix A for details about the orderings used for other datasets.
Figure I shows the accuracy of our collaborator selection algorithm. For this experiment, we per- form adaptation between a target domain and each available candidate, and based on the target accuracy after each pairwise adaptation, we obtain the collaborator that yields the highest accuracy for each target domain. This serves the ground truth for our algorithm. Thereafter, we run our collaborator selection algorithm on each ordering and obtain the optimal collaborator based on Al- gorithm 2 By comparing the output of our algorithm with the ground truth, we obtain Figure [ which shows that on average, our algorithm has a selection accuracy of 82%. On further analysis, found that our algorithm primarily makes mistakes when two candidates have similar domain discrepancy (estimated using the W, distance) with the target domain and we pick the second most optimal collaborator. In such scenarios, although the top-1 collaborator selection accuracy drops,
7

---

Under review as a conference paper at ICLR 2020
it typically does not impact the target error significantly because adapting with the second-mos optimal domain also yields a good adaptation performance.
RMNNST
Mie2Mic
Dight
Office Caltech
DomainÑer
O2
Oz
O+
0:
O:
O
O:
Oz
Os
0:
0:
O2
O,
No Adaptation
56.23
36.23
15.80
73.11
75.18
14.7
73.42
57.80
93.41
87.15
92.52
19.09
39.73
43.69
79.1
76.0
78.41
90.10
Labeled Source
53.62
54.53
46.47
73.07
N0.96
79.96
66.12
45
91.98
95.57
Malti-Cill Collbbeallbrrtttrr
56.88
37 58
17.8 84
74
75.19
75. 37
60.63
64
62.91
87
88.
02:37
25.5
24.1
MDDA (Ouns)
96.08
70.84
79.19
78.07
80.12
79.96
$1.87
89.03
64.83
98.82
89.58
96.01
34.21
34.34
Meal 86.18
71.77
79.52
78.07
80.96
79.96
83.84
89.03
69.33
95
82
92.09
96.01
34.28
54.34
Table I:
Target accuracy under different techniques of selecting a collaborator using ADDA. Ideal refers to the best achievable performance if the most optimal collaborator is picked for each target.
Table shows, for three different orderings, the mean accuracy across all target domains with MDDA and other baselines. We observe that in majority of the scenarios, MDDA outperforms the labeled-source adaptation baseline which is commonly used in uDA methods. This confirms our key intuition that labeled source domain is not always optimal for domain adaptation, and demon- strates the value of more flexible and decentralized approach like MDDA. Interestingly, we observe that while the multi-collaborator baseline is less accurate than MDDA in general, its performance is particularly poor on Rotated MNIST in which the number of collaborators are quite high while a detailed investigation of this finding is outside the scope of this paper, we surmise that this technique does not scale well the number of collaborator domains increase. MDDA, on the contrary, only performs pairwise adaptation once an optimal collaborator is selected, as such it scales gracefully as the number of target domains increase.
The results presented so far used the adversarial loss formulation from Eq. and 2 which was proposed in ADDA. In Table 2, we evaluate the applicability of MDDA to methods that use other adversarial loss formulations such as (i) when a Gradient Reversal Layer (GRL) is used to compute the mapping loss. (ii) when Wasserstein Distance is used loss metric for domain discriminator (Shen et al. (2018) and (iii) a recently proposed uDA technique called CADA (Zou et al. (2019) which enforces consensus between source and target features. More details about these techniques and their optimization objectives are provided in Appendix A.
We observe that while different techniques yield different target accuracies, MDDA can work in conjunction with all of them to improve the overall accuracy over the conventional labeled source
baseline.
RMNIST (O:)
Digit (O,)
ADDA
GRI
WaeDA
CADA
|| ADDA
GRI.
WheeDA
CADA
60
No Adaptation
Randam
42.89
36,23
30.31
36.23
43.78
16.23
57.15
36:23
78.41
73.42
70.22
73.42
74.91
73.42
76.70
23.42
-0-
Labeled Source
53.62
45.60
37.99
37.65
$1.46
76.66
37.31
65.22
(DDA((Uus)
86.08
63.49
85.6
78.37
89.03
85.24
79.45
76.79
KM
MON
DC
ON
Ideal
56.38
68.92
$6.91
79.50
39.92
85.42
79.9
78.1
Figure 1: Mean accurncy collabora- for selection for three orderings for Ro- lated MNIST (RM), Digits (DI). Mic2Mic (M2M), Office (OC) and Dem)anNeu(DD).
Table 2: Mean target accuracy for a random or- der of domains for different adaptation methods.
Performance of Distributed Domain Adaptation. We now evaluate our Lazy-synchronized DA algorithm from a domain adaptation lens. Our key objective is to evaluate the target-domain accuracy and communication trade-off, i.e., can we save communication costs associated with distributed training while ensuring similar levels of target domain accuracy as the conventional -on-tiitribtted DA algorithms. In Figure P we plot the target domain accuracy with the number of adversarial training steps for three uDA tasks. It can be observed that even with lazy synchronization (sync-up step size p = 4), we can achieve similar target accuracy and convergence rate as non-distributed and synchronized training methods, while incurring only 25% of the communication costs (as are exchanging data every 4 steps). We also note that the distributed training baseline (DPSGD) performs poorly for adversarial training primarily because of the non-overlapping label spaces on each node (with respect to the domain label).
lens.
Table exxaadds this finding to show the target domain accuracy for 10 adaptation tasks under differ- ent training mechanisms. Further, we also report the average time taken to perform the adversarial training for different datasets. Note that the training time is the sum of local computation time and
8

---

Under review as a conference paper at ICLR 2020
Retated MNISTTO
Office Cwhechip
Target Accuracy (99)
70
Targe Accuracy (96)
1oo0 Accurooy marryyy (8))
60-
-00
10
Syndrenvee
2000
8000 10000
500
1000
1500
2000
10000
15000
of Training Steps
of Treining Steps
Training Steps
Figure 2: Comparison of target domain accuracy and training convergence across non-distributed and distributed training methods. For Lazy-synchronized approach, we use sync-up step p= 4. For Mic2Mic, the PSGD baseline did not improve the accuracy, as such we omit it from the figure.
communication time across nodes. In conventional non-distributed DA, it is assumed that the do- main datasets available on the same node, such there is no communication required during the training, which expectedly results in the fastest training process, at the expense of user privacy. In the full-synccroonzeed case, the amount of communication equals the size of discriminator gradients multiplied by the number of training steps, whereas in the lazy-synchronized case, the communica- tion amount is one pth of the amount of fully synchronized case, where p is the sync-up step. Based on the amount of data communicated across nodes, we estimate the communication time by assum- ing a bandwidth of 40Mbps which is roughly the average upload speed for broadband connections globally]] The computation time is measured for each training step and aggregated to get the total computation time. By adding the total computation time with the total communication time, we obtain the total training time reported in Table 3
As shown in Table 3 the Lazy-Synchronizer approach achieves target accuracy close to non- distributed and sully-yyncronnized approaches. For example, in the D + A task in Office-Caltech, our approach- - while ensuring that domain data remains private results 91.77% target domain accuracy, which is indeed close to the 92.01 % and 92.13% accuracy of non-distributed and fully- synchronized training mechanism. Further, we observe that the training time of our approach is significantly less than the nully-yynchronized approach due the reduced communication.
RMNIST
Offic-Caltect
Digts
Domainhet
Mic2Mic
Trsining
[] 00-60
150
180
kimin)
W-C
D+
Uimio
M-M--11 -1
Sya -
Unmi)) [|
S--R
Q R
[omn]
C- R
R-CC
(imin)
SmanosOWily
[]
32.68
61.51
37.81
$5.28
57.56
79.53
|| $4.44
90.25
6965
69:11
Distibered
6961
91.74
92.01
0.07
[]
7802
77.M
Syschosizsd
69:30
91.31
207
81.5
81.1
22.5
Tayy Syrcheorord
48.4
90.6
18
9036
91.77
153
$1.66
19778
57
||
5325 00077
52
77.4
81.00
10
Table 3: Target Domain Accuracy and Training Time (averaged across two adaptation tasks) for each dataset. As expected, the Non-Distributed approach is the fastest as does not require any communication during training, at the expense of user privacy. Our Synchronied approach (p=4) provides the best trade-off between accuracy and training time, without requiring the exchange of raw domain data.
5 Limitations and Conclusion
We introduced novel perspective on uDA research and explored practical challenges associated with deploying uDA methods in real-world MI. systems. Our proposed framework MDDA the first-ever solution aiming to make uDA work in a decentralized and distributed manner.
As uDA rapidly evolving field, we did not study every class of uDA algorithms (e.g., those which combine feature-level adaptations with instance-level adaptations) in this paper. We also made an assumption that target domains are introduced sequentially in the system, however there could indeed be other ways in which MI. models would evolve in practice (e.g., multiple target domains join together or in batches). We leave those scenarios as future work. Finally, our method provides user privacy by only sending discriminator gradicnts across nodes. While no prior work has shown that discriminator gradients can leak raw data, we do not discount the possibility that privacy attacks could be developed on MDDA in the future. We leave detailed privacy study of MDDA as a future work.

---

Under review as a conference paper at ICLR 2020
References
Yoshinori Aono, Takuya Hayashi, Lihua Wang, Shiho Moriai, et al. ing via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, 13(5):1333-145, 2017.
deep learn-
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory of learning from different domains. Machine learning, 79(1- 2010.
Andreea Bobu, Eric Tzeng. Judy Hoffman, and Trevor Darrell. Adapting to continuously shifting domains. 2018.
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp. Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konecny, Stefano Mazzocchi, H Brendan McMahan, al. Towards federated learning at scale: System design. arXiv preprint arXiv:1902.01046, 2019.
Wei Dai, Abhimanu Kumar, Jinliang Wei, Qirong Ho, Garth Gibson, and Eric P Xing. High- performance distributed ml scale through parameter server consistency models. In Twenty- Ninth AAAI Conference Artificial Intelligence, 2015.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409. 1409.7999, 2014.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarte training of neural net- works. The Journal of Machine Learning Research, 17(1):0996 2016.
Robin Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. arXiv preprint 171.v:171,.07557, 2017.
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael Cree. Regularisation of neural networks by enforcing lipschitz continuity. arXiv preprint 1EXXi:18044044 2018.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans. In Advances in neural information processing systems, pp- 5767-5777, 2017.
Judy Hoffman, Eric Tzeng, Taesung Park, Jun- Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International Conference Machine Learning, pp. 1994-2003, 2018.
Ehsan Hosseini-Asl, Yingbo Zhou, Caiming Xiong, and Richard Socher. A multi- discriminator cyclegan for unsupervised non-parallel speech domain adaptation. arXiv preprint arXiv: 00522, 2018.
Jakub Konečný, Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint ariiv:1610.05490 2016.
Mu Li. David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski. James Long. Eugene J Shekita, and Bor- Yoing Su. Scaling distributed machine learning with the param- eter server. In / //th [USENIX] Symposium on Operating Systems Design and Implementation ((OSDI) 14), pp. 583-553999 598, 2014.
Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsich, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? case study for decentralized parallel stochastic gradient descent. In Advances in Neural Information Processing Systems, pp. 5330-5340, 201
Xiangru Lian, Wei Zhang, Ce Zhang, and Ji Liu. Asynchronous decentralized parallel stochastic gradient descent. arXiv preprint 17000:/7 2017b.
Yang Liu, Tianjian Chen, and Qiang Yang. Secure federated transfer learning. arXiv preprint arXiv: arXiv:1812.0888 03337, 2018.
10

---

Under review as a conference paper at ICLR 2020
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. arXiv preprint arXiv: 1.2.0099, 2015.
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael Jordan. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 2208 2217.22777222 JMLR. org, 2017.
Akhil Mathur, Anton Isopoussu, Nadia Berthouze, Nicholas D Lane, and Fahim Kawsar. Unsuper- vised domain adaptation for robust sensory systems. In Proceedings of the 2019 ACM Interna- fional Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019
ACM International Symposium on Wearable Computers, pp- 505-509. ACM, 2019.
Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 691-706. IEEE, 2019.
RV Mises and Hilda Pollaczek- Geiringe.. Praktische verfahren der gleichungsauffösung. ZAMM- Journal of Applied Mathematics and Mechanics/Zeischriff für Angewandte Mathematik und Mechanik, 9(2)1122-164 1929.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprchensive privacy analysis of deep learning: Stand-alone and federated learning under passive and active white-box inference attacks, arXiv preprint arXiv: 011..112..0111 2018.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning, 2.
Xingchao Peng. Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. arXiv preprint arXiv: 18/2.01754, 2018.
levgen Redko, Amaury Habrard, and Mare Sebban. Theoretical analysis of domain adaptation with optimal transport. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 737-753. Springer, 2017.
Swami Sankaranarayanan. Yogesh Balaji, Carlos D Castillo, and Rama Chellappa. Generate to adapt: Aligning domains using generative adversarial networks. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 8503-8512, 2018.
Alexander Sergeev and Mike Del Balso. Horovod: fast and easy distributed deep learning in tensor- 2018.
flow. arXiv preprint
Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. Wasserstein distance guided representation learning for domain adaptation. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, and Ji Liu. Communication compression for decentralized training. Advances Neural Information Prucessing Systems, pp. 7652-7662, 2018a.
Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu. D22: Decentralized training over decentralized data. arXiv preprint arXiv: 070667 2018b.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.
Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
Markus Wulfmeier, Alex Bewley, and Ingmar Posner. Addressing appearance change in outdoor robotics with adversarial domain adaptation. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1551-1558. IEEE, 2017.

---

Under review as a conference paper at ICLR 2020
Qiang Yang. Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):12, 2019.
Han Zhao, Shanghang Zhang. Guanhang Wu, José MF Moura, Joao P Costeira, and Geoffrey J Gordon. Adversarial multiple source domain adaptation. In Advances Neural Information Processing Systems, pp. 8559-8570, 2018.
Han Zou, Yuxun Zhou, Jianfei Yang, Huihan Liu, Hari Prasanna Das, and Costas Spanos. Con- sensus adversarial domain adaptation. In Proceedings of the AAAI Conference Artificial Intel- ligence, volume 33, pp. 5997-6004, 2019.
A
Appendix
Here we provide details about our experiment setup and model architectures. We are also in the process of obtaining necessary clearances to release source code to the community.
A.1 Domain Orderings.
As we mentioned in our problem formulation, the order of target domains joining the system can effect the decisions of our collaborator selecting algorithm. and lead to different target accuracies. To measure the effect of target orders, we reported three different orders for each data set in our experiments. We specify these order as follows:
O
O₂
O₂
RMNIST
0,30,60,90,11 120.0099,,220,15000 150, 180,210,240,270.300
2.180,210,20..70..000 330,30,60,90,120,
30.0,180,210,240.333 60,90,270,300,122,,I1
Mic2Mic
C.V.R,U
R.U.C.V
V.R,U.C
Digits
syhn.mnist.mnist modiffedd synth yynth.digits.usp.
synth digits.usps.mnin modff.nndii
mnist maniifiedd.mist.sshhn.
usps.synth.digic
Office-Caltech
W.A.D.C
D.C.A.W
A.D.W.C
DomainNet
S,R.I.Q
S.Q.I.R
R.I.Q.S
Table 4: Ordering of source and target used in our experiments. Domains in bold correspond to the labeled source domain, which is introduced first in the system.
A.2 Model Architectures and Hyperparameters.
We now describe the neural architectures used for each dataset along with the hyperparameters used in supervised and adversarial learning.
We use the well-known LeNet architecture for this dataset as shown below. The model was trained for each source domain with a learning rate of 10-4 using the Adam optimizer and batch size of 32.
Rotated MNIST:
Conv2D (filters 20, kernel_size = 5, (esiivation//relu)))
MaxPooling2D (pocl_si ze = 2, Sttrides 2), Conv2D (filterss 50, kernel_size = 5, act ineivvation/c relti))=/relu)), MaxPooling2D spool_siaa size 2, strides 2), Flatten(), Dense (500, activation relu'), Dense (10, activations so softmax')
In order to enforce the Lipschitz continuity, added spectral norn regularization during the train- ing process. In the adversarial training process, we used the ADDA losses to perform domain adaptation with learning rate of 10-5 for the target extractor and 10-1 for the discriminator.
12

---

Under review as a conference paper at ICLR 2020
Office-Caltech: We used Keras Inception- V3 pre-trained ImageNet as the base model for this task. We added bottleneck layer and final classification layer. The model was trained for each source domain with learning rate of 10 using the Adam optimizer and batch size of 32.
InceptionV3 (include Setpluude_tp-Falseeee input_shaper (299, 299, 3) avg=' pool'), Dense (256, activation-' relu') Dense (10, activation- softmax')
In order to enforce the Lipschitz continuity, we added spectral norm regularization during the train- ing process. In the adversarial training process, we used the ADDA losses to perform domain adaptation with learning rate of 10-5 for the target extractor and le-4 for the discriminator.
We used Keras ResNet50-v2 pre-trained on ImageNet as the base model for this task. The model was trained for each source domain with learning rate of le-5 using the Adam optimizer and a batch size of 64.
DomainNet:
ResNet50V2 phcllude_top=Falle Dense (245, activation-" softmax')
input_shapes (224, 224,3), avg='pool'),
In order to enforce the Lipschitz continuity, we added spectral norm regularization during the train- ing process. In the adversarial training process, we used the ADDA losses perform domain adaptation with a learning rate of 10-6 for the target extractor and 10-t 4 for the discriminator.
Mic2Mic: We mainly used three convolutional layers for this task. The model was trained for each source domain with learning rate of 10-5 using the Adam optimizer and batch size of 64.
Conv2D (filtees 64, kernel size = (8, (0) activation- (erivattion'relu))
MaxPooling2D (pocl_sioe)) Conv2D (filters 128, ternel_size = 14, ostivation=rrelu)) (1, 4)),
(2, 2),
MaxPooling2D (pool_ssipp
Conv2D (filters 512, sarnel_size = (2,2)),
Flatten
Dense (256, activation='ellu" Dense (31)
In order to enforce the Lipschitz continuity, we added spectral norm regularization during the train- ing process. In the adversarial training process, used the ADDA losses perform domain adaptation with a learning rate of 10-6 for the target extractor and 10-4 for the discriminator.
Digits: We constructed neural network with three convolutional layer and some additional tech- niques like Dropout, ButchNormalization, for this task. The model was trained for each source domain with learning rate of 10 using the Adam optimizer and batch size of 64.
inputs = keras. Input (shape= (32, 3), name= img') x Conv2D (filters 64, kernel_ = 5, strides=2) (inputs) (x, training-is.ttraiiing. x Dropout (0.1) (0.1) (x, crainingsis_trainning]
BatchNormalization:
ReLU () (x)
Conv2D (filters = 128, kernel_size = 5, strides=1) (x)
Batcchoomalizzaiinn))) (x, trainingris_ttraiinng
Dropout (0.3) (0.3) (x, trainingsis_trriiinng)
ReLU () (x)
Conv2D (filters = 256, kernel_size = 5, strides=1) (x)
batchhormalizztionn () (x, trainingsis
=
Dropout (0.5) (x, trainingsis_traiinnn))
X ReLU () (x)
Flatten () (x)
X Dense (512) (x)
BatchNormalization (x, trainingsis_ttraiiing
X ReLU() () (x)
Dropout (0.5) (x, trainingsis__treining
13

---

Under review as a conference paper at ICLR 2020
outputs Dense (10) (x)
In order to enforce the Lipschitz continuity, we added spectral norm regularization during the train- ing process. In the adversarial training process, used the ADDA losses perform domain adaptation with a learning rate of 10-6 for the target extractor and 10 for the discriminator.
A.3 Architecture Diagram
Figure 3 shows the general architecture of our distributed suzysyycchoonize training technique. As explained in Algorithm I and $3.22 we split each constituent neural network in the adversarial train- ing framework across source and target nodes. Consequently, the extractor, domain discriminator and task classifter have source components and target components. Knowledge exchange between the nodes done through exchanging the discriminator gradients in the sync-up steps.
Gradients
p(e)
Source
Source Discriminator
discriinnaoo Vy(Dg. 410)
Extractor
In non-sync steps: Accumulate local gradients sync-up steps: Aggregate accumulated gradients, then update discriminators
p(n)
target discriminator Vg(Dy, g/a)) + Gradients of target extractor Vg((E, (
Target Extractor
Target Discriminator
Apply Vg((Ey- 4!"") to target extractor every step
Figure Distributed adversarial training architecture of MDDA
A.4 Details of Domain Adaptation Algorithms
We now discuss the adversarial training formulation of the various uDAA algorithms with which we evaluated the efficacy of our proposed MDDA framework. As shown in T.ble we evaluate our method with four domain adaptation techniques: ADDA (Tzeng et al. (2017). Gradient Reversal (Ganin et al. (2016)) and Wasserstein DA (Shen al. (201 8)), and CADA (Zou et al. (211)) Below are the adversarial training formulations of these techniques as proposed in their original papers.
ADDA. Following the same notations used earlier in the paper, the adversarial loss formulations of ADDA can be represented mathematically as:
min Ladup EEs (D((((()))))]] Ex [log(1 D(Er(x:))
min Laduas = - En (og(((( Ex
(8)
The Discriminator D is optimized using Ladvp where the domain data from source and target do- mains are assigned different domain labels (0 and 1). update the feature extractor Fr, ADDA proposes to invert the domain labels, which results in the loss formulation given in Tradex--
In order to run ADDA in a distributed manner, we decompose the discriminator D into Dy and Dr which results in the following loss functions:
LadvDs = Ex (IsSs
(9)
Ladupy
Ex [log(1 (log( (Fr(()))
(10)
14

---

Under review as a conference paper at ICLR 2020
min Ladvar = -Bg,N (log((((r((()))))
(11)
Pr
Thereafter, we compute local gradients for Ds and Dr,
Vg(Ds, = dDs
(12)
òLadvpr
Vg(Dr, 8Dr
(13)
and aggregate them in the sync-up step as shown in Algorithm The aggregated gradients are used optimize Dg and Dr, while ËT is optimized using the loss function in Equation
Gradient Reversal. The Gradient Reversal approach uses the same loss formulation for the dis- criminators Dy and Dr ADDA as shown in Equations
10
However, to update the target extractor Er. it leverages the gradient reversal strategy, resulting in the following loss function.
min Lodvar
Ludenr
Fr
(14)
Er [log(1 Dr(Er(x)))
The computation of local gradients for Dg and Dr follow the same process shown in Equations 12 and 13
Wasserstein DA. In this recently published technique, authors the Wasserstein distance as the loss function for discriminator. Wasserstein distance between two datasets is defined as
Xx,,e
Σ D(Es(z.)) -
1 Σ ((Er(z))
(15)
The x,~Xa
The JI
where and are the number of samples in the dataset. The discriminator loss is computed as:
min Ladve - - NoN Ri (asserstein((x..:r] + +
(16)
where Lorad is the gradient penalty used enforce the Lipschitz constraint on the discriminator. Further, the target extractor is optimized using the following loss function:
min Ladexi EEg,NXs. Wasserstein((,,.2) Ey
(17)
We use the same strategy to compute local gradients for Dg and Dy as shown in Eq 12 and
13
CADA. Consensus Adversarial Domain Adaptation a technique recently proposed by Zou et al. 2019) which enforces the source and target extractors to arrive at consensus in the feature space through adversarial training. It the same loss formulation for the discriminators Dg and Dr as However, the key difference is that CADA optimizes both the source and target feature extractors in the training process, until the discriminator can no longer distinguish the features from source and target domains.
ADDA shown in Equations 7 9 10
min Ladvan Es
Ee, [log( Ds (Fs(())))))
(18)
min CadvMr Ex, (og(Dr(Er(xx)) Er
(19)
Thereafter, a shared classifier is trained on the sourced labeled data by keeping the source feature extractor fixed. The shared classifier can be used with the target extractor to make predictions.
15

---

Under review as a conference paper at ICLR 2020
K
min Las - -E ) (Xgs)ss Cshured
[k= (ogel(lC (Es(x))
16


Reviews:
[{"review_id": "rylZ6nz59B", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I read the authors response. I am satisfied with the explanations on the privacy party. However, decentralized training part is still unsatisfactory since the empirical evaluation is not really decentralized. Back of the envelope calculations are at best correlated to the actual times spent by each node.  Hence, the numbers in Table 3 are not physical numbers, rather result of an idealized network. Moreover, the x-axis of Figure 2 being training step is still not acceptable. Decentralized and centralized methods should be compared in terms of time which is the only fair metric. I stick to my original decision.\n\n-----\n\nThe manuscript is proposing a method for domain adaptation in a private and distributed setting where there are multiple target domains and they are added in a sequential manner. The proposed method considers only the domain adaptation methods in which the source model training and the target model training are done separately. In this setting, existing adapted models can be used as a source domain since a trained model suffices for adaptation. One major contribution of the paper is proposing a straightforward but successful method to choose which domain to adapt from. The main algorithmic tool is estimating Wasserstein distance and choosing the closest domain. The second contribution is distributed training setting for privacy and decentralization.\n\nChoosing which model to adapt from is an interesting contribution. The proposed setting is definitely sensible and the proposed method is theoretically sound. Hence, I consider this as a valuable contribution to the domain adaptation literature. Moreover, results suggest that it also results in significant performance improvement.\n\nPrivacy and decentralized learning part has major issues. First of all, the privacy and learning private models is a sub-field of machine learning with a large literature. Authors do not discuss any of these existing work. Second of all, authors do not specify the definition of privacy they are using. Only guarantee the  algorithm provides is not passing data around. However, this is clearly not enough. Passing gradients might result in sharing sensitive data. The actual data can be reconstructed (upto some accuracy) using the gradients passed between nodes. Therefore, either a justification or a privacy guarantee result is needed. Both of these are major issues which need to be fixed.\n\nDecentralized learning is also an important problem which have been studied significantly. Related work section is missing majority of recent and existing work on distributed learning and federated learning. Moreover, empirical study is very counter intuitive. Results are given in terms of accuracy vs number of training steps. The important metrics are amount of massages passed and the total time of the distributed training. Many distributed algorithms trade off having less accurate gradients (consequently having higher number of gradient updates) with less message passing. Hence, I am not sure how to understand the distributed domain adaptation experiments. I am not even sure what the time in Table 3 actually means since it is clearly not even experimented in a distributed setting.\n\nIn summary, the submission is addressing an important problem. Moreover, the contribution on collaborator selection is interesting and seems to be working well. However, the private and decentralized learning parts are rather incomplete from related work and experiment sense. Moreover, I am also not sure can we call this method private or not.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "rkgK7eQYcH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper focuses on the problem of unsupervised domain adaptation in practical machine learning systems. To address the problems in current unsupervised domain adaptation methods, the authors propose to a novel multi-step framework which works in a decentralized and distributed manner. Experimental results show the effectiveness of the proposed approach. \n\nThis paper is well-motivated and the proposed method is novel for unsupervised domain adaptation. The paper is well-supported by theoretical analysis, however, the improvements are not that significant on some experimental results. For the above reasons, I tend to accept this paper but wouldn't mind rejecting it. \n\nQuestions: \n1. The experiments do not really show the superiority of the proposed method compared to the common centralized approaches as they have similar performances on both collaborator selection and distributed domain adaptation. Can you convince the readers more with some other experiments?\n2. What is the weakness of such decentralization models?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "SyehsyHxKr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n\n###Summary###\n\nThis paper tackles unsupervised domain adaptation in a decentralized setting.  The high-level observation is that the conventional unsupervised domain adaptation has two bottlenecks, namely excessive centralization and poor support for distributed domain datasets. The paper proposes Multi-Step Decentralized Domain Adaptation (MDDA) to transfer the knowledge learned from the source domain to the target domain without sharing the data.  \n\nThe paper also explores explore a proposition: in addition to adapting from the labeled source, can uDA leverage the knowledge from other target domains, which themselves may have undergone domain adaptation in the past. \n\nThe proposed MMDA method contains a feature extractor (E), a domain discriminator (D) and task classifier (C) for each domain. The target domain components are initialized with the respective source components. The source domain discriminator D_s target domain discriminator D_t are synchronized by exchanging and averaging the gradients. The paper also proposes Lazy Synchronization to reduce the communication cost of the algorithm.\n\nThe paper also proposes Wasserstein distance guided collaborator selection schema to perform the domain adaptation task. \n\nThe paper performs experiments on five image and audio datasets: Rotated MNIST, Digits, and Office-Caltech, DomainNet and Mic2Mic. \n\nThe baselines used in this paper include \"Labeled Source\", \"Random Collaborator\", and \"Multi-Collaborator\". The experimental results demonstrate that the proposed method can outperform the baselines on some of the experimental settings. The paper also provides a detailed analysis of the model and experimental results. \n\n### Novelty ###\n\nThis paper does not propose a new domain adaptation algorithm. However, the paper introduces some interesting tricks to solve the MMDA task such as the lazy synchronization between the source domain discriminator and the target domain discriminator. \n\n###Clarity###\n\nSeveral critical explanations are missing from the paper:\n1) When training the source domain discriminator D_s and target domain discriminator D_t, if the features between the source domain and target domain cannot be shared with each other, how to train the D_s and D_t. For example, the D_s cannot get access to the features from the target domain, how to train D_s? \n2) How is the target classifier C_t updated when there are no labels for the target domain?\n3) As far as I understand, the domain discriminator is this paper is trained adversarially. The detailed adversarial training step is unclear.  \n\n###Pros###\n\n1) The paper proposes an interesting transfer learning schema where the data between the source and target domain can not be shared with each other to protect the data-privacy.\n\n2) The paper provides extensive experiments on multiple standard domain adaptation benchmarks, especially the most recent dataset such as the DomainNet. \n\n3) The paper provides detail empirical analysis to demonstrate the effectiveness of the proposed methods. \n\n###Cons###\n\n1) The most critical issue of this paper is that some explanations are missing, e.g. how are D_s, D_t, C_t trained? Refer to the #Clarity.\n\n2) The presentation and writing of this paper need polish. The author should do more relative surveys to motivate the authors. One critical relevant reference of this paper is:\n\"Secure Federated Transfer Learning\", Yang Liu et al \n\nhttps://arxiv.org/pdf/1812.03337.pdf\n\n3) The baselines used in this paper is also trivial. It is desirable to compare the proposed method with state-of-the-art domain adaptation methods.\n\nBased on the summary, cons, and pros, the current rating I am giving now is \"reject\". I would like to discuss the final rating with other reviewers, ACs.\nTo improve the rating, the author should explain the questions I proposed in the #Clarity", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "ByxBx9sFOS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper focuses on the problem of domain adaptation among multiple domains when some domains are not available on the same machine. The paper builds a decentralized algorithm based on previous domain adaptation methods.\n\nPros: \n1. The problem is novel and practical. Previous domain adaptation assumes that source and target domains are available but it can happen when the source and target domains have connection issues.\n2. The method exploits asynchronizing accumulating and synchronizing update, which reduces the cost of communication between domains.\n3. The paper proposes to use Wasserstein distance to select the optimal domain as the source domain for the target domain. \n4. The experimental results show that the proposed method outperforms baselines.\n\nCons:\n1. The asynchronizing accumulating and synchronizing update is not novel. It has been used in other communities such as reinforcement learning.\n\nOverall, the paper is good and it is technically sound. The contribution is not significant to the community but providing a new perspective for domain adaptation. I vote for weak accept.\n\nThank Reviewer1 for reminding. I think the paper still has some novelty and the comments address my concerns. I do  not change my score. Also, I'm not unhappy if the paper is rejected. It is more like a borderline paper.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a solution to the decentralized privacy preserving domain adaptation problem. In other words, how to adapt to a target domain without explicit data access to other existing domains. In this scenario the authors propose MDDA which consists of both a collaborator selection algorithm based on minimal Wasserstein distance as well as a technique for adapting through sharing discriminator gradients across domains. 

The reviewers has split scores for this work with two recommending weak accept and two recommending weak reject. However, both reviewers who recommended weak accept explicitly mentioned that their recommendation was borderline (an option not available for ICLR 2020). The main issues raised by the reviewers was lack of algorithmic novelty and lack of comparison to prior privacy preserving work. The authors agreed that their goal was not to introduce a new domain adaptation algorithm, but rather to propose a generic solution to extend existing algorithms to the case of privacy preserving and decentralized DA.  The authors also provided extensive revisions in response to the reviewers comments. Though the reviewers were convinced on some points (like privacy preserving arguments), there still remained key outstanding issues that were significant enough to cause the reviewers not to update their recommendations. 

Therefore, this paper is not recommended for acceptance in its current form. We encourage the authors to build off the revisions completed during the rebuttal phase and any outstanding comments from the reviewers. 
Decision: Reject