Original: D OMAIN A DAPTIVE M ULTIBRANCH N ETWORKS
Revised: D OMAIN A DAPTIVE M ULTIBRANCH N ETWORKS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We tackle unsupervised domain adaptation by accounting for the fact that different domains may need to be processed differently to arrive to a common feature representation effective for recognition.
Revised: We tackle unsupervised domain adaptation by accounting for the fact that different domains may need to be processed differently to arrive to a common feature representation effective for recognition.

Original: To this end, we introduce a deep learning framework where each domain undergoes a different sequence of operations, allowing some, possibly more complex, domains to go through more computations than others.
Revised: To this end, we introduce a deep learning framework where each domain undergoes a different sequence of operations, allowing some, possibly more complex, domains to go through more computations than others.

Original: This contrasts with state-of-the-art domain adaptation techniques that force all domains to be processed with the same series of operations, even when using multi-stream architectures whose parameters are not shared.
Revised: This contrasts with state-of-the-art domain adaptation techniques that force all domains to be processed with the same series of operations, even when using multi-stream architectures whose parameters are not shared.

Original: As evidenced by our experiments, the greater ﬂexibility of our method translates to higher accuracy.
Revised: As evidenced by our experiments, the greater ﬂexibility of our method translates to higher accuracy.

Original: Furthermore, it allows us to handle any number of domains simultaneously.
Revised: Furthermore, it allows us to handle any number of domains simultaneously

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: While deep learning has ushered in great advances in automated image understanding, it still suffers from the same weaknesses as all other machine learning techniques: when trained with images obtained under speciﬁc conditions, deep networks typically perform poorly on images acquired under different ones.
Revised: While deep learning has ushered in great advances in automated image understanding, it still suffers from the same weaknesses as all other machine learning techniques: when trained with images obtained under speciﬁc conditions, deep networks typically perform poorly on images acquired under different ones.

Original: This is known as the domain shift problem: the changing conditions cause the statistical properties of the test, or target, data, to be different from those of the training, or source, data, and the network’s performance degrades accordingly.
Revised: This is known as the domain shift problem: the changing conditions cause the statistical properties of the test, or target, data, to be different from those of the training, or source, data, and the network’s performance degrades accordingly.

Original: Domain adaptation aims to address this problem, especially when annotating images from the target domain is difﬁcult, expensive, or downright infeasible.
Revised: Domain adaptation aims to address this problem, especially when annotating images from the target domain is difﬁcult, expensive, or downright infeasible.

Original: The dominant trend is to map images to features that are immune to the domain shift, so that the classiﬁer works equally well on the source and target domains (Fernando et al., 2013; Ganin & Lempitsky, 2015; Sun & Saenko, 2016).
Revised: The dominant trend is to map images to features that are immune to the domain shift, so that the classiﬁer works equally well on the source and target domains (Fernando et al., 2013; Ganin & Lempitsky, 2015; Sun & Saenko, 2016).

Original: In the context of deep learning, the standard approach is to ﬁnd those features using a single architecture for both domains (Tzeng et al., 2014; Ganin & Lempitsky, 2015; Sun & Saenko, 2016; Yan et al., 2017; Zhang et al., 2018).
Revised: In the context of deep learning, the standard approach is to ﬁnd those features using a single architecture for both domains (Tzeng et al., 2014; Ganin & Lempitsky, 2015; Sun & Saenko, 2016; Yan et al., 2017; Zhang et al., 2018).

Original: Intuitively, however, as the domains have different properties, it is not easy to ﬁnd one network that does this effectively for both.
Revised: Intuitively, however, as the domains have different properties, it is not easy to ﬁnd one network that does this effectively for both.

Original: A better approach is to allow domains to undergo different transformations to arrive at domain-invariant features.
Revised: A better approach is to allow domains to undergo different transformations to arrive at domain-invariant features.

Original: This has been the focus of recent work (Tzeng et al., 2017; Berm ´ udez-Chac ´ on et al., 2018; Rozantsev et al., 2018; 2019), where source and target data pass through two different networks with the same architecture but different weights, nonetheless related to each other.
Revised: This has been the focus of recent work (Tzeng et al., 2017; Berm´udez-Chac´on et al., 2018; Rozantsev et al., 2018; 2019), where source and target data pass through two different networks with the same architecture but different weights, nonetheless related to each other.

Original: In this paper, we introduce a novel, even more ﬂexible paradigm for domain adaptation, that allows the different domains to undergo different computations , not only in terms of layer weights but also in terms of number of operations, while selectively sharing subsets of these computations.
Revised: In this paper, we introduce a novel, even more ﬂexible paradigm for domain adaptation, that allows the different domains to undergo different computations , not only in terms of layer weights but also in terms of number of operations, while selectively sharing subsets of these computations.

Original: This enables the network to automatically adapt to situations where, for example, one domain depicts simpler images, such as synthetic ones, which may not need as much processing power as those coming from more complex domains, such as images taken in-the-wild.
Revised: This enables the network to automatically adapt to situations where, for example, one domain depicts simpler images, such as synthetic ones, which may not need as much processing power as those coming from more complex domains, such as images taken in-the-wild.

Original: Our formulation reﬂects the intuition that source and target domain networks should be similar because they solve closely related problems, but should also perform domain-speciﬁc computations to offset the domain shift.
Revised: Our formulation reﬂects the intuition that source and target domain networks should be similar because they solve closely related problems, but should also perform domain-speciﬁc computations to offset the domain shift.

Original: To turn this intuition into a working algorithm, we develop a multibranch architecture that sends the data through multiple network branches in parallel.
Revised: To turn this intuition into a working algorithm, we develop a multibranch architecture that sends the data through multiple network branches in parallel.

Original: What gives it the necessary ﬂexibility are trainable gates that are tuned to modulate and combine the outputs of these branches, as shown in
Revised: What gives it the necessary ﬂexibility are trainable gates that are tuned to modulate and combine the outputs of these branches, as shown in

Original: Fig.
Revised: Fig.

Original: Assigning to each domain its own set of gates allows the global network to learn what set of  computations should be carried out for each one.
Revised: Assigning to each domain its own set of gates allows the global network to learn what set of [Figure] computations should be carried out for each one.

Original: As an additional beneﬁt, in contrast to previous strategies for untying the source and target streams (Rozantsev et al., 2018; 2019), our formulation naturally extends to more than two domains.[Figure]
Revised: As an additional beneﬁt, in contrast to previous strategies for untying the source and target streams (Rozantsev et al., 2018; 2019), our formulation naturally extends to more than two domains.

Original: In other words, our contribution is a learning strategy that adaptively adjusts the speciﬁc computation to be performed for each domain.
Revised: In other words, our contribution is a learning strategy that adaptively adjusts the speciﬁc computation to be performed for each domain.

Original: To demonstrate that it constitutes an effective approach to extracting domain-invariant features, we implement it in conjunction with the popular domain classiﬁer-based method of Ganin & Lempitsky (2015).
Revised: To demonstrate that it constitutes an effective approach to extracting domain-invariant features, we implement it in conjunction with the popular domain classiﬁer-based method of Ganin & Lempitsky (2015).

Original: Our experiments demonstrate that our Domain Adaptive Multibranch Networks, which we will refer to as DAMnets, not only outperform the original technique of Ganin & Lempitsky (2015), but also the state-of-the-art strategy for untying the source and target weights of Rozantsev et al.
Revised: Our experiments demonstrate that our Domain Adaptive Multibranch Networks, which we will refer to as DAMNets, not only outperform the original technique of Ganin & Lempitsky (2015), but also the state-of-the-art strategy for untying the source and target weights of Rozantsev et al.

Original: (2019), which relies on the same domain classiﬁer.
Revised: (2019), which relies on the same domain classiﬁer.

Original: We will make our code publicly available upon acceptance of the paper.
Revised: We will make our code publicly available upon acceptance of the paper.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Domain Adaptation.
Revised: Domain Adaptation.

Original: Domain adaptation has achieved important milestones in recent years (Dai et al., 2007; Gretton et al., 2009; Pan et al., 2010; Fernando et al., 2013; Sun et al., 2016; Shu et al., 2018), with deep learning-based methods largely taking the lead in performance.
Revised: Domain adaptation has achieved important milestones in recent years (Dai et al., 2007; Gretton et al., 2009; Pan et al., 2010; Fernando et al., 2013; Sun et al., 2016; Shu et al., 2018), with deep learning-based methods largely taking the lead in performance.

Original: The dominant approach to deep domain adaptation is to learn a domain-invariant data representation.
Revised: The dominant approach to deep domain adaptation is to learn a domain-invariant data representation.

Original: This is commonly achieved by ﬁnding a mapping to a feature space where the source and target features have the same distribution.
Revised: This is commonly achieved by ﬁnding a mapping to a feature space where the source and target features have the same distribution.

Original: In Tzeng et al.
Revised: In Tzeng et al.

Original: (2014); Long et al.
Revised: (2014); Long et al.

Original: ; Yan et al.
Revised: ; Yan et al.

Original: (2017), the distribution similarity was measured in terms of Maximum Mean Discrepancy (Gretton et al., 2007), while other metrics based on second- and higher-order statistics were introduced in Koniusz et al.
Revised: (2017), the distribution similarity was measured in terms of Maximum Mean Discrepancy (Gretton et al., 2007), while other metrics based on second- and higher-order statistics were introduced in Sun &

Original: (2017); Sun et al. (2017); Sun & Saenko (2016).
Revised: Saenko (2016); Koniusz et al. (2017); Sun et al.

Original: In Saito et al.
Revised: In Saito et al.

Original: (2018), the distribution alignment process was disambiguated by exploiting the class labels, and in H¨ausser et al.(2017); Shkodrani et al.
Revised: (2018), the distribution alignment process was disambiguated by exploiting the class labels, and in H ¨ ausser et al. Shkodrani et al.

Original: by leveraging anchor points associating embeddings between the domains.
Revised: by leveraging anchor points associating embeddings between the domains.

Original: Another popular approach to learning domain-invariant features is to train a classiﬁer to recognize the domain from which a sample was drawn, and use adversarial training to arrive to features that the classiﬁer can no longer discriminate (Ganin et al., 2016; 2017; Tzeng et al., 2015).
Revised: Another popular approach to learning domain-invariant features is to train a classiﬁer to recognize the domain from which a sample was drawn, and use adversarial training to arrive to features that the classiﬁer can no longer discriminate (Tzeng et al., 2015; Ganin et al., 2016; 2017).

Original: This idea has spawned several recent adversarial domain adaptation classiﬁcation (Hu et al., 2018;
Revised: This idea has spawned several recent adversarial domain adaptation classiﬁcation (Hu et al., 2018;

Original: Zhang et al., 2018), semantic segmentation (Chen et al., 2018; Hoffman et al., 2018; Hong et al., 2018), and active learning (Su et al., 2019) techniques, and we will use such a classiﬁer.
Revised: Zhang et al., 2018), semantic segmentation (Hoffman et al., 2018; Chen et al., 2018; Hong et al., 2018), and active learning (Su et al., 2019) techniques, and we will use such a classiﬁer.

Original: Closest in spirit to our approach are those that do not share the weights of the networks that process the source and target data (Berm´udez-Chac´on et al., 2018; Rozantsev et al., 2018; 2019; Tzeng et al., 2017).
Revised: Closest in spirit to our approach are those that do not share the weights of the networks that process the source and target data (Tzeng et al., 2017; Berm´udez-Chac´on et al., 2018; Rozantsev et al., 2018; 2019).

Original: In Tzeng et al.
Revised: In Tzeng et al.

Original: (2017), the weights were simply allowed to vary freely.
Revised: (2017), the weights were simply allowed to vary freely.

Original: In Berm´udez-Chac´on et al. ; Rozantsev et al.
Revised: In Rozantsev et al. ; Berm ´ udez-Chac ´ on et al.

Original: (2018), it was shown that regularizing them to remain close to each other was beneﬁcial.
Revised: (2018), it was shown that regularizing them to remain close to each other was beneﬁcial.

Original: More recently, Rozantsev et al.
Revised: More recently, Rozantsev et al.

Original: (2019) proposed to train small networks to map the source weights to the target ones.
Revised: (2019) proposed to train small networks to map the source weights to the target ones.

Original: While these methods indeed untie the source and target weights, the source and target data still undergo the same computations, i.e., number of operations.
Revised: While these methods indeed untie the source and target weights, the source and target data still undergo the same computations, i.e., number of operations.

Original: In this paper, we argue that the amount of computation, that is, the network capacity, should adapt to each domain and reﬂect their respective complexities.
Revised: In this paper, we argue that the amount of computation, that is, the network capacity, should adapt to each domain and reﬂect their respective complexities.

Original: We rely on a domain classiﬁer as in Tzeng et al.
Revised: We rely on a domain classiﬁer as in Tzeng et al.

Original: ; Ganin et al.
Revised: ; Ganin et al.

Original: However, we do not force the source and target samples to go through the same transformations, which is counterintuitive since they display different appearance statistics.
Revised: However, we do not force the source and target samples to go through the same transformations, which is counterintuitive since they display different appearance statistics.

Original: Instead, we start from the premise that they should undergo different computations and use domain-speciﬁc gates to turn this premise into our DAMnet architecture.
Revised: Instead, we start from the premise that they should undergo different computations and use domain-speciﬁc gates to turn this premise into our DAMNet architecture.

Original: Dynamic Network Architectures.
Revised: Dynamic Network Architectures.

Original: As the performance of a neural network is tightly linked to its structure, there has been a recent push towards automatically determining the best architecture for the problem at hand.
Revised: As the performance of a neural network is tightly linked to its structure, there has been a recent push towards automatically determining the best architecture for the problem at hand.

Original: While neural architecture search techniques (Zoph & Le, 2017; Liu et al., 2018; 2019; Pham et al., 2018; Zoph et al., 2018; Real et al., 2019; Noy et al., 2019) aim to ﬁnd one ﬁxed architecture for a given dataset, other works have focused on dynamically adapting the network structure at inference time (Graves, 2016;
Revised: While neural architecture search techniques (Zoph & Le, 2017; Liu et al., 2018; 2019; Pham et al., 2018; Zoph et al., 2018; Real et al., 2019; Noy et al., 2019) aim to ﬁnd one ﬁxed architecture for a given dataset, other works have focused on dynamically adapting the network structure at inference time (Graves, 2016;

Original: Ahmed & Torresani, 2017; Shazeer et al., 2017;
Revised: Ahmed & Torresani, 2017; Shazeer et al., 2017;

Original: Veit & Belongie, 2018; Wu et al., 2018).
Revised: Veit & Belongie, 2018; Wu et al., 2018).

Original: In particular, in Ahmed & Torresani (2017); Shazeer et al.
Revised: In particular, in Ahmed & Torresani (2017); Shazeer et al.

Original: (2017); Veit & Belongie (2018); Bhatia et al.
Revised: (2017); Veit & Belongie (2018); Bhatia et al.

Original: (2019), gates were introduced for this purpose.
Revised: (2019), gates were introduced for this purpose.

Original: While our DAMnets also rely on gates, their role is very different: ﬁrst, we work with data coming from different domains, whereas these gated methods, with the exception of Bhatia et al.
Revised: While our DAMNets also rely on gates, their role is very different: ﬁrst, we work with data coming from different domains, whereas these gated methods, with the exception of Bhatia et al.

Original: (2019), were all designed to work in the single-domain scenario.
Revised: (2019), were all designed to work in the single-domain scenario.

Original: Second, and more importantly, these techniques aim to deﬁne a different computational path for every test sample .
Revised: Second, and more importantly, these techniques aim to deﬁne a different computational path for every test sample .

Original: By contrast, we seek to determine the right computation for each domain .
Revised: By contrast, we seek to determine the right computation for each domain .

Original: Another consideration is that we freeze our gates for inference while these methods must constantly update theirs.
Revised: Another consideration is that we freeze our gates for inference while these methods must constantly update theirs.

Original: We believe this to be illsuited to domain adaptation, particularly because learning to adapt the gates for the target domain, for which only unlabeled data is available, is severely under-constrained.
Revised: We believe this to be illsuited to domain adaptation, particularly because learning to adapt the gates for the target domain, for which only unlabeled data is available, is severely under-constrained.

Original: This lack of supervision may be manageable when one seeks to deﬁne operations for a whole domain, but not when these operations are sample-speciﬁc.
Revised: This lack of supervision may be manageable when one seeks to deﬁne operations for a whole domain, but not when these operations are sample-speciﬁc.

Original:  M ETHOD
Revised:  M ETHOD

Original: We now describe our deep domain adaptation approach, which automatically adjusts the computations that the different domains undergo.
Revised: We now describe our deep domain adaptation approach, which automatically adjusts the computations that the different domains undergo.

Original: We ﬁrst introduce the multibranch networks that form the backbone of our DAMnet architecture and then discuss training in the domain adaptation scenario.
Revised: We ﬁrst introduce the multibranch networks that form the backbone of our DAMNet architecture and then discuss training in the domain adaptation scenario.

Original: 3. M ULTIBRANCH N ETWORKS
Revised: 3. M ULTIBRANCH N ETWORKS

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Equation] [Equation] K ( i ) j

Original: Let us ﬁrst consider a single domain.
Revised: Let us ﬁrst consider a single domain.

Original: In this context, a traditional deep neural network can be thought of as a sequence of N f operations f ( i ) ( · ) 1 ≤ i ≤ N f , each transforming the output of the previous one.
Revised: In this context, a traditional deep neural network can be thought of as a sequence of N f operations f ( i ) ( · ) 1 ≤ i ≤ N f , each transforming the output of the previous one.

Original: Given an input image x , this can be expressed as
Revised: Given an input image x , this can be expressed as

Original: [Equation]
Revised: [Equation]

Original: As a general convention, each operation f ( i ) ( · ) can represent either a single layer or multiple ones.
Revised: As a general convention, each operation f ( i ) ( · ) can represent either a single layer or multiple ones.

Original: Our formulation extends this deﬁnition by replacing each f ( i ) by multiple parallel computations, as shown in Fig.
Revised: Our formulation extends this deﬁnition by replacing each f ( i ) by multiple parallel computations, as shown in Fig.

Original: More speciﬁcally, we replace each f ( i ) by a computational unit { f ( i ) 1 , . . .
Revised: More speciﬁcally, we replace each f ( i ) by a computational unit { f ( i ) 1 , . . .

Original: , f ( i ) K } consisting of K parallel branches .
Revised: , f ( i ) K } consisting of K parallel branches .

Original: Note that this K can be different at each stage of the network and should therefore be denoted as K ( i ) .
Revised: Note that this K can be different at each stage of the network and should therefore be denoted as K ( i ) .

Original: However, to simplify notation, we drop this index below.
Revised: However, to simplify notation, we drop this index below.

Original: Given this deﬁnition, we write the output of each computational unit as
Revised: Given this deﬁnition, we write the output of each computational unit as

Original: [Equation]
Revised: [Equation]

Original:  where ˆΣ( · ) is an aggregation operator that could be deﬁned in many ways.
Revised: (2) where ˆΣ( · ) is an aggregation operator that could be deﬁned in many ways.

Original: It could be a simple summation that gives all outputs equal importance, or, at the opposite end of the spectrum, a multiplexer that selects a single branch and ignores the rest.
Revised: It could be a simple summation that gives all outputs equal importance, or, at the opposite end of the spectrum, a multiplexer that selects a single branch and ignores the rest.

Original: To cover the range between these two alternatives, we introduce learnable gates that enable the network to determine what relative importance the different branches should be given.
Revised: To cover the range between these two alternatives, we introduce learnable gates that enable the network to determine what relative importance the different branches should be given.

Original: Our gates perform a weighted combination of the branch outputs.
Revised: Our gates perform a weighted combination of the branch outputs.

Original: Each gate is controlled by a set of K activation weights { φ ( i ) j } 1 ≤ j ≤ K , and a unit returns
Revised: Each gate is controlled by a set of K activation weights { φ ( i ) j } 1 ≤ j ≤ K , and a unit returns

Original: [Equation]
Revised: [Equation]

Original: If ∀ j, φ ( i ) j = 1 , the gate performs a simple summation.
Revised: If ∀ j, φ ( i ) j = 1 , the gate performs a simple summation.

Original: If φ ( i ) j
Revised: If φ ( i ) j

Original: = 1 for a single j and 0 for the others, it behaves as a multiplexer.
Revised: = 1 for a single j and 0 for the others, it behaves as a multiplexer.

Original: The activation weights φ ( i ) j enable us to modulate the computational graph of network block f ( i ) .
Revised: The activation weights φ ( i ) j enable us to modulate the computational graph of network block f ( i ) .

Original: To bound them and encourage the network to either select or discard each branch in a computational unit, we write them in terms of sigmoid functions with adaptive steepness.
Revised: To bound them and encourage the network to either select or discard each branch in a computational unit, we write them in terms of sigmoid functions with adaptive steepness.

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: where the g ( i ) j s are learnable unbounded model parameters, and π ( i ) controls the plasticity of the activation—the rate at which φ ( i ) j varies between the extreme values 0 and 1 for block i .
Revised: where the g ( i ) j s are learnable unbounded model parameters, and π ( i ) controls the plasticity of the activation—the rate at which φ ( i ) j varies between the extreme values 0 and 1 for block i .

Original: During training, we initially set π ( i ) to a small value, which enables the network to explore different gate conﬁgurations.
Revised: During training, we initially set π ( i ) to a small value, which enables the network to explore different gate conﬁgurations.

Original: We then apply a cooling schedule on our activations, by progressively increasing π ( i ) over time, so as to encourage the gates to reach a ﬁrm decision.
Revised: We then apply a cooling schedule on our activations, by progressively increasing π ( i ) over time, so as to encourage the gates to reach a ﬁrm decision.

Original: Note that our formulation does not require (cid:80) Kj =1 φ ( i ) j = 1 , that is, we do not require the aggregated output x ( i ) to be a convex combination of the branch outputs f ( i ) j
Revised: Note that our formulation does not require (cid:80) Kj =1 φ ( i ) j = 1 , that is, we do not require the aggregated output x ( i ) to be a convex combination of the branch outputs f ( i ) j

Original: ( x ( i − 1) ) .
Revised: ( x ( i − 1) ) .

Original: This is deliberate because allowing the activation weights to be independent from one another provides additional ﬂexibility for the network to learn general additive relationships.
Revised: This is deliberate because allowing the activation weights to be independent from one another provides additional ﬂexibility for the network to learn general additive relationships.

Original: Finally, a Multibranch Network is the concatenation of multiple computational units, as shown in
Revised: Finally, a Multibranch Network is the concatenation of multiple computational units, as shown in

Original: Fig.
Revised: Fig.

Original: For the aggregation within each unit f ( i ) to be possible, the f ( i ) j s’ outputs must be of matching shapes.
Revised: For the aggregation within each unit f ( i ) to be possible, the f ( i ) j s’ outputs must be of matching shapes.

Original: Furthermore, as in standard networks, two computational units can be attached only if the output shape of the ﬁrst one matches the input shape of the second.
Revised: Furthermore, as in standard networks, two computational units can be attached only if the output shape of the ﬁrst one matches the input shape of the second.

Original: Although it would be possible to deﬁne computational units at any point in the network architecture, in practice, we usually take them to correspond to groups of layers that are semantically related.
Revised: Although it would be possible to deﬁne computational units at any point in the network architecture, in practice, we usually take them to correspond to groups of layers that are semantically related.

Original: For example, one would group a succession of convolutions, pooling and non-linear operations into the same computational unit.
Revised: For example, one would group a succession of convolutions, pooling and non-linear operations into the same computational unit.

Original: 3. D OMAIN A DAPTIVE M ULTIBRANCH N ETWORKS 3.2. T WO D OMAINS
Revised: 3. D OMAIN A DAPTIVE M ULTIBRANCH N ETWORKS 3.2. T WO D OMAINS

Original: S ou r ce do m a i n

Original: T a r g e t do m a i n f ( 2 )f ( 2 )f ( 2 ) 2 f ( 1 )f ( 1 )f ( 1 )f ( 1 )f ( 1 )f ( 1 )f ( 2 )f ( 2 )f ( 2 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 ) 4 f ( 1 )f ( 1 )

Original: While both domains share the same computational units, their outputs are obtained by different aggregations of their inner operations, e.g., in the ﬁrst unit, the source domain does not use the middle two operations, whereas the target domain does; by contrast, both exploit the fourth operation.

Original: In essence, this scheme adapts the amount of computation that each domain is subjected to.

Original: Our goal is to perform domain adaptation, that is, leverage a large amount of labeled images, X s = { x s 1 , . . .
Revised: Our goal is to perform domain adaptation, that is, leverage a large amount of labeled images, X s = { x s 1 , . . .

Original: , x s N } with corresponding annotations Y s = { y s 1 , . . .
Revised: , x sN } with corresponding annotations Y s = { y s 1 , . . .

Original: , y sN } , drawn from a source domain, to train a model for a target domain, whose data distribution is different and for which we only have access to unlabeled images X t
Revised: , y sN } , drawn from a source domain, to train a model for a target domain, whose data distribution is different and for which we only have access to unlabeled images X t

Original: = { x t 1 , . . .
Revised: = { x t 1 , . . .

Original: , x tM } .
Revised: , x tM } .

Original: To this end, we extend the gated networks of Section 3.1 by deﬁning two sets of gates, one for the source domain and one for the target one.
Revised: To this end, we extend the gated networks of Section 3.1 by deﬁning two sets of gates, one for the source domain and one for the target one.

Original: Let { ( φ s ) ( i ) j } Kj =1 and { ( φ t ) ( i ) j } Kj =1 be the corresponding source and target activation weights for computational unit f
Revised: Let { ( φ s ) ( i ) j } Kj =1 and { ( φ t ) ( i ) j } Kj =1 be the corresponding source and target activation weights for computational unit f

Original: ( i ) , respectively.
Revised: ( i ) , respectively.

Original: Given a sample x d coming from a domain d ∈ { s, t } , we take the corresponding output of the i -th computational unit to be
Revised: Given a sample x d coming from a domain d ∈ { s, t } , we take the corresponding output of the i -th computational unit to be

Original: [Equation]
Revised: [Equation]

Original: Note that under this formulation, the domain identity d of the sample is required in order to select the appropriate ( φ d ) ( i ) .
Revised: Note that under this formulation, the domain identity d of the sample is required in order to select the appropriate ( φ d ) ( i ) .

Original: The concatenated computational units forming the DAMnet encode sample x from domain d into a feature vector z = f ( x , d ) .
Revised: The concatenated computational units forming the DAMNet encode sample x from domain d into a feature vector z = f ( x , d ) .

Original: Since the gates for different domains are set independently from one another, the outputs of the branches for each computational unit are combined in a domainspeciﬁc manner, dictated by the activation weights ( φ d ) ( i ) j .
Revised: Since the gates for different domains are set independently from one another, the outputs of the branches for each computational unit are combined in a domainspeciﬁc manner, dictated by the activation weights ( φ d ) ( i ) j .

Original: Therefore, the samples are encoded to a common space, but arrive to it through potentially different computations.
Revised: Therefore, the samples are encoded to a common space, but arrive to it through potentially different computations.

Original: Fig.
Revised: Fig.

Original: 3 depicts this process.
Revised: 3 depicts this process.

Original: Ultimately, the network can learn to share weights for computational unit f
Revised: Ultimately, the network can learn to share weights for computational unit f

Original: ( i ) by setting ( φ s ) ( i )
Revised: ( i ) by setting ( φ s ) ( i )

Original: j
Revised: j

Original: = ( φ t ) ( i ) j , ∀ j .
Revised: = ( φ t ) ( i ) j , ∀ j .

Original: It can also learn to fully untie the weights by having A Si ∩ A Ti = ∅ , where
Revised: It can also learn to fully untie the weights by having A Si ∩ A Ti = ∅ , where

Original: A Si and A Ti denote the set of non-zero activations in the two domains.
Revised: A Si and A Ti denote the set of non-zero activations in the two domains.

Original: Finally, in contrast to Tzeng et al. (2017); Berm´udez-Chac´on et al.
Revised: Finally, in contrast to Tzeng et al. (2017); Berm´udez-Chac´on et al.

Original: ; Rozantsev et al.
Revised: ; Rozantsev et al.

Original: (2018; 2019), it can learn to use more computation for one domain than for the other by setting ( φ s ) ( i ) j > 0 for two different branches f
Revised: (2018; 2019), it can learn to use more computation for one domain than for the other by setting ( φ s ) ( i ) j > 0 for two different branches f

Original: ( i ) j while having only a single non-zero ( φ t ) ( i ) j , for a particular computational unit f
Revised: ( i ) j while having only a single non-zero ( φ t ) ( i ) j , for a particular computational unit f

Original: ( i ) .
Revised: ( i ) .

Revised: S ou r ce do m a i n

Revised: T a r g e t do m a i n f ( 2 )f ( 2 )f ( 2 ) 2 f ( 1 )f ( 1 )f ( 1 )f ( 1 )f ( 1 )f ( 1 )f ( 2 )f ( 2 )f ( 2 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 )f ( 3 ) 4 f ( 1 )f ( 1 )

Original: The above formulation treats all branches for each computational unit as potentially sharable between domains.
Revised: While both domains share the same computational units, their outputs are obtained by different aggregations of their inner operations, e.g., in the ﬁrst unit, the source domain does not use the middle two operations, whereas the target domain does; by contrast, [Table] [Table] essence, this scheme

Revised: [Table]

Revised: amount

Revised: [Table]

Revised: is subjected to.

Revised: The above formulation treats all branches for each computational unit as potentially sharable between domains.

Original: However, it is sometimes desirable not to share at all .
Revised: However, it is sometimes desirable not to share at all .

Original: For example, batchnormalization layers that accumulate and update statistics of the data over time, even during the forward pass, are best exposed to a single domain to learn domain-speciﬁc statistics.
Revised: For example, batchnormalization layers that accumulate and update statistics of the data over time, even during the forward pass, are best exposed to a single domain to learn domain-speciﬁc statistics.

Original: We allow for this by introducing computational units whose gates are ﬁxed, yet domain speciﬁc, and that therefore act as multiplexers.
Revised: We allow for this by introducing computational units whose gates are ﬁxed, yet domain speciﬁc, and that therefore act as multiplexers.

Original: After the last computational unit, a small network p y operates directly on the encodings and returns the class assignment ˆ y = p
Revised: After the last computational unit, a small network p y operates directly on the encodings and returns the class assignment ˆ y = p

Original: y ( z ) , thus subjecting the encodings for all samples to the same set of operations.
Revised: y ( z ) , thus subjecting the encodings for all samples to the same set of operations.

Original: 3.2. M ULTIPLE D OMAINS
Revised: 3.2. M ULTIPLE D OMAINS

Original: The formulation outlined above extends naturally to more than two domains, by assigning one set of gates per domain.
Revised: The formulation outlined above extends naturally to more than two domains, by assigning one set of gates per domain.

Original: This enables us to exploit annotated data from different source domains, and even to potentially handle multiple target domains simultaneously.
Revised: This enables us to exploit annotated data from different source domains, and even to potentially handle multiple target domains simultaneously.

Original: In this generalized case, we introduce governing sets of gates with activations φ d 1 , . .
Revised: In this generalized case, we introduce governing sets of gates with activations φ d 1 , . .

Original: . , φ d D for D different domains.
Revised: . , φ d D for D different domains.

Original: They act in the same way as in the two-domain case and the overall architecture remains similar.
Revised: They act in the same way as in the two-domain case and the overall architecture remains similar.

Original: 3.2. T RAINING
Revised: 3.2. T RAINING

Original: When training our models, we jointly optimize the gate parameters ( g d ) ( i ) j , from Eq. 4, along with the other network parameters using standard back-propagation.
Revised: When training our models, we jointly optimize the gate parameters ( g d ) ( i ) j , from Eq. 4, along with the other network parameters using standard back-propagation.

Original: To this end, we make use of a composite loss function, designed to encourage correct classiﬁcation for labeled samples from the source domain(s) and align the distributions of all domains, using labeled and unlabeled samples.
Revised: To this end, we make use of a composite loss function, designed to encourage correct classiﬁcation for labeled samples from the source domain(s) and align the distributions of all domains, using labeled and unlabeled samples.

Original: This loss can be expressed as
Revised: This loss can be expressed as

Original: [Equation]
Revised: [Equation]

Original:  where (cid:96) and u are the sets of labeled and unlabeled samples, respectively, and where we assumed, without loss of generality, that the samples are ordered.
Revised: [Equation] ) , (6) where (cid:96) and u are the sets of labeled and unlabeled samples, respectively, and where we assumed, without loss of generality, that the samples are ordered.

Original: The ﬁrst term in this loss, L y ( y , ˆ y ) , is the standard cross-entropy, which compares the groundtruth class probabilities y with the predicted ones
Revised: The ﬁrst term in this loss, L y ( y , ˆ y ) , is the standard cross-entropy, which compares the groundtruth class probabilities y with the predicted ones

Original: ˆ y = p
Revised: ˆ y = p

Original: y ( z ) , where, as discussed in Section 3.2.1, z
Revised: y ( z ) , where, as discussed in Section 3.2.1, z

Original: = f ( x , d ) is the feature encoding of sample x from domain d .
Revised: = f ( x , d ) is the feature encoding of sample x from domain d .

Original: For the second term, which encodes distribution alignment, we rely on the domain confusion strategy of Ganin & Lempitsky (2015), which is commonly used in existing frameworks.
Revised: For the second term, which encodes distribution alignment, we rely on the domain confusion strategy of Ganin & Lempitsky (2015), which is commonly used in existing frameworks.

Original: Speciﬁcally, for D domains, we make use of an auxiliary domain classiﬁer network p d that predicts a D -dimensional vector of domain probabilities ˆ d given the feature vector z .
Revised: Speciﬁcally, for D domains, we make use of an auxiliary domain classiﬁer network p d that predicts a D -dimensional vector of domain probabilities ˆ d given the feature vector z .

Original: Following the gradient reversal technique of Ganin & Lempitsky (2015), we express the second term in our loss as L d ( d , ˆ d ) = − (cid:80) Di
Revised: Following the gradient reversal technique of Ganin & Lempitsky (2015), we express the second term in our loss as L d ( d , ˆ d ) = − (cid:80) Di

Original: =1 d i log(ˆ d i ) , where d is the
Revised: =1 d i log(ˆ d i ) , where d is the

Original: D -dimensional binary vector encoding the ground-truth domain, d i indicates the i -th element of d , and ˆ d = p d ( R ( z )) , with R the gradient reversal pseudofunction of Ganin & Lempitsky (2015) that enables to incorporate adversarial training directly into back-propagation.
Revised: D -dimensional binary vector encoding the ground-truth domain, d i indicates the i -th element of d , and ˆ d = p d ( R ( z )) , with R the gradient reversal pseudofunction of Ganin & Lempitsky (2015) that enables to incorporate adversarial training directly into back-propagation.

Original: That is, with this loss, standard back-propagation trains jointly the domain classiﬁer to discriminate the domains and the feature extractor f ( · ) to produce features that fool this classiﬁer.
Revised: That is, with this loss, standard back-propagation trains jointly the domain classiﬁer to discriminate the domains and the feature extractor f ( · ) to produce features that fool this classiﬁer.

Original: When training is complete and the gates have reached a stable state, the branches whose activations are close to zero are deactivated.
Revised: When training is complete and the gates have reached a stable state, the branches whose activations are close to zero are deactivated.

Original: This prevents the network from performing computations that are irrelevant and allows us to obtain a more compact network to process the target data.
Revised: This prevents the network from performing computations that are irrelevant and allows us to obtain a more compact network to process the target data.

Original:  E VALUATION 4. B ASELINES
Revised:  E VALUATION 4. B ASELINES

Original: Since we rely on the domain confusion loss to train our model, we treat the Domain-Adversarial
Revised: Since we rely on the domain confusion loss to train our model, we treat the Domain-Adversarial

Original: Neural Network (DANN) method of Ganin & Lempitsky (2015), as our ﬁrst baseline.
Revised: Neural Network (DANN) method of Ganin & Lempitsky (2015), as our ﬁrst baseline.

Original: To demonstrate the beneﬁts of our approach over simply untying the source and target stream parameters, we compare our approach against the Residual Parameter Transfer (RPT) method of Rozantsev et al.
Revised: To demonstrate the beneﬁts of our approach over simply untying the source and target stream parameters, we compare our approach against the Residual Parameter Transfer (RPT) method of Rozantsev et al.

Original: (2019), which constitutes the state of the art in doing so.
Revised: (2019), which constitutes the state of the art in doing so.

Original: Note that RPT also relies on the domain confusion loss, which makes our comparison fair.
Revised: Note that RPT also relies on the domain confusion loss, which makes our comparison fair.

Original: In addition, we report the results of directly applying a model trained on the source domain to the target, without any domain adaptation, which we refer to as “No DA”.
Revised: In addition, we report the results of directly applying a model trained on the source domain to the target, without any domain adaptation, which we refer to as “No DA”.

Original: We also provide the oracle accuracy of a model trained on the fully-labeled target domain, referred to as “ On TD ”.
Revised: We also provide the oracle accuracy of a model trained on the fully-labeled target domain, referred to as “ On TD ”.

Original: 4. I MPLEMENTATION D ETAILS
Revised: 4. I MPLEMENTATION D ETAILS

Original: We adapt different network architectures to the multibranch paradigm for different adaptation problems.
Revised: We adapt different network architectures to the multibranch paradigm for different adaptation problems.

Original: For all cases, we initialize our networks’ parameters by training the original versions of those architectures on the source domains, either from scratch, for simple architectures, or by ﬁne-tuning weights learned on ImageNet, for very deep ones.
Revised: For all cases, we initialize our networks’ parameters by training the original versions of those architectures on the source domains, either from scratch, for simple architectures, or by ﬁne-tuning weights learned on ImageNet, for very deep ones.

Original: We then set the parameters of all branches to the values from the corresponding layers.
Revised: We then set the parameters of all branches to the values from the corresponding layers.

Original: We perform this training on the predeﬁned training splits, when available, or on 75% of the images, otherwise.
Revised: We perform this training on the predeﬁned training splits, when available, or on 75% of the images, otherwise.

Original: The initial values of the gate parameters are deﬁned so as to set the activations to 1 K , for each of the K branches.
Revised: The initial values of the gate parameters are deﬁned so as to set the activations to 1 K , for each of the K branches.

Original: This prevents our networks from  favoring a particular branch for any domain.
Revised: This prevents our networks from initially favoring a particular branch for any domain.

Original: To train our networks, we use Stochastic Gradient Descent with a momentum of 0.9 and a variable learning rate deﬁned by the annealing schedule of Ganin & Lempitsky (2015) as µ p = µ 0 (1+ α · p ) β , where p is the training progress, relative to the total number of training epochs, µ 0 is the initial learning rate, which we take to be 10 − 2 , and α = 10 and β = 0 .
Revised: To train our networks, we use Stochastic Gradient Descent with a momentum of 0.9 and a variable learning rate deﬁned by the annealing schedule of Ganin & Lempitsky (2015) as µ p = µ 0 (1+ α · p ) β , where p is the training progress, relative to the total number of training epochs, µ 0 is the initial learning rate, which we take to be 10 − 2 , and α = 10 and β = 0 .

Original: 75 as in Ganin & Lempitsky (2015).
Revised: 75 as in Ganin & Lempitsky (2015).

Original: We eliminate exploding gradients by (cid:96) 2 -norm clipping.
Revised: We eliminate exploding gradients by (cid:96) 2 -norm clipping.

Original: Furthermore, we modulate the plasticity of the activations at every gate as π ( i ) =
Revised: Furthermore, we modulate the plasticity of the activations at every gate as π ( i ) =

Original: 1 − p , that is, we make π ( i ) decay linearly as training progresses.
Revised: 1 − p , that is, we make π ( i ) decay linearly as training progresses.

Original: As data preprocessing, we apply mean subtraction, as in Ganin & Lempitsky (2015).
Revised: As data preprocessing, we apply mean subtraction, as in Ganin & Lempitsky (2015).

Original: We train for 200 epochs, during which the network is exposed to all the image data from the source and target domains, but only to the annotations from the source domain(s).
Revised: We train for 200 epochs, during which the network is exposed to all the image data from the source and target domains, but only to the annotations from the source domain(s).

Original: Our “ On TD ” oracle is trained on either the preset training splits, when available, or our deﬁned training data, and evaluated on the corresponding test data.
Revised: Our “ On TD ” oracle is trained on either the preset training splits, when available, or our deﬁned training data, and evaluated on the corresponding test data.

Original: For the comparison to this oracle to be meaningful, we follow the same strategy for our DAMnets.
Revised: For the comparison to this oracle to be meaningful, we follow the same strategy for our DAMNets.

Original: That is, we use the unlabeled target data from the training splits only and report results on the testing splits.
Revised: That is, we use the unlabeled target data from the training splits only and report results on the testing splits.

Original: This protocol differs from that of Rozantsev et al.
Revised: This protocol differs from that of Rozantsev et al.

Original: (2019), which relied on a transductive evaluation, where all the target images, training and test ones, were seen by the networks during training.
Revised: (2019), which relied on a transductive evaluation, where all the target images, training and test ones, were seen by the networks during training.

Original: 4. I MAGE R ECOGNITION
Revised: 4. I MAGE R ECOGNITION

Original: We evaluate our method in the task of image recognition for which we use several domain adaptation benchmark problems: Digits , which comprises three domains: MNIST (LeCun et al., 1998),
Revised: We evaluate our method in the task of image recognition for which we use several domain adaptation benchmark problems: Digits , which comprises three domains: MNIST (LeCun et al., 1998),

Revised: MNIST-M (Ganin & Lempitsky, 2015), and SVHN (Netzer et al., 2011); Ofﬁce (Saenko et al.,

Revised: [Table]

Original: MNIST-M (Ganin & Lempitsky, 2015), and SVHN (Netzer et al., 2011); Ofﬁce (Saenko et al., 2010), which contains three domains: Amazon, DSLR, and Webcam; Ofﬁce-Home (Venkateswara et al., 2017), with domains Art, Clipart, Product, and Real; and VisDA17{+VisDA
+} (Peng et al., 2018), with
Revised: 2010), which contains three domains: Amazon, DSLR [Table] and Webcam; Ofﬁce-Home (Venkateswara et al., 2017), with domains Art, Clipart, Product, and Real; and {+VisDA
+} (Peng et al., 2018), with

Original: Synthetic and Real images.
Revised: Synthetic and Real images.

Original: As all these are well studied benchmark datasets, we provide full descriptions and image examples evidencing the different degrees of domain shift in Appendix B.[Table]
Revised: As all these are well studied benchmark datasets, we provide full descriptions and image examples evidencing the different degrees of domain shift in Appendix B.

Original: Setup.
Revised: Setup.

Original: As discussed in Section 3, our method is general and can work with any  network architecture.
Revised: As discussed in Section 3, our method is general and can work with any feed-forward network architecture.

Original: To showcase this, for the digit recognition datasets, we apply it to the LeNet and SVHNet architectures (Ganin & Lempitsky, 2015), which are very simple convolutional networks, well suited for small images.
Revised: To showcase this, for the digit recognition datasets, we apply it to the LeNet and SVHNet architectures (Ganin & Lempitsky, 2015), which are very simple convolutional networks, well suited for small images.

Original: Following Ganin & Lempitsky (2015), we employ LeNet when using the synthetic datasets MNIST and MNIST-M as source domains, and SVHNet when SVHN acts as source domain.
Revised: Following Ganin & Lempitsky (2015), we employ LeNet when using the synthetic datasets MNIST and MNIST-M as source domains, and SVHNet when SVHN acts as source domain.

Original: We extend these architectures to multibranch ones by deﬁning the computational units as the groups of consecutive convolution, pooling and non-linear operations deﬁned in the original model.
Revised: We extend these architectures to multibranch ones by deﬁning the computational units as the groups of consecutive convolution, pooling and non-linear operations deﬁned in the original model.

Original: For simplicity, we use as many branches within each computational unit as we have domains, and all branches from a computational unit follow the same architecture, which we provide in Appendix A, Figures 1 and 2.
Revised: For simplicity, we use as many branches within each computational unit as we have domains, and all branches from a computational unit follow the same architecture, which we provide in Appendix A, Figures 1 and 2.

Original: As backbone network to process all the rest of the datasets, we use a ResNet-50 (He et al., 2016), with the bottleneck layer modiﬁcation of Rozantsev et al.
Revised: As backbone network to process all the rest of the datasets, we use a ResNet-50 (He et al., 2016), with the bottleneck layer modiﬁcation of Rozantsev et al.

Original: While many multibranch conﬁgurations can be designed for such a deep network, we choose to make our gated computational units coincide with the layer groupings deﬁned in He et al.
Revised: While many multibranch conﬁgurations can be designed for such a deep network, we choose to make our gated computational units coincide with the layer groupings deﬁned in He et al.

Original: (2016), namely conv1, conv2 x, conv3 x, conv4 x, and conv5 x.
Revised: (2016), namely conv1, conv2 x, conv3 x, conv4 x, and conv5 x.

Original: The resulting multibranch network is depicted in Appendix A, Figure 4.
Revised: The resulting multibranch network is depicted in Appendix A, Figure 4.

Original: We feed our DAMnets images resized to 224 × 224 pixels, as expected by ResNet-50.
Revised: We feed our DAMNets images resized to 224 × 224 pixels, as expected by ResNet-50.

Original: Results.
Revised: Results.

Original: The results for the digit recognition and Ofﬁce-Home datasets are provided in Table 1.
Revised: The results for the digit recognition and Ofﬁce-Home datasets are provided in Table 1.

Original: Results for Ofﬁce and VisDA17 datasets are presented in Appendix D. Our approach outperforms the baselines in all cases.
Revised: Results for Ofﬁce and VisDA17 datasets are presented in Appendix D. Our approach outperforms the baselines in all cases.

Original: For the Digits datasets, in addition to the traditional two-domain setup, we also report results when using two source domains simultaneously.
Revised: For the Digits datasets, in addition to the traditional two-domain setup, we also report results when using two source domains simultaneously.

Original: Note that the reference method RPT (Rozantsev et al., 2019) does not apply to this setting, since it was designed to transform a single set of source parameters to the target ones.
Revised: Note that the reference method RPT (Rozantsev et al., 2019) does not apply to this setting, since it was designed to transform a single set of source parameters to the target ones.

Original: Altogether, our method consistently outperforms the others.
Revised: Altogether, our method consistently outperforms the others.

Original: Note that the ﬁrst two columns correspond to the combinations reported in the literature.
Revised: Note that the ﬁrst two columns correspond to the combinations reported in the literature.

Original: We believe, however, that the SVHN (cid:46) MNIST one is quite artiﬁcial, since, in practice, one would typically annotate simpler, synthetic images and aim to use real ones at test time.
Revised: We believe, however, that the SVHN (cid:46) MNIST one is quite artiﬁcial, since, in practice, one would typically annotate simpler, synthetic images and aim to use real ones at test time.

Original: We therefore also report synthetic (cid:46) SVHN cases, which are much more challenging.
Revised: We therefore also report synthetic (cid:46) SVHN cases, which are much more challenging.

Original: The multi-source version of our method achieves a significant boost over the baselines in this scenario.
Revised: The multi-source version of our method achieves a significant boost over the baselines in this scenario.

Original: To further demonstrate the potential of our approach in this setting, we replaced its backbone with the much deeper ResNet-50 network and applied it on upscaled versions of the images.
Revised: To further demonstrate the potential of our approach in this setting, we replaced its backbone with the much deeper ResNet-50 network and applied it on upscaled versions of the images.

Original: As shown in the column indicated by a (cid:63) , this allowed us to achieve an accuracy close to 80%, which is remarkable for such a difﬁcult adaptation task.
Revised: As shown in the column indicated by a (cid:63) , this allowed us to achieve an accuracy close to 80%, which is remarkable for such a difﬁcult adaptation task.

Original: On Ofﬁce-Home , the gap between DAMnet and the baselines is again consistent across the different domain pairs.
Revised: On Ofﬁce-Home , the gap between DAMNet and the baselines is again consistent across the different domain pairs.

Original: Note that, here, because of the relatively large number of classes, the overall performance is low for all methods.
Revised: Note that, here, because of the relatively large number of classes, the overall performance is low for all methods.

Original: Importantly, our results show that we gain performance by training on more than one source domain, and by leveraging all synthetic domains to transfer to the real one, our approach reaches an accuracy virtually equal to that of using full supervision on the target domain.
Revised: Importantly, our results show that we gain performance by training on more than one source domain, and by leveraging all synthetic domains to transfer to the real one, our approach reaches an accuracy virtually equal to that of using full supervision on the target domain.

Original: Despite our best efforts, we were unable to obtain convincing results for RPT using the authors’ publicly available code, as results for this dataset were not originally reported for RPT.
Revised: Despite our best efforts, we were unable to obtain convincing results for RPT using the authors’ publicly available code, as results for this dataset were not originally reported for RPT.

Original: Gate dynamics.
Revised: Gate dynamics.

Original: To understand the way our networks learn the domain-speciﬁc branch assignments, we track the state of the gates for all computational units over all training epochs.
Revised: To understand the way our networks learn the domain-speciﬁc branch assignments, we track the state of the gates for all computational units over all training epochs.

Original: In Figure 4, we plot the corresponding evolution of the gate activations for the DSLR+Webcam (cid:46)
Revised: In Figure 4, we plot the corresponding evolution of the gate activations for the DSLR+Webcam (cid:46)

Original: Amazon task on Ofﬁce.
Revised: Amazon task on Ofﬁce.

Original: Note that our DAMnet leverages different branches over time for each domain before reaching a ﬁrm decision.
Revised: Note that our DAMNet leverages different branches over time for each domain before reaching a ﬁrm decision.

Original: Interestingly, we can see that, with the exception of the ﬁrst unit, which performs low-level computations, DSLR and Webcam share all branches.
Revised: Interestingly, we can see that, with the exception of the ﬁrst unit, which performs low-level computations, DSLR and Webcam share all branches.

Original: By contrast, Amazon, which has a signiﬁcantly different appearance, mostly uses its own branches, except in two compu- tational units.
Revised: By contrast, Amazon, which has a signiﬁcantly different appearance, mostly uses its own branches, except in two computational units.

Original: This evidences that our network successfully understands when domains are similar and can thus use similar computations.
Revised: This evidences that our network successfully understands when domains are similar and can thus use similar computations.

Original: 4. O BJECT DETECTION
Revised: 4. O BJECT DETECTION

Original: [Table]
Revised: [Table]

Original: We evaluate our method for the detection of drones from video frames, on the UAV-200 dataset (Rozantsev et al., 2018), which contains examples of drones both generated artiﬁcially and captured from real video footage.
Revised: We evaluate our method for the detection of drones from video frames, on the UAV-200 dataset (Rozantsev et al., 2018), which contains examples of drones both generated artiﬁcially and captured from real video footage.

Original: Full details and example images are provided in Appendix B.
Revised: Full details and example images are provided in Appendix B.

Original: Setup.
Revised: Setup.

Original: Our domain adaptation leverages both the synthetic examples of drones, as source domain, and the limited amount of annotated real drones, as target domain, as well as the background negative examples, to predict the class of patches from the validation set of real images.
Revised: Our domain adaptation leverages both the synthetic examples of drones, as source domain, and the limited amount of annotated real drones, as target domain, as well as the background negative examples, to predict the class of patches from the validation set of real images.

Original: We follow closely the supervised setup and network architecture of Rozantsev et al.
Revised: We follow closely the supervised setup and network architecture of Rozantsev et al.

Original: (2019), including the use of AdaDelta as optimizer, cross-entropy as loss function, and average precision as evaluation metric.
Revised: (2019), including the use of AdaDelta as optimizer, cross-entropy as loss function, and average precision as evaluation metric.

Original: Our multibranch computational units are deﬁned as groupings of successive convolutions, nonlinearities, and pooling operations.
Revised: Our multibranch computational units are deﬁned as groupings of successive convolutions, nonlinearities, and pooling operations.

Original: The details of the architecture are provided in Appendix A, Figure 3.
Revised: The details of the architecture are provided in Appendix A, Figure 3.

Original: Results.
Revised: Results.

Original: Our method considerably surpasses all the others in terms of average precision, as shown in Table 2, thus validating DAMnets as effective models for leveraging synthetic data for domain adaptation in real-world problems
Revised: Our method considerably surpasses all the others in terms of average precision, as shown in Table 2, thus validating DAMNets as effective models for leveraging synthetic data for domain adaptation in real-world problems.

Original: . 4. DAM NET AS A GENERAL FEATURE EXTRACTOR
Revised: 4. DAMN ET AS A GENERAL FEATURE EXTRACTOR

Original: [Table]
Revised: [Table]

Original: We validate the effectiveness of our method as a feature extractor, by combining it with the Maximum Classiﬁer Discrepancy (MCD) method of Saito et al.
Revised: We validate the effectiveness of our method as a feature extractor, by combining it with the Maximum Classiﬁer Discrepancy (MCD) method of Saito et al.

Original: As MCD operates on the extracted encodings, we replace the encoding strategy that MCD uses, which is the same as DANN, with our
Revised: As MCD operates on the extracted encodings, we replace the encoding strategy that MCD uses, which is the same as DANN, with our

Original: DAMnet.
Revised: DAMNet.

Original: Or, in other words, we replace the domain classiﬁer in our approach with the corresponding MCD term.
Revised: Or, in other words, we replace the domain classiﬁer in our approach with the corresponding MCD term.

Original: Speciﬁcally, we use a single computational unit with two branches, each of which replicates the architectures proposed in Saito et al.
Revised: Speciﬁcally, we use a single computational unit with two branches, each of which replicates the architectures proposed in Saito et al.

Original: We present the results of combining MCD with DAMnet in Table 3.
Revised: We present the results of combining MCD with DAMNet in Table 3.

Original: In all tested scenarios, we obtain improvements over using MCD as originally proposed.
Revised: In all tested scenarios, we obtain improvements over using MCD as originally proposed.

Original: 4. B RANCH ARCHITECTURES
Revised: 4. B RANCH ARCHITECTURES

Original: To obtain more insights about speciﬁc branch decisions, we evaluate the effects of adding extra branches to the network, as well as using branches with different capacities.
Revised: To obtain more insights about speciﬁc branch decisions, we evaluate the effects of adding extra branches to the network, as well as using branches with different capacities.

Original: 4.6. B RANCHES WITH DIFFERENT CAPACITIES
Revised: 4.6. B RANCHES WITH DIFFERENT CAPACITIES

Original: When computational units are composed of branches of different capacities, DAMnets often assign branches with more capacity to more complex domains.
Revised: When computational units are composed of branches of different capacities, DAMNets often assign branches with more capacity to more complex domains.

Original: To exemplify this, we trained a modiﬁed multibranch SVHNet for adaptation between MNIST and SVHN.
Revised: To exemplify this, we trained a modiﬁed multibranch SVHNet for adaptation between MNIST and SVHN.

Original: Instead of the identical branches originally used, we replace the second branch in each computational unit with a similar branch where the convolution operation is performed by 1x1 rather than 5x5 kernels.
Revised: Instead of the identical branches originally used, we replace the second branch in each computational unit with a similar branch where the convolution operation is performed by 1x1 rather than 5x5 kernels.

Original: These second branches, with  25 times fewer parameters each, are mostly used by the simpler domain—MNIST in this case.
Revised: These second branches, with [Figure] 25 times fewer parameters each, are mostly used by the simpler domain—MNIST in this case.

Original: We provide the gate evolution that reﬂects this in Appendix C, Figures 5 and 6.[Figure]
Revised: We provide the gate evolution that reﬂects this in Appendix C, Figures 5 and 6.

Original: 4.6. A DDITIONAL BRANCHES
Revised: 4.6. A DDITIONAL BRANCHES

Original: We explore the effects of using more branches than domains, so as to provide the networks with alternative branches from where to choose.
Revised: We explore the effects of using more branches than domains, so as to provide the networks with alternative branches from where to choose.

Original: In particular, we explore the case where K = D + 1 .
Revised: In particular, we explore the case where K = D + 1 .

Original: We evaluate multibranch LeNet and ResNet architectures under this setting.
Revised: We evaluate multibranch LeNet and ResNet architectures under this setting.

Original: We show the gate activation evolution in Appendix C, Figures 7 and 8.
Revised: We show the gate activation evolution in Appendix C, Figures 7 and 8.

Original: During the training process, we have observed that the networks quickly choose to ignore extra ﬂows when K > D .
Revised: During the training process, we have observed that the networks quickly choose to ignore extra branches when K > D .

Original: This suggests that they did not contribute to the learning of our feature extraction.
Revised: This suggests that they did not contribute to the learning of our feature extraction.

Original: We did not ﬁnd experimental evidence to support that K > D is beneﬁcial.
Revised: We did not ﬁnd experimental evidence to support that K > D is beneﬁcial.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have introduced a domain adaptation approach that allows for adaptive, separate computations for different domains.
Revised: We have introduced a domain adaptation approach that allows for adaptive, separate computations for different domains.

Original: Our framework relies on computational units that aggregate the outputs of multiple parallel operations, and on a set of trainable domain-speciﬁc gates that adapt the aggregation process to each domain.
Revised: Our framework relies on computational units that aggregate the outputs of multiple parallel operations, and on a set of trainable domain-speciﬁc gates that adapt the aggregation process to each domain.

Original: Our experiments have demonstrated the beneﬁts of this approach over the state-of-the-art weight untying strategy; the greater ﬂexibility of our method translates into a consistently better accuracy.
Revised: Our experiments have demonstrated the beneﬁts of this approach over the state-of-the-art weight untying strategy; the greater ﬂexibility of our method translates into a consistently better accuracy.

Original: Although we only experimented with using the same branch architectures within a computational unit, our framework generalizes to arbitrary branch architectures, the only constraint being that their outputs are of commensurate shapes.
Revised: Although we only experimented with using the same branch architectures within a computational unit, our framework generalizes to arbitrary branch architectures, the only constraint being that their outputs are of commensurate shapes.

Original: An interesting avenue for future research would therefore be to automatically determine the best operation to perform for each domain, for example by combining our approach with neural architecture search strategies.
Revised: An interesting avenue for future research would therefore be to automatically determine the best operation to perform for each domain, for example by combining our approach with neural architecture search strategies.

Original: Appendices A M ULTIBRANCH A RCHITECTURES
Revised: Appendices A M ULTIBRANCH A RCHITECTURES

Original: Below, we provide the network architectures and detailed building blocks of our Domain Adaptive
Revised: Below, we provide the network architectures and detailed building blocks of our Domain Adaptive

Original: Multibranch Networks, for the single source domain case (D = 2).
Revised: Multibranch Networks, for the single source domain case (D = 2).

Original: Each computational unit is enclosed by dotted lines.
Revised: Each computational unit is enclosed by dotted lines.

Original: The input and output shapes for all layer groupings are provided.
Revised: The input and output shapes for all layer groupings are provided.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: S o u r c e
Revised: S o u r c e

Original: D o m a i n
Revised: D o m a i n

Original: T a r g e t
Revised: T a r g e t

Original: D o m a i n conv1 conv conv _x conv _x conv _x conv _x conv _x conv _x conv _x conv _x
Revised: D o m a i n conv1 conv conv _x conv _x conv _x conv _x conv _x conv _x conv _x conv _x

Original: [Table]
Revised: [Table]

Original: classifier
Revised: classifier

Original: [Table]
Revised: [Table]

Original: classifier bottleneck 
Revised: classifier bottleneck conv

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: ReLU +
Revised: [Figure] [Table] +

Original: Conv 1x

Original: [Table]
Revised: [Table]

Original: BatchNorm
Revised: BatchNorm

Original: ReLU
Revised: [Table]

Original: Conv 
Revised: Conv 1x

Original: [Table]
Revised: [Table]

Original: BatchNorm

Original: ReLU 
Revised: ReLU +

Original: Conv1x [Table] BatchNorm Conv 1x
Revised: Conv

Original: [Table]
Revised: [Table]

Original: ch

Original: BatchNorm
Revised: BatchNorm

Original: ReLU +

Original: Conv 1x

Original: [Table]

Original: BatchNorm

Original: ReLU
Revised: ReLU

Original: Conv 3x

Original: [Table]
Revised: [Table]

Original: BatchNorm
Revised: BatchNorm

Original: ReLU
Revised: ReLU

Original: Conv 1x

Original: [Table]
Revised: [Table]

Original: BatchNorm conv _{b,c ,d} 28x28x

Original: [Table]

Original: conv _a

Original: [Table]

Original: ReLU +

Original: [Table]

Original: ReLU

Original: [Table]

Original: Conv

Original: [Table]

Original: BatchNorm

Original: ReLU +

Original: [Table]

Original: BatchNorm

Original: ReLU

Original: [Table]

Original: BatchNorm conv _{
Revised: BatchNorm conv _{

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: , f}

Original: [Table]
Revised: [Table]

Original: conv _a

Original: [Table]
Revised: [Table]

Original: ReLU +

Original: Conv

Original: [Table]
Revised: [Table]

Original: BatchNorm

Original: [Table]
Revised: [Table]

Original: BatchNorm

Original: ReLU

Original: [Table]
Revised: [Table]

Original: BatchNorm

Original: Conv 1x

Original: [Table]

Original: [Table]

Original: BatchNorm
Revised: BatchNorm

Original: ReLU +

Original: Conv

Original: [Table]
Revised: [Table]

Original: BatchNorm

Original: ReLU

Original: [Table]
Revised: [Table]

Original: BatchNorm

Original: [Table]
Revised: [Table]

Original: BatchNorm conv _{

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: B B ENCHMARK DATASET DESCRIPTIONS B. D IGIT RECOGNITION
Revised: B B ENCHMARK DATASET DESCRIPTIONS B. D IGIT RECOGNITION

Original: MNIST (LeCun et al., 1998) consists of black and white images of handwritten digits from 0 to 9.
Revised: MNIST (LeCun et al., 1998) consists of black and white images of handwritten digits from 0 to 9.

Original: All images are of size 28 × 28 pixels.
Revised: All images are of size 28 × 28 pixels.

Original: The standard training and testing splits contain 60,000 and 10,000 examples, respectively.
Revised: The standard training and testing splits contain 60,000 and 10,000 examples, respectively.

Original: MNIST-M (Ganin & Lempitsky, 2015) is synthetically generated by randomly replacing the foreground and background pixels of random MNIST samples with natural images.
Revised: MNIST-M (Ganin & Lempitsky, 2015) is synthetically generated by randomly replacing the foreground and background pixels of random MNIST samples with natural images.

Original: Its image size is 32 × 32 , and the standard training and testing splits contain 59,001 and 9,001 images, respectively.
Revised: Its image size is 32 × 32 , and the standard training and testing splits contain 59,001 and 9,001 images, respectively.

Original: SVHN (Netzer et al., 2011), the Street View House Numbers dataset, consists of natural scene images of numbers acquired from Google Street View.
Revised: SVHN (Netzer et al., 2011), the Street View House Numbers dataset, consists of natural scene images of numbers acquired from Google Street View.

Original: Its images are also of size 32 × 32 pixels, and its preset training and testing splits are of 73,257 and 26,032 images, respectively.
Revised: Its images are also of size 32 × 32 pixels, and its preset training and testing splits are of 73,257 and 26,032 images, respectively.

Original: The SVHN images are centered at the desired digit, but contain clutter, visual artifacts, and distractors from its surroundings.
Revised: The SVHN images are centered at the desired digit, but contain clutter, visual artifacts, and distractors from its surroundings.

Original: MNIST
Revised: MNIST

Original: MNIST-M
Revised: MNIST-M

Original: SVHN
Revised: SVHN

Original: B. O BJECT RECOGNITION
Revised: B. O BJECT RECOGNITION

Original: Ofﬁce (Saenko et al., 2010) is a multiclass object recognition benchmark dataset, containing images of 31 categories of objects commonly found in ofﬁce environments.
Revised: Ofﬁce (Saenko et al., 2010) is a multiclass object recognition benchmark dataset, containing images of 31 categories of objects commonly found in ofﬁce environments.

Original: It contains color images from three different domains: 2,817 images of products scraped from Amazon, 498 images acquired using a DSLR digital camera, and 795 images captured with a webcam.
Revised: It contains color images from three different domains: 2,817 images of products scraped from Amazon, 498 images acquired using a DSLR digital camera, and 795 images captured with a webcam.

Original: The images are of arbitrary sizes and aspect ratios.
Revised: The images are of arbitrary sizes and aspect ratios.

Original: Backpack Bicycle Bikehelmet Bookcase
Revised: [Figure]

Original: Bottle
Revised: Bottle

Original: Calculator
Revised: Calculator

Original: Chair
Revised: Chair

Original: Desklamp
Revised: Desklamp

Original: Computer
Revised: Computer

Original: File cabinet Headphones
Revised: File cabinet Headphones

Original: Amazon
Revised: Amazon

Original: DSLR
Revised: DSLR

Original: Webcam
Revised: Webcam

Original: Ofﬁce-Home (Venkateswara et al., 2017) comprises a larger corpus of color, arbitrarily-sized images from 65 different classes of objects commonly found in ofﬁce and home environments, coming from four different domains.
Revised: Ofﬁce-Home (Venkateswara et al., 2017) comprises a larger corpus of color, arbitrarily-sized images from 65 different classes of objects commonly found in ofﬁce and home environments, coming from four different domains.

Original: It contains 2,427 images extracted from paintings (Art), 4,365 clipart images (Clipart), 4,439 pho- tographs of products (Product), and 4,357 pictures captured with a regular consumer camera (Real world). 
Revised: It contains 2,427 images extracted from paintings (Art), 4,365 clipart images (Clipart), 4,439 pho- tographs of products (Product), and 4,357 pictures captured with a regular consumer camera (Real world). Alarmclock

Original: [Figure]
Revised: [Figure]

Original: [Table]

Original: Art
Revised: Art

Original: Clipart
Revised: Clipart

Original: Product
Revised: Product

Original: Realworld
Revised: Realworld

Original: VisDA 2017 (Peng et al., 2018) includes images of diverse sizes from 12 different categories, coming from two different domains: 55,368 synthetic renders of 3D models, and 152,397 photographs of the real-world objects.
Revised: VisDA 2017 (Peng et al., 2018) includes images of diverse sizes from 12 different categories, coming from two different domains: 55,368 synthetic renders of 3D models, and 152,397 photographs of the real-world objects.

Original: It is larger than the other two datasets, and exhibits a more signiﬁcant domain shift.
Revised: It is larger than the other two datasets, and exhibits a more signiﬁcant domain shift.

Original: [Figure]
Revised: [Figure]

Revised: Bus

Revised: Car

Revised: Horse

Revised: Knife

Revised: Motorcycle

Revised: Person

Original: Plant
Revised: Plant

Original: [Figure]
Revised: Skateboard Train

Original: Truck
Revised: Truck

Original: [Figure]
Revised: [Figure]

Revised: Synthetic

Original: B. O BJECT DETECTION
Revised: B. O BJECT DETECTION

Original: UAV-200 aggregates 200 images of real drones and around 33,000 synthetic ones, as well as around 190,000 patches obtained from the background of the video, which do not contain drones, used as negative examples.
Revised: UAV-200 aggregates 200 images of real drones and around 33,000 synthetic ones, as well as around 190,000 patches obtained from the background of the video, which do not contain drones, used as negative examples.

Original: All examples are of size 40 × 40 pixels.
Revised: All examples are of size 40 × 40 pixels.

Original: We evaluate performance on a validation set comprising 3,000 positive and 135,000 negative patches.
Revised: We evaluate performance on a validation set comprising 3,000 positive and 135,000 negative patches.

Original: [Figure]
Revised: [Figure]

Original: C A DDITIONAL EXPERIMENTS
Revised: C A DDITIONAL EXPERIMENTS

Original: [Figure]
Revised: [Figure]

Original: mnist sv h n

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: D F ULL RESULTS
Revised: D F ULL RESULTS

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: MNIST-M (cid:46) SVHN
Revised: MNIST-M (cid:46) SVHN

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: MNIST + MNIST-M
Revised: MNIST + MNIST-M

Original: (cid:46) SVHN
Revised: (cid:46) SVHN

Original: [Table] [Table]
Revised: [Table] [Table]

Original: n/a
Revised: n/

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: MNIST + MNIST-M
Revised: MNIST + MNIST-M

Original: (cid:46) SVHN (cid:63)
Revised: (cid:46) SVHN (cid:63)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: n/
Revised: n

Original: [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: Ofﬁce

Original: [Table]
Revised: [Table]

Original: Webcam (cid:46)
Revised: Webcam (cid:46)

Original: Amazon 41.
Revised: Amazon 41.

Original: DSLR (cid:46) Amazon 34.
Revised: DSLR (cid:46) Amazon 34.

Original: DSLR + Webcam (cid:46) Amazon 45.
Revised: DSLR + Webcam (cid:46) Amazon 45.

Original: n/a 68.
Revised: n/a 68.

Original: Ofﬁce-Home
Revised: Ofﬁce-Home

Original: Art (cid:46) Product 37.
Revised: Art (cid:46) Product 37.

Original: Clipart (cid:46) Product 36.
Revised: Clipart (cid:46) Product 36.

Original: Clipart (cid:46)
Revised: Clipart (cid:46)

Original: Art 29.
Revised: Art 29.

Original: Real world (cid:46)
Revised: Real world (cid:46)

Original: Art 50.
Revised: Art 50.

Original: Art (cid:46) Real world 53.
Revised: Art (cid:46) Real world 53.

Original: Clipart (cid:46) Real world 43.
Revised: Clipart (cid:46) Real world 43.

Original: Product (cid:46) Real world 46.
Revised: Product (cid:46) Real world 46.

Original: Clipart + Product (cid:46)
Revised: Clipart + Product (cid:46)

Original: Real world 53.
Revised: Real world 53.

Original: n/a 72.
Revised: n/a 72.

Original: Art + Clipart + Product (cid:46)
Revised: Art + Clipart + Product (cid:46)

Original: Real world 58.
Revised: Real world 58.

Original: n/a 77.
Revised: n/a 77.

Original: VisDA
Revised: VisDA

Original: Synthetic (cid:46) Real 35.
Revised: Synthetic (cid:46) Real 35.

Original: Real (cid:46)
Revised: Real (cid:46)

Original: Synthetic 51.
Revised: Synthetic 51.

Original: UAVSynthetic (cid:46) Real ∗ 0.
Revised: UAVSynthetic (cid:46) Real ∗ 0.

Original: [Table]
Revised: [Table]

Revised: [Table]

Original:  Evaluated with 
Revised: and Rozantsev [Table] ) (cid:63) Evaluated with a ResNet-50 ∗

Original: [Table]
Revised: [Table]

Original: Results
Revised: reported

Original: [Table]
Revised: [Table]