Original: D ISTRIBUTION M ATCHING P ROTOTYPICAL N ETWORK FOR U NSUPERVISED D OMAIN A DAPTATION
Revised: D ISTRIBUTION M ATCHING P ROTOTYPICAL N ETWORK FOR U NSUPERVISED D OMAIN A DAPTATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: State-of-the-art Unsupervised Domain Adaptation (UDA) methods learn transferable features by minimizing the feature distribution discrepancy between the source and target domains.
Revised: State-of-the-art Unsupervised Domain Adaptation (UDA) methods learn transferable features by minimizing the feature distribution discrepancy between the source and target domains.

Original: Different from these methods which do not model the feature distributions explicitly, in this paper, we explore explicit feature distribution modeling for UDA.
Revised: Different from these methods which do not model the feature distributions explicitly, in this paper, we explore explicit feature distribution modeling for UDA.

Original: In particular, we propose Distribution Matching Prototypical Network (DMPN) to model the deep features from each domain as Gaussian mixture distributions.
Revised: In particular, we propose Distribution Matching Prototypical Network (DMPN) to model the deep features from each domain as Gaussian mixture distributions.

Original: With explicit feature distribution modeling, we can easily measure the discrepancy between the two domains.
Revised: With explicit feature distribution modeling, we can easily measure the discrepancy between the two domains.

Original: In DMPN, we propose two new domain discrepancy losses with probabilistic interpretations.
Revised: In DMPN, we propose two new domain discrepancy losses with probabilistic interpretations.

Original: The ﬁrst one minimizes the distances between the corresponding Gaussian component means of the source and target data.
Revised: The ﬁrst one minimizes the distances between the corresponding Gaussian component means of the source and target data.

Original: The second one minimizes the pseudo negative log likelihood of generating the target features from source feature distribution.
Revised: The second one minimizes the pseudo negative log likelihood of generating the target features from source feature distribution.

Original: To learn both discriminative and domain invariant features, DMPN is trained by minimizing the classiﬁcation loss on the labeled source data and the domain discrepancy losses together.
Revised: To learn both discriminative and domain invariant features, DMPN is trained by minimizing the classiﬁcation loss on the labeled source data and the domain discrepancy losses together.

Original: Extensive experiments are conducted over two UDA tasks.
Revised: Extensive experiments are conducted over two UDA tasks.

Original: Our approach yields a large margin in the Digits Image transfer task over state-of-the-art approaches.
Revised: Our approach yields a large margin in the Digits Image transfer task over state-of-the-art approaches.

Original: More remarkably, DMPN obtains a mean accuracy of 81.4% on VisDA 2017 dataset.
Revised: More remarkably, DMPN obtains a mean accuracy of 81.4% on VisDA 2017 dataset.

Original: The hyper-parameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.
Revised: The hyper-parameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Recent advances in deep learning have signiﬁcantly improved state-of-the-art performance for a wide range of applications.
Revised: Recent advances in deep learning have signiﬁcantly improved state-of-the-art performance for a wide range of applications.

Original: However, the improvement comes with the requirement of a massive amount of labeled data for each task domain to supervise the deep model.
Revised: However, the improvement comes with the requirement of a massive amount of labeled data for each task domain to supervise the deep model.

Original: Since manual labeling is expensive and time-consuming, it is therefore desirable to leverage or reuse rich labeled data from a related domain.
Revised: Since manual labeling is expensive and time-consuming, it is therefore desirable to leverage or reuse rich labeled data from a related domain.

Original: This process is called domain adaptation, which transfers knowledge from a label rich source domain to a label scarce target domain (Pan & Yang, 2009).
Revised: This process is called domain adaptation, which transfers knowledge from a label rich source domain to a label scarce target domain (Pan & Yang, 2009).

Original: Domain adaptation is an important research problem with diverse applications in machine learning, computer vision (Gong et al., 2012; Gopalan et al., 2011; Hoffman et al., 2014; Saenko et al., 2010) and natural language processing (Collobert et al., 2011; Glorot et al., 2011).
Revised: Domain adaptation is an important research problem with diverse applications in machine learning, computer vision (Gong et al., 2012; Gopalan et al., 2011; Hoffman et al., 2014; Saenko et al., 2010) and natural language processing (Collobert et al., 2011; Glorot et al., 2011).

Original: Traditional methods try to solve this problem via learning domain invariant features by minimizing certain distance metric measuring the domain discrepancy, for example Maximum Mean Discrepancy (MMD) (Gretton et al., 2009; Pan et al., 2008; 2010) and second-order statistics (Sun & Saenko, 2016).
Revised: Traditional methods try to solve this problem via learning domain invariant features by minimizing certain distance metric measuring the domain discrepancy, for example Maximum Mean Discrepancy (MMD) (Gretton et al., 2009; Pan et al., 2008; 2010) and correlation distance (Sun & Saenko, 2016).

Original: Then labeled source data is used to learn a model for the target domain.
Revised: Then labeled source data is used to learn a model for the target domain.

Original: Recent studies have shown that deep neural networks can learn more transferable features for domain adaptation (Glorot et al., 2011;
Revised: Recent studies have shown that deep neural networks can learn more transferable features for domain adaptation (Glorot et al., 2011;

Original: Yosinski et al., 2014).
Revised: Yosinski et al., 2014).

Original: Consequently, adaptation layers have been embedded in the pipeline of deep feature learning to learn concurrently from the source domain supervision and some specially designed domain discrepancy losses (Tzeng et al., 2014;
Revised: Consequently, adaptation layers have been embedded in the pipeline of deep feature learning to learn concurrently from the source domain supervision and some specially designed domain discrepancy losses (Tzeng et al., 2014;

Original: Long et al., 2015; Sun & Saenko, 2016; Zellinger et al., 2017).
Revised: Long et al., 2015; Sun & Saenko, 2016; Zellinger et al., 2017).

Original: However, none of these methods explicitly model the feature distributions of the source and target data to measure the discrepancy.
Revised: However, none of these methods explicitly model the feature distributions of the source and target data to measure the discrepancy.

Original: Inspired from the recent works by Wan et al.
Revised: Inspired from the recent works by Wan et al.

Original: (2018) and Yang et al.
Revised: (2018) and Yang et al.

Original: (2018), which have shown that modeling feature distribution of a training set improves classiﬁcation performance, we explore explicit distribution modeling for UDA.
Revised: (2018), which have shown that modeling feature distribution of a training set improves classiﬁcation performance, we explore explicit distribution modeling for UDA.

Original: We model the feature distributions as Gaussin mixture distributions, which facilitates us to measure the discrepancy between the source and target domains.
Revised: We model the feature distributions as Gaussin mixture distributions, which facilitates us to measure the discrepancy between the source and target domains.

Original: Our proposed method, i.e., DMPN, works as follows.
Revised: Our proposed method, i.e., DMPN, works as follows.

Original: We train a deep network over the source domain data to generate features following a Gaussian mixture distribution.
Revised: We train a deep network over the source domain data to generate features following a Gaussian mixture distribution.

Original: The network is then used to assign pseudo labels to the unlabeled target data.
Revised: The network is then used to assign pseudo labels to the unlabeled target data.

Original: To learn both discriminative and domain invariant features, we ﬁne-tune the network to minimize the cross-entropy loss on the labeled source data and domain discrepancy losses.
Revised: To learn both discriminative and domain invariant features, we ﬁne-tune the network to minimize the cross-entropy loss on the labeled source data and domain discrepancy losses.

Original: Speciﬁcally, we propose two new domain discrepancy losses by exploiting the explicit Gaussian mixture distributions of the deep features.
Revised: Speciﬁcally, we propose two new domain discrepancy losses by exploiting the explicit Gaussian mixture distributions of the deep features.

Original: The ﬁrst one minimizes the distances between the corresponding Gaussian component means between the source and target data.
Revised: The ﬁrst one minimizes the distances between the corresponding Gaussian component means between the source and target data.

Original: We call it Gaussian Component Mean Matching (GCMM).
Revised: We call it Gaussian Component Mean Matching (GCMM).

Original: The second one minimizes the negative log likelihood of generating the target features from the source feature distribution.
Revised: The second one minimizes the negative log likelihood of generating the target features from the source feature distribution.

Original: We call it Pseudo Distribution Matching (PDM).
Revised: We call it Pseudo Distribution Matching (PDM).

Original: Extensive experiments on Digits Image transfer tasks and synthetic-to-real image transfer task demonstrate our approach can provide superior results than state-of-the-art approaches.
Revised: Extensive experiments on Digits Image transfer tasks and synthetic-to-real image transfer task demonstrate our approach can provide superior results than state-of-the-art approaches.

Original: We present our proposed method in Section 3, extensive experiment results and analysis in Section 4 and conclusion in Section 5.
Revised: We present our proposed method in Section 3, extensive experiment results and analysis in Section 4 and conclusion in Section 5.

Original:  R ELATED WORKS
Revised:  R ELATED WORKS

Original: Domain adaptation is an important research problem with diverse applications in machine learning, computer vision (Gong et al., 2012; Gopalan et al., 2011; Hoffman et al., 2014; Saenko et al., 2010) and natural language processing (Collobert et al., 2011; Glorot et al., 2011).
Revised: Domain adaptation is an important research problem with diverse applications in machine learning, computer vision (Gong et al., 2012; Gopalan et al., 2011; Hoffman et al., 2014; Saenko et al., 2010) and natural language processing (Collobert et al., 2011; Glorot et al., 2011).

Original: According to the survey Pan & Yang (2009), traditional domain adaptation methods can be organized into two categories: feature matching and instance re-weighting.
Revised: According to the survey Pan & Yang (2009), traditional domain adaptation methods can be organized into two categories: feature matching and instance re-weighting.

Original: Feature matching aims to reduce the domain discrepancy via learning domain invariant features by minimizing certain distance metric, for example Maximum Mean Discrepancy (MMD) (Gretton et al., 2009; Pan et al., 2008; 2010), second-order statistics (Sun & Saenko, 2016) and et al.
Revised: Feature matching aims to reduce the domain discrepancy via learning domain invariant features by minimizing certain distance metric, for example Maximum Mean Discrepancy (MMD) (Gretton et al., 2009; Pan et al., 2008; 2010), correlation distance (Sun & Saenko, 2016), Central Moment Discrepancy (CMD) Zellinger et al. (2017) and et al.

Original: Then labeled source data is used to learn a model for the target domain.
Revised: Then labeled source data is used to learn a model for the target domain.

Original: Instance re-weighting aims to reduce the domain discrepancy via re-weighting the source instances according to their importance weights with respect to the target distribution (Huang et al., 2007).
Revised: Instance reweighting aims to reduce the domain discrepancy via re-weighting the source instances according to their importance weights with respect to the target distribution (Huang et al., 2007).

Original: In the era of deep learning, studies have shown that deep neural networks can learn more transferable features for domain adaptation (Glorot et al., 2011; Yosinski et al., 2014), therefore, domain adaptation layers have been embedded in the pipeline of deep feature learning to learn concurrently from the source domain supervision and some specially designed domain discrepancy losses (Tzeng et al., 2014;
Revised: In the era of deep learning, studies have shown that deep neural networks can learn more transferable features for domain adaptation (Glorot et al., 2011; Yosinski et al., 2014), therefore, domain adaptation layers have been embedded in the pipeline of deep feature learning to learn concurrently from the source domain supervision and some specially designed domain discrepancy losses (Tzeng et al., 2014;

Original: Long et al., 2015; Sun & Saenko, 2016; Zellinger et al., 2017).
Revised: Long et al., 2015; Sun & Saenko, 2016; Zellinger et al., 2017).

Original: Some recent works Ganin &
Revised: Some recent works Ganin &

Original: Lempitsky (2014), Tzeng et al.
Revised: Lempitsky (2014), Tzeng et al.

Original: (2017), Long et al.
Revised: (2017), Long et al.

Original: (2018) add a domain discriminator into the deep feature learning pipeline, where a feature generator and a domain discriminator are learned adversarially to generate domain invariant features.
Revised: (2018) add a domain discriminator into the deep feature learning pipeline, where a feature generator and a domain discriminator are learned adversarially to generate domain invariant features.

Original: All these works can be categorized as the feature matching type of domain adaptation method.
Revised: All these works can be categorized as the feature matching type of domain adaptation method.

Original: However, none of them models the feature distributions of the source and target data for distribution matching.
Revised: However, none of them models the feature distributions of the source and target data for distribution matching.

Original: In this paper, we show that explicitly modeling the feature distributions enables us to measure the domain discrepancy more easily and helps us to propose new domain discrepancy losses.
Revised: In this paper, we show that explicitly modeling the feature distributions enables us to measure the domain discrepancy more easily and helps us to propose new domain discrepancy losses.

Original: Prototypical network (PN) was ﬁrst proposed in Snell et al.
Revised: Prototypical network (PN) was ﬁrst proposed in Snell et al.

Original: (2017) for few shot learning, which shows that learning PN is equivalent to performing mixture density estimation on the deep features with an exponential density.
Revised: (2017) for few shot learning, which shows that learning PN is equivalent to performing mixture density estimation on the deep features with an exponential density.

Original: Recently, in Wan et al.
Revised: Recently, in Wan et al.

Original: (2018)’s
Revised: (2018)’s

Original: and
Revised: and

Original: Yang et al.
Revised: Yang et al.

Original: (2018)’s works, it has been shown that modeling the deep feature distribution of a training set as Gaussian mixture distribution improves classiﬁcation performance.
Revised: (2018)’s works, it has been shown that modeling the deep feature distribution of a training set as Gaussian mixture distribution improves classiﬁcation performance.

Original: As Gaussian density belongs to one type of exponential density, the models proposed in Wan et al.
Revised: As Gaussian density belongs to one type of exponential density, the models proposed in Wan et al.

Original: (2018)’s
Revised: (2018)’s

Original: and
Revised: and

Original: Yang et al.
Revised: Yang et al.

Original: (2018)’s works are variants of PN.
Revised: (2018)’s works are variants of PN.

Revised: However, the two works study the classiﬁcation problem in a single domain, which is different from our work on the problem of domain adaptation.

Original: In Pan et al.
Revised: In Pan et al.

Original: (2019), prototypical networks are ﬁrst applied for domain adaptation.
Revised: (2019), prototypical networks are ﬁrst applied for domain adaptation.

Original: Multi-granular domain discrepancy minimization at both class-level and sample-level are employed in Pan et al.
Revised: Multi-granular domain discrepancy minimization at both class-level and sample-level are employed in Pan et al.

Original: (2019) to reduce the domain difference and achieves state-of-the-art results in various domain adaptation tasks.
Revised: (2019) to reduce the domain difference and achieves state-of-the-art results in various domain adaptation tasks.

Original: However, in Pan et al.
Revised: However, in Pan et al.

Original: (2019)’s work, the deep feature distribution is modeled implicitly when they apply PN for UDA, in our work, we explicitly model the deep feature distribution as Gaussian mixture distribution for UDA.
Revised: (2019)’s work, the deep feature distribution is modeled implicitly when they apply PN for UDA, in our work, we explicitly model the deep feature distribution as Gaussian mixture distribution for UDA.

Original:  D ISTRIBUTION M ATCHING P ROTOTYPICAL N ETWORKS
Revised:  D ISTRIBUTION M ATCHING P ROTOTYPICAL N ETWORKS

Original: In Unsupervised Domain Adaptation (UDA), we are given N s labeled samples D s = { ( x si , y si ) } N s i = in the source domain and N t unlabeled samples D t = { x ti } N t i =1 in the target domain.
Revised: In Unsupervised Domain Adaptation (UDA), we are given N s labeled samples D s = { ( x si , y si ) } N s i = in the source domain and N t unlabeled samples D t = { x ti } N t i =1 in the target domain.

Original: The source and target samples share the same set of labels and are sampled from probability distributions P s and P t respectively with P s (cid:54) =
Revised: The source and target samples share the same set of labels and are sampled from probability distributions P s and P t respectively with P s (cid:54) =

Original: P t .
Revised: P t .

Original: The goal is to transfer knowledge learnt from the labeled source domain to the unlabeled target domain.
Revised: The goal is to transfer knowledge learnt from the labeled source domain to the unlabeled target domain.

Original: 3. D EEP F EATURE D ISTRIBUTION M ODELING
Revised: 3. D EEP F EATURE D ISTRIBUTION M ODELING

Original: We model the deep embedded features of the source data as a Gaussian mixture distribution where the Gaussian component means act as the prototypes for each class.
Revised: We model the deep embedded features of the source data as a Gaussian mixture distribution where the Gaussian component means act as the prototypes for each class.

Original: Let { µ sc } Cc =1 and { Σ sc } Cc =1 be the Gaussian component means and covariance matrices of the Gaussian mixture distribution, then the posterior distribution of a class y given the embedded feature f can be expressed as in Eqn.where f = F ( x, θ ) , F : X → R d is the embedding function with parameter θ and d is the dimension of the embedded feature, p ( c ) is the prior probability of class c and C is the total number of classes.
Revised: Let { µ sc } Cc =1 and { Σ sc } Cc = be the Gaussian component means and covariance matrices of the Gaussian mixture distribution, then the posterior distribution of a class y given the embedded feature f can be expressed as in Eqn. 1 where f = F ( x , θ ) , F : X → R d is the embedding function with parameter θ and d is the dimension of the embedded feature, p ( c ) is the prior probability of class c and C is the total number of classes.

Original: [Equation]
Revised: [Equation]

Original: With labeled source data D s = { ( x si , y si ) }
Revised: With labeled source data D s = { ( x si , y si ) }

Original: N s i =1 ,following Wan et al. (2018), a classiﬁcation loss L cls can be computed as the cross-entropy between the posterior probability distribution and the one-hot class label as shown in Eq. 
Revised: N s i =1 , a classiﬁcation loss L cls can be computed as the cross-entropy between the posterior probability distribution and the one-hot class label as shown in Eq. 2 and following Wan et al.

Original: 2 and a log likelihood regularization term L lkd can be deﬁned as inEq.
Revised: (2018), a log likelihood regularization term L lkd can be deﬁned as in

Revised: Eq. 3, where f si = F ( x si , θ ) .

Original: [Equation]
Revised: [Equation]

Original: The ﬁnal loss function L GM for training a network with Gaussian mixture feature distribution is deﬁned as L GM =
Revised: The ﬁnal loss function L GM for training a network with Gaussian mixture feature distribution is deﬁned as L GM =

Original: L cls ( D ) + ϕ L lkd ( D ) , where ϕ is a non-negative weighting coefﬁcient.
Revised: L cls ( D ) + ϕ L lkd ( D ) , where ϕ is a non-negative weighting coefﬁcient.

Original: Notice, the distribution parameters { µ sc } Cc =1 and { Σ sc } Cc =1 are learned automatically from data.
Revised: Notice, the distribution parameters { µ sc } Cc =1 and { Σ sc } Cc =1 are learned automatically from data.

Original: 3. G AUSSIAN C OMPONENT M EAN M ATCHING
Revised: 3. G AUSSIAN C OMPONENT M EAN M ATCHING

Original: To match the deep feature distributions between the source and target data, we propose to match the corresponding Gaussian component means between them.
Revised: To match the deep feature distributions between the source and target data, we propose to match the corresponding Gaussian component means between them.

Original: We utilize the network learnt on the labeled source data to assign pseudo labels to target samples.
Revised: We utilize the network learnt on the labeled source data to assign pseudo labels to target samples.

Original: As such, we denote the target samples with pseudo labels as ˆ D t = { ( x ti , ˆ y ti ) } N t i =1 .
Revised: As such, we denote the target samples with pseudo labels as ˆ D t = { ( x t i , ˆ y t i ) } N t i =1 .

Original: We empirically estimate the Gaussian component means { µ es c } C c =1 1 and { µ et c } C c =1 as follows:
Revised: We empirically estimate the Gaussian component means { µ esc } Cc =1 1 and { µ etc } Cc =1 as follows:

Original: [Equation]
Revised: [Equation]

Original: where D sc and ˆ D tc denote the sets of source/target samples from class c  .
Revised: where D sc and ˆ D tc denote the sets of source/target samples from class c , f ti = F ( x ti ; θ ) .

Original: The GaussianComponent
Revised: The Gaussian

Original:  Mean Matching (GCMM) loss L GCMM is deﬁned as follows:
Revised: Component Mean Matching (GCMM) loss L GCMM is deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: where ||·|| is the L 2 norm between two vectors.
Revised: where ||·|| is the L 2 norm between two vectors.

Original: Intuitively, if the source features and target features follow the same Gaussian mixture distribution, then the Gaussian component means of the same class from the two domains will be the same.
Revised: Intuitively, if the source features and target features follow the same Gaussian mixture distribution, then the Gaussian component means of the same class from the two domains will be the same.

Original: Thus minimizing L GCMM helps to reduce the domain discrepancy.
Revised: Thus minimizing L GCMM helps to reduce the domain discrepancy.

Original: Better illustrated in Fig.
Revised: Better illustrated in Fig.

Original: 3. P SEUDO D ISTRIBUTION M ATCHING
Revised: 3. P SEUDO D ISTRIBUTION M ATCHING

Original: On the pseudo labeled target data ˆ D t , we further propose to match the embedded target feature distribution with the source Gaussian mixture feature distribution via minimizing the following pseudo negative log likelihood loss, which we denoted as L PDM :
Revised: On the pseudo labeled target data ˆ D t , we further propose to match the embedded target feature distribution with the source Gaussian mixture feature distribution via minimizing the following pseudo negative log likelihood loss, which we denoted as L PDM :

Original: [Equation]
Revised: [Equation]

Original: Minimizing L PDM 2 maximizes the likelihood of the pseudo labeled target features on the source
Revised: Minimizing L PDM 2 maximizes the likelihood of the pseudo labeled target features on the source

Original: Gaussian mixture feature distribution.
Revised: Gaussian mixture feature distribution.

Original: To achieve that, the network is enforced to learn an embedding function which produces similar embedded feature distributions between the source data and target data.
Revised: To achieve that, the network is enforced to learn an embedding function which produces similar embedded feature distributions between the source data and target data.

Original: Otherwise, this term will induce a large loss value and dominate the overall objective function to be minimized.
Revised: Otherwise, this term will induce a large loss value and dominate the overall objective function to be minimized.

Original: Therefore, minimizing L PDM helps to reduce the domain discrepancy.
Revised: Therefore, minimizing L PDM helps to reduce the domain discrepancy.

Original: As we are using pseudo labeled target data to calculate this domain discrepancy loss function, we term it as Pseudo Distribution Matching (PDM) loss.
Revised: As we are using pseudo labeled target data to calculate this domain discrepancy loss function, we term it as Pseudo Distribution Matching (PDM) loss.

Revised: Furthermore, while minimizing GCMM loss brings the source and target feature distribution closer, minimizing PDM loss shapes the target feature distribution to be similar as the source Gaussian mixture distribution.

Revised: Thus, these two loss functions complement each other to reduce the distribution discrepancy.

Original: Better illustrated in Fig.
Revised: Better illustrated in Fig.

Original: 3. O PTIMIZATION
Revised: 3. O PTIMIZATION

Original: The overall training objective of DMPN can be written as follows:
Revised: The overall training objective of DMPN can be written as follows:

Original: [Equation]
Revised: [Equation]

Original: where minimizing the ﬁrst two terms of the objective function helps the model to learn discriminative features with the supervision from the labeled source data, and minimizing the last two terms helps to match the embedded feature distributions between the source and target domains so that the learned classiﬁer from the labeled source data can be directly applied in the target domain.
Revised: where minimizing the ﬁrst two terms of the objective function helps the model to learn discriminative features with the supervision from the labeled source data, and minimizing the last two terms helps to match the embedded feature distributions between the source and target domains so that the learned classiﬁer from the labeled source data can be directly applied in the target domain.

Original: The whole model is illustrated in Fig.
Revised: The whole model is illustrated in Fig.

Original: Training Procedure.
Revised: Training Procedure.

Original: To train DMPN, we ﬁrst pre-train a network with labeled source data on L GM .
Revised: To train DMPN, we ﬁrst pre-train a network with labeled source data on L GM .

Original: Then mini-batch gradient descent algorithm is adopted for further optimization of the network on
Revised: Then mini-batch gradient descent algorithm is adopted for further optimization of the network on

Original: Eqn. 7, where half of the samples in the mini-batch are from labeled source data D s and the other half are from unlabeled target data D t .
Revised: Eqn. 7, where half of the samples in the mini-batch are from labeled source data D s and the other half are from unlabeled target data D t .

Original: To obtain pseudo labels for the unlabeled target data, we use the learned source distribution parameters to calculate the class probabilities for each target data point as in Eqn. 1 and assign the class with the largest probability as the pseudo label.
Revised: To obtain pseudo labels for the unlabeled target data, we use the learned source distribution parameters to calculate the class probabilities for each target data point as in Eqn. 1 and assign the class with the largest probability as the pseudo label.

Original: To remedy the error of the self-labeling, we took similar approach as in French et al.
Revised: To remedy the error of the self-labeling, we took similar approach as in French et al.

Original: (2018) and Pan et al.
Revised: (2018) and Pan et al.

Original: (2019) to ﬁlter unlabeled target data points whose maximum predicted class probability is smaller than some threshold.
Revised: (2019) to ﬁlter unlabeled target data points whose maximum predicted class probability is smaller than some threshold.

Original: Apart from that, we also propose to weight the contribution of each sample to the discrepancy loss based on the predicted probability.
Revised: Apart from that, we also propose to weight the contribution of each sample to the discrepancy loss based on the predicted probability.

Original: In this way, less conﬁdently predicted target samples will make smaller contributions in the training process.
Revised: In this way, less conﬁdently predicted target samples will make smaller contributions in the training process.

Original: Inference.
Revised: Inference.

Original: For inference, we ﬁrst apply the learned embedding function F on the target data, then we will use the learned distribution parameters to calculate the class probabilities for each target data point as in Eqn. 1.
Revised: For inference, we ﬁrst apply the learned embedding function F on the target data, then we will use the learned distribution parameters to calculate the class probabilities for each target data point as in Eqn. 1.

Original: Finally, we output the class with the largest probability for each target data point as our prediction.
Revised: Finally, we output the class with the largest probability for each target data point as our prediction.

Original: 3. T HEORETICAL I NSIGHT
Revised: 3. T HEORETICAL I NSIGHT

Original: There is another type of domain adaptation problem, called Supervised Domain Adaptation (SDA) in the literature.
Revised: There is another type of domain adaptation problem, called Supervised Domain Adaptation (SDA) in the literature.

Original: In SDA, we are provided with a large amount of labeled source data and a small amount of labeled target data, the goal is to ﬁnd a hypothesis that works well in the target domain.
Revised: In SDA, we are provided with a large amount of labeled source data and a small amount of labeled target data, the goal is to ﬁnd a hypothesis that works well in the target domain.

Original: By employing pseudo labeled target data in the training process, our method can be considered as working on a generalized problem of SDA, where the labeled target data is noisy.
Revised: By employing pseudo labeled target data in the training process, our method can be considered as working on a generalized problem of SDA, where the labeled target data is noisy.

Original: Ben-David et al.
Revised: Ben-David et al.

Original: (2010) has proved that we can bound the target error of a domain adaptation algorithm that minimizes a convex combination of empirical source and target error in SDA as follows:
Revised: (2010) has proved that we can bound the target error of a domain adaptation algorithm that minimizes a convex combination of empirical source and target error in SDA as follows:

Original: [Equation]
Revised: [Equation]

Original: (8) where (cid:15) γ ( h ) =
Revised: (8) where (cid:15) γ ( h ) =

Original: γ(cid:15) t ( h ) + (1 − γ ) (cid:15) s ( h ) is the convex combination of the source and target error with γ ∈ [0 , 1] , (cid:15) s =
Revised: γ(cid:15) t ( h ) + (1 − γ ) (cid:15) s ( h ) is the convex combination of the source and target error with γ ∈ [0 , 1] , (cid:15) s =

Original: E x ∼ P s [ | h ( x ) − f s ( x ) | ] , (cid:15) t =
Revised: E x ∼ P s [ | h ( x ) − f s ( x ) | ] , (cid:15) t =

Original: E x ∼ P t [ | h ( x ) − f t ( x ) | ] , f s and f t are the labeling function in the source and target domains respectively, h is a hypothesis in class H , d [Equation] P [Equation]
Revised: E x ∼ P t [ | h ( x ) − f t ( x ) | ] , f s and f t are the labeling function in the source and target domains respectively, h is a hypothesis in class H , d H ∆ H ( P s , P t )

Original: , P [Equation] [Equation] [Equation]
Revised: = 2 sup h,h (cid:48) ∈H | P x ∼ P s [ h ( x ) (cid:54)

Original: P

Original: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: h (cid:48)

Original:  x )] − P x ∼ [Equation] [ h ( x ) (cid:54) = h (cid:48)
Revised: ( x )] − P x ∼ P t [ h ( x ) (cid:54) = h (cid:48)

Original: ( x )] | measures the domain discrepsancy in the hypothesis space H and λ = (cid:15) s ( h ∗ ) + (cid:15) t ( h ∗ ) is the combined error in two domains of the joint ideal hypothesis 
Revised: ( x )] | measures the domain discrepsancy in the hypothesis space H and λ = (cid:15) s ( h ∗ ) + (cid:15) t ( h ∗ ) is the combined error in two domains of the joint ideal hypothesis h ∗ = arg min h ∈H (cid:15) s ( h ) + (cid:15) t

Original: [Equation]
Revised: ( h ) .

Original: Denote the noise ratio of the target labeling function to be ρ , the convex combination of the source and noisy target error as ˜ (cid:15) γ ( h ) =
Revised: Denote the noise ratio of the target labeling function to be ρ , the convex combination of the source and noisy target error as ˜ (cid:15) γ ( h ) =

Original: γ(cid:15) t (cid:48)
Revised: γ(cid:15) t (cid:48)

Original: ( h )
Revised: ( h )

Original: + (1 − γ ) (cid:15) s ( h ) , where (cid:15) t (cid:48)
Revised: + (1 − γ ) (cid:15) s ( h ) , where (cid:15) t (cid:48)

Original: ( h ) is the target error on the noisy target labeling function, then we can bound the target error as follows:
Revised: ( h ) is the target error on the noisy target labeling function, then we can bound the target error as follows:

Original: [Equation]
Revised: [Equation]

Original: In summary, this bound is decomposed into three parts: the domain discrepancy d H ∆ H , the error λ of the ideal joint hypothesis and the noise ratio ρ of the pseudo labels.
Revised: In summary, this bound is decomposed into three parts: the domain discrepancy d H ∆ H , the error λ of the ideal joint hypothesis and the noise ratio ρ of the pseudo labels.

Original: In DMPN, we minimize the ﬁrst term through minimizing the domain discrepancy losses.
Revised: In DMPN, we minimize the ﬁrst term through minimizing the domain discrepancy losses, as d H ∆ H is small when the source features and target features have similar distribution and minimizing the domain discrepancy losses makes the source and target feature to distribute similarly.

Original: The second term is assumed to be small, as otherwise there is no classiﬁer that performs well on both domains.
Revised: The second term is assumed to be small, as otherwise there is no classiﬁer that performs well on both domains.

Original: Finally, during training, as we continuously improve the accuracy of the classiﬁer for target data, we get more and more accurate predictions, thus reducing the noise ratio ρ .
Revised: Finally, during training, as we continuously improve the accuracy of the classiﬁer for target data, we get more and more accurate predictions, thus reducing the noise ratio ρ .

Original: We empirically verify that ρ is decreasing in Section 4.2.
Revised: We empirically verify that ρ is decreasing in Section 4.2.

Original:  E XPERIMENTS 4. D ATASETS AND E XPERIMENTAL S ETTINGS
Revised:  E XPERIMENTS 4. D ATASETS AND E XPERIMENTAL S ETTINGS

Original: Digits Image Transfer.
Revised: Digits Image Transfer.

Original: For the Digits Image transfer task, we consider the MNIST (M), USPS (U) and SVHN (S) datasets.
Revised: For the Digits Image transfer task, we consider the MNIST (M), USPS (U) and SVHN (S) datasets.

Original: Both the MNIST (M) and USPS (U) image datasets contains handwritten digits from ‘0’ to ‘9’.
Revised: Both the MNIST (M) and USPS (U) image datasets contains handwritten digits from ‘0’ to ‘9’.

Original: The MNIST dataset consists of 70k images and the USPS dataset has 9.3k images.
Revised: The MNIST dataset consists of 70k images and the USPS dataset has 9.3k images.

Original: Unlike MNIST and USPS, the SVHN (S) dataset is a real-world Digits dataset of house numbers in Google street view images and contains 100k cropped Digits images.
Revised: Unlike MNIST and USPS, the SVHN (S) dataset is a real-world Digits dataset of house numbers in Google street view images and contains 100k cropped Digits images.

Original: We follow the standard evaluation protocol (Tzeng et al., 2017; Pan et al., 2019).
Revised: We follow the standard evaluation protocol (Tzeng et al., 2017; Pan et al., 2019).

Original: We consider three directions of adaptation: M → U, U → M and S → M. For the transfer between MNIST and USPS, we sample 2k images from MNIST training set (60,000) and 1.8k images from USPS training set (7,291) for adaptation and evaluation is reported on the standard test sets: MNIST (10,000), USPS (2,007).
Revised: We consider three directions of adaptation: M → U, U → M and S → M. For the transfer between MNIST and USPS, we sample 2k images from MNIST training set (60,000) and 1.8k images from USPS training set (7,291) for adaptation and evaluation is reported on the standard test sets: MNIST (10,000), USPS (2,007).

Original: For
Revised: For

Original: S → M, we use the whole training set SVHN (73,257) and MNIST (60,000) for adaptation and evaluation is reported on the standard test set MNIST (10,000).
Revised: S → M, we use the whole training set SVHN (73,257) and MNIST (60,000) for adaptation and evaluation is reported on the standard test set MNIST (10,000).

Original: In addition, we use the same CNN architecture, namely a simple modiﬁed version of LeCun et al.
Revised: In addition, we use the same CNN architecture, namely a simple modiﬁed version of LeCun et al.

Original: (2 conv-layer LeNet) as Pan et al.
Revised: (2 conv-layer LeNet) as Pan et al.

Original: and
Revised: and

Original: Tzeng et al.
Revised: Tzeng et al.

Original: (2017) for fair comparison 3 .
Revised: (2017) for fair comparison 3 .

Original: Synthetic-to-Real Image Transfer.
Revised: Synthetic-to-Real Image Transfer.

Original: For synthetic-to-Real image transfer, we use the VisDAdataset for evaluation.
Revised: For synthetic-to-Real image transfer, we use the VisDAdataset for evaluation.

Original: In total, VisDA 2017 contains over 280k images across 12 categories in the combined training, validation, and testing domains.
Revised: In total, VisDA 2017 contains over 280k images across 12 categories in the combined training, validation, and testing domains.

Original: The training domain consists of 152k synthetic images which are generated by rendering 3D models of the same object categories from different angles and under different lighting conditions.
Revised: The training domain consists of 152k synthetic images which are generated by rendering 3D models of the same object categories from different angles and under different lighting conditions.

Original: The validation domain includes 55k images by cropping object in real images from COCO.
Revised: The validation domain includes 55k images by cropping object in real images from COCO.

Original: And the testing domain consists of 72k images cropped from video frmaes in YT-BB.
Revised: And the testing domain consists of 72k images cropped from video frmaes in YT-BB.

Original: As the labels for the testing data are unavailable, we use the training data as our source domain and the validation data as our target domain.
Revised: As the labels for the testing data are unavailable, we use the training data as our source domain and the validation data as our target domain.

Original: Following Pan et al.
Revised: Following Pan et al.

Original: (2019), we adopt 50-layer ResNet pre-trained on ImageNet as our basic CNN architecture.
Revised: (2019), we adopt 50-layer ResNet pre-trained on ImageNet as our basic CNN architecture.

Revised: Ofﬁce-Home Transfer.

Revised: (Venkateswara et al., 2017) consists of 15,500 images in 65 object classes in ofﬁce and home settings, forming four extremely dissimilar domains: Artistic images (Ar), Clip

Revised: Art (Cl), Product images (Pr), and Real-World images (Rw).

Original:  Implementation Details.
Revised: By permuting the 4 domains, we can generate 12 different domain adaptation tasks. Our method perform the best in all the transfer tasks compared to state-of-the-art UDA methods. The experiment results and training details for this experiment are in the Appendix A.3 due to space constraint. Implementation Details.

Original: Following Wan et al.
Revised: Following Wan et al.

Original: (2018), we assume the covariance matrices { Σ sc } C c = to be diagonal and the prior probability to be p ( c )
Revised: (2018), we assume the covariance matrices { Σ sc } Cc = to be diagonal and the prior probability to be p ( c )

Original: = 1 /C when pre-training the network on the labeled source data.
Revised: = 1 /C when pre-training the network on the labeled source data.

Original: The three trade-off parameters ϕ , α and β in Eqn.
Revised: The three trade-off parameters ϕ , α and β in Eqn.

Original: 7 are simply set to be 0.1, 1, 0.1.
Revised: 7 are simply set to be 0.1, 1, 0.1.

Original: We strictly follow Pan et al.
Revised: We strictly follow Pan et al.

Original: and set the embedding dimension d as 10/512 for
Revised: and set the embedding dimension d as 10/512 for

Original: Digits/synthetic-to-real image transfer.
Revised: Digits/synthetic-to-real image transfer.

Original: We implement DMPN with Pytorch.
Revised: We implement DMPN with Pytorch.

Original: We use ADAM with 0.0005 weight decay and 0.9/0.999 momentum for training and set the mini-batch size to be 128/ in Digits/synthetic-to-real image transfer.
Revised: We use ADAM with 0.0005 weight decay and 0.9/0.999 momentum for training and set the mini-batch size to be 128/ in Digits/synthetic-to-real image transfer.

Original: We train the network for 350 epochs for the Digits Image transfer tasks.
Revised: We train the network for 350 epochs for the Digits Image transfer tasks.

Original: The learning rate is initially set to be 1e-5 for the covariance matrices and 1e-3 for the other parameters 4 and is decayed by 0.1 at epoch 150 and 250.
Revised: The learning rate is initially set to be 1e-5 for the covariance matrices and 1e-3 for the other parameters 4 and is decayed by 0.1 at epoch 150 and 250.

Original: For the synthetic-to-real image transfer, we ﬁx the learning rate to be 1e-6 and train the network for 100 epochs 4 .
Revised: For the synthetic-to-real image transfer, we ﬁx the learning rate to be 1e-6 and train the network for 100 epochs 4 .

Original: Finally, for the Digits Image transfer tasks, we apply weighted PDM loss to remedy the labeling error, where each sample is weighted by p i − 0 .
Revised: Finally, for the Digits Image transfer tasks, we apply weighted PDM loss to remedy the labeling error, where each sample is weighted by the maximum predicted class probability.

Original: 9 , where p i is the maximum predicted class probability for sample i , 0.1 is the random predicted class probability so that a random prediction will have weight 0 and division of 0.9 is to make sure the range of the weight is between 0 and 1.

Original: For the synthetic-to-real image transfer task, we apply ﬁltering to remedy the labeling error, where only target examples with maximum predicted probability over 0.8 is used for training.
Revised: For the synthetic-to-real image transfer task, we apply ﬁltering to remedy the labeling error, where only target examples with maximum predicted probability over 0.8 is used for training.

Original: Following the standard, for Digits
Revised: Following the standard, for Digits

Original: Image transfer tasks, we adopt the classiﬁcation accuracy on target domain as evaluation metric and for synthetic-to-real image transfer, we use the average per class accuracy for evaluation metric.
Revised: Image transfer tasks, we adopt the classiﬁcation accuracy on target domain as evaluation metric and for synthetic-to-real image transfer, we use the average per class accuracy for evaluation metric.

Original: We will publish our code upon acceptance.
Revised: We will publish our code upon acceptance.

Original: Compared Methods.
Revised: Compared Methods.

Original: To demonstrate the beneﬁts of our proposed method, we compare it with the following approaches: (1) Source-only directly exploits the classiﬁcation model trained on source domain to classify target samples.
Revised: To demonstrate the beneﬁts of our proposed method, we compare it with the following approaches: (1) Source-only directly exploits the classiﬁcation model trained on source domain to classify target samples.

Original: (2) RevGrad (Ganin & Lempitsky, 2014) trains domain invariant features via adding a domain classiﬁer in the deep feature learning pipeline via gradient reversal.
Revised: (2) RevGrad (Ganin & Lempitsky, 2014) trains domain invariant features via adding a domain classiﬁer in the deep feature learning pipeline via gradient reversal.

Original: (3) DC (Tzeng et al., 2014) minimizes MMD along with deep feature learning for domain invariant features.
Revised: (3) DC (Tzeng et al., 2014) minimizes MMD along with deep feature learning for domain invariant features.

Original: (4) DAN (Long et al., 2015) applies multiple variants of MMD to align feature representations from multiple layers.
Revised: (4) DAN (Long et al., 2015) applies multiple variants of MMD to align feature representations from multiple layers.

Original: (5) RTN (Long et al., 2016) extends DAN by adapting classiﬁers through a residual transfer module.
Revised: (5) RTN (Long et al., 2016) extends DAN by adapting classiﬁers through a residual transfer module.

Original: (6) ADDA (Tzeng et al., 2017) separates the source feature learning and target feature learning using different networks and use a domain discriminator to learn domain invariant features.
Revised: (6) ADDA (Tzeng et al., 2017) separates the source feature learning and target feature learning using different networks and use a domain discriminator to learn domain invariant features.

Original: (7) JAN (Long et al., 2017) aligns the joint distribution of the network activation of multiple layers across domains.
Revised: (7) JAN (Long et al., 2017) aligns the joint distribution of the network activation of multiple layers across domains.

Original: (8) MCD (Saito et al., 2018) employs task-speciﬁc decision boundaries to align the distributions of source and target domains.
Revised: (8) MCD (Saito et al., 2018) employs task-speciﬁc decision boundaries to align the distributions of source and target domains.

Original: (9) CDAN+E (Long et al., 2018) adds a conditional adversarial classiﬁer on the deep feature learning pipeline to learn domain invariant features.
Revised: (9) CDAN+E (Long et al., 2018) adds a conditional adversarial classiﬁer on the deep feature learning pipeline to learn domain invariant features.

Original: (10) S-En+Mini-aug (French et al., 2018) modiﬁes the mean teacher variant of temporal ensembling for UDA.
Revised: (10) S-En+Mini-aug (French et al., 2018) modiﬁes the mean teacher variant of temporal ensembling for UDA.

Original: (11) TPN (Pan et al., 2019) is the ﬁrst work to apply PN for UDA.
Revised: (11) TPN (Pan et al., 2019) is the ﬁrst work to apply PN for UDA.

Original: TPN gen is the variant trained only with general-purpose domain discrepancy loss.
Revised: TPN gen is the variant trained only with general-purpose domain discrepancy loss.

Original: (12) DMPN is our proposed method.
Revised: (12) DMPN is our proposed method.

Original: DMPN GCMM and DMPN PDM are trained only with GCMM loss and PDM loss respectively.
Revised: DMPN GCMM and DMPN PDM are trained only with GCMM loss and PDM loss respectively.

Original: (13) Train-on-target is an oracle that trained on labeled target samples. 
Revised: (13) Train-on-target is an oracle that trained on labeled target samples. [Table]

Original: [Table]
Revised: [Table]

Original: 4. E XPERIMENTAL A NALYSIS
Revised: E XPERIMENTAL A NALYSIS

Original: Performance Comparison.
Revised: Performance Comparison.

Original: Table 1 shows the results of all methods for the two tasks.
Revised: Table 1 shows the results of all methods for the two tasks.

Original: Overall, our proposed method achieves superior results than all the existing methods.
Revised: Overall, our proposed method achieves superior results than all the existing methods.

Original: For the Digits Image transfer tasks, DMPN has improved the accuracy for M → U, U → M and S → M by 2.6%, 0.7% and 3.8% respectively compared to the second best.
Revised: For the Digits Image transfer tasks, DMPN has improved the accuracy for M → U, U → M and S → M by 2.6%, 0.7% and 3.8% respectively compared to the second best.

Original: We have made great advancement considering the second best accuracy results are already quite high.
Revised: We have made great advancement considering the second best accuracy results are already quite high.

Original: For the task S → M, due to convergence reasons, we have added batch normalization layers to the original CNN architectures.
Revised: For the task S → M, due to convergence reasons, we have added batch normalization layers to the original CNN architectures.

Original: For fair comparison, we have re-run some experiments on other methods by adding batch normalization layers to them.
Revised: For fair comparison, we have re-run some experiments on other methods by adding batch normalization layers to them.

Original: For methods whose public code are not available, we simply report the accuracy results with the original
Revised: For methods whose public code are not available, we simply report the accuracy results with the original

Original: CNN architecture.
Revised: CNN architecture.

Original: For ADDA, adding batch normalization layer has improved its accuracy result from 76.0% to 83.6%, which has an increase of 7.6% of accuracy.
Revised: For ADDA, adding batch normalization layer has improved its accuracy result from 76.0% to 83.6%, which has an increase of 7.6% of accuracy.

Original: However, we doubt adding batch normalization layers will have the same effect on TPN, as TPN already has a quite high accuracy.
Revised: However, we doubt adding batch normalization layers will have the same effect on TPN, as TPN already has a quite high accuracy.

Original: Nonetheless, we think our accuracy result of 96.8% on this task will be difﬁcult for the other methods to surpass even with batch normalization layers.
Revised: Nonetheless, we think our accuracy result of 96.8% on this task will be difﬁcult for the other methods to surpass even with batch normalization layers.

Original: For the Synthetic-to-real image transfer task, we only compare with methods without extensive data augmentations and our method has increased the state-of-the-art single model mean accuracy by 1.0%.
Revised: For the Synthetic-to-real image transfer task, we only compare with methods without extensive data augmentations and our method has increased the state-of-the-art single model mean accuracy by 1.0%.

Original: TPN gen reduces the domain discrepancy via minimizing the pairwise Reproducing Kernel Hilbert Space (RKHS) distances among the corresponding prototypes of the source-only, target-only and source-target data.
Revised: TPN gen reduces the domain discrepancy via minimizing the pairwise Reproducing Kernel Hilbert Space (RKHS) distances among the corresponding prototypes of the source-only, target-only and source-target data.

Original: In DMPN GCMM we minimize the L 2 distance between the corresponding Gaussian component means of the source and target data.
Revised: In DMPN GCMM we minimize the L 2 distance between the corresponding Gaussian component means of the source and target data.

Original: The L 2 distance can be viewed as the distance in a Linear RKHS space.
Revised: The L 2 distance can be viewed as the distance in a Linear RKHS space.

Original: The calculation of our proposed GCMM loss is much simpler than the general purpose loss, yet with explicitly modeling of the feature distributions, DMPN GCMM has a gain of accuracy of 3.0%, 1.1%, 6.3% and 6.6% on the M → U, U
Revised: The calculation of our proposed GCMM loss is much simpler than the general purpose loss, yet with explicitly modeling of the feature distributions, DMPN GCMM has a gain of accuracy of 3.0%, 1.1%, 6.3% and 6.6% [Figure]

Original:  → M, S → M and synthetic-to-real image transfer tasks respectively compared with TPN gen , showing that our method is more effective.
Revised: [Figure] on the M → U, U → M, S → M and synthetic-to-real image transfer tasks respectively compared with TPN gen , showing that our method is more effective.

Original: Ablation Analysis.
Revised: Ablation Analysis.

Original: In Table 1, combining GCMM loss and PDM loss helps to increase the accuracy results, showing that the two domain discrepancy losses are compatible to each other.
Revised: In Table 1, combining GCMM loss and PDM loss helps to increase the accuracy results, showing that the two domain discrepancy losses are compatible to each other.

Original: DMPN GCMM performs better than or similar to almost all other domain adaptation methods and DMPN PDM performs better than most of them.
Revised: DMPN GCMM performs better than or similar to almost all other domain adaptation methods and DMPN PDM performs better than most of them.

Original: Convergence Analysis.
Revised: Convergence Analysis.

Original: Figure 2 (a) shows the training progress of DMPN.
Revised: Figure 2 (a) shows the training progress of DMPN.

Original: The GCMM loss and
Revised: The GCMM loss and

Original: PDM loss keep decreasing with more training epochs.
Revised: PDM loss keep decreasing with more training epochs.

Original: The prediction accuracy on the unlabeled[Figure] [Figure] target data keeps increasing.
Revised: The prediction accuracy on the unlabeled target data keeps increasing.

Original: And the noise ratio ρ decreases along the training process, from the initial value of 38.6% decreases to 22.9%, which supports our theoretical analysis in Section 3.5.
Revised: And the noise ratio ρ decreases along the training process, from the initial value of 38.6% decreases to 22.9%, which supports our theoretical analysis in Section 3.5.

Original: Figure 3 shows the t-SNE visualizations of the source and target embedded features during training, which shows that target classes are becoming increasingly well discriminated by the source classiﬁer.
Revised: Figure 3 shows the t-SNE visualizations of the source and target embedded features during training, which shows that target classes are becoming increasingly well discriminated by the source classiﬁer.

Original: Sensitivity Analysis.
Revised: Sensitivity Analysis.

Original: Figure 2 (b)-(d) shows the sensitivity analysis on the hyper-parameters α , β and ϕ with the other hyper-parameters ﬁxed.
Revised: Figure 2 (b)-(d) shows the sensitivity analysis on the hyper-parameters α , β and ϕ with the other hyper-parameters ﬁxed.

Original: Overall, the experiment results show that we can get similar accuracy results or even better when changing the hyper-parameters in a certain range, showing that our method is robust against hyper-parameter changes.
Revised: Overall, the experiment results show that we can get similar accuracy results or even better when changing the hyper-parameters in a certain range, demonstrating that our method is robust against hyper-parameter changes.

Revised: The sensitivity analysis on the conﬁdence threshold is in the Appendix A.2, which shows our method is robust against threshold value.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper, we propose Distribution Matching Prototypical Network (DMPN) for Unsupervised
Revised: In this paper, we propose Distribution Matching Prototypical Network (DMPN) for Unsupervised

Original: Domain Adaptation (UDA) where we explicitly model and match the deep feature distribution of the source and target data as Gaussian mixture distributions.
Revised: Domain Adaptation (UDA) where we explicitly model and match the deep feature distribution of the source and target data as Gaussian mixture distributions.

Original: Our work ﬁlls the gap in UDA where stateof-the-art methods assume the deep feature distributions of the source and target data are unknown when minimizing the discrepancy between them.
Revised: Our work ﬁlls the gap in UDA where stateof-the-art methods assume the deep feature distributions of the source and target data are unknown when minimizing the discrepancy between them.

Original: We propose two new domain discrepancy losses based on the Gaussian mixture distributions of the deep features called Gaussian Component Mean
Revised: We propose two new domain discrepancy losses based on the Gaussian mixture distributions of the deep features called Gaussian Component Mean

Original: Matching (GCMM) and Pseudo Distribution Matching (PDM).
Revised: Matching (GCMM) and Pseudo Distribution Matching (PDM).

Original: Extensive experiments verify the effectiveness of our proposed method and domain discrepancy losses.
Revised: Extensive experiments verify the effectiveness of our proposed method and domain discrepancy losses.

Original: Finally, a post-hoc hyperparameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.
Revised: Finally, a post-hoc hyperparameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.

Original: A A PPENDIX 
Revised: A A PPENDIX A. D ETAILED R ESULTS ON S YNTHETIC - TO - REAL IMAGE TRANSFER

Revised: [Table] [Table] TPN task 89.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: TPN 93.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: DMPN

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: PDM

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: aug

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: target

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: A. S ENSITIVITY ANALYSIS ON CONFIDENCE THRESHOLD

Revised: Fig.

Revised: 4 shows the sensitivity analysis of our method on different values of conﬁdence threshold on

Revised: VisDA 2017 dataset.

Revised: The experiment results show that we can get similar accuracy results or even better when changing the conﬁdence threshold in a certain range, demonstrating that our method is robust against hyper-parameter changes.

Revised: A. O FFICE -H OME T RANSFER

Revised: Table 3 presents experiment results of state-of-the-art UDA methods and our method on OfﬁceHome dataset.

Revised: Our method gives the best accuracy results in all transfer tasks, showing the effectiveness of our method.

Revised: In this experiment, we train the network for 100 epochs.

Revised: The learning rate is initially set to be 1e-5 for all the parameters and is decayed by 0.1 at epoch 60 and 80.

Revised: Following Long et al.

Revised: (2018), α and β vary in each training epoch and are calculated by α = 1 − e − γp 1+ e − γp

Revised: and β = 0 .

Original: {+1 ∗ 1 − e − γp 
+}
Revised: {+1 ∗ 1 − e − γp 
+}

Revised: + e − γp respectively, where γ is set to be the default value 10, p is the training process changing from 0 to 1.

Original: [Table]
Revised: [Table]