Distribution Matching Prototypical Network for Unsupervised Domain Adaptation
Authors: Lei Zhu
Wei Wang
Mei Hui Zhang
Beng Chin Ooi
Chang Yao
Venue:  (2020)
Keywords: Deep Learning
Unsupervised Domain Adaptation
Distribution Modeling

Abstract: State-of-the-art Unsupervised Domain Adaptation (UDA) methods learn transferable features by minimizing the feature distribution discrepancy between the source and target domains. Different from these methods which do not model the feature distributions explicitly, in this paper, we explore explicit feature distribution modeling for UDA. In particular, we propose Distribution Matching Prototypical Network (DMPN) to model the deep features from each domain as Gaussian mixture distributions. With explicit feature distribution modeling, we can easily measure the discrepancy between the two domains. In DMPN, we propose two new domain discrepancy losses with probabilistic interpretations. The first one minimizes the distances between the corresponding Gaussian component means of the source and target data. The second one minimizes the pseudo negative log likelihood of generating the target features from source feature distribution. To learn both discriminative and domain invariant features, DMPN is trained by minimizing the classification loss on the labeled source data and the domain discrepancy losses together. Extensive experiments are conducted over two UDA tasks. Our approach yields a large margin in the Digits Image transfer task over state-of-the-art approaches. More remarkably, DMPN obtains a mean accuracy of 81.4% on VisDA 2017 dataset. The hyper-parameter sensitivity analysis shows that our approach is robust w.r.t hyper-parameter changes.

Under review as a conference paper at ICLR 2020
DISTRIBUTION MATCHING PROTOTYPICAL NETWORK FOR UNSUPERVISED DOMAIN ADAPTATION
Anonymous authors Paper under double-bling review
ABSTRACT
State-of-the-art Unsupervised Domain Adaptation (UDA) methods learn trans- ferable features by minimizing the feature distribution discrepancy between the source and target domains. Different from these methods which do not model the feature distributions explicitly, in this paper, we explore explicit feature distribu- tion modeling for UDA. In particular, we propose Distribution Matching Prototyp- ical Network (DMPN) to model the deep features from each domain as Gaussian mixture distributions. With explicit feature distribution modeling, we can easily measure the discrepancy between the two domains. In DMPN, we propose two new domain discrepancy losses with probabilistic interpretations. The first one minimizes the distances between the corresponding Gaussian component means of the source and target data. The second one minimizes the pseudo negative log likelihood of generating the target features from source feature distribution. To learn both discriminative and domain invariant features, DMPN is trained by minimizing the classification loss on the labeled source data and the domain dis- crepancy losses together. Extensive experiments tasks. Our approach yields a large margin in the Digits Image transfer task over state-of-the-art approaches. More remarkably, DMPN obtains 81.4% on VisDA 2017 dataset. The hyper-parameter sensitivity analysis shows that our approach robust w.r.t hlper-paaameter changes.
conducted over two UDA
accuracy of
INTRODUCTION
Recent advances in deep learning have significantly improved state-of-the-art performance for wide range of applications. However, the improvement comes with the requirement of massive amount of labeled data for each task domain to supervise the deep model. Since manual labeling is expensive and cime-consuming, it therefore desirable to leverage or reuse rich labeled data from a related domain. This process is called domain adaptation, which transfers knowledge from label rich source domain to a label scarce target domain (Pan & Yang 2009).
Domain adaptation is important research problem with diverse applications in machine learning. computer vision (Gong et al.) 2012 Gopalan et al.) 2011 Hoffman et al., 2014 Saenko et al., 2010)
and natural language processing (Collobert et al. 2011 Glorot et al. [001]] Traditional methods
try to solve this problem via learning domain invariant features by minimizing certain distance met- ric measuring the domain discrepancy, for example Maximum Mean Discrepancy (MMD) Gretton et al., 2009: Pan et al., 2008 2010) and correlation distance (Sun & Saenko 2016). Then labeled source data used learn model for the target domain. Recent studies have shown that deep neu- ral networks can learn more transferable features for domain adaptation (Glorot et al. [001] Yosinski al. 2014). Consequently, adaptation layers have been embedded in pipeline of deep feature learning to learn concurrently from the source domain supervision and some specially designed do- main discrepancy losses (Tzeng et al.
et al. 2015 Sun & Saenko 2016 Zellinger et al.
2017).
However, none of these methods explicitly model the feature distributions of the source and target data to measure the discrepancy. Inspired from the recent works by Wan al. (2018) and Yang al. 2018), which have shown that modeling feature distribution of training set improves classification performance, explore explicit distribution modeling for UDA. We model the feature distributions

---

Under review as a conference paper at ICLR 2020
as Gaussin mixture distributions, which facilitates to measure the discrepancy between the source and target domains.
Our proposed method, i.c., DMPN, works as follows. We train deep network over the source domain data to generate features following a Gaussian mixture distribution. The network is then used assign pseudo labels to the unlabeled target data. To learn both discriminative and domain invariant features, we fine-tune the network to minimize the cross-entropy loss on the labeled source data and domain discrepancy losses. Specifically, we propose two new domain discrepancy losses by exploiting the explicit Gaussian mixture distributions of the deep features. The first one minimizes the distances between the corresponding Gaussian component means between the source and target data. We call it Gaussian Component Mean Matching (GCMM). The second one minimizes the negative log likelihood of generating the target features from the source feature distribution. We call it Pseudo Distribution Matching (PDM). Extensive experiments on Digits Image transfer tasks and synthetie-to-real image transfer task demonstrate our approach can provide superior results than slate-of-the-an approaches. We present our proposed method in Section 3 extensive experiment results and analysis in Section 4 and conclusion in Section 5
2 RELATED WORKS
Domain adaptation is an important research problem with diverse applications in machine learn- ing, computer vision (Gong et al. 2012 Gopalan et al.
Hoffman et al. 2014, Saenko et al.) 2010) and natural language processing Collobert et al. 2011 Glorot et al. 2011). According to the survey Pan & Yang (2009), traditional domain adaptation methods can be organized into two categories: feature matching and instance re-weighting. Feature matching aims to reduce the do- main discrepancy via learning domain invariant features by minimizing certain distance metric, for example Maximum Mean Discrepancy (MMD) (Giretton et al.) 2009 Pan et al. 2008 2010), corre- lation distance (Sun & Saenko, 2016). Central Moment Discrepancy (CMD) Zellinger et al. (2017) and et al. Then labeled source data is used to learn a model for the target domain. Instance re- weighting aims reduce the domain discrepancy via re-weighting the source instances according to their importance weights with respect to the target distribution (Huang et al. 2007).
2011:
In the era of deep learning, studies have shown that deep neural networks can learn more transferable features for domain adaptation (Glorot et al. 2011 Yosinski et al. 2014). therefore, domain adapta- tion layers have been embedded in the pipeline of deep feature learning to learn concurrently from the source domain supervision and some specially designed domain discrepancy losses (Tzeng et al., 2014 Long al. 2015) Sun & Saenko [016 Zellinger et al. (017]. Some recent works Ganin & Lempitsky 2014). Tzeng et al. (2017). Long et al. (2018) add domain discriminator into the deep feature learning pipeline, where feature generator and a domain discriminator are learned ad- versarially to generate domain invariant features. All these works can be categorized the feature matching type of domain adaptation method. However, none of them models the feature distribu- tions of the source and target data for distribution matching. In this paper, we show that explicitly modeling the feature distributions enables us to measure the domain discrepancy more easily and helps us to propose new domain discrepancy losses.
Prototypical network (PN) was first proposed in Snell et al. (2017) for few shot learning, which shows that learning PN is equivalent performing mixture density estimation on the deep features with an exponential density. Recently, in WWan et al. (2018)'s 's and (Yang et al. (2018)'s works, it has been shown that modeling the deep feature distribution of a training set as Gaussian mixture distri- bution improves classification performance, As Gaussian density belongs to one type of exponential density, the models proposed in Wan et al. (2018)'s 's and Yang et al. 20188 's works are variants of PN. However, the two works study the classification problem in single domain, which is different from our work on the problem of domain adaptation. In Pan et al. (2019), prototypical networks are first applied for domain adaptation. Multi-granule class-level and sample-lveel are employed in Pan et al. (2019) to reduce the domain difference and achieves state-of- mate-off-heeart results in various domain adaptation tasks. However, in Pan et al. (219))ss 's work, the deep feature distribution is modeled implicitly when they apply PN for UDA, in our work, explicitly model the deep feature distribution as Gaussian mixture distribution for UDA.
domain discrepancy minimization at both
2

---

Under review as a conference paper at ICLR 2020
3 DISTRIBUTION MATCHING PROTOTYPICAL NETWORKS
In Unsupervised Domain Adaptation (UDA), we are given N labeled samples D = {(x], vf) H in the source domain and N' unlabeled samples D' = { in the target domain. The source and target samples share the same set of labels and sampled from probability distributions P and p respectively with pa * pt. The goal is to transfer knowledge learnt from the labeled source domain to the unlabeled target domain.
3.1 DEEP FEATURE DISTRIBUTION MODELING
We model the deep embedded features of the source data Gaussian mixture distribution where the Gaussian component means act as the prototypes for each class. Let {pr)a= and {E:} be the Gaussian component means and covariance matrices of the Gaussian mixture distribution, then the posterior distribution of a class given the embedded feature f can be expressed as in Eqn. [ where dimension of the embedded feature, p(c) is the prior probability of class and C' is the total number of classes.
Fax, 0). F X - Re is the embedding function with parameter and d is the
pluls) Nffiug Ej)p(v)
(1)
Ne ME, EE)p(c
With labeled source data D = [(a]. yf D() a classification loss Lets can be computed as cross-entropy between the posterior probability distribution and the one-hot class label as shown in Eq. and following Wan et al. (2018). a log likelihood regularization term Likd can be defined as in Eq.
where f Flw
N
N(fi ug Dipppus N H E))p(c)
(ID" =
log
(2)
N
N log N Nft: E
Lika(TD")
N
The final loss function LGM for training a network with Gaussian mixture feature distribution is defined as LGM = (D) + (D), where & is non-negative weighting coefficient. Notice, the distribution parameters {u() and (Ea)a= are learned automatically from data.
3.2 GAUSSIAN COMPONENT MEAN MATCHING
To match the deep feature distributions between the source and target data, we propose to match the corresponding Gaussian component means between them. We utilize the network learnt on the labeled source data to assign pseudo labels to target samples. As such, we denote the target samples with pseudo labels Tie (a N We empirically estimate the Gaussian component means
{pe) I and C Je=1 as follows:
Σ
Σ f
(4)
D
x'CD:
where D& and Di denote the sets of source/target samples from class = Component Mean Matching (GCMM) loss LOOMM is defined as follows:
The Gaussian
(GccMs((pes) 1e ( (1))))))) Σ 1 10l
where ||-|| is the L norm between two vectors. Intuitively, if the source features and target features follow the same Gaussian mixture distribution, then the Gaussian component means of the same class from the two domains will be the same. Thus minimizing LOCMM helps to reduce the domain discrepancy. Better illustrated in Fig.
Te-1 is different from (4))C= the latter learned directly from data and are used to assign pseudo lahels for target data.

---

Under review as a conference paper at ICLR 2020
Class 1
GCMM Loss PDM Loss
raw data
Embedding
Class 2
Class 3
Figure I: Illustration of the overall training objective. This figure displays the model after finish pre-training it with the labeled source data on LGM. Different colors represent different classes. Dotted ellipses represent Gaussian mixture distribution of the source embedded features. The amor- phous shapes represent pseudo labeled target feature distribution before we optimize the network further on the overall objective function Eqn. 7 GCMM loss tries to bring the corresponding Gaussian component means between the source data and pscudo labeled target data closer, repre- sented by the black two-way arrows. Minimizing GCMM brings the feature distributions of the source and target domains closer, thus reducing the domain discrepancy. PDM loss tries to match the pseudo target feature distribution to the source Gaussian mixture distribution, represented by the colored one-way arrow. Minimizing PDM increases the likelihood of target features the source feature distribution, thus reducing the domain discrepancy. Best viewed in color.
3.3 PSEUDO DISTRIBUTION MATCHING
On the pseudo labeled target data Di, we further propose to match the embedded target feature dis- tribution with the source Gaussian mixture feature distribution via minimizing the following pseudo negative log likelihood loss, which we denoted as LPDM:
CPDM (ID*)
Σ
log Nf 2g,)
(6)
DDA
Minimizing CrDad maximizes the likelihood of the pseudo labeled target features on the source Gaussian mixture feature distribution. To achieve that, the network is enforced to learn an embed- ding function which produces similar embedded feature distributions between the source data and target data. Otherwise, this term will induce large loss value and dominate the overall objective function to be minimized. Therefore, minimizing LPDM helps to reduce the domain discrepancy. As we using pseudo labeled target data to caleulate this domain discrepancy loss function, term it as Pseudo Distribution Matching (PDM) loss. Furthermore, while minimizing GCMM loss brings the source and target feature distribution closer, minimizing PDM loss shapes the target fea- ture distribution to be similar as the source Gaussian mixture distribution. Thus, these two loss functions complement each other reduce the distribution discrepancy, Better illustrated in Fig.
3.4 OPTIMIZATION
The overall training objective of DMPN can be written as follows:
min
Las (TD*) ) + (TT") + aLCcMM (u
c Je (pg)C)) ( + (LPDM((D)) (7)
0, (
where minimizing the first two terms of the objective function helps the model to learn discriminative features with the supervision from the labeled source data, and minimizing the last two terms helps to match the embedded feature distributions between the source and target domains that the learned classifier from the labeled source data can be directly applied in the target domain. The whole model is illustrated in Fig.
Training Procedure. To train DMPN, we first pre-train a network with labeled source data on LaM- Then mini-batch gradient descent algorithm is adopted for further optimization of the network on
Nottce,, gradient from LrdM does not mack-propagate to update (M3C and (EAC We learn source distribution parameters only from labeled source data.
4

---

Under review as a conference paper at ICLR 2020
Eqn. 7 where half of the samples in the mini- are from labeled source data D and the other half are from unlabeled target data D'. To obtain pseudo labels for the unlabeled target data, we use the learned source distribution parameters to calculate the class probabilities for each target data point as in Eqn. D and assign the class with the largest probability as the pseudo label. To remedy the error of the self-labeling, (211)) to filter unlabeled target data points whose maximum predicted class probability is smaller than some threshold. Apart from that, we also propose to weight the contribution of each sample to the discrepancy loss based on the predicted probability. In this way, less confidently predicted target samples will make smaller contributions in the training process.
took similar approach as in French al. (2018) and [Pan et al.]
Inference. For inference, we first apply the learned embedding function F on the target data, then we will use the learned distribution parameters to calculate the class probabilities for each target data point as in [qn.[] [ Finally, we output the class with the largest probability for each target data point as our prediction.
3.5 THEORETICAL INSIGHT
There is another type of domain adaptation problem, called Supervised Domain Adaptation (SDA) in the literature. In SDA, we are provided with a large amount of labeled source data and a small amount of labeled target data, the goal is find hypothesis that works well in the target domain. By employing pseudo labeled target data in the training process, our method can be considered as working on a generalized problem of SDA, where the labeled target data is noisy. [Ben-Davdd eet al. (2010) has proved that we can bound the target error of a domain adaptation algorithm that minimizes convex combination of empirical source and target error in SDA as follows:
(e,(t) cx(h)] s (1 (1- G P') A)
(8)
where (h) = (h) + (1 ()-7)c,,1) is the convex combination of the source and target er- Ez- [1(z)) f'(z)],, f and ft are a hypothesis in class H. dHAH(( PP*, Pt) = 2 supp,, Ex PPZ- [mxx) * h (x)] - Pa [h(x) * h' (a:)] measures the do- main discrepsancy in the hypothesis space H and = e, c,()))))))))))))²" + (l()) is the combined error two domains of the joint ideal hypothesis h" arg min e, (h) + (h).
ror with y E [0, 1], E, Ex-P- [h(() f*(x)]..
the labeling function in the source and target domains respectively,
Denote the noise ratio of the target labeling function to be p. the convex combination of the source and noisy target error as E, (h) ye, (h) + (1 - -)),,(t where e, (h) is the target error on the noisy target labeling function, then we can bound the target error as follows:
(2,(h) ex (h)l = hee (h) + (1 - (1) - (h) + (h)(h) - (h)(
- (c,(() (h()) + + (h) - chh)
(9)
s (1 - >) dHa P*) + X) p7
In summary, this bound is decomposed into three parts: the domain discrepancy dHaN. the error A of the ideal joint hypothesis and the noise ratio p of the pseudo labels. In DMPN, minimize the first term through minimizing the domain discrepancy losses, as dHAH is small when the source features and target features have similar distribution and minimizing the domain discrepancy losses makes the source and target feature to distribute similarly. The second term is assumed to be small, as otherwise there is no classifier that performs well on both domains. Finally, during training, we continuously improve the accuracy of the classifier for target data, we get more and more accurate predictions, thus reducing the noise ratio p. We empirically verify that p is decreasing in Section 4.2
EXPERIMENTS
4.1 DATASETS AND EXPERIMENTAL SETTINGS
Digits Image Transfer. For the Digits Image transfer task, we consider the MNIST (M), USPS (U) and SVHN (S) datasets. Both the MNIST (M) and USPS (U) image datasets contains handwritten digits from '0' to '9'. The MNIST dataset consists of 70k images and the USPS dataset has 9.3k
5

---

Under review as a conference paper at ICLR 2020
images. Unlike MNIST and USPS, the SVHN (S) dataset is a real-world Digits dataset of house numbers in Google street view images and contains 100k cropped Digits images. We follow the standard evaluation protocol (Tzeng et al. 2017: Pan et al. 2019). We consider three directions of adaptation: M + U, U + M and S -> M. For the transfer between MNIST and USPS, we sample 2k images from MNIST training set (60,000) and 1.8k images from USPS training set (7,291) for adaptation and evaluation is reported on the standard test sets: MNIST (10,000), USPS (2,007). For
+ M, we use the whole training set SVHN (73,257) and MNIST (60,000) for adaptation and evaluation is reported on the standard test set MNIST (10,000). In addition, we use the same CNN architecture, namely a simple modified version of LeCun et al. (1998) (2 conv-layer LeNet) as Pan et al. (211)) and (Tznng et al. (0117) for fair comparisor
Synthetic-to-Real Image Transfer. For synthetic-to-Ret image transfer, we use the VisDA 2017 dataset for evaluation. In total, VisDA 2017 contains over 280k images across 12 categories in the combined training, validation, and testing domains. The training domain consists of 152k synthetic images which are generated by rendering 3D models of the same object categories from different angles and under different lighting conditions. The validation domain includes 55k images by crop- ping object in real images from COCO. And the testing domain consists of 72k images cropped from video frmaes in YT-BB. As the labels for the testing data are unavailable, we use the training data as our source domain and the validation data our target domain. Following |an al. (2019). we adopt 50-layer ResNet pre-trained on ImageNet as our basic CNN architecture.
Office-Home Transfer. (Venkateswara al. 2017) consists of 15,500 images in 65 object classes in office and home settings, forming four extremely dissimilar domains: Artistic images (Ar), Clip Art (CI), Product images (Pr), and Real-World images (Rw). By permuting the 4 domains, we can generate 12 different domain adaptation tasks. Our method perform the best in all the transfer tasks compared state-of-the-as art UDA methods. The experiment results and training details for this experiment are in the Appendix A.3 due to space constraint.
c=1 Implementation Details. Following Wan et al. (2018). we assume the covariance matrices {E to be diagonal and the prior probability to be p(c) 1/C² when pre-training the network on the labeled source data. The three trade-off parameters yo, a and / in Eqn. 7 are simply set to be 0.1, 1. 0.1. We strictly follow Pan et al.] (2019) and set the embedding dimension d as 10/512 for Digits/synthetic-to-
image transfer. We implement DMPN with Pytorch. We use ADAM with 0.0005 weight decay and 0.9/0.999 momentum for training and set the mini-batch size to be 128/120 in Digiis/syntheeic-to-r to-rall image transfer. We train the network for 350 epochs for the Digits Image transfer tasks. The learning rate is initially set to be le-5 for the covariance matrices and 1e-3 for the other parameters" and is decayed by 0.1 at epoch 150 and 250. For the synthetictto-real image transfer, we fix the learning rate to be 1e-6 and train the network for 100 epoch #l Finally, for the Digits Image transfer tasks, we apply weighted PDM loss to remedy the labeling error, where each sample is weighted by the maximum predicted class probability. For the synthetic-to-real image transfer task, we apply filtering to remedy the labeling error, where only target examples with maximum predicted probability over 0.8 is used for training. Following the standard, for Digits Image transfer tasks, we adopt the classification accuracy on target domain as evaluation metric and for synthetic-to-real image transfer, we use the average per class accuracy for evaluation metric. We will publish our code upon acceptance.
Compared Methods. To demonstrate the benefits of our proposed method, we compare it with the following approaches: (1) Source-only directly exploits the classification model trained source domain to classify target samples. (2) ReyGrad (Ganin & Lempitsky 2014) trains domain invariant features via adding domain classifier in the deep feature learning pipeline via gradient reversal. (3) DC (Tzeng et al., 2014) minimizes MMD along with deep feature learning for domain invariant features. (4) DAN (Long al. (015) applies multiple variants of MMD align feature repre- sentations from multiple layers. (5) RTN (Long et al. (016)) extends DAN by adapting classifiers through residual transfer module. (6) ADDA (Tzeng et al. 2017) separates the source feature learning and target feature learning using diffferntt networks and use a domain discriminator to learn domain invariant features. (7) JAN (Long et al.) 2017) aligns the joint distribution of the network
Foor the transfer + M. we insert hatch normalization layers after each layer in the original architecture (2 conv-layer L.Neet,, as the model fails to converge without them. For fair comparison, we also add batch normalization layers for other compared method.
*I .aarinng rates are chosen based on the validation results from pre-training on laheled source data.
6

---

Under review as a conference paper at ICLR 2020
activation of multiple layers across domains. (8) MCD Saito et al. 2018 employs task-specific de- cision boundaries to align the distributions of source and target domains. (9) CDAN+E (Long et al. 2018) adds conditional adversarial classifier on the deep feature learning pipeline learn domain invariant features. (10) S-En+Mini-aug (French et al. 2018)) modifies the mean teacher variant of temporal ensembling for UDA. (11) TPN (Pan et al.] 2019) the first work to apply PN for UDA. TPN gen. is the variant trained only with general-purpose domain discrepancy loss. (12) DMPN is our proposed method. DMPN m and DMPN PDM are trained only with GCMM loss and PDM loss respectively. (13) Train-on-target is an oracle that trained on labeled target samples.
Table I: Classification accuracy (%) of different methods for (a) Digits Image transfer across MNIST (M), USPS (U) and SVHN (S), and (b) Synthetic-to-rea image transfer on VisDA 2017 dataset. We draw the results directly from the original paper if experimental settings are the same. For Digits Image transfer, results with ** are trained without batch normalization layers (See Section 4.2 for detail). For synthetic-to-red image transfer, only report the mean accuracy due to space constraint. The detailed per class accuracy results are available the Appendix
A.I A.I
(b) Synthetic-to-real image transfer
(a) Digits Image transfer
Method Source-only
Mean Accuracy
Method
M + U
U
M
S + M
55.3
Source-only
75.2
57.1
64.4
RevGrad
58.6
RevGrad
77.1
73.0
73.9
DC
55.5
DC
77.1
73.0
67
DAN
59.8
DAN
80.3
77.8
73.7
RTN
63.8
RTN
82.0
81.2
75.3*
JAN
66.5
ADDA
89.4
90.1
83.6
CDAN+E
70.0
JAN
84.4
83.4
78.4"
MCD
71.9
MCD
90.0
88.5
81.8
S-En+Mini -ug
74.2
TPN
91.3
93,5
90.2
gen
TPN gen
73.6
TPN
92.1
94.1
93.0"
TPN
80.4
DMPNGCMM
94.3
94.6
96.5
DMPNGCMM
80.2
DMPNPDM
91.8
89.1
96.6
DMPNPDM
81.3
DMPN
94.7
94.8
96.8
DMPN
81.4
Train-on-target
97.3
96.7
99.3
Train-on-target
95.8
4.2 EXPERIMENTAL ANALYSIS
Performance Comparison. Table shows the results of all methods for the two tasks. Overall, our proposed method achieves superior results than all the existing methods. For the Digits Image trans- fer tasks, DMPN has improved the accuracy for M - U, U M and - M by 2.6%, 0.7% and 3.8% respectively compared to the second best. We have made great advancement considering the second best accuracy results are already quite high. For the task S - M, due to convergence reasons, have added batch normalization layers to the original CNN architectures. For fair comparison, have re-run some experiments on other methods by adding batch normalization layers to them. For methods whose public code not available, we simply report the accuracy results with the original CNN architecture. For ADDA, adding batch normalization layer has improved its accuracy result from 76.0% 83.6%, which has an increase of 7.6% of accuracy. However, we doubt adding batch normalization layers will have the same effect on TPN, as TPN already has quite high accuracy. Nonetheless, we think our accuracy result of 96.8% on this task will be difficult for the other meth- ods to surpass even with batch normalization layers. For the Synthetic-to-reat image transfer task, we only compare with methods without extensive data augmentations and our method has increased the state-of-the-ar single model mean accuracy by 1.0%. TPN reduces the domain discrepancy via minimizing the pairwise Reproducing Kernel Hilbert Space (RKHS) distances among the cor- responding prototypes of the source-only, target-only and source-target data. In DMPNGCMM minimize the L distance between the corresponding Gaussian component means of the source and target data. The L distance can be viewed the distance in Linear RKHS space. The calculation of our proposed GCMM loss is much simpler than the general purpose loss, yet with explicitly mod- eling of the feature distributions, DMPN has gain of accuracy of 3.0%, 1. %, 6.3% and 6.6%

---

Under review as a conference paper at ICLR 2020
-072
mitakouupa
miiatshuppa
107
uapa-rmmma
ovhn--mmis
uxpaormmnt
pite-omnist
Accuracy
synthetic
meist-Pppp synthaticorral
syothete
360-
oir
olt
020
556
alc
1.6
Number of Epootre
018 0.000 0.1)
Bl 3a-0,a=0.1)
s0-1,8-11
(a)
(b)
(c)
(d)
Figure 2: (a): GCMM loss, PDM loss and Accuracy with the increase of training epochs on VisDA. (b)-(d): Sensitivity analysis on hyper-parameters 3 and yp respectively.
Source Taraet
Source Tardet
Source
Tardet
Source
Tardet
Source Taraet
(a) Epoch:0
(b) Epoch:20
(c) Epoch:50
(d) Epoch:80
(e) Epoch:
Figure 3: (a)-(e): The t-SNE visualizations of features generated by DMPN with the increase of training epochs on VisDA 2017.
on the M + U. M. M and synthetic-to-real image transfer tasks respectively compared with TPN gen. showing that our method is more effective.
Ablation Analysis. In T combining GCMM loss and PDM loss helps increase the accuracy results, showing that the two domain discrepancy losses are compatible to each other. DMPN performs better than or similar almost all other domain adaptation methods and DMPNPDM per- forms better than most of them.
Convergence Analysis. Figure 2 (a) shows the training progress of DMPN. The GCMM loss and PDM loss keep decreasing with more training epochs. The prediction accuracy on the unlabeled target data keeps increasing. And the noise ratio o decreases along the training process, from the initial value of 38.6% decreases to 22.9%, which supports our theoretical analysis in Section 3.5 Figure 3 shows the t-SNE visualizations of the source and target embedded features during training. which shows that target classes are becoming increasingly well discriminated by the source classifier.
Sensitivity Analysis. Figure 2 (b)-(d) shows the sensitivity analysis on the hyper-parameters a, 3 and with the other hyper-parameters fixed. Overall, the experiment results show that we can get similar accuracy results or even better when changing the paramterrreeters in a certain range, demonstrating that method is robust against pprem-rrreettr changes. The sensitivity analysis on the confidence threshold in the Appendix A.2 which shows our method is robust against threshold value.
CONCLUSIONS
In this paper, propose Distribution Matching Prototypical Network (DMPN) for Unsupervised Domain Adaptation (UDA) where we explicitly model and match the deep feature distribution the source and target data Gaussian mixture distributions. Our work fills the gap in UDA where state- of-the-art methods assume the deep feature distributions of the source and target data are unknown when minimizing the discrepancy between them. We propose two new domain discrepancy losses based on the Gaussian mixture distributions of the deep features called Gaussian Component Mean Matching (GCMM) and Pseudo Distribution Matching (PDM). Extensive experiments verify the effectiveness of our proposed method and domain discrepancy losses. Finally, a post-hoe hyper- parameter sensitivity analysis shows that our approach robust w.r.t hyper- parameter changes.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory learning from different domains. Machine learning, 79(1-2):151-175
2010.
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. Journal of machine learning research, 12(Aug):2493-2555 253377772277... 2011.
Geoffrey French, Michal Mackiewicz, and Mark H. Fisher. Self-ensembling for visual domain
adaptation. In ICLR, 2018.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. arXiv preprint 1409. 0090, 2014.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Domain adaptation for large-scale sentiment classification: deep learning approach. In Proceedings of the 28th international conference on machine learning (ICML-/I), pp. 513-520, 2011.
Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2066-2073. IEEE, 2012.
Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. Domain adaptation for object recognition: An unsupervised approach. 201/ international conference on computer vision, pp. 999-1006. IEEE, 2011.
Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. Covariate shift by kernel mean matching. Dataset shift in machine learning, 3(4):, 2009.
Judy Hoffman, Sergio Guadarrama, Eric S Tzeng, Ronghang Hu, Jeff Donahue, Ross Girshick, Trevor Darrell, and Kate Saenko. Lsda: Large scale detection through adaptation. In Advances in Neural Information Processing Systems, pp. 3536-3544, 2014.
Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex J Smola. Cor- recting sample selection bias by unlabeled data. In Advances in neural information processing systems, pp. 601-608, 2007.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-22333 2324, 1998.
Mingsheng Long, Yue Cao, Jianmin Wang. and Michael Jordan, Learning transferable features 2015.
with deep adaptation networks. arXiv preprint
Mingsheng Long. Han Zhu, Jianmin Wang, and Michael Jordan. Unsupervised domain adaptation with residual transfer networks. Advances in Neural Information Processing Systems, pp. 136- 144, 2016.
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael Jordan. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 2208 2217.227.222 JMLR. org, 2017.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems, pp. 1640-1650, 2018.
Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10)) 1359, 2009.
Sinno Jialin Pan, James T Kwok, Qiang Yang, et al. Transfer learning via dimensionality reduction. In AAAI, volume 8, pp. 677-682, 2008.
Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 22(():199-210 2010.

---

Under review as a conference paper at ICLR 2020
Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong- Wah Ngo, and Tao Mei. Transferrable proto- typical networks for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2019.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In European conference on computer vision, pp. 213-226. Springer, 2010.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier dis- crepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pp. 3723-3732, 2018.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pp. 4077-4087, 2017.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European Conference on Computer Vision, pp. 443-450. Springer, 2016.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprini arXiv: 1412.3474, 2014.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2018-5027, 2017.
Weitao Wan, Yuanyi Zhong, Tianpeng Li, and Jiansheng Chen. Rethinking feature distribution for loss functions in image classification. 2018 IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 9117-9126, 2018.
Hong-Ming Yang, Xu- Yao Zhang, Fangying Yin, and Chenglin Liu. Robust classification with convolutional prototype learning. 2018 IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3474-3482, 2018.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? Advances in neural information processing systems, pp. 3320-3322 2014.
Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Plate Platz. Central moment discrepancy (emd) for domain-invarian representation learn- ing. arXiv preprint 7rX1v:1702.08811, 2017."
10

---

Under review as a conference paper at ICLR 2020
A APPENDIX
A.1
DETAILED RESULTS ON SYNTHETIC-TO-REES IMAGE TRANSFER
Table 2: Classification accuracy (%) of different methods for Synthetic-to-real image transfer on
VisDA 2017 dataset
Method
plane
beycl
bus
horse
knife
meycl
person
plant
sktbrd
train
truck
Source-only
70.6
51.8
55.8
68.9
77.9
7.6
93.3
34.5
81.1
27.9
88.6
5.6
55.3
RevGrad
75,9
70.5
65.3
17.3
72.8
38.6
58.0
77.2
72.5
40.4
70.4
44.7
58.6
DC
63.6
38.4
71.2
61.4
71.4
10.9
86.6
43.5
70.2
47.7
79.8
21.6
55.5
DAN
61.7
54.8
77.7
32.2
75.0
80.8
78.3
46.9
66.9
34.5
79.6
29.1
59.8
RTN
79.5
59.6
78.0
82.7
82.0
84.7
54.7
81.6
34.5
74.2
63.8
JAN
92.1
81.4
72.5
81.5
70.5
79.7
44.6
74.2
24.6
66.5
MCD
87.0
60.9
83.7
64.0
88.9
79.6
84.7
76.9
88.6
40.3
83.0
25.8
71.9
TPN am
94.5
86.8
76.8
49.7
92.1
12.5
84.7
75.2
92.1
86.8
84.1
47.4
73.6
TPN
89.2
62,8
71.7
83,5
90.6
24.6
88.8
91.1
89.8
74.7
69.1
36.1
72.7
TPN
93.7
85.1
69.2
81.6
93.5
61.9
89.3
81.4
93.5
81.6
84.5
49.9
80.4
DMPNoCAM
94.1
83.0
73.7
54.7
90.7
89.2
80.5
79.1
91.0
89.9
82.8
53.2
80.2
DMPNFMM
94.8
85.4
80.0
90.9
84.6
78.7
78.6
90.7
92.4
88.8
55.5
81.3
DMPN
94.6
84.9
75.7
88.1
80.6
78.6
91.3
86.7
55.6
81.4
S-En=Mimi-aug
92.9
84.9
71.6
41.2
88.8
92.4
67.5
63.5
84.5
71.8
83.2
48.1
74.2
S-En+Test-auu
96.3
87.9
84.7
55.7
95.9
95.2
88.6
77.4
93.3
92.8
87.5
38.2
82.8
Train-on-aan targett
99.5
91.9
97.3
96.8
98.3
98.5
94.1
96.2
99.0
98.2
97.9
82.3
95.8
A.2 SENSITIVITY ANALYSIS ON CONFIDENCE THRESHOLD
82.0
81.5
Accuracy (%)
80.0
0.8
0.7
0.8
0.9
Confidence Threshold
Figure 4: Sensitivity analysis on confidence threshold.
Fig. 4 shows the sensitivity analysis of method different values of confidence threshold VisDA 2017 dataset. The experiment results show that we can get similar accuracy results or even better when changing the confidence threshold in certain range, demonstrating that our method is changes.
robust against
A.3 OFFICE-HOME TRANSFER
Table presents experiment results of state-of-the-ar UDA methods and our method on Office- Home dataset. Our method gives the best accuracy results in all transfer tasks, showing the effec-
tiveness of our method. this experiment, we train the network for 100 epochs. The learning rate
is initially set to be le-5 for all the parameters and is decayed by 0.1 at epoch 60 and 80. Follow-
ing Long et al. (2018). and / vary in each training epoch and are calculated by I+e
1-e-7P and
I+e -p respectively, where y is set to be the default value 10, p is the training process changing from 0 to 1.
0.1 1-e-2F
11

---

Under review as a conference paper at ICLR 2020
Table 3: Classification accuracy (%) of different methods on Office- Home dataset
Method
Ar-CI
Ar-Pr
Ar-Rw
CI-Ar
Cl-Pr
Cl-Rw
Pr-Ar
Pe-CI
Rw-Rw
Rw-AA
Rw--CI
Rw-Pr
Avg
Souroc-only
34.9
3000
38.0
37.1
11.9
16.2
38.5
3122
60.4
33.9
11.2
39.9
16.1
DAN
43.6
37.0
67.9
45.8
56.5
60.4
44.1)
43.6
67.7
63.1
51.5
74.3
$6.3
JAN
45.9
61.2
68.9
50.4
59.7
61.0
45.8
43.4
70.3
63.9
52.4
36.8
58.3
CDAN-E
70.6
57.6
70.0
70.0
57.4
50.9
77.3
56.7
81.6
65.8
TPN
33.4
71.4
76.1
59.0
70.8
70.5
61.8
54.3
50.7
71.3
38.4
$4.4
67.7
12


Reviews:
[{"review_id": "SygAfV3gqH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper develops a new method for adapting models trained on labeled data from some source domain to unlabeled data in a target domain. The authors accomplish this by adapting a technique from [1] and [2] enforcing that the deep features learned during training approximately follow a Gaussian mixture distribution. With the learned features in this form, the authors ensure domain adaptation by minimizing the discrepancy between the distributions arising from the source and target datasets.\n\nStrengths:\n + The paper's experiments show an improvement in the model's performance relative to past work, utilizing a large number of comparison models.\n + The use of explicit distributional information within the learned representations seems like a good fit for the task at hand, and the authors' experiments back this up.\n\nWeaknesses:\n - The proposed method for unsupervised domain adaptation is very similar to the prototypical networks approach in [3], with the primary difference being a loss term incentivizing a Gaussian mixture distribution over features.\n - While the authors achieve improved performance over [3], the gains in classification accuracy on the target dataset aren't especially huge (~1-3%).\n - The paper is a bit hard to follow, and would be improved by giving a more explicit comparison of the methods used here to past work, especially [1] and [3].\n\n\n[1] Weitao Wan, Yuanyi Zhong, Tianpeng Li, and Jiansheng Chen. Rethinking feature distribution for loss functions in image classification. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9117–9126, 2018.\n\n[2] Hong-Ming Yang, Xu-Yao Zhang, Fangying Yin, and Chenglin Liu. Robust classification with convolutional prototype learning. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3474–3482, 2018.\n\n[3] Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong-Wah Ngo, and Tao Mei. Transferrable prototypical networks for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2239–2247, 2019.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJenFJCaFB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "<Paper summary>\nThe authors proposed Distribution Matching Prototypical Network (DMPN) for unsupervised domain adaptation. DMPN extracts features from the input data and models them as Gaussian mixture distributions. By explicitly modeling the distributions that the features follow, the discrepancy between the distribution of source data and that of target data can be easily evaluated. DMPN is trained by jointly minimizing two kinds of loss, which are classification loss on the source data and domain discrepancy loss that is calculated via the explicit models. Experimental results on two popular benchmark datasets validate the advantage of DMPN over other state-of-the-art methods. \n\n<Review summary>\nThe proposed method seems simple but empirically performs well. The paper is well written and easy to follow, so we can maybe easily implement it. However, I have several concerns mainly about the details and theories of the proposed method, which makes my score a bit lower than the border line. Given clarifications in an author response, I would be willing to increase the score.\n\n<Details>\n* Strength\n + The motivation of using ProtoNet for domain adaptation seems reasonable.\n + The proposed method performs well in the experiments.\n + The paper, especially the experiment section, is well written and easy to follow.\n\n\n* Weakness and concerns\n - Several points on the proposed loss (GCMM and PDM) are not sufficiently discussed.\n  -- Why do we need two kinds of loss? These losses seem to play almost same role. Since PDM loss corresponds to target-side log likelihood regularization term (Eq. (3)), I wonder if we really need GCMM loss. \n  -- Since the authors explicitly model the feature distributions by Gaussian mixtures (GMs), it might be possible to calculate a standard divergence between source and target data distributions by using the parameters of GMs. Compared with such a straightforward approach, the proposed method seems to be ad-hoc and is not theoretically validated. What term of divergence (or distance) does it minimize?\n  -- When a certain class does not appear in pseudo-labeled target data, how can we calculate GCMM loss? (specifically, \\mu^{et}_c)\n  -- Are Eq. (3) and Eq. (6) correct? These are defined as total loss, not average, over each domain. It means that the scale of the coefficients for these terms changes according to the number of training data, but the sensitivity analysis in Fig. 2 does not show such effect.\n  -- Since the proposed losses heavily depend on the pseudo labels on the target data, it should be important to carefully set a proper threshold for the confidence. Is the proposed method sensitive against the change of this threshold? If so, how can we tune it?\n  -- How can we know p(c) in advance?\n\n - The theory shown in 3.5 is not sufficiently validated. \n  -- The authors state ````we minimize the first term through minimizing the domain discrepancy losses,\" but it is not sufficiently supported, because the relationship between the proposed losses and H-delta-H divergence is not clear. \n\n - I am concerned about whether the proposed method works well with harder datasets such as Office-Home dataset, because each class data are modeled by a simple Gaussian distribution in the proposed method. \n\n\n* Minor concerns that do not have an impact on the score\n - Using both f^s_i and F(x^s_i; \\theta) is confusing.\n - Typo in Eq. (7): PMD -> PDM\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJldN2FWqH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces Distribution Matching Prototypical Network (DMPN) for Unsupervised Domain Adaptation (UDA). The proposed method explicitly models the feature distribution as a Gaussian mixture model in both source and target domains. Then the method aligns the target distribution with the source distribution by minimizing losses, which are called Gaussian Component Mean Matching (GCMM) and Pseudo Distribution Matching (PDM).\n\nThis paper should be rejected because (1) the novelty of the main idea is marginal, and (2) the performance gain over the baseline methods is also marginal.\n\nPan et al. already proposed the idea of transferring the knowledge from the source to the target using the prototype of each class. It is required to explain why explicit modeling performs better than implicit modeling of prototypes by theory or practice.\n\nIn table 2, the proposed method seems better than TPN, but in the appendix, by comparing then in each category, the proposed method wins six categories, whereas TPN also wins six categories. Therefore, it is hard to say the proposed DMPN is more effective than another method.\n\nEach prototype is modeled using a mean and a covariance matrix. Why the authors don't use the estimated covariance matrix to measure the distance in eq.5?\n\nBecause the proposed method uses pseudo-labeling for the target domain, it seems that the weights to determine unreliable examples are crucial. The paper should show the sensitivity of ways to determine the weights. What happens if values of 0.1 and 0.9 are changed in (pi-0.1)/0.9 on page 6?", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses the problem of unsupervised domain adaptation and proposes explicit modeling of the source and target feature distributions to aid in cross-domain alignment. 

The reviewers all recommended rejection of this work. Though they all understood the paper’s position of explicit feature distribution modeling, there was a lack of understanding as to why this explicit modeling should be superior to the common implicit modeling done in related literature. As some reviewers raised concern that the empirical performance of the proposed approach was marginally better than competing methods, this experimental evidence alone was not sufficient justification of the explicit modeling. There was also a secondary concern about whether the two proposed loss functions were simultaneously necessary. 

Overall, after reading the reviewers and authors comments, the AC recommends this paper not be accepted. 
Decision: Reject