Rethinking Joint Maximum Mean Discrepancy for Visual Domain Adaptation
Authors: Wei Wang
Haifeng Xia
Chao Huang
Zhengming Ding
Cong Wang
Haojie Li
Xiaochun Cao
Venue: NeurIPS 2025 oral ()
Keywords: domain adaptation
JMMD
HSIC
feature discrimination
graph embedding
Primary area: general_machine_learning

Abstract: In domain adaption (DA), joint maximum mean discrepancy (JMMD), as a famous distribution-distance metric, aims to measure joint probability distribution difference between the source domain and target domain, while it is still not fully explored and especially hard to be applied into a subspace-learning framework as its empirical estimation involves a tensor-product operator whose partial derivative is difficult to obtain. To solve this issue, we deduce a concise JMMD based on the Representer theorem that avoids the tensor-product operator and obtains two essential findings. First, we reveal the uniformity of JMMD by proving that previous marginal, class conditional, and weighted class conditional probability distribution distances are three special cases of JMMD with different label reproducing kernels. Second, inspired by graph embedding, we observe that the similarity weights, which strengthen the intra-class compactness in the graph of Hilbert Schmidt independence criterion (HSIC), take opposite signs in the graph of JMMD, revealing why JMMD degrades the feature discrimination. This motivates us to propose a novel loss JMMD-HSIC by jointly considering JMMD and HSIC to promote discrimination of JMMD. Extensive experiments on several cross-domain datasets could demonstrate the validity of our revealed theoretical results and the effectiveness of our proposed JMMD-HSIC.

Rethinking Joint Maximum Mean Discrepancy for Domain Adaptation
Wei Wang' Haifeng Xia' Chao Huang!*
Zhengming Ding²
Cong Wang' Haojic Li Xiaochun Cao
Shenzhen Campus of Sun Yat-sen University DDeaatment of Computer Science, Tulane University University of California, San Francisco "Shandong University of Science and Technology
(wangwei29, xiahf5, huangch253, caoxi (aooxiioccunnmmaaa mammil.yyuu........ edu. en zding1 @tulane..eeeeee.eee .dd supercong440gaiil. com hjliRadust.
Abstract
In domain adaption (DA), joint maximum mean discrepancy (JMMD), as a fa- mous distributionddistance metric, aims to measure joint probability distribution difference between the source domain and target domain, while it is still not fully explored and especially hard to be applied into subspace-learning framework as its empirical estimation involves tensor- prrsouttrooucttto operator whose partial deriva- tive is difficult to obtain. To solve this issue, we deduce concise JMMD based on the Representer theorem that avoids the tensor-product operator and obtains two essential findings. First, we reveal the uniformity JMMD by proving that previous marginal, class conditional, and weighted class conditional probability distribution distances are three special cases of JMMD with different label re- producing kernels. Second, inspired by graph embedding, similarity weights, which strengthen the intra-class compactness in the graph Hilbert Schmidt independence criterion (HSIC), take opposite signs in the graph of JMMD, revealing why JMMD degrades the feature discrimination. This motivates to propose novel loss JMMD-HS HICC by jointly considering JMMD and HSIC promote discrimination of JMMD. Extensive experiments on several cross-domain datasets could demonstrate the validity of our revealed theoretical results and the effectiveness of our proposed IMMD-HSIC.
observe that the
Introduction
Domain adaptation (DA) has emerged as an effective technology solve the well-known problem of domain discrepancy that frequently occurs in reality [15)]. Many promising approaches have been suggested mitigate this issue from different perspectives 1 A major issue in DA how to formulate favorable probability distribution distance that can be applied to measure the proximity of two different probability distributions, thus numerous probability distribution-distanne metrics have been proposed over the years. For example, the Quadratic and Bullback--eeible distances derived from the Bregman divergence and generated by different convex functions are introduced to match two different probability distributions explicitly (1] However, extending them into different models may be inflexible since they are parametric and require an intermediate density estimation [12]. The Wasserstein distance derived from the optimal transport problem exploits transportation plan to align two different marginal [33, class conditional [14] or joint [15] probability distributions, but it might be inconvenient to be applied into subspace-learning framework because it often comes
down to a complex bi-level optimization problem [6
*Coresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

JMMD
domain
target domain
(100)) 1 w0rr
TTAA0) @ verr)
space
label
tr( (1eM ( M))())))))
A
category
feature space
KM K /K"
graph embedding with JMMD
dot-prodiact
tensor-product
JMMD-HSIC subspace-learning
((")
((Y)
KT
O L (K ())
T' & AOM
graph embedding with HSIC
label space
HSIC
draw close push away
common feature
(b)
Figure 1: The overview of our revealed theoretical results and proposed JMMD-HSIC. (a) We map the features (upper part) and labels (lower part) of the source and target domains into the RKHS, respectively; (b) The application of JMMD (upper part) and HSIC (lower part) in a subspace-learning framework; (c) The graph embedding interpretation JMMD (upper part) and HSIC (lower part) in a yubspace-learning domains after subspace learning. '_' the module JMMD means the feature-label dependence difference, and '+' in the module of HSIC means separately considering feature-label dependence the two domains. to and are feature and label mappings for RKHS. and B are projection matrices for projected RKHS. (X) @ o(Y) is censor-produce operator for the covariance. B) is the deduced concise JMMD/HSIC in projected RKHS.
framework; (d) Learned feature representations of the source and target
tr(B K XX (KYY @ M ) xXXB)
Gretton el al. propose metric of maximum mean discrepancy (MMD), which empirically estimates the distance between two probability distributions reproducing kernel Hilbert space (RKHS)
[17]. Due to its simplicity and solid theoretical foundation, it has been applied into wide range
of problems, such as deep generative models [18] and variational autoencoders [19] etc. Although MMD could establish marginal [12], class conditional (20), and weighted class conditional (21) probability distribution distances, it is still not fully explored about the joint probability distribution distance (i.e., joint maximum mean discrepancy. JMMD) and is hard to be applied into subspace- learning framework, as its empirical estimation involves a tensor- proouutt operator whose partial derivative is difficult to obtain.
Specifically, when JMMD is considered in subspace-learning framework, we project data features and their corresponding labels into RKHS (Fig- [a)) and aim to exploit a projected RKHS by feature projection matrix T with infinite dimensions (the upper part of Fig. [bb). In the projected RKHS the common feature space, the fcature-label dependence difference between the source domain and target domain is minimized, and their joint probability distributions [22) are aligned accordingly (Fig. dd). Here, the covariance that involves tensor-product operator describes the feature-label dependence. However, one issue remains to be overcome in this process: the partial derivative with respect to the
T is hard to obtain. In this paper, based the Representer theorem 233, we deduce a concise JMMD that avoids the tensor produut operator and optimizes a finite-dimensional B instead of T, where the Representer theorem represents T (x) by finite values of kemel function with corresponding coefficients from B, and some matrix operation properties make the tensor- tensor-product disappear. Therefore, the derivative can be easily obtained.
With the concise JMMD, we obtain two essential findings. Firstly, reveal the uniformity of JMMD by proving that previous popular marginal, class conditional, and weighted class conditional probability distribution distances are its three special cases with label kernels K' K² and K3 and we also prove that they are reproducing kernels. This finding could provide theoretical guidance to refine JMMD by designing label kernels for different problems in DA. Moreover, recent work indicates that the procedure of distribution alignment degrades the feature discrimination unexpectedly 24 25

---

but lacks theoretical support. To reveal this, similar to JMMD, we deduce a concise Hilbert Schmidt independence criterion (HSIC) (26., which models the feature-label dependence and maximizes the dependence to improve feature discrimination (the features have better dixcrimination ability if the feature-label dependence larger and vice versa) (the lower part of Fig. [b). We design label reproducing kernel K for HSIC to strengthen the intra-class compactness or feature discrimination. Then, we explore the relationship between JMMD and HSIC inspired by the graph embedding viewpoint to better understand the reason for feature discrimination degradation in JMMD.
Specifically, we observe that the similarity weights, which strengthen intra-class compactness in the graph of HSIC, take opposite signs in the graph of JMMD. Thus, JMMD degrades the discrimination unexpectedly. As shown in Fig. [c), in the feature space, JMMD tries to push data points from the same classes in the same domain further and draw those from the same classes in different domains closer. In contrast, in the feature space, HSIC aims to draw data points from the same classes in the same domain closer (intra-class compactness). To this end, we propose a novel loss JMMD-HSIC by jointly considering JMMD and HSIC. Therefore, we can improve the discrimination of JMMD, leading to better DA capacity (Fig. [d).. The whole pipeline of this paper is briefly depicted in Fig. 1 and our main contributions are summarized below,
Based on the Representer theorem, we deduce a concise JMMD that avoids the tensor- product operator, and the derivative can be easily obtained so that it can be applied into a subspace-learinng framework.
With the concise JMMD, we reveal its uniformity by proving that previous distribution distances are its special cases with different label reproducing kernels. This finding yields theoretical guidance for refining JMMD by designing label kernels for different problems.
To better understand the reason for feature discrimination degradation in JMMD, we reveal the relationship between JMMD and HSIC inspired by graph embedding. Then, novel loss JMMD-HSIC is proposed to promote discrimination of JMMD.
2 Related Work
2.1 Maximum Mean Discrepancy
Gretton al. introduce a probability fistribution-distann metric MMD 17. Along this direction, Pan et al. incorporate MMD into a subspace-learning framework to learn some transfer components [12 across the source domain and target domain. Duan et al. embed MMD into a multiple kernel learning framework to jointly learn kernel function and robust classifier (27) by minimizing the
structural risk functional and the distribution mismatch. Long el al. weightee source instances
irrelevant to target ones to realize more positive knowledge transfer based on MMD (28). Ghifary et al. employ MMD to deal with distribution bias in simple neural network
(29] Tzeng el al. propose
domain confusion loss based on MMD in deep neural network [30]. Long al. present deep
adaptation network where the multi-kernel MMD is applied into all task-specific layers [3I].
To model more complicated probability distribution distances, Long et al. propose a class-wise MMD align class conditional probability distributions 20), which can be conveniently optimized in many subspace-learning frameworks [32 (3]. To deal with an imbalanced dataset, Wang et al. establish a weighted class-wise MMD where class prior probabilities are introduced [2I. Concerning the label distribution shift problem, Yan et al. construct a weighted MMD where source instances are multiplied by special weights (34] Moreover, Wang el al. raise dynamic balanced MMD to quantitatively account for relative importance between the marginal and class conditional probability distribution distances [35]. Deng et al. propose an extended MMD to simultancously minimize the intra-class dispersion and maximize the inter-class compactness [36].
Zhang et al. estimate the joint probability distribution discrepancy based on the Bayesian law and propose a discriminative joint probability distribution discrepancy (37]. In contrast, Long et al. estimate the uncentered feature-label covariance in the source and target domains, and consider the difference in covariance between the two domains as the joint probability distribution discrepancy. Then, they propose JMMD and construct transfer network accordingly [22]. However, JMMD is still not fully explored and is very hard to be applied to subspace-learning framework as it involves tensor- product operator whose partial derivative difficult to obtain. This paper theoretically deduces concise JMMD and obtains two essential findings
3

---

2.2 Hilbert Schmidt Independence Criterion
HSIC measures dependence between two random variables [26]. To preserve important data features, Dorri el al. regard original and transformed data features as two random variables and maximize their dependence during distribution alignment [38] [9]. Yan et al. minimize dependence between projected features and domain features (e.g., background information) to learn a robust domain- invariant subspace [40]. Inspired by classifier adaptation, Wang et al. propose a projected HSIC to maximize feature-label dependence in reconstruction DA framework [4]] Similarly, we aim to maximize the feature- dependence while we further explore the relationship between JMMD and HSIC, and design a label reproducing kernel K for HSIC in this work. Besides, HSIC is computed on the whole domains
4
while we separately compute HSIC on the source and target domains.
2.3 Graph Embedding
Graph embedding as prevalent technology has been applied to many problems due to its advantage in discovering complicated data structures and relationships, such as subspace-learning methods 442 43 and graph convolutional networks [44] 45 etc. A lot of
-aasdd DA approaches aim to establish a similarity graph and exploit a feature mapping to embed the graph vertices into a desirable low-dimensional subspace so that some important data structures (local manifold structure [35], local discrimination structure [24], etc.) could be respected elegantly during the distribution alignment process. This paper does not focus on leveraging the graph embedding technique in the DA problem. Instead, we analyze the relationship between JMMD and HSIC inspired by graph embedding to illustrate the reason for feature discrimination degradation in JMMD.
Rethinking JMMD
In this paper, the bold uppercase letter X denotes matrix, while the bold lowercase letter x is column vector of X. Moreover, x is the i-th column vector of X and Xij is the value of i-th row and j-th column of X. @ and @ are the tensor-product and dot- dot-product operators.
Domain Adaptation. Given a labeled source domain 2 := "X,,") and an unlabeled target domain Y"), where Xt E Im y E RCx=". 'm' is the feature dimension, "n is the number
D' :=== (X', of source/target instances (n² + n¹ n) and 'C' the number of shared categories. DA assumes that the two domains follow different joint probability distributions, Le., Ps(X", Y²) & P'(X", Y"), but share the same feature and label spaces, and Y is not available during the training process. Y is the probability soft label, i.e., the probability of a data sample x; that belongs to a class 'c' is Yci- Notably, the hard label (one-hot label) is a special case of Y where Yei =1 and Ya =0 (C *c) if x belongs to the e-th class. DA aims to design distribution-disture between their joint probability distributions so that the classifier trained on a source domain can be generalized into another target domain.
metric minimize the divergence
Reproducing Kernel Hilbert Space. RKHS is a Hilbert space (?() of function f - on domain Ω and its inner-product and Hilbert-Schmidt norm are (;-) and || ||H-- The evaluation functional fx) can be reproduced by reproducing kernel function k(x, x x,, i.e. (f(-), k(x, :)) = fx). and the RKHS takes its name from this so-called reproducing kernel function. le k(x,)) can be viewed implicit mapping (xx (infinite dimensions) where 1((, xT) (x)(x, #6(x 1/H-
3.1 A Concise JMMD
In DA. JMMD as a famous probability cnniiibutiondiistanne metric is still not fully explored and hard to be applied into a subspace-lcarning framework as its empirical estimation involves a tensor-product operator whose partial derivative is nontrivial to obtain [2). In this section, we first deduce concise JMMD and reveal its uniformity. Then, we deduce a concise HSIC to explore the relationship between JMMD and HSIC inspired by graph embedding, revealing why JMMD degrades the feature discrimination. Finally, we propose a novel loss JMMD-HSIE to improve the discrimination of JMMD and apply it into a general subspace-learning framework.
Given a domain D sampled from a joint probability distribution P(X, Y) 46 we project the data feature and its corresponding label y into RKHS, i.c., (x(x and Φ(y), where &/ and are the

---

feature and label mappings. Then. we utilize the uncentered covariance between the feature and label to represent a joint probability distribution, i.e., Cxy (x)( @ o(y)). Here, we estimate the covariance Cxy of domain 'D with the expectation Ex or mean μXY of all samples' covariance (x) @ ((y) in a RKHS. The covariance essentially describes the dependence between feature and label. Due to the impossibility of obtaining all possible samples in domain 'D (an infinite number), JMMD [22] adopts the maximum likelihood estimate principle and utilizes finite samples to empirically estimate μ and pxy: for source domain and target domain, and then minimizes the loss of I jixyy= Il to draw joint probability distributions the two domains closer. Formally, JMMD and its concise form in RKHS is defined as below,
(H Y"), P'(X, Y")) ||xx /xx 1 tn(K (K (MM)),
where KXX KYY Raxa are the kernel matrices and they are computed by kij k* (x,, x ky 1 y, Y- and K- are feature and label kernels. M' R is the MMD matrix for JMMD. Remarkably, the nonlinear functions & and & do not need to be explicit, and the tensor- product operator disappears, more details could be found in Section A of the supplementary material. To incorporate JMMD into a subspace-learning framework, deduce the concise form of JMMD in a projected RKHS as below,
1 EE (
& o()))) - ! EE ((x)) & (())) = tr(B K KY O B)
(2)
As shown in the left side of Eq. (2) the dimension of feature projection matrix T E RRoxd is infinite since 1o is an infinite mapping. and a tensor-pront tensor-product operator involved so that is nontrivial to obtain the partial derivative with respect to T. To overcome this issue, we utilize the Representer theorem and some matrix operation properties to obtaín the right side of Eq. 2 Remarkably, the infinitediimensional T does not need to be optimized because we resort to optimize finite- dimensional B, and the censor-produs operator disappears. Therefore, it is easy to obtain the partial derivative with respect to B, and JMMD could be applied into subspace-learingg framework. More details about the proof of Eq. can be found in Section B of the supplementary material.
3.2 The Uniformity of JMMD
In this section, we introduce theorem to illustrate that JMMD is a unified form of existing popular marginal, class conditional, and weighted class conditional probability distribution distances.
Theorem The marginal, class conditional, and weighted class conditional probability distribution distances are three special cases of JMMD with label reproducing kernels K K2 and K*, and more details about these three distances could be found in Section C of the supplementary material. K 1 Ixx is a matrix whose elements are all with size of n n, and K², K are defined as below,
(n'"")/("''n'',,
X DRE, ,x D&E
1, E DE DMc
(n'n))/("''n'),
x E Die, ,x E Die
1, E DE,,,, DLe
kij
(n'n")/(">'''')
X; E D&e, E Die
kij
1. E Dse,x, E Die
(n'
E Die, ,xx Dea
1. E D&s
0, otherwise,
0, otherwise,
where the superscript 's/t." denotes data points from the e-th class in the source/target domain. The proof of this theorem could be found in Section D of the supplementary material, and we also prove that K21 K2 and K are the reproducing kernels in Section E of the supplementary material. Notably, Theorem M yields theoretical guidance to refine JMMD by designing more delicate label kernels for different problems in DA, and we will leave this open direction in our future work.
5

---

3.3 A Concise HSIC
HSIC [26] also utilizes the covariance to establish the feature-label dependence, and aims to maximize the dependence for given domain to improve its feature discrimination. A problem is that the domain- specific dependence may be decreased or the feature discrimination is degraded unexpectedly when minimizing JMMD. In the next section, we will illustrate the reason for discrimination degradation in JMMD from the graph embedding viewpoint. Following JMMD, we utilize finite samples to empirically estimate Croy- and Cxyy, and maximize these two terms separately. Similarly, HSIC involves tensor- producct operator whose derivative is hard to obtain, thus we deduct concise HSIC in an RKHS and concise HSIC in projected RKHS as follows,
M")),
tr(B K KY O K
(4)
Notably, the concise HSIC consistent with JMMD. Thus, it is easy to jointly consider them in concise form, which will be introduced in 3.5 Besides, we design a label kernel for HSIC as below,
(-n'n')/(u"''***, X, E D i
()))(n(e-1))((n33 (n) N E D5o,i=)
kg
(-nb))/(n"''**),
X E D }j (n'n'(n'< - -1))
(5)
X, D i=j
0, otherwise.
There are some advantages with k*: 1) (4) will be a minimization problem which can be easily analyzed from the graph embedding viewpoint: 2) In 3.4 we will illustrate that the intra-class compactness (diserimination) of source domain and target domain can be improved with label kernel K 3) K is a reproducing kernel which is proved in Section E of the supplementary material.
3.4 A Graph Embedding Viewpoint
In this section, we reveal that JMMD degrades the feature discrimination from the graph embedding viewpoint. Given a data matrix X = [x,,... Xn). we establish a nearest neighbor graph G with vertices, where each vertex denotes a data point. Let W be the weight matrix of G, and Wij measures the similarity weight between and xj in original feature space (the larger Wij is and the closer they are and vice versa). Graph embedding technique 42 tries to find a desirable feature representation of X so that it could respect the relationship between each two data points in the original feature space. Formally, aims to minimize the following objective function,
E Ei (wyll 2 zy 21 tr ZZLZ
(6)
where Z is the embedding representation of X. L Q W is the graph Laplacian matrix where Q diag(g, (q .4) is diagonal matrix and q; = - Wi. Inspired by graph embedding, we
regard L' = K2 & M and L" = Kd @ M as two graph Laplacian matrices. Then, we reveal the similarity weight matrices W and will of JMMD and HSIC as follows,
-1/(n8on50),
x E Dee x E Des
-1/(n'ento,
x E Die X E Die
1/(n>">ee) Xi E D Da 1/(nt"n'2) x E Dic, x E Die
w/
1/(n59nt9),
x; E De , E Die
1/(n'119), x E DIC, x E Dre
0, otherwise.
0, otherwise,

---

Then, the concise JMMD 2 and HSIC
in projected RKHS can be rewritten as follows,
EL Dj
B KXX
B B 1P) E
B k
B
B
(8)
where TK k is the embedded feature representation and similar to 6 Specifically, the distance between B kxxx and B xxx should be closer if wi > O but further if Wi since the goal is minimize (8) From 7 observe that JMMD will push two data points from the same classes in the same domain further. and draw those from the same classes different domains closer. HSIC will draw data points from the same classes in the same domain closer (intra-class compactness). These observations illustrate that JMMD degrades feature discrimination as the similarity weights which strengthen intra-class compactness in the graph of HSIC, take opposite signs in the graph of minimization problem.
JMMD. Besides, the designed K makes 4
3.5
The Proposed JMMD-HSIC
From the above analysis, we consider JMMD and HSIC by jointly minimizing 2 and (4) to propose JMMD-HSIC. With little abuse of notations, we denote K' and K" for label kernels of JMMD and HSIC, then IMMD-HSEE in an RKHS and projected RKHS are finalized as follows,
tr(K XX K C
M' + 8K" ))), tr K
K O
M' + +OK" O "MMM K
where 8 aims to balance the relative importance between JMMD and HSIC, and we can adopt K' K² or K3 for K' and K for K.. To validate our revealed theoretical results and the effectiveness of JMMD-HSIC. we incorporate into a general subspace-learning framework, i.e. arg min (*(B) L(B). where L (B) is the JMMD-HSIC loss in projected RKHS, and its partial derivative with
respect to B is 2K XX (K' ② M + sk" O M XXB. B. C(B) is general form for other losses.
4 Experiments
4.1 Datasets and Experimental Settings
To validate revealed theoretical results and the effectiveness of JMMD-HSIC, conduct extensive experiments on four benchmark datasets in cross-domain object recognition. D Office Caltech10 [47] consists of four domains, i.e., Amazon, Dslr, Webcam, Caltech; D²: ImageCLEF-DA includes three domains, Le.. Caltech-256, ImageNet ILSVRC 2012, Pascal VOC 2012; D Office-31 48 contains three domains, i.e., Amazon, Dslr, Webcam; D Office-Home 40 has four domains,
ie., Art, Clipart, Product, Real-world.
As this paper mainly focuses on the problem that JMMD is hard to be applied into subspace- learning framework, we incorporate the proposed JMMD-HSIC into three subspace-learning-base DA approaches. i.e., joint distribution adaptation (JDA) (20] selective pseudo-labeling (SPL) (50), and optimal graph learning- baasnd label propagation (OGL [5I]. We abbreviate these three variants IDA+MMD-HSICC
respectively. For fair comparison, all syper-parameters remain consistent with the three approaches. Regarding 8, we uniformly set it to 0.5 for JDA+JMMD-HSS datasets for the other two variants after trials. On Officel0-Calteche with 800 dimensions 477 and the DECAF-6 features with 4096 dimensions [52]. On the other three datasets, we utilize the ResNer-50 features with 2048 dimensions (53). Moreover, we adopt K2 for JMMD due to its superiority and K for HSIC.
SPL+JMMD-HSIC. and OGL WGLLP+JMMD-HSS
while assigning different values on the corresponding
we use the SURF features
4.2 Results
We compare our proposed approach with existing state-of -fee--fftthhtteee art shallow (SPL [50] PGCD 54
RMMD (55) and deep DA approaches (BSP+MetaReg (56), DRDA (57. RSDA-MSTE AMMTN 58
our proposed approach better than the baseline methods SPL and OGI,2P on average, and has achieved 1.4% and 0.9/0.8% improvements the two datasets, respectively. Besides, OGL 2 P+JMMD-HSII could
Deffreys-DD (59] OGI 3p (51) on D and D As can be seen from Tabs. 1 and 2
7

---

Table I: Comparison results on Office-31 with ResNet-50 features. A, D, W in the second row denotes domains of Amazon, Dslr, and Webcam, respectively.
Source
Amazon
Dsir
Webcam
Venue
Avg.
Target
W
A
W
PGCD S4
TIP'23
95.2
94.0
76.4
99.0
76.5
100.0
90.2
RMMD-I 55
TNNLS'23
90.4
88.4
74.1
98.7
74.8
99.8
87.7
BSP+MetaReg [56]
23
96.2
95.2
76.8
99.2
74.6
100.0
90.3
DRDA [57]
TIP'23
94.5
95.8
75.6
98.8
76.6
100.0
90.2
RSDA-MSTN [58]
TPAMI'24
96.1
95.9
77.8
99.3
78.2
100.0
91.2
Jeffreys-DD [59
NeurlPS 24
95.9
94.9
76.0
99.1
74.6
100.0
90.1
SPL [50]
20
93.0
92.7
76.4
98.7
76.8
99.8
89.6
SPL+JMMD-HSIC
95.8
95.5
78.5
99.1
77.0
100.0
91.0
OGL [S
TIP'25
96.2
95.5
77.5
98.7
76.8
99.4
90.7
IMM---IIMI
96.5
95.8
78.8
99.3
78.8
100.0
91.5
Table 2: Comparison results on Office-Home with ResNet-50 features. A, C, R in the second row denotes domains of Artistic, Clipart, Product, and Real-War Geal-World, respectively.
Source
Artistic
Clipart
Product
Real World
Venue
Avg.
Target
A
R
C
R
C
PGCD [54]
TIP'23
57.7 71.
68.8
RMMD-1 55
TNNLS'
79.3 784477779336..67.887773070
69.5
BSP+MetaReg
TKDE'23
7..07...7..66..7777733
71.3
DRDA [57]
TIP'23
RSDA-MSTN 58
TPAMI'24
Jeffreys-DD [59]
24
SPL [50]
AAAI'20
77.8
78.0 1...088..66..53.1822.888808
SPL+JMMD-HSIC
77.1
85.8
OGL [SII
TIP'25
57.8
78.8 82.
68
68. 68. 56.6
81.6
80.4
56.6
85.0
72.8
OGL'P+JMMD-HSIC
58.3
79.6996666686 82.5
69.3
81 80.9 69 57. 83.3 73.4 61.2
85.3 73.6
achieve the best average results among all compared approaches, which has achieved 0.3% and 0.5%
improvements compared with the second-best methods, i.e., RSDA-MSTN, The comparison results on the other datasets could be found Section F of the supplementary material. Generally speaking, these results can show the effectiveness and competitiveness of our proposed JMMD-HSIC.
4.3 Feature Visualization
To further show the results of JMMD-HSIC we visualize the feature distributions using the t-SNE algorithm common practice in this field [31 [22] Fig. shows the related results for JMMD, HSIC, and JMMD-HSIC in the SPL framework. The better the matching of points with the same color but different shapes, the smaller the distribution difference; disregarding shape, the tighter the clustering of points with the same color, the better the discriminability. As illustrated in Fig. 2aa), the original features perform badly both the distribution alignment and discrimination. From Fig. 2bb), JMMD tries to align the feature distributions of the source domain and target domain, but damages the discrimination greatly, As depicted in Fig. 2 c). HSIC aims to enhance the discriminative structure in both the source and target domains. However, it exhibits poor distribution alignment, as highlighted by the dashed boxes where points of the same color but different shapes are distributed in different regions. As shown in Fig. d)) the feature distributions are matched better compared to HSIC (as highlighted by the dashed boxes where points of the same color different shapes are distributed in the same region) and data points from the same classes tend to be closer (better discrimination) compared to JMMD. These observations could validate our revealed theoretical results and the effectiveness of JMMD-HSIC compared to JMMD and HSIC.

---

Table 3: Ablation study with different losses on Office-31 (average accuracy on tasks) and Office- Home (average accuracy on 12 tasks) with ResNet-50 features.
SPL.
OGL
Dataset
SPL
P
JMMD
HSIC
JMMD-HSIC
JMMD
HSIC
JMMD-HSIC
Office-31
89.6
87.3
89.9
91.0
90.7
88.5
90.6
91.5
Office-Home
71.0
68.7
69.0
71.9
72.8
71.3
71.5
73.6
Table 4: Ablation study with different label kernels Office-31 (average accuracy 6 tasks) and Office-Homm (average accuracy on 12 tasks) with ResNet-50 features.
SPL+JMMD-HSIC
OGL POMLL+++JMMHHHN
Dataset
SPL
OGL'P
K-
K
K
K
K
K
Office-31
89.6
85.6
91.0
90.9
90.7
86.7
91.5
91.5
Office-Home
71.0
64.9
71.9
71.8
72.8
69.8
73.6
73.6
4.4
Ablation Study
We further validate our revealed theoretical results by inspecting the feature distribution distance (JMMD metric) and feature-label independence. The smaller the JMMD metric, the better the distribution alignment; the smaller the feature-label independence. the better the discriminability. We run the methods of PCA, JDA+JMMD, JDA+HSIC and JDA+JMMD-HSICC on the dataset of D with SURF features and utilize four different classifiers (I (I-nearest neighbor (I-NN). SVM, label propagation (L.) (60 and nearest class prototype (NCP) (50) and two different label forms (hard and soft). We report average results of the two metrics on all DA tasks. Then, we compute these two metrics of each method on their embedded feature representations. Note that, in order to compute the true distance or metric, we have use the ground-truth labels instead of the pseudo ones. However, the ground-truth target labels are only used for verification, not for training procedure [20]. As shown in Fig. 3 we could obtain the following observations. With JMMD, the JMMD metric is the smallest among the four methods but the feature-label independence is the largest among them, which indicates good distribution alignment but compromises discriminative structure. Conversely, with HSIC, the feature-labe independence the smallest among the four methods but the JMMD metric the largest among them, which suggests consideration of discriminative structure but overlooking distribution alignment. In contrast, the proposed JMMD-HSIC strikes a good balance between distribution alignment and discriminability, which could lead better DA capacity. Moreover, PCA performs poorly in both aspects. In Tab. 3] we conduct ablation experiments using different loss funetions on baselines of SPL and OGL"p "P and have the following observations. Considering that the classifiers employed in SPL and OGL frameworks require higher feature discriminability. can observe that HSIC outperforms JMMD. Since the Office-31 dataset has smaller distribution discrepancies, the performance improvement of HSIC over JMMD is more significant on the Office- 31 dataset (2.6%, 2.1%) compared the Hfice-Home dataset (0.3%, 0.2%) Our proposed loss function achieves optimal results. Although both SPL and OGL also incorporate loss functions for feature distribution alignment and discriminative feature learning, our approach provides more precise balancing between these two objectives by fundamentally analyzing how JMMD compromises feature discriminability. In other words, we more effectively mitigate the impact of distribution alignment on feature discriminability. Consequently, our method achieves superior performance compared to SPL and *GL These observations could validate our revealed theoretical results and the effectiveness of proposed JMMD-HSIC compared to JMMD and HSIC.
In Tab. we conduct ablation experiments to validate why we adopt the label kernel K² It can be observed that the marginal distribution kernel K3, which neglects label information, leads to inaccurate distribution alignment and thus performs the worst. The results of the weighted class- conditional distribution kernel K3 and the conditional distribution kemel K² are nearly identical because the class imbalance issue in the experimental dataset used in this paper not severe-wherean the weighted cass-condiiional distribution kernel is specifically designed to address class imbalance.

---

original features
SPL-JMMDD
(0) SPL-BIIC
-MM----M-IICC
Figure 2: Feature visualization of the DA task Amazon (source domain) Webcam (target domain) from Office-31 dataset for SPL+JMMD, SPL+ and SPL+JMMDD-HSIC Different colors represent various classes, and "o" and 'x' represent source domain and target domain, respectively.
440
4.11
(x) Different classifiers and labels
(b) Different classifiers and labels
6
Figure 3: Quantitative analysis for the JMMD (distribution distance, (a)) and HSIC (feature-label
independence, (b)) metrics and sensitivity analysis for d (c).
4.5 Sensitivity of Hyper- Parrme-arr
We conduct the sensitivity analysis of 8 in SPL+JMMD-HSIC to validate that the optimal results could be achieved under stable range. We report average accuracy results of SPL+JMMD-HSIC on D'. D2. and D with deep features. We plot average classification accuracy w.z.., different values of 8 in Fig. c(c), and choose 8 2]. which reflects the importance of HSIC for discrimination reinforcement in JMMD. It can be observed that the average accuracy often achieves its optimal value on wide range for each dataset, which could display the stability of 5. Notably, the average results of D with different values of 5 fluctuate between 68. 76.7%, smaller than those of D ~ D Therefore, do not plot the change for D observe the fluctuation trends more distinctively. The rationale behind selecting 8 within the range of [0.1. 2] is as follows: we are gradually mitigating the negative effects of JMMD on discriminability when 8 E [0.1, 1]. We completely eliminate the negative effects of JMMD, and gradually promote discriminability when d E [1, 2]. Therefore, the model performs better when 8 takes larger values within this range.
5 Conclusions
JMMD is still not fully explored and is especially hard to be applied to subspace-learning framework. To overcome this problem, we deduce a concise JMMD and obtain two essential findings, i.e., the uniformity of JMMD and the reason for feature discrimination degradation in JMMD. To strengthen the discrimination of JMMD, we jointly consider JMMD and HSIC to propose novel loss dubbed as JMMD-HSIC. Comprehensive tests carried out some benchmark datasets could validate revealed theoretical results, and show promising performance with our proposed JMMD-HSI future work, we plan to explore: 1) designing an advanced label kernel to handle more diverse domain adaptation scenarios; 2) extending the framework to more complex visual tasks; 3) designing novel label kernels by systematically analyzing how different classifiers vary in their sensitivity to feature distribution alignment and feature discriminability.
For
10

---

Acknowledgments
This work was supported in part by the the National Key R&D Program of China (Grant No.2022ZD01 19200), National Natural Science Foundation of China (Grant No.62306343, Guangdong Basic and Applied Basic Research Foundation (Grant No.2025A11510111222 No.2025A1515011399) China Postdoctoral Science Foun- Shenzhen Science and Technol- ogy Program (No.20231 121173390000000 2023A008), Shenzhen General Research Project NN0.ICY12004120222554007777
No.62301621, No. 62066402,,4)
dation (Grant No.2025TT80435,
No. 2024M77277)))
References
[1] Ye Wang, Junyang Chen, Mengzhu Wang, Hao Li, Wei Wang, Houcheng Su, Zhihui Lai, Wei Wang, and Zhenghan Chen. A closer look at classifier adversarial domain generalization, In Proceedings of the 31 ACM International Conference Multimedia, pages 280-289, 2023.
[2] Wei Wang, Mengzhu Wang. Xiao Dong, Long Lan, Quannan Zu, Xiang Zhang, and Cong Wang. Class-specific and self-learning local manifold structure for domain adaptation. Pattern Recognition, 142:109654 2023.
[3] Wei Wang, Hanyang Li. Ke Shi, Chao Huang, Yang Cao, Cong Wang, and Xiaochun Cao. Optimal graph learning and nuclear norm maximization for deep cross-domain robust label propagation. In Proceedings of the Thirty-Thii International Joint Conference Artificial Intelligence, pages 1407-1415, 2024.
[4] Wei Wang, Hanyang Li, Cong Wang, Chao Huang, Zhengming Ding, Feiping Nic, and Xiaochun Cao. Deep label propagation with nuclear norm maximization for visual domain adaptation. IEEE Transactions on Image Processing, 34:1 1214-12552 2025.
[5] Wangzi Qi, Wei Wang, Chao Huang. Jie Wen, and Cong Wang. Batch singular value polar- ization and weighted semantic augmentation for universal domain adaptation. In Forty-first International Conference on Machine Learning, 2024.
[6] Yang Zou, Zhiding Yu, B. V. K. Vijaya Kumar, and Jinsong Wang. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the 15th European Conference on Computer Vision, volume 11207, pages 297-313, 2018.
[7] Mehmet Pilanci and Elif Vural. Domain adaptation on graphs by learning aligned graph bases. IEEE Transactions on Knowledge and Data Engineering, 34(2):587-61) 2022.
[8] Yaroslav Ganin and Victor S. Lempitsky. Unsupervised domain adaptation by backpropagation. In Proceedings of the 32nd International Conference on Machine Learning. volume 37, pages
1 1180-188,, 2015.
[9] Léo Gautheron, levgen Redko, and Carole Lartizien. Feature selection for unsupervised domain adaptation using optimal transport. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, volume 11052, pages 759-776, 2018.
[10] Lei Zhang, Shanshan Wang, Guang-Bin Huang, Wangmeng Zuo, Jian Yang, and David Zhang. Manifold criterion guided transfer learning via intermediate domain generation. IEEE Transac- tions on Neural Networks and Learning Systems, (0(12)37759937777 3773, 2019.
[11] Si Si, Dacheng Tao, and Bo Geng. Bregman divergence- regularization for transfer subspace learning. IEEE Transactions on Knowledge and Data Engineering, 22(7):929-942, 2010.
[12) Sinno Jialin Pan, Ivor W. Tsang, James T. Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks and Learning Systems, 29(2):199-210. 1.11.
[13] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamoniy, Optimal transport for domain adaptation. IEEE Transactions on Puttern Analysis and Machine Intelligence, 35(9):1853-1865 2017.
11

---

[14] Alain Rakotomamonjy, Rémi Flamary, Gilles Gasso, M. El Alaya, Maxime Berar, and Nicolas Courty. Optimal transport for conditional domain matching and label shift. Machine Learning. 111(5):1651-1670 2022.
[15] Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal transportation for domain adaptation. In Proceedings of the Annual Conference on Neural Information Processing Systems, pages 3730-3739, 2017.
[16] Rémi Flamary, Marco Cuturi, Nicolas Courty, and Alain Rakotomamonjy. Wasserstein discrim- inant analysis. Machine Learning, (0((1))1123-1945 2018.
[17] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Berhard Schölkopf, and Alexander J. Smola. A kernel method for the cro-sample-problene In Proceedings of the Anmual Conference on Neural Information Processing Systems, pages 513-520, 2006.
[18] Yong Ren, Yucen Luo, and Jun Zhu. Improving generative moment matching networks with distribution partition. In Proceedings of the AAAI Conference Artificial Intelligence, pages 9403-9410, 2021.
[19] Shengjia Zhao, Jiaming Song, and Stefano Ermon. Infovae: Balancing learning and inference variational autoencoders. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 5885-5892, 2019.
[20] Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S. Yu. Transfer feature learning with joint distribution adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pages 2200- 2207, 2013.
[21] Jindong Wang, Yiqiang Chen, Shuji Hao, Wenjie Feng, and Zhiqi Shen. Balanced distribution adaptation for transfer learning. In Proceedings of the IEEE International Conference Data Mining, pages 1129-11 134, 2017.
[22] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pages 2208-2217 2017.
[23] Bernhard Schölkopf and Alexander J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA, 2001.
[24] Wei Wang. Zhihui Wang. Haojie Li, Juan Zhou, and Zhengming Ding. Adaptive local neighbors for transfer discriminative feature learning. In Proceedings of the 24th European Conference on Artificial Intelligence, volume 325, pages 1602, 2020.
[25] Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. Transferability vs. discrim- inability: batch spectral penalization for adversarial domain adaptation. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pages 1081-1090, 2019.
[26] Arthur Gretton, Olivier Bousquet, Alexander J. Smola, and Bernhard Schölkopf. Measuring statistical dependence with hilbert-schmidt norms. Proceedings of the 16th International Conference on Algorithmic Learning Theory, volume 3734, pages 63-77, 2005.
[27] Lixin Duan, Ivor W. Tsang, and Dong Xu. Domain transfer multiple kernel learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3):465-479, 2012.
[28] Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S. Yu. Transfer joint matching for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1410-1417, 2014.
[29] Muhammad Ghifary, W. Bastiaan Kleijn, and Mengjie Zhang. Domain adaptive neural networks for object recognition. In Proceedings of the 13th Pacific Rim International Conference on Artificial Intelligence, volume 8862, pages 998-904,70 2014.
[30] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: maximizing for domain invariance. CoRR, abs/1412.3474, 2014.
12

---

[31] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. Learning transferable features with deep adaptation networks. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pages 97-105, 2015.
[32] Shuang Li, Chi Harold Liu, Limin Su, Binhui Xie, Zhengming Ding, C. L. Philip Chen, and Dapeng Wu. Discriminative transfer feature and label consisteney for cross-domain image classification. IEEE Transactions on Neural Networks and Learning Systems, 3 31(11):4842-45 -456, 2020.
[33] Min Meng, Mengcheng Lan, Jun Yu, and ligang Wu. Coupled knowledge transfer for visual data recognition. IEEE Transactions on Circuits and Systems for Video Technology, 3165):777-11788 2021.
[34] Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, and Wangmeng Zuo. Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 954, 2017.
[35] Jindong Wang, Wenjie Feng. Yiqiang Chen, Han Yu, Meiyu Huang, and Philip S. Yu. Visual domain adaptation with manifold embedded distribution alignment. In Proceedings of the ACM Multimedia, pages 402-410, 2018
[36] Wanxia Deng, Qing Liao, Lingjun Zhao, Deke Guo, Gangyao Kuang, Dewen Hu, and Li Liu. Joint clustering and discriminative feature alignment for unsupervised domain adaptation. IEEE Transactions on Image Processing,
2021.
[37] Wen Zhang and Dongrui Wu. Discriminative joint probability maximum mean discrepancy (DJP- MMD) for domain adaptation. In IEEE International Joint Conference on Neural Networks, pages 1-8, 2020.
[38] Fatemeh Dorri and Ali Ghodsi. Adapting component analysis. In Proceedings of the 12th IEEE International Conference on Data Mining, pages 846-851, 2012.
[39] Fatemeh Dorri and Ali Ghodsi. Minimizing the discrepancy between source and target domains by learning adapting components. Journal of Computer Science and Technology, 29(1): 105-115, 2014.
[40] Ke Yan, Lu Kou, and David Zhang. Learning Nomain-invarian subspace using domain features and independence maximization. IEEE Transactions on Cybernetics, (111):2282299 2018.
[41] Shanshan Wang, Lei Zhang, Wangmeng Zuo, and Bob Zhang. Class-specific reconstruction transfer learning for visual recognition across domains. IEEE Transactions on Image Processing, 29:2424-244 2438, 2020.
[42] Miao Zheng, Jiajun Bu, Chun Chen, Can Wang, Lijun Zhang, Guang Qiu, and Deng Cai. Graph regularized sparse coding for image representation. IEEE Transactions Image Processing, 2377): 2011.
[43] Bin Zhang, Qianyao Qiang, Fei Wang, and Feiping Nie. Flexible niew unsupervised graph embedding. IEEE Transactions Image Processing, 30:4143-4156, 2021.
[44] Xinhong Ma, Tianzhu Zhang, and Changsheng Xu. Gcan: Graph convolutional adversarial network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8266-8276, 2019.
[45] Ming Jin, Heng Chang, Wenwu Zhu, and Somayeh Sojoudi. Power up! robust graph convo- lutional network via graph powering. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8004-8012, 2021.
[46] Charles R. Baker. Joint measures and cross-covariance operators. Transactions of the American Mathematical Society,
1973.
[47] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 2066-2073, 2012.
13

---

[48] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell, Adapting visual category models to new domains. In Proceedings of the 1 11th European Conference on Computer Vision, volume 6314, pages 213-226, 2010.
[49] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition. pages 5385-5394, 2017.
[50] Qian Wang and Toby P. Breckon. Unsupervised domain adaptation via structured predic- tion based selective pseudo-labeling. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6243-6250, 2020.
[51] Wei Wang, Mengzhu Wang, Chao Huang, Cong Wang, Jie Mu, Feiping Nie, and Xiaochun Cao. Optimal graph learning- based label propagation for cross-domain image classification. IEEE Transactions on Image Processing, 34:1529 1544, 2025.
[52] Alex Krizhevsky, llya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In Proceedings of the Annual Conference Neural Information Processing Systems, pages 1106-1114, 2012.
[53] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 770-778, 2016.
[54] Wenxu Wang, Zhencai Shen, Daoliang Li, Ping Zhong, and Yingyi Chen. Probability- -based graph embedding cross-domain and class discriminative feature learning for domain adaptation. IEEE Transactions on Image Processing, 32:72-87, 2023.
(55) Wei Wang, Haojie Li, Zhengming Ding, Feiping Nie, Junyang Chen, Xiao Dong, and Zhihui Wang. Rethinking maximum mean discrepancy for visual domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 34(1):264-277 2766477-72222 2023.
[56] Shuang Li, Wenxuan Ma, Jinming Zhang, Chi Harold Liu, Jian Liang, and Guoren Wang. Meta-reweighted regularization for unsupervised domain adaptation. JEEE Transactions on Knowledge and Data Engineering, 35(3):2781-2777 -22995 2023.
[57] Zenan Huang, Jun Wen, Siheng Chen, Linchao Zhu, and Nenggan Zheng. Discriminative radial domain adaptation. IEEE Transactions on Image Processing, 32:1 12:1419-143,, 2023.
[58] Xiang Gu, Jian Sun, and Zongben Xu. Unsupervised and semi-supervised robust spherical space domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(3):1757-1774, 2024.
[59] Ziqiao Wang and Yongyi Mao. On $fS-divergence principled domain adaptation: An improved framework. In Advances in Neural Information Processing Systems, 2024.
[60] Feiping Nie, Shiming Xiang, Yun Liu, and Changshui Zhang. A general graph-based semi- supervised learning with novel class discovery. Neural Computing and Applications, 19(4):549- 555, 2010.
[61] Xianda Zhang, Matrix Analysis and Applications. Tsinghua University Press, 2013.
[62] Wei Wang, Baopu Li, Mengzhu Wang, Feiping Nie, Zhihui Wang, and Haojie Li. Confidence regularized label propagation based domain adaptation. IEEE Transactions on Circuits and Systems for Video Technology, 22(6):3319-3338 2022.
[63] Shuhao Cui, Shuhui Wang, Junbao Zhuo Liang Li, Qingming Huang, and Qi Tian. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situa- tions. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 3940-3949 2020.
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: This paper aims to fully explore the distribution--isttacee metric, i.e., JMMD, in the field of DA.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [No]
15

---

Justification:
We did not discuss the limitations of this work due to space constraints.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specifiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [No]
Justification: We provided complete proof for the theoretical results in the supplementary file due to space constraints.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provided comprehensive information to facilitate the reproducibility of our work.
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provided the references for the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
stteeeesubmiiiiooooooooooo. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https c/iips.cc/public//uue
Codeeuumiiii wettsbbmmssiiioi ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
We have specified all the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We have conducted sensitivity analysis of hyper-parameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification:
Our experiments do not require significant computational resources.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
18

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and ensured that the paper conforms, in every respect, with the Code.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
19

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- ce-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
20

---

Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: This paper focuses a statistical cittibution-distann metric JMMD in traditional machine learning and does not involve LLM.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our L.M policy http:p//earrrppeeccoooffeeerrree for what should or should be described.
21

---

A A Concise JMMD in an RKHS
Theorem In an RKHS, JMMD could be rewritten as the following concise form,
Dx P'(X,, Y¹), P'(x', x9) = -1 EE (V(x)) @ ((xx))) # E (v(x)) 3 (800)0)))
=In(K>((K*) M²))
(10)
where Dy denotes a distance metric between two joint probability distributions, i.e., P and Pt, We empirically estimate JMMD with the following steps: i) utilize v and é to map features and labels from the source domain and target domain to the RKHS, respectively; ii) we calculate the mean of the tensor product between feature and lahel for each domain; iii) compute the difference between these two means. Notably, x and xj are the i-th and j-th column vectors X.
Proof:
For convenience, we define ["xx,, y*) and I (x(x,,y) y") shown in the following equations,
('x", y*) (x)) @ (vvi), (xx)) o(x }
E Reex""
(11)
[' y²) (xx)) @ o(yl), TM ,(x)))..., 2 x
Rox!
Then, 10 could be rewritten as below,
Du P'(X' Y"), P'(X', x2) = - (V(X)) e d(1)))
#o2 Ej= [M(X)) @ o(x))) 1|2
n
(xx', T (x yy)
yx) (n) ('(,',)))
-oe ment
))
Mexi
min
(12)
where 11x1 and 1 are two column vectors whose elements are all ones with sizes of n² and n'. Moreover, we have the following equations,
( X V''
(13)
r'(''y') x²) R 3 KM
(14)
yt) xxx KYY
(15)
[' yt) ( Tx* 3²) = K x C YYY
(16)
where K x'x by X'X A2x (x)
k
E RRa xn' RB xn'
are the kernel matrices and they are computed
k XY s; T)..... Here x- and k. are feature and label kernels.
Therefore, we can rewrite as below.
122 using the feature kernel matrix XXX and the label kernel matrix YY
22

---

P' )) Pr Pi(X' Y")
D/
xx xxx
K rY
K
kY
Lexcllxx n'n"
-he Ses nin'
(17)
K k
k
K
Kv
Lixc 1 n/n
Lax L win
= tr K M').
where KXX E R and KYY
E Roxa are feature and label kernel matrices for all source and target
domains, and n = n² + n'.
According ur(A A BC) tr A(B O C)) where A, B and C are symmetric matrices 61 thus
tr(K kr(K* M k (K @ MM)) M' calculated below.
1/(n'n'),
X, E D
mj 1/(n'n), X.X E E 11/(n''', otherwise.
(18)
where DD and Dr denote source and target domains.
B A Concise JMMD in Projected RKHS
Theorem In a projected RKHS, JMMD could be rewritten as the following concise form,
Du Ps Y.), P'(X', Y9)
1/f Σn
T xxx) 2 o(x))) Ejz (T" ((x) @ )(x))) l1-4
Ei (brv(x)) (T (xx))) @ o(())) Enn (Ii ( @ xox))) |
ir B K xX
K YY
& M K*))
(19)
where T E Ro is the feature projection matrix and 'd' is the dimension in the embedded subspace. Different from (T),, we project y (xx)) and t (xx) into an embedded subspace, and then empirically estimate JMMD.
Proof:
We begin by introducing the Representer theorem 23 below,
Theorem 4 (Representer theorem) It says that any function can be decomposed into finite values of kernel function with corresponding coefficients 233
T Txx(()= -i= (D,b*(x,xi)
(20)
En (D,(b((())) (b(((()), ++xx)))),,xx))))) = Ein (x)(xx))x(()))))
where b, 1 and definite projection matrix B [bT :b gaxd
23

---

For convenience, we define ('(x",y) and (x('xy') as shown in the following equations according the Representer theorem,
y*(x
[2in bIb(()) (i (xx)) @ @(r)), (Zi bIV(() @ d6 R
(21)
('(x')''==
(ELibr b(x(()) (xx)) @ o(r)), .. , bx((())) (((()) @ 265%)) E R
where b; [Rdx1 and definite projection matrix B :.. ...... b E Roxd (n n² + n²).
Then, (99) could be rewritten as below,
Dx
1 Dm (E
(x) e0 (
2j=1 (E (x) (x() e (cs))
-1(10(0) ('(x',')) ]
le laxi
- Txx nin!
(x(x; y9) O'(xt, 8t) 1)
(x
-tr
M 1 ut'
('x
l 17 nin
l I Lixa whnt
(x(x'))))) ('(x',y)) ]
11x1 1axa nin"
Lexe ntnn
tt 0(xx)))T(xxx))) ('xx"9)TT(x..)
1sxa n'a
-oo wint
)
(xxx,)) (xxt"")
(22)
Similar to the proof of Theorem 2 we rewrite (* ( ( ... using feature and label kernels. First.
(E by (x)) & o(x1), (EL, (x(x))))((X)) @ <(ri))
bebx)) & o(>)), (E by(()) (x+) & <(ri))
( (E bie(())) ((( (x)) @ o(x)), (E T) & d(yi))
(23)
where ( denotes the inner product between two vectors. Moreover, we have the following equation,
24

---

( (Ein brb((i) bros() (x)) o(r)), (Ei- (xi((i) () @ (lym))
((E (x ) (CEi bioi(x)) Job(x)) ((Y))
(v(x)) (E
((En (x)((x)) (x)) @ o(ym))
(w(x))(C ((xx))(Ei>> (E) brybxi (xi) (x)))) s0
(x., xxi) ] T so k- vm)
(x) xi)
(x xn)
BB
k* (x), Xx), (x2, xk), (x,,
[x) x1), x2),
(x xnn
BB
(x)
(x2 xx), (x,. (y) ym)
XX
BB K Ko
k Nm). ym):
(24)
where the subscripts (i, )) and k) denote the i-th row vector and the k-th column vector of a given matrix, respectively. Then, we can obtain the following equation,
K XX (
BB XX
(11,81)
K XX (1.,) BB T XX
k (N1,
xx
(2.o)
BB K xx
(.1)
(y2, F1)
xx
(2,))
BB T K K Nw)
(25)
XX (m),e)
BB (o.1) R- (1
KXX (m)
BB T
Similarly, (6'x
XX
BB xxx
K kv (1o441)844))
XX 1, BB K k (y (ym41.8a)
(,
KXX
BB XXX R. (Y0442)
XXX 22
BB KXX YY (Fm42.80)
(0-+2.))
(1.
(o.n)
(26)
K (n.0]
BB KXX
+1 (yn:
xX BB
XX k (y..
k
(n.0]
(* (x
KXX
BB KXX
K kY (y Na44))
K (1.
BB T kY (y.,
K xx
(2.)
BB
K
xx
XY No441))))
XX BB
KXX K. (y2,
(27)
K
(
XX
BB Xx
Ko YY KY (ynr)
K
XX (..
BB KXX KY Ny)
(e(x,))
25

---

KXX
1,'+ ., ,B (..)) kY (yo
K ot 1.,)
, BB Kom k (1
BB K kY ky (1
K 1 2+
BB K( k. Nn)
(m+22,))
(28)
K
XX n.a]
BB K k Y1)
K
XXX
BB KXX kV Wo Nw)
According to
25
28p. we could obtain the following equation,
0'xx' 0'xx))) ()(xi)) "'xx,")""(''")""
- Aexxa aent
)
D2 = ir (r(
"'x'"))T'xx'") 0'(x', ((xx))))
Laxi Lexa afa
-t(BBKk K " B
(29)
C Probability Distribution Distances
C.1 Maximum Mean Discrepancy
The maximum mean discrepancy (MMD) establishes the mean embedding of the marginal probabilitty distribution RKHS endowed by the kernel Y-x (feature mapping v)), and using finite samples to empirically estimate the distance between μx (mean embedding of source domain) and Mx (mean embedding of target domain) with the Hilbert-Schmide norm the following equation,
De P'(')) ||B [x(x)))))) E3 (w(X') 1/-
11100- ux
2
(30)
Enn ((x)) /| tr Ir(KXX
1| Σ (xx))
where XXX
Iaax and x,xx xxx (xx(xxxx) x)). Besides, the MMD matrix MM can be (X) (X)
computed below,
x, D= 1/(n'n',,
1/(n'n'),
m M
x, D D
(31)
-1/(pfn"),
otherwise.
Moreover, the MMD in a projected RKHS is tr(B XXX XM
C.2 Class- Wise Maximum Mean Discrepancy
The class-wise maximum mean discrepancy (CMMD) constructs the sum of MMD for each specific class the following equation.
26

---

Dz (2(X' P'(X)19)
R (((x)) - IB (x(x -ce=1 1x -xx
(32)
Σ ip
Ei oxx)) पoب 1 =Ee= tn(K**M00
where the MMD matrix c can be computed as below,
1/(n8188.0)
E DEC, Xg E DD
1/(nben)9)
x E Die, Xj E Die
C.c mj
-1/(n)/n)9),
E D Xj E Die E Die, x D
(33)
-1/(n
0, otherwise.
Similarly, the CMMD in projected RKHS is Ee tr K K
C.3 Weighted Class-Wits Maximum Mean Discrepancy
To deal with class imbalanced dataset, the weighted class-wise maximum mean discrepancy (WCMMD) introduces the class prior probability P(Y) into the CMMD (2I), which pays more attention on the large-size categories and is formulated as the following equation,
Σ
p P'(y=zc n²
Ei (xx)
P'ry=c) nl
Σ Σj= (xx)))
1
7
(34)
>
1/2 Σnz (xx) - + Σ 2 (xx)
= - ur(K**MWCS)
7
where WC,e can be computed with the following equation,
1/(n'n"), x x XE D . E Des
1/(n'n'),
x; E Xj E Die
WC,c mij
x E -1/(n'n"),
X Die
(35)
-1/(n'n"), x E Die X E Drs
0, otherwise.
Similarly, the WCMMD in projected RKHS is DC tr(B XXX NCC
D The Uniformity of JMMD
Theorem The marginal, class conditional and weighted class conditional probability distribution distances are three special cases of JMMD with label reproducing kernels K² K2 and K² K² 1 Laxn is matrix whose elements are all I with the size of X and K2, K are defined as below.
27

---

x, EE E xj E Di
(n
Xi X Du, Xj E Die
ki
E E D .x E Die
(36)
(n'n')/n"''"**)
E E Die D
0, otherwise,
x EED E Dee
x REDI, Die
kj
E E Die
(37)
1. M E D
0, otherwise,
where the superscript 's/t,c' denotes data points from the e-th class in the source/target domain. Proof:
As proved before, the formulations of concise JMMD are tr( (KYY ④ M²)) (in a RKHS) and tr(B x @ M K (in projected RKHS). Moreover, the formulations of marginal prob- ability distribution distance are tr(K*XMMM (in a RKHS) and tr(B' T X**MMM<
(in a projected RKHS). The formulations of class conditional probability distribution distance are tr(K Xr(KXXMC.. (in RKHS) and tr(B' K X B) (in projected RKHS). The formulations of weighted class condi- tional probability distribution distance are tr(k** M WC.,) (in a RKHS) and tr(B (in projected RKHS). It is easy to verify that K2 K M' WW,F, Therefore, the marginal, class conditional and weighted class conditional probability distribution distances are three special cases of JMMD with different label reproducing
" M
② M'
MM K" O M' M and
kernels K| K2 and K3, We will prove K!, K² and K3 are the reproducing kernels in next Subsection.
Reproducing Kernels
Theorem Kl, K2, K3 and K are the reproducing kernels, where K¹, K², K3 are defined Theorem
5
Proof:
we only have to prove that the Gram matrices G G² and G corresponding to K' K2 and K are semi cemi-positive definite matrices. In fact, the Gram matrices 22 and G3 can be decomposed into the following equation,
K' K² and k According to the Mercer's theorem 23
G2 G²= GZ.,
G G-=C G...
(38)
is obvious that the sum of several semi-positive definite matrices is also semi-positive definite matrix, thus only have prove that the Gram matrices G G2,e and G3.e are semi-positive definite. G 2 and 3 can be decomposed into the following equations,
G ===pi "T 2 = =2
3.
33.T
(39)
where p² 1, column vector whose elements are all 1, and 2.0 E TR. 33. E R. could be defined as below.
28

---

n² n//n"e,
x; E Di
2,0 Pi
n'/nhe,
x; E Dio
(40)
0, otherwise,
1, X; D&c
Pj
3,0
1, X; Die
(41)
0, otherwise,
where Pj
2/3.c is the value of the i-th component of 2/3,c For Vx E IR" and x of 0, we have,
x G x p'p' T x= 1
(42)
= =x1 +22 +... + xa)²200 0.
x G = x p2..p2..Txx
(43)
(xip) 2.
x2p2 2.c + xnpp 2.02 2 0.
and,
x G p3
(44)
(x1P) 3.c + x2P2 3.0 + x0p3 2 0.
Therefore, G¹. G² and G are semi-positive definite matrices. Then, Kt, K² and K$ are the reproducing kernels.
k²: For Vx Rº and x + 0, due to Wij 2 0, we have,
n
x GG's
Wij(X: x,)² 2 O,
(45)
where G is the Gram matrix of Kd and W defined as below,
(n'"")/(~**"*>))
x E D&C, D ,x E Dse
Wij (n'n)//n''"''')
x E Die E X E Dis
(46)
0, otherwise.
Therefore, G is semi-positive matrix and K" is the reproducing kernel.
Experiments
We run the IDA-MMDDISSCCOUUJMMAD---SS and adopt the classifiers of 1-nearest neighbor (I-NN), support vector machines (SVME. label propagation (LF)) [60] and nearest class prototype
"htps://www.ssic.mtu ntw.siiiiiieieeneoooo www.t//www//////.....ttoc cjlin/libsvm
29

---

Table 5: Ablation study using different classifiers/lablls on the Office dataset with SURF features.
Classifier
1-NN
SVM
LP
NCP
Original Features 40.9
47.7
48.3
45.7
Label
Hard
Hard
Soft
Hard
Soft
Hard
Soft
JDA+JMMD
47.7
50.2
49.2
54.2
52.8
47.8
41,4
JDA+HSIC
47.9
49.0
48.5
54.0
52.8
48.8
46.8
JDA+Our
49.6
50.9
49.9
55.4
54.3
49.6
47.1
Table Comparison average results our proposed SPL+JMMD-HSIC with state-of-the-art DA methods Office 1- dataset with DECAF-6 features. A, C, D, W in the second row denotes domains of Amazon, Caltech, DsIr, and Webcam, respectively.
Source
Amazon
Caltech
Dslr
Webcam
Venue
Avg.
Target
C
D
W
A
W
A
C
A
C
PGCD
54
TIP'23
86.5
90.4
84.1
92.5
92.4
91.2
92.5
87.6 100.0 91.6
85.3
100.0
91.2
RMMD-II 55
TNNLS 23
88.4
91.7
92.9
93.4
96.8
95.9
93.6
88.9
100.0
92.2
88.9
100.0
93.6
SPL 50
AAAI'0
87.4
89.2
95.3
92.7
98.7
93.2
92.9
88.6
98.6
92.0
87.0
100.0
93.0
SPL+Our
90.0
96.8
93.9
93.7
99.4
93.9
93.8
90.3
100.0
93.3
89.4
99.4
94.5
OGL [11
TIP'25
89.7
97.5
91.9
94.3
98.7
95.9
94.2
90.2
99.3
94.6
89.5
100.0
94.6
OGL'P+Our
90.2 97.8 93.2
95.3
99.2
95.7
94.6 90.5 100.0 94.2 89.3
100.0
95.0
Table 7: Comparison of average results our proposed SPL+JMMD- with state-of-the-ar DA methods on ImageCLEF-DA dataset with ResNet-50 features. C, I, in the second row denotes domains haltech-256, ImageNet ILSVRC, and Pascal VOC, respectively.
Caltech
ImageNet
Pascal
Source
Venue
-256
ILSVRC
VOC
Avg.
Target
I
P
C
P
C
RMMD-I 55
TNNLS'23
93.2
78.3
95.7
79.5
95.5
92.0
89.0
RSDA-MSTN 58
TPAMI'24
93.3
79.3
97.8
80.5
96.8
94.2
90.3
SPL [50]
AAAI'0
95.7
80.5
96.7
78.3
96.3
94.5
90.3
SPL+Our
96.3
81.4
96.7
80.5
96.7
95.0
91.1
2GL 51
TIP'25
95.8
81.2
96.8
82.2
97.2
95.7
91.5
OGL2P+Our
96.5
81.8
97.4
83.5
97.8
96.0
92.2
30

---

(NCFF) [50] on the Office-Caltech1 10 dataset with SURF features (average classification results on 12 DA tasks). As can be seen from Tab. []] JMMD and HSIC perform better than the original features as JMMD matches the distributions of the source domain and target domain, and HSIC enhances domain-specific diseriminative structures. The proposed JMMD-HSIC could achieve the best results no matter what classifiers or labels are, which shows the effectiveness of JMMD-HSIC and indicates that it is necessary to jointly consider JMMD and HSIC for better DA capacity. Here, the symbol "Soft' denotes the probability soft label and the symbol 'Hard" is the hard (one-hot) label. Notably, I-NN could not produce a soft label thus only 'Hard' reported. It can be seen that the performance of the 'Soft' label is worse than that of the 'Hard' label, and it may be because the performance heavily depends on the quality of predicted soft labels of the target domain [62
63
We compare our proposed approach with existing state-of of sef-the-and art shallow (SPL [50] PGCD [54] RMMD (551) and deep DA approaches (RSDA-MSTH) (58), OGI (11) on D and D As can be seen from Tabs. 6 and 7 our proposed approach is better than the baseline methods SPL. and OGL'² on average, and has achieved 1. 1.5%/0.4% and 0.8/0.7% improvements on the two datasets, respectively. Besides, all compared approaches, which has achieved 0.4% and 0.7% improvements compared with the second-best methods, i.e., OGL'P. Generally speaking, these results can show the effectiveness and competitiveness of our proposed JMMD-HSIC.
could achieve the best average results among
31


Reviews:
[{"review_id": "vDPTb3aPeD", "reviewer": "Reviewer_5L6J", "summary": "The paper addresses the problem of domain adaptation in which the task is to reduce the discrepancy between the joint probability distribution between the source and target domain. The paper builds upon the joint maximum mean discrepancy (JMMD) metric and discusses its limitations of being applied into a subspace-learning framework as its empirical estimation involves a tensor-product operator whose partial derivative is difficult to obtain. The authors deduce a concise JMMD based on the Representer theorem that avoids the tensor-product operator and obtain two essential findings. They reveal the uniformity of JMMD by proving that previous marginal, class conditional, and weighted class conditional probability distribution distances are three special cases of JMMD with different label reproducing kernels. Moreover the authors also discuss the relationship between JMMD and HSIC based on graph embeddings. Based on these observations, the paper proposes a novel loss JMMD-HSIC by jointly considering JMMD and HSIC to promote discrimination of JMMD. Experiments on four benchmarks are shown.", "questions": "All of my concerns are listed above in the weaknesses section. The paper is well written and explained. The experiments section of the paper can be made stronger.", "limitations": "No. The authors have mentioned that they did not discuss the limitations because of space constraints, but there is no limitation discussed in the supplementary as well.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n* The paper is well written and all the concepts are well explained.\n* The relationship analysis and the joint optimization of JMMD and HSIC is novel.\n* The graph embedding viewpoint discussed in Section 3.4 showing findings like JMMD degrades the feature discrimination is interesting and insightful.\n\n\n\n\n**Weaknesses:**\n* It would be great to have convergence analysis for the loss function - including speed, time, etc.\n* Robustness analysis: are the reported values averaged over multiple runs? If yes/no, what do the standard deviations look like? How robust is the loss function?\n* Ablation study: Although the authors have provided some ablation studies in Figure 3, it would be great to check the values in Table 1 or 2 by individually employing the JMMD and HSIC components.\n* What projection algorithm was used for visualizing the features in Figure 2.\n* Typo: Figure 1 (c) - draw close and push away legends using the same symbol.", "was_revised": true, "final_justification": "The authors have addressed all of my concerns, and I am raising my score. I hope the additional information from the rebuttal is reflected in the final version of the paper.", "rebuttal": "**Weakness 1 (Convergence Analysis for the Proposed Loss Function):** **(a) Differentiable.** Since the proposed loss function is differentiable, the final objective function to be optimized is also convergent as long as the applied subspace-learning-based domain adaptation framework is convergent. **(b) Closed-Form Solution.** In this manuscript, the proposed loss function is applied into JDA [15], SPL [45], and OGL2P [46], and the resulting objective functions to be optimized all admit closed-form solutions, ensuring convergence. **(c) Computational Complexity.** The computational complexity of the proposed loss function is $O(\\text{Cn + Cn}^\\text{2})$, where $\\text{C}$ is the number of classes and $\\text{n}$ is the total number of samples from both the source and target domains. Since $\\text{C}$ is generally much smaller than the sample size $\\text{n}$, the complexity mainly depends on the number of samples. **(d) Time/Speed.** We analyze the time required for the algorithm to compute the proposed loss function (**loss construction**) and optimize the corresponding objective function (**objective optimization**) on two datasets and two subspace-learning-based domain adaptation frameworks (SPL and OGLP2P), with the results recorded in the following two tables. \n\n**(a)** It can be observed that due to the use of MATLAB, which provides highly efficient matrix computation capabilities, the objective optimization and the loss construction exhibit highly efficient runtime performance. **(b)** Since the Office-Home dataset is larger in scale, it requires more time compared to the Office-31 dataset. **(c)** the subspace-learning-based domain adaptation framework applied in this manuscript belongs to a two-stage domain adaptation approach. That is, deep learning is first employed to extract a promising feature representation, followed by further refinement using subspace-learning methods. Since subspace-learning methods entail significantly lower computational overhead compared to deep learning training, they hold substantial research value and have consistently garnered sustained attention from researchers. **(d)** These results not only validate the efficiency of the two-stage domain adaptation approach but also highlight the necessity of addressing the challenges in applying JMMD within subspace-learning frameworks.\n\n| 6 tasks of the Office-31 dataset | Time/Speed (Average (seconds)) |\n| ------------- |:-------------:|\n| SPL (objective optimization)      | 0.357 |\n| SPL (loss construction)     | 0.075 |\n| OGL2P (objective optimization) | 1.330 |\n| OGL2P (loss construction) | 0.085 |\n\n| 12 tasks of the Office-Home dataset | Time/Speed (Average (seconds)) |\n| ------------- |:-------------:|\n| SPL (objective optimization)      | 1.253 |\n| SPL (loss construction)     | 0.303 |\n| OGL2P (objective optimization) | 0.557 |\n| OGL2P (loss construction) | 0.310 |\n\n**Weakness 2 (Robustness Analysis for the Proposed Loss Function):** The results reported in this manuscript **are not averaged over multiple runs**, for the following reasons: The proposed loss function is applied into three shallow subspace-learning-based domain adaptation methods. Since no stochastic processes are involved, the results obtained from each run are deterministic, with a standard deviation of 0. Referring to the original experimental results in these three papers (JDA [15], SPL [45], OGLP [46]), none of them record or mention the standard deviation. \n\nThe loss function we designed is robust, free from stochastic processes, and ensures that the same input will always produce the same output. **In Section 6 of the supplementary material (Lines 116-127) and Figure 3 of the main text (Lines 300-301)**, we conduct ablation studies on Office10-Caltech10 dataset comparing JMMD, HSCI, and our proposed JMMD-HSCI. Different classifiers are employed to validate the robustness of the proposed loss function. Additionally, in our response to your raised **Weakness 6**, we have provided a detailed discussion regarding the limitations of the proposed loss function.\n\n**Weakness 3 (Ablation Study by Individually Employing the JMMD and HSIC Components):** **In Section 6 of the supplementary material (Lines 116-127)**, **ablation experiments comparing JMMD, HSCI and our JMMD-HSCI** are performed on Office10-Caltech10 dataset, with multiple classifiers verifying the loss's robustness. As per your suggestions, we present the experimental results individually using JMMD and HSIC in the following two tables. **(a)** Considering that the classifiers employed in SPL and OGL2P frameworks require higher feature discriminability, we can observe that HSIC outperforms JMMD. **(b)** Since the Office-31 dataset has smaller distribution discrepancies, the performance improvement of HSIC over JMMD is more significant on the Office-31 dataset (2.6%, 2.1%) compared to the Office-Home dataset (0.3%, 0.2%). **(c)** Our proposed loss function achieves optimal results. Although both SPL and OGL2P methods also incorporate loss functions for feature distribution alignment and discriminative feature learning, our approach provides more precise balancing between these two objectives by fundamentally analyzing how JMMD compromises feature discriminability. In other words, we more effectively mitigate the impact of distribution alignment on feature discriminability. Consequently, our method achieves superior performance compared to SPL and OGL2P.\n\n| 6 tasks of the Office-31 dataset | Accuracy (Average (%)) |\n| ------------- |:-------------:|\n| SPL | 89.6 |\n| SPL+JMMD | 87.3 |\n| SPL+HSIC | 89.9 |\n| SPL+JMMD-HSIC | **91.0** |\n| OGL2P | 90.7 |\n| OGL2P+JMMD | 88.5 |\n| OGL2P+HSIC | 90.6 |\n| OGL2P+JMMD-HSIC | **91.5** |\n\n| 12 tasks of the Office-Home dataset | Accuracy (Average (%)) |\n| ------------- |:-------------:|\n| SPL | 71.0 |\n| SPL+JMMD | 68.7 |\n| SPL+HSIC | 69.0 |\n| SPL+JMMD-HSIC | **71.9** |\n| OGL2P | 72.8 |\n| OGL2P+JMMD | 71.3 |\n| OGL2P+HSIC | 71.5 |\n| OGL2P+JMMD-HSIC | **73.6** |\n\n**Weakness 4 (Projection Algorithm for Feature Visualization in Figure 2 (Lines 300-301)):** In Figure 2, we employ the **t-SNE** algorithm for feature visualization, which has been widely adopted in domain adaptation field [45,46].\n\n**Weakness 5 (Typo in Figure 1 (c) (Lines 36-37)):** Thank you for identifying this typo. We will correct it in the revised manuscript.\n\n**Limitation (Limitation Discussions for the Proposed Loss Function):** The limitations of our proposed loss are as follows: **(a) Domain Adaptation Setting.** The proposed JMMD-HSIC can only address the closed-set domain adaptation setting, where the label spaces of the source domain and target domain are identical. However, we do not explore its application in more complex domain adaptation scenarios, such as open-set domain adaptation [1], partial domain adaptation [2], weakly-supervised domain adaptation [3], etc. **(b) Task.** This submission only investigates the application of JMMD-HSCI in cross-domain classification tasks, while its potential in other more complex tasks, such as semantic segmentation and object detection, remains under-explored. **(c) Classifier.** Although this study has verified that JMMD can improve performance across different classifiers, how to design JMMD kernels for different classifiers has not been sufficiently explored.\n\n[1] Saito K., et al. Open Set Domain Adaptation by Backpropagation. ECCV, 2018.\n\n[2] Cao Z. J., et al. Partial Adversarial Domain Adaptation. ECCV, 2018.\n\n[3] Shu Y., etal. Transferable Curriculum for Weakly-Supervised Domain Adaptation. AAAI, 2019."}, {"review_id": "sYIZLTgRdA", "reviewer": "Reviewer_NVEg", "summary": "The paper exploits the JMMD for UDA problem. To address the partial derivative problem caused by the tensor-product operator, the paper deduces a concise JMMD based on the Representer theorem. It analyzes the connection between marginal, conditional, and weighted conditional MMD. Also, the paper provides a graph embedding viewpoint for JMMD with HSIC.", "questions": "For the statement “hard to be applied into a subspace learning framework”, what is subspace learning? There is no clear definition in the paper.  \n\n“as its empirical estimation involves a tensor-product operator whose partial derivative is difficult to obtain.” In the modern deep learning framework, like PyTorch, can the auto-differentiation tools solve this problem? \n\n\nIn Figures 2c and d, why are the features of different classes overlapped?", "limitations": "The paper does not discuss the limitations of the proposed method.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: The experimental results show the benefit of JMD-HSIC. Feature visualization of SPL+JMMD-HSIC is interesting. \n\nWeaknesses:\n\nWriting is hard to follow. \n\n1. Using the Representer theorem to formulate the kernel of JMMD is not new, e.g., [1]. \n\n[1]. Long M, Wang J, Ding G, et al. Transfer feature learning with joint distribution adaptation[C]//Proceedings of the IEEE international conference on computer vision. 2013: 2200-2207.\n\n2. Regarding Theorem 1, why is the uniformity of JMMD important for UDA? In UDA, essentially, you want to align the joint distribution $p(x,y)$. Using different characteristic kernels can reveal marginal, class-conditional, and weighted class conditional distributions from a joint distribution. Many previous MMD-based methods already attempt to align the joint distribution rather than the pure marginal or conditional distribution. Can you use the combination of the three? An ablation study on different chosen kernels would be good. \n\n\n3. In lines 227-229, “From (7), we observe that JMMD will push two data points from the same classes in the same domain further, and draw those from the same classes in different domains closer. HSIC will draw two data points from the same classes in the same domain closer (intra-class compactness).” Can you elaborate on the details? It is not easy to derive such a conclusion. \n\n4. The graph embedding viewpoint has been discussed in [2]. Can you discuss the connection of the two? Specifically, [2] has the same conclusion and a more detailed explanation for $w_{i,j}^J$. \n\n[2]. Chen Y, Song S, Li S, et al. A graph embedding framework for maximum mean discrepancy-based domain adaptation algorithms[J]. IEEE Transactions on Image Processing, 2019, 29: 199-213.", "was_revised": true, "final_justification": "Most of my concerns are addressed. So, I raise my score accordingly.", "rebuttal": "**Weaknesses 1 (the Novelty of JMMD's Kernel Formulated by the Representer Theorem):** **First,** The motivation of this submission mainly comes from a paper published at ICML’17 [3] (precisely the authors of the reference you mentioned [1]), and they applied JMMD into a deep neural network. They argued that JMMD is difficult to be applied into shallow domain adaptation methods. The following sentence is originated from the ICML paper: **“· · · while in conventional shallow domain adaptation methods the joint distributions are not easy to manipulate and match.”** **Second,** JDA (Joint Distribution Adaptation) proposed in the reference you mentioned [1] defines the joint distribution difference as the combination of the marginal and class conditional MMD (i.e., their summation), which differs from our definition of joint distribution difference. From the uniformity of JMMD proved in our work, JDA actually represents a special case of JMMD. Consequently, **the kernel obtained by JDA through the Representer theorem corresponds to the sum of the marginal kernel and class conditional kernel, rather than the JMMD kernel.** **Third,** their definitions of marginal and class conditional MMD do not involve complex tensor product operations, making their problem inherently less challenging than ours. **Fourth,** when employing the Representer theorem to derive the JMMD kernel (thereby eliminating its complicated tensor-product operation), we not only uncover its uniformity but also explain—from a graph embedding perspective—the fundamental reason why it compromises feature discriminability. This insight could lead us to propose a novel and more robust loss function, such as the proposed JMMD-HSIC.\n\n**Weaknesses 2 (the Importance of JMMD's Uniformity for UDA):** **First,** as addressed in our reply to your raised **Weakness 1**, our definition of joint distribution difference differs from theirs. By proving the uniformity of JMMD, we identify their method as a special case of ours. This does not imply equivalence, since we can improve JMMD by exploring its limitations and defining more robust kernels. For instance, this submission reveals how JMMD compromises feature discriminability and accordingly designs the JMMD-HSIC kernel. **Second,** our approach combines not the three components but the class conditional distribution kernel and our newly designed HSIC-based kernel. The rationale for selecting the class conditional distribution kernel is twofold: **(a)** Marginal distribution kernel neglect label information, often leading to imprecise distribution alignment; **(b)** While weighted class-conditional kernel can handle class imbalance issue, our experimental datasets exhibit minimal class imbalance, making their results nearly indistinguishable from the class conditional kernel. **Third,** following your suggestion, we have validated these claims through ablation studies, as shown in the following two tables. **Fourth,** the uniformity of JMMD enables our to develop more robust loss functions (as demonstrated by our JMMD-HSIC) by identifying its limitations.\n\n| 6 tasks of the Office-31 dataset | Accuracy (Average (%)) |\n| ------------- |:-------------:|\n| SPL | 89.6 |\n| SPL+JMMD-HSIC (marginal kernel) | 85.6 |\n| SPL+JMMD-HSIC (class conditional kernel) | **91.0** |\n| SPL+JMMD-HSIC (weighted class conditional kernel) | 90.9 |\n| OGL2P | 90.7 |\n| OGL2P+JMMD-HSIC (marginal kernel) | 86.7 |\n| OGL2P+JMMD-HSIC (class conditional kernel) | **91.5** |\n| OGL2P+JMMD-HSIC (weighted class conditional kernel) | **91.5** |\n\n| 12 tasks of the Office-Home dataset | Accuracy (Average (%)) |\n| ------------- |:-------------:|\n| SPL | 71.0 |\n| SPL+JMMD-HSIC (marginal kernel) | 64.9 |\n| SPL+JMMD-HSIC (class conditional kernel) | **71.9** |\n| SPL+JMMD-HSIC (weighted class conditional kernel) | 71.8 |\n| OGL2P | 72.8 |\n| OGL2P+JMMD-HSIC (marginal kernel) | 69.8 |\n| OGL2P+JMMD-HSIC (class conditional kernel) | **73.6** |\n| OGL2P+JMMD-HSIC (weighted class conditional kernel) | **73.6** |\n\n**Weaknesses 3 (Details about the Sentences in Lines 227-229):** **First,** the primary objective of graph embedding is to minimize the objective function (6) (Lines 219-220). When the weight wij between samples i and j is positive, we should reduce the feature distance (**draw them closer**) between them to prevent the objective function from increasing; conversely, when wij is negative, we should increase their feature distance (**push them further**) to decrease the objective function value. **Second,** through derivation, we obtain (8) (Lines 224-225) and find it maintains the same form as the objective function in (6). From the JMMD (side of (8)) and its corresponding weights (left side of (7) (Lines 223-224)), we observe that when samples belong to the same domain and same class, the weight is negative, indicating we should increase the distance or **push them further between two data points from the same classes in the same domain**. Conversely, when samples are from different domains but share the same class, the weight is positive, suggesting we should decrease their distance or **draw them closer between two data points from the same classes in the same domain**. Similarly, analyzing the HSIC (right side of (8)) and its weights (right side of (7)), we find positive weights occur when samples belong to the same domain and the same class, meaning we should decrease the distance or **draw them closer between two data points from the same classes in the same domain**. \n\n**Weaknesses 4 (the Connections between [2] and Ours):** The connections of the two are as follows: **Special Case Analysis:** Their work actually reveals a special case within our JMMD framework - specifically how marginal MMD and class conditional MMD may compromise discriminability. Our formulation generalizes to more distribution metrics satisfying JMMD's concise form, including the weighted class conditional MMD and the proposed JMMD-HSCI. **Different Solution Strategies:** Our approach fundamentally differs in addressing discriminability degradation. We introduce the JMMD-HSIC kernel with proven its reproducibility, maintaining its validity as a special case of JMMD. Their approach lacks strong theoretical foundation as it modifies the marginal MMD and class conditional MMD empirically through observation.\n\n**Question 1 (the Definition of Subspace Learning in this Submission):** In response to your raised **Weakness 2**, we clarify that the motivation of this work originates from an ICML'17 paper, where the authors argued that JMMD is difficult to apply to shallow domain adaptation methods. The following sentence is originated from the ICML paper: **· · · while in conventional shallow domain adaptation methods the joint distributions are not easy to manipulate and match.** Therefore, the subspace-learning methods referred to in this submission specifically denote shallow subspace-learning frameworks such as Principal Component Analysis (PCA) and Non-negative Matrix Factorization (NMF), etc.\n\n**Question 2 (Hard to Obtain Partial Derivative):** In response to your raised **Question 1**, we have provided clarification regarding the subspace-learning framework mentioned in this submission. The primary objective of this work is to address the challenge of obtaining partial derivatives when applying JMMD into the shallow subspace-learning frameworks **not the deep subspace-learning frameworks**.\n\n**Question 3 (Classes Overlapped in Figure 2c and d (Lines 300-301)):** Under the domain adaptation setting where target domain labels are unavailable during training, we follow the common practice in the field by employing pseudo-labels (model predictions) to iteratively update the pseudo-labels and our proposed loss function. However, for the visualization in this Figure, we adopted the ground-truth labels following Reference [1]. **The inherent unreliability of pseudo-labels remains a critical bottleneck limiting domain adaptation performance. Consequently, it is inherently challenging to learn features that can effectively separate different classes according to their true labels. The observed inter-class overlap primarily stems from the discrepancy between pseudo-labels and ground-truth labels for these samples.**\n\n**Limitation (Limitation Discussions for the Proposed Method):** The limitations of our proposed loss are as follows: **(a) Domain Adaptation Setting.** The proposed JMMD-HSIC can only address the closed-set domain adaptation setting, where the label spaces of the source domain and target domain are identical. However, we do not explore its application in more complex domain adaptation scenarios, such as open-set domain adaptation [4], partial domain adaptation [5], weakly-supervised domain adaptation [6], etc. **(b)** Task. This submission only investigates the application of JMMD-HSCI in cross-domain classification tasks, while its potential in other more complex tasks, such as semantic segmentation and object detection, remains under-explored. **(c)** Classifier. Although this study has verified that JMMD can improve performance across different classifiers, how to design JMMD kernels for different classifiers has not been sufficiently explored.\n\n[1] Long M.S., et al. Transfer Feature Learning with Joint Distribution Adaptation. ICCV, 2013.\n\n[2] Chen Y. M., et al. A Graph Embedding Framework for Maximum Mean Discrepancy-based Domain Adaptation Algorithms. TIP, 2019.\n\n[3] Long M.S., et al. Deep Transfer Learning with Joint Adaptation Networks. ICML, 2017.\n\n[4] Saito K., et al. Open Set Domain Adaptation by Backpropagation. ECCV, 2018. \n\n[5] Cao Z. J., et al. Partial Adversarial Domain Adaptation. ECCV, 2018. \n\n[6] Shu Y., etal. Transferable Curriculum for Weakly-Supervised Domain Adaptation. AAAI, 2019."}, {"review_id": "AFiK7jyKEF", "reviewer": "Reviewer_Cj1Z", "summary": "The distribution-distance metric has always been a critical issue in domain adaptation. In this paper, the authors first address the challenge of applying JMMD in subspace learning frameworks through a series of theoretical derivations. Based on this result, the authors uncover the uniformity of JMMD. Furthermore, from the perspective of graph embedding, the authors compare the relationship between HSIC and JMMD and explain why feature discrimination is compromised during the process of minimizing JMMD. Finally, the authors propose a novel loss function (JMMD-HSIC) and conduct extensive experiments to validate the correctness of the theoretical results and the effectiveness of the proposed loss function.", "questions": "Please refer to the weaknesses.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is highly readable and easy to understand. \n2. The supplementary materials provide thorough theoretical derivations, making both the theoretical findings and experimental results highly credible. \n3. The distribution-distance metric is a crucial issue in domain adaptation, and the theoretical results in this paper not only resolve the difficulty of applying JMMD in subspace learning frameworks but also reveal the uniformity of JMMD and the reason for its degradation of discriminability. These contributions demonstrate strong originality and significantly advance the field of domain adaptation. \n4. The experimental results are comprehensive, effectively validating the correctness of the theoretical results and the effectiveness of the proposed method.\n\nWeaknesses:\n1. The authors should include more discussion to explore how the theoretical findings could inspire future work. \n2. Figure 1 should provide more detailed descriptions to clarify the motivation of this study and the workflow of the proposed method.\n3. In domain adaptation, the labels of the target domain are unavailable, whereas JMMD requires labels from both domains. How does this paper address this issue? If pseudo-labels are adopted, how is the correct modeling of this loss ensured? \n4. Can the proposed loss function be effectively applied in deep learning frameworks?", "was_revised": true, "final_justification": "The authors have fully addressed all my concerns. I have also reviewed the comments from other reviewers, particularly #Reviewer 5L6J and #Reviewer J6vh, who both highlighted the significant strengths of this submission. Therefore, I maintain my recommendation to accept this submission.", "rebuttal": "**Weaknesses 1 (Future Work Inspired by the Theoretical Findings in this Submission):** In our responses to **#Reviewers 1, 2, and 4**, we have thoroughly discussed three key limitations of the proposed loss function. To address these limitations in future work, we plan to: **(a)** designing an advanced label kernel to handle more diverse domain adaptation scenarios; **(b)** extending the framework to more complex visual tasks; **(c)** designing novel label kernels by systematically analyzing how different classifiers vary in their sensitivity to feature distribution alignment and feature discriminability.\n\n**Weaknesses 2 (Detailed Descriptions for the Motivation and Workflow of the Proposed Method):** In our responses to **#Reviewers 1 and 2,** we clarify that the motivation of this work originates from an ICML'17 paper, where the authors argued that JMMD is difficult to apply to shallow domain adaptation methods. The following sentence is originated from the ICML paper: **· · · while in conventional shallow domain adaptation methods the joint distributions are not easy to manipulate and match.**\n \nWe will provide the following detailed descriptions for Figure 1 (Lines 36-37) in our revised manuscript: The overview of our revealed theoretical results and proposed JMMD-HSIC. (a) We map the features (upper part) and labels (lower part) of the source and target domains into the RKHS, respectively; (b) The application of JMMD (upper part) and HSIC (lower part) in a subspace-learning framework; (c) The graph embedding interpretation of JMMD (upper part) and HSIC (lower part) in a subspace-learning framework; (d) Learned feature representations of the source and target domains after subspace learning. '$−$' in the module of JMMD means the feature-label dependence difference, and '$+$' in the module of HSIC means separately considering feature-label dependence in the two domains. $\\textbf{X}$ and $\\textbf{Y}$ are data feature and label matrices. $\\psi$ and $\\phi$ are feature and label mappings for a RKHS. $\\textbf{T}$ and $\\textbf{B}$ are projection matrices for a projected RKHS. $\\textbf{T}\\psi(\\textbf{X})\\otimes \\phi (\\textbf{Y})$ is a tensor-product operator for the covariance. $\\text{tr}(\\textbf{B}^{\\top}\\textbf{K}^{\\textbf{X}\\textbf{X}}(\\textbf{K}^{\\textbf{Y}\\textbf{Y}}\\odot\\textbf{M}^{\\text{J}/\\text{H}})\\textbf{K}^{\\textbf{X}\\textbf{X}}\\textbf{B})$ is the deduced concise JMMD/HSIC in a projected RKHS.\n\n**Weaknesses 3 (Pseudo-Labels Issue for the Proposed Loss Function):** **(a)** As you rightly pointed out, our model employs pseudo-labeling (prediction results) and trains the model by iteratively updating both the pseudo-labels and our designed loss function. To address the inherent unreliability of pseudo-labels, this submission adopts classifiers specifically designed by two subspace-learning-based domain adaptation techniques (SPL [45], OGL2P [46]). **(b) In Section 6 of the supplementary material (Lines 116-127)**, ablation experiments comparing JMMD, HSCI and our JMMD-HSCI are performed on Office10-Caltech10 dataset, with multiple classifiers verifying the loss's robustness. Furthermore, we compare the performance of two distinct label formats (one-hot hard labels and probabilistic soft labels). As referenced in [4], this challenge can be mitigated by designing a confidence-regularized classifier that alleviates both the over-confidence problem of hard labels and the excessive noise introduced by probabilistic soft labels. Ultimately, this approach enhances the model's robustness against pseudo-label unreliability. **(c)** In the following two tables, we have also applied the proposed loss function to the shallow subspace learning-based domain adaptation framework (CRLP-L21) described in [4], demonstrating performance improvements compared with CRLP-L21. **(d)** For future work, we plan to explore: (1) confidence-based filtering methods to discard unreliable pseudo-labels, (2) enhancing the loss function's robustness to pseudo-label issue, and (3) designing more advanced classifiers to obtain more reliable pseudo-labels.\n\n**Weaknesses 4 (the Applications of Our Proposed Loss Function in Deep Learning Frameworks):** Since this study primarily addresses the challenge of applying JMMD into shallow subspace-learning-based domain adaptation frameworks, we initially do not implement the proposed loss function in end-to-end deep domain adaptation frameworks. Following your suggestion, we have now applied the proposed loss into three different deep learning frameworks (JAN [1], DSAN [2], DLP-NNM [3]), with experimental results on both the Office-31 and Office-Home datasets presented in the following two tables. It can be observed that the proposed loss function still leads to varying degrees of performance improvement across these three frameworks.\n\n| 6 tasks of the Office-31 dataset | Venue |  Accuracy (Average (%)) |\n| -------------|:------------- |:-------------:|\n| JAN | ICML'17 | 84.3 |\n| JAN+JMMD-HSIC | - | **86.2** |\n| DSAN | TNNLS'20 | 88.4 |\n| DSAN+JMMD-HSIC | - | **89.8** |\n|  CRLP-DA21 | TCSVT'21 | 90.4 |\n|  CRLP-DA21+JMMD-HSIC | - | **91.3** |\n| DLP-NNM | TIP'25 | 90.9 |\n| DLP-NNM+JMMD-HSIC | - | **91.4** |\n\n| 12 tasks of the Office-Home dataset | Venue |  Accuracy (Average (%)) |\n| -------------|:------------- |:-------------:|\n| JAN | ICML'17 | 58.3 |\n| JAN+JMMD-HSIC | - | **60.5** |\n| DSAN | TNNLS'20 | 67.6 |\n| DSAN+JMMD-HSIC | - | **69.6** |\n| CRLP-DA21 | TCSVT'21 | 71.4 |\n| CRLP-DA21+JMMD-HSCI | - | **72.9** |\n| DLP-NNM | TIP'25 | 73.7 |\n| DLP-NNM | - | **74.3** |\n\n[1] Long M.S., et al. Deep Transfer Learning with Joint Adaptation Networks. ICML, 2017.\n\n[2] Zhu Y. C., et al. Deep Subdomain Adaptation Network for Image Classification. TNNLS, 2020.\n\n[3] Wang W., et al. Deep Label Propagation with Nuclear Norm Maximization for Visual Domain Adaptation. TIP, 2025.\n\n[4] Wang W. et al. Confidence Regularized Label Propagation Based Domain Adaptation. TCSVT, 2021."}, {"review_id": "cdUtcgMVWX", "reviewer": "Reviewer_J6vh", "summary": "This paper conducts research on a JMMD, joint maximum mean discrepancy. \nThey mention that the empirical estimation of JMMD includes a tensor-product operator, which makes it difficult to obtain the partial derivative. \nThey conduct a theoretical analysis and have two findings: 1) the uniformity of JMMD, 2) similarity weights, which strengthen the intra-class compactness in the graph of Hilbert Schmidt independence criterion (HSIC), take opposite signs in the graph of JMMD, revealing the reason why JMMD degrades the feature discrimination. \nWith these findings, they propose a novel loss, JMMD-HSIC. \nExtensive results prove the effectiveness of the proposed method.", "questions": "See weakness 2-4.", "limitations": "No. I think the authors can have discussions on the limitations of the proposed loss, maybe it will unveil other important factors in domain adaptation.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros:\n1. This paper pays attention to an interesting problem, JMMD. JMMD is a very famous distance in the domain adaptation research community. The authors reflect on the limitations of this classical distance, which is new to me. \n2. Avoiding tensor-product operators helps obtain the partial derivative. I think this kind of modification is good. \n3. Now we know that some domain alignment methods will hurt discriminability, but we have not found the root cause. I think this paper opens a good topic. \n4. I think it is good and interesting to tackle domain adaptation from a theoretical perspective. \n\nCons:\n1. I think the paper can be polished, especially the abs and introduction section. The authors can use some bold texts to highlight key ideas. \n2. I think the novelty of this paper is ok for me, but not very high. The authors can have more discussions on the novelty of the paper. \n3. I want to see more extensive expeirment results, eg. combination with more methods. \n4. The authors can have discussions on the limitations or failure cases of the proposed method.", "was_revised": true, "final_justification": "After reading the rebuttal and other reviews, I think this paper can be accepted. I raise my score to 5.", "rebuttal": "**Weaknesses 1 (Paper Writing):** According to your suggestions, we will polish the revised manuscript to enhance the readability.\n\n**Weaknesses 2 (More Discussions on the Novelty of this Submission):** Thanks for your recognition of our work. Following your suggestions, we will enhance the revised manuscript by including more discussion on our innovative contributions, such as the limitations of the proposed loss function and future research directions (as addressed in our responses to your raised **Weaknesses 4 and Limitation**).\n\n**Weaknesses 3 (More Experimental Results, e.g., Combination with More Methods):** Following your suggestion, we have supplemented more experimental results: in response to the raised **Weakness 1 by #Reviewer 1**, we include runtime evaluations for computing the proposed loss function and optimizing its corresponding objective; to address the raised **Weakness 3 by #Reviewer 1**, we perform ablation studies by individually using JMMD and HSIC; in response to the raised **Weakness 2 by #Reviewer 2**,  we conduct ablation studies by using different JMMD kernels for the proposed JMMD-HSIC. Furthermore, in **Section 6 of the supplementary materials**, we provide: (1) robustness experiments of the proposed method with different classifiers (Table 1 in Lines 117-118), (2) comparative experimental results on the Office10-Caltech10 dataset (Table 2 in Lines 117-118), and (3) comparative experimental results on the ImageCLEF-DA dataset (Table 3 in Lines 135-136).\n\nMoreover, we have incorporated the proposed loss function into both a shallow subspace-learning-based domain adaptation method (CRLP-DA21 [3]) and three end-to-end deep adaptation frameworks (JAN [1], DSAN [2], and DLP-NNM [4]), as shown in the following two tables. Although the primary objective of this work is to address the challenges of applying JMMD in shallow subspace-learning methods, our final loss function demonstrates successful integration with end-to-end deep adaptation approaches. Notably, JAN is the pioneering work published in ICML'17 that first apply JMMD into end-to-end deep adaptation frameworks, but their original paper identified a critical issue. The following sentence is originated from the ICML paper: **· · · while in conventional shallow domain adaptation methods the joint distributions are not easy to manipulate and match.** This issue serves as a key motivation for our work. DSAN represents another significant contribution that specifically addresses how to define class conditional distribution discrepancy in deep adaptation frameworks. CRLP-DA21 and DLP-NNM offer representative solutions that mitigate the impact of pseudo-labeling issues on loss functions requiring target domain pseudo-labels through innovative classifier designs. Our experimental results show that the proposed loss function achieves varying degrees of performance improvement when applied to all four of these methods, which can demonstrate the effectiveness of our proposed loss function.\n\n| 6 tasks of the Office-31 dataset | Venue |  Accuracy (Average (%)) |\n| -------------|:------------- |:-------------:|\n| JAN | ICML'17 | 84.3 |\n| JAN+JMMD-HSIC | - | **86.2** |\n| DSAN | TNNLS'20 | 88.4 |\n| DSAN+JMMD-HSIC | - | **89.8** |\n|  CRLP-DA21 | TCSVT'21 | 90.4 |\n|  CRLP-DA21+JMMD-HSIC | - | **91.3** |\n| DLP-NNM | TIP'25 | 90.9 |\n| DLP-NNM+JMMD-HSIC | - | **91.4** |\n\n| 12 tasks of the Office-Home dataset | Venue |  Accuracy (Average (%)) |\n| -------------|:------------- |:-------------:|\n| JAN | ICML'17 | 58.3 |\n| JAN+JMMD-HSIC | - | **60.5** |\n| DSAN | TNNLS'20 | 67.6 |\n| DSAN+JMMD-HSIC | - | **69.6** |\n| CRLP-DA21 | TCSVT'21 | 71.4 |\n| CRLP-DA21+JMMD-HSCI | - | **72.9** |\n| DLP-NNM | TIP'25 | 73.7 |\n| DLP-NNM+JMMD-HSCI | - | **74.3** |\n\n**Weaknesses 4 and Limitation (Limitations of the Proposed Method and Future Work):** The limitations of our proposed loss function are as follows: **(a) Domain Adaptation Setting.** The proposed JMMD-HSIC can only address the closed-set domain adaptation setting, where the label spaces of the source domain and target domain are identical. However, we do not explore its application in more complex domain adaptation scenarios, such as open-set domain adaptation [5], partial domain adaptation [6], weakly-supervised domain adaptation [7], etc. **(b) Task.** This submission only investigates the application of JMMD-HSCI in cross-domain classification tasks, while its potential in other more complex tasks, such as semantic segmentation and object detection, remains under-explored. **(c) Classifier.** Although this study has verified that JMMD can improve performance across different classifiers, how to design JMMD kernels for different classifiers has not been sufficiently explored. For future work, we plan to explore: **(1)** designing an advanced label kernel to handle more diverse domain adaptation scenarios; **(2)** extending the framework to more complex visual tasks; **(3)** designing novel label kernels by systematically analyzing how different classifiers vary in their sensitivity to feature distribution alignment and feature discriminability.\n\n[1] Long M.S., et al. Deep Transfer Learning with Joint Adaptation Networks. ICML, 2017.\n\n[2] Zhu Y. C., et al. Deep Subdomain Adaptation Network for Image Classification. TNNLS, 2020.\n\n[3] Wang W. et al. Confidence Regularized Label Propagation Based Domain Adaptation. TCSVT, 2021.\n\n[4] Wang W., et al. Deep Label Propagation with Nuclear Norm Maximization for Visual Domain Adaptation. TIP, 2025.\n\n[5] Saito K., et al. Open Set Domain Adaptation by Backpropagation. ECCV, 2018.\n\n[6] Cao Z. J., et al. Partial Adversarial Domain Adaptation. ECCV, 2018.\n\n[7] Shu Y., etal. Transferable Curriculum for Weakly-Supervised Domain Adaptation. AAAI, 2019."}]

Author rebuttal:

Meta-review: This paper was reviewed by 4 experts in the field and received three "Accept" and one "Borderline Accept" as the final ratings. The reviewers agreed that the joint optimization of joint maximum mean discrepancy (JMMD) and Hilbert Schmidt independence criterion (HSIC) is a novel idea, the paper is well-written with sound theoretical derivations, the experimental analysis is comprehensive, and the results are promising. 

Reviewer 5L6J mentioned a point about the convergence analysis of the proposed loss function, which was addressed convincingly by the authors in the rebuttal. The authors also conducted experiments on the Office-31 and Office-Home datasets to analyze the convergence speed of their loss function. Concerns were also raised regarding the individual contributions of the JMMD and HSIC components; the authors have conducted ablation studies on the Office-31 and Office-Home datasets and have demonstrated the results with each component separately, and the benefits of combining them. The authors have also conducted experiments to show that their proposed loss function can be applied to train deep neural networks, in response to a comment from Reviewer Cj1Z. 

The reviewers, in general, have a positive opinion about the paper and its contributions; during the post-rebuttal discussion period, all of them have mentioned that their concerns were appropriately addressed. Based on the reviewers' feedback, the decision is to recommend the paper for acceptance to NeurIPS 2025. The reviewers have provided some valuable comments, such as discussions about the limitations of the proposed method, and how the theoretical findings could inspire future work. The authors are encouraged to include these in the final version of their paper. Given the practical usefulness of domain adaptation, the novelty of the proposed solution, the sound theoretical analysis, the comprehensive experiments, the promising results, and the consistent positive feedback from the reviewers, the AC feels this is a top quality paper, and recommends an oral presentation at NeurIPS 2025. We congratulate the authors on the acceptance of their paper!
Decision: Accept (oral)