Title: Kernel Trace Distance: Quantum Statistical Metric between Measures through RKHS Density Operators
Venue: NeurIPS 2025
Accepted: no
Keywords: Integral Probability Metrics

kernel methods

robustness
Abstract: Distances between probability distributions are a key component of many statistical machine learning tasks, from two-sample testing to generative modeling, among others. We introduce a novel distance between measures that compares them through a Schatten norm of their kernel covariance operators. We show that this new distance is an integral probability metric that can be framed between a Maximum Mean Discrepancy (MMD) and a Wasserstein distance. In particular, we show that it avoids some pitfalls of MMD, by being more
discriminative and robust to the choice of hyperparameters. Moreover, it benefits from some compelling properties of kernel methods, that can avoid the curse of dimensionality for their sample complexity. We provide an algorithm to compute the distance in practice by introducing an extension of kernel matrix for difference of distributions that could be of independent interest. Those advantages are illustrated by robust approximate Bayesian computation under contamination as well as particle flow simulations.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel distance between probability distributions called kernel trace distance and demonstrates that it is an Integral Probability Metric. The new distance measure is also robust to contamination and has statistical convergence rates that do not depend on the dimension. The authors demonstrate the practicality on particle gradient flows and Approximate Bayesian Computation.\n\nThe kernel trace distance is defined as the Schatten 1-norm (the sum of the singular values) of the difference between the density operators corresponding to each probability distribution (i.e. the difference between the sums of the singuar values of each density operator). The density operator is associated with the given kernel covariance operator which generalized the classical covariance matrix to infinite-dimensional feature space.\n\nBecause the density operator is infinite-dimensional, computing the singular values of these operators is not tractable. Drawing inspirations from the original kernel trick, the authors propose to take samples from both probability distributions in consideration. The union of the samples is denoted as $(z_k)_{k = 1, \\cdots, r}$. Assuming all samples are distinct (which ensures linear independence of the samples), the authors derive a novel kernel trick for deriving the difference kernel matrix. The claim is that the singular values of this alternate difference kernel matrix can be used to compute the singular values of the difference of the density operators; the authors provide a proof.\n\nThe authors also prove nice properties of the proposed distance metrics: 1) the proposed distance is an IPM; 2) the proposed distance (the Schatten 1-norm of the difference) lower bounds the Schatten 2-norm of the difference of the density operators; 3) the proposed distance is upper bounded by a related Wasserstein distance; 4) the proposed distance is related to other kernelized quantum divergences; 5) the proposed distance is more discriminative. This is shown in Figure 1 which demonstrates that the increase in the bandwidth parameter should decrease the difference to zero but this is more pronounced in the proposed distance and the MMD does have a peak in the middle. The authors extend this to analysis in which the distance between two mixture distributions are considered and demonstrate that the proposed metric is more discriminative (Proposition 3.5); 6) the proposed distance is more robust. An $\\epsilon$ contmination does not stray too much from the true distance, whereas the similar contamination can impact distances such as the Wasserstein distance.\n\nLastly, the authors discuss convergence. The first part is the convergence when considering that the eigenvalues are truncated to the top ones. The second is due to the approximation of the density operator with a finite number of samples from both probability distributions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: I think this manuscript is well-written and the authors did make considerable efforts to make rather theoretical aspects accessible to practitioners in kernel methods. The flow of the paper is logical, starting from discussing the proposed distance in relation to the distance metrics that are already in use and proving nice properties such as discriminative power and convergence. The authors derive a new kernel trick which I think is very interesting. I did go through some of the proofs, both in the main writeup and appendix and they seem to be correct.\n\nWeaknesses: the paper is largely theoretical (with significant results!) so I think rather limited results in experimental section can be understandable. One small complaint in the experiment section is the choice of the parameter (such as using the Gaussian kernel with a fixed bandwidth). I will also make additional comments regarding this in the question section.\n\nPresentation-wise: there are some things that could be highlighted such as the discriminative power of the kernel, e.g. Figure 1 which I think is an interesting figure should deserve more attention. But from casual reading, this is not easy to see (I was just able to gather this from my third pass over the paper).\n\nQUESTIONS:\n1. Are there some principled ways of choosing the hyperparameters of the kernel used in the experiment?\n2. Compared to the other distance metrics, do you see any tradeoff in terms of incurred computational requirement, other than the standard cost required for SVD computation on the difference kernel matrix?\n3. Do the authors see any sparsity tricks that could be applied for the difference kernel matrix? How would this change the properties of the distance metric?\n4. I did find Section 3.2 a bit long and the part of the reason is that it combines the discussion between distance between two unimodal Gaussians and the distance between two mixture distributions. There needs to be some pause between these topics.\n5. Do the author have the reference for the proof of robustness for MMD (Line 262)?\n6. Do the author have plans for verifying some of the assumptions used in analysis of the convergence rate section, for example, Assumption P and E on some datasets?\n\nLIMITATIONS:\nThere are no significant limitations. I would encourage the authors to pursue a follow-up paper on the application because there are so many directions you can take this, such as using the distance in generative neural network, etc. This is an interesting submission with many applications.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a new distance between probability measures defined using a Schatten norm on the kernel covariance operators. This novel distance has advantages over MMD while not suffering from the curse of dimensionality present for Wasserstein distances. Theoretical properties of the distance are studied, including its robustness and efficient estimation. Experiments on Approximate Bayesian Computation and Particle Flow are given.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The newly formulated distance strikes a balance between MMD and Wasserstein distance. In particular, it is efficient to estimate, robust, and has an integral probability formulation.\n- This work contributes nicely to a growing family of probability metrics defined on kernel covariance operators.\n\nWeaknesses:\n\n- In the zoo of distances, while this has some intriguing properties, it is hard to get a full picture of when to use this distance versus when not to. Some comparisons are given, but they are not informative about what settings this is most useful for.\n- The experiments are light and somewhat glossed over. While they show advantages for the proposed distance, they are not extensive enough to explore the advantages/disadvantages of the proposed distance.\n- The particle flow experiment is mentioned in the main body but no qualitative/quantitative results are given in the main body of the text.\n\nQUESTIONS:\n- How restrictive are the assumptions on the decay rate of eigenvalues of the operators that are used in the statistical rate proofs?\n- How do different choices of $p$ in the Schatten norm affect the kernel trace distance?\n- Are there other estimators of this distance that one can define using, for example, the IPM formulation and neural networks?\n- Can the authors offer more guidance on how to use the theory developed here for choosing this distance over other existing distances in the zoo of metrics over probability measures?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new distance between probability distributions based on the Schatten norm of their kernel covariance operators. It lies between MMD and Wasserstein distance, offering better discriminative power and robustness to hyperparameters. The method retains the efficiency of kernel-based approaches, helping avoid the curse of dimensionality. Practical computation is enabled via a new kernel matrix formulation, with applications shown in Bayesian inference and particle simulations.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. Unlike prior methods that rely on approximations, the proposed approach leverages kernel features in the dual space to derive a closed-form expression for the metric using the kernel trick, enhancing both theoretical clarity and computational efficiency.\n2. The authors provide formal guarantees for their proposed distance and demonstrate its utility across a range of numerical applications beyond GANs, indicating greater generality and robustness.\n3. The proposed distance $d_{\\text{KT}}$ is shown to be an integral probability metric (IPM) with desirable theoretical properties, including robustness to contamination and dimension-independent convergence rates, making it well-suited for high-dimensional and noisy data settings.\n\n**Weaknesses**\n1. As stated in lines 58–67, the proposed metric is the same as that in Mroueh et al. [2017], but the authors use kernel features directly in the dual formulation and derive a closed-form expression for the metric by leveraging a kernel trick. Furthermore, the motivation for using this metric differs from previous approaches, as stated in lines 38–40, which note that MMD has several pitfalls due to the different energies of mean embeddings. However, these pitfalls are not clearly explained, and it remains unclear why the proposed metric can effectively avoid them.\n2. The notation $k^2$, which first appears in Proposition 2.2, is not clearly defined.\n3. The advantage of the KT metric over MMD appears to stem from its \"normalized\" formulation. While MMD accounts for the internal energy associated with the variance of the distributions, doesn't this suggest that the internal energy form may better reflect the distribution and more effectively distinguish between two distributions with the same parameter?\n4. the theoretical analysis mainly compare the value of different metrics, however in statistical testing, how large is the value is not that important, since we always compare the value of test statistic with a testing threshold, normally, when the test satistic have larger scale the testing thresholds will be larger. To compare MMD and the KT , it is important to analyse how they performs compared to testing thresholds. \n5. MMD was originally proposed for the two-sample testing task, so more comparisons between KT and MMD within this context would be appreciated. Similarly, the KT metric was first introduced by Mroueh et al. [2017] for evaluating generative models, and additional experiments in this setting would also be valuable.\n6. The experiment in Table 1 does not clearly demonstrate the advantages of KT. For smaller values of $\\epsilon$, KT appears to underperform, while MMD yields better results. In contrast, for larger $\\epsilon$, KT shows improved performance, whereas MMD results in higher MSE. More experiments with varying values of $\\epsilon$ would be appreciated.\n\nQUESTIONS:\nNO\n\nLIMITATIONS:\nYES", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper defines a new kernel trace distance $d_{KT}$ between probability measures. The idea is to embed each distribution as its expected second moment matrix/operator after applying the kernel embedding map, then taking the Schatten 1-norm between them. They show how to compute this explicitly using SVD and the kernel trick. Under some assumptions on the kernel, they show that it is IPM over a class of bounded function with some Lipschitzness-like structure, allowing them to upper bound $d_{KT}$ by TV and some Wasserstein distances. Compared to MMD with the squared kernel (corresponding to the Schatten 2-norm above), they show that it has some nicer normalization properties but still exhibits fast empirical convergence under some standard decay assumptions. Their results are verified with some small experiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- This is an interesting definition explored through a few different lenses, with advantages over Wasserstein and MMD distances in certain regimes. I occasionally have qualms with this \"style\" of paper, introducing a new statistical distance, and while I still have some issues, I think this paper is better than the average one of this flavor, through pretty careful comparisons with existing distances.\n- The paper was nice to read, I followed all of their arguments on my first read.\n- The convergence rates are perhaps the most technical piece of the paper but are well-explained and appear correct to me. Dimension-free rates are always nice.\n\nWeaknesses:\n- The question asked on Page 1 --- \"How large such a function space could be before the curse of dimensionality kicks in?\" --- is rather ill-posed. There are function classes $\\mathcal{F}$ which exhibit empirical convergence rates $\\mathbb{E}[\\\\|\\\\hat{\\\\mu}\\_n - \\\\mu\\\\|\\_{\\\\mathcal{F}}] \\\\lesssim n^{-1/k}$ for every $k$ between 2 and $d$. For example, one can take $\\mathcal{F} = \\mathcal{F}_k$ to contain all $f:\\mathbb{R}^d \\to \\mathbb{R}^d$ of the form $f = g \\circ U$, where $U$ is an orthogonal projection onto a $k$ dimensional subspace and $g$ is 1-Lipschitz. The resulting IPM is the k-dimensional max-sliced (or projection-robust) 1-Wasserstein distance, which exhibits empirical convergence at rate $n^{-1/k}$ when $\\mu$ has bounded second moments.\n- In general, comparing statistical distances is always a bit subtle, and it is tricky to say when one is better than another. E.g. robustness is a useful property when data is contaminated, but it may lead to overlooking differences when data is clean. Fast statistical rates lead to fewer samples needed to estimate to a given precision, but guarantees at a given precision are often incomparable between different distances. I have some questions below which would help convince me further of the kernel trace distance's utility.\n- The computational complexity is okay, but not great. Worst-case, between empirical measures on $n$ points it looks like $O(n^3)$ instead of $O(n^2)$ for MMD.\n\nMinor nits:\n- I believe Prop 3.6 has a simpler proof (not that it is complicated) without factor of 2, since $|d_{KT}(P_\\varepsilon,Q) - d_{KT}(P,Q)| \\leq d_{KT}(P_\\varepsilon, P) \\leq TV(P_\\varepsilon,P) \\leq \\varepsilon$\n- Line 87, \"explicit\" usually not a verb\n- In Assumption 0, I assume \"completely\" should be \"complete\", unless \"completely separable\" is its own property\n- Line 156, \"note\" confused me, used for definition rather than observation as standard. Same thing on Line 266\n- For Assumption 2, I was not familiar with \"simple\" meaning multiplicity one. Apparently somewhat standard, so that's fine. However, if so, the inequalities which follow should be strict.\n\nQUESTIONS:\n1. Since there are so many choices for statistical distances, I personally favor those with some operational meaning or relation to downstream tasks of external interest. E.g. most f divergences tell you something about rates of fundamental testing questions, W1 tells you about estimating Lipschitz test functions and W2 gives rise to rich metric geometry. I don't think this is strictly necessary for acceptance (but it would definitely push me over the bar) - can you think of any way of motivating $d_{KT}$ from first principles? The analogies from quantum are nice but pretty hand-wavy.\n2. For transparency, are there any settings where you think $d_{KT}$ should *not* be used over MMD?\n3. When working in $\\mathbb{R}^d$ with Euclidean structure, we usually center covariance matrices by the mean. You could analogously consider $\\tilde{\\Sigma}\\_\\mu = \\mathbb{E}\\_\\mu[(\\varphi(x) - \\mathbb{E}\\_\\mu[\\varphi(x)])(\\varphi(x) - \\mathbb{E}\\_\\mu[\\varphi(x)])^\\top] = \\Sigma\\_\\mu - \\mathbb{E}\\_\\mu[\\varphi(x)] \\mathbb{E}\\_\\mu[\\varphi(x)]^\\top$ for your setting. Is this a reasonable alternative, and how might it compare to your choice?\n4. Do you believe that the rates from Cor 4.3 are tight up to log factors? I would guess so for the second rate at least. I don't think lower bounds are a top priority but they would of course be welcome.\n\nLIMITATIONS:\nSee second question.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new statistical divergence inspired by \"quantum statistics\". \nFor a given \"feature map\" $\\varphi(x)$, the new divergence \"kernel trace (KT) distance\" between two measures $\\mu$ and $\\nu$ is defined as the Shatten 1-norm of the difference between the covariances of the feature map with respect to the two measures: \n$\\|\\|\\Sigma\\_\\mu-\\Sigma\\_\\nu\\|\\|\\_1$, where $\\Sigma\\_\\mu\\triangleq \\mathbb{E}_{x\\sim \\mu} [\\varphi(x)\\varphi(x')^*]$.\nWhen the feature map $\\varphi(x)$ is associated with a PSD kernel $k(x,x')$, i.e., $k(x,x')=\\langle \\varphi(x),\\varphi(x')\\rangle$, then the KT distance can be computed by the kernel trick, similar to the standard kernel methods (Section 2.3).\nThe relations of the KT distance to IPMs (TV and Wasserstein distances) and other quantum statistical divergences (kernel BW and kernel KL divergences) are also established (Section 3.1). Its robustness is also discussed. \nThe convergence rate of the empirical measure to the underlying measure in the KT distance is established in Section 4. Under a certain spectrum decay assumption of the target distribution, the parametric rate $O(n^{-1/2})$ is nearly attained. As a byproduct, the result also readily implies the dimension-free rate of the KBW distance via its equivalence to the KT distance.\nThe authors applied the KT distance for ABC and particle flow experiments.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\nThe paper is overall well written and easy to understand (except a few parts).\nThe writing is technically solid and mostly easy to understand.\nThe idea of considering the Schatten-1 norm of the difference between the covariances sounds natural and elegant.\nThe proposed distance has a better discriminative power than MMD. \nIts relationship to other divergences are well studied.\n\n## Weaknesses\nEven though the exposition is overall easy to understand, I believe some parts can be revised to improve readability.\nIn particular, I cannot understand what the authors wish to convey in Section 3.2.\nAlso, the writing in `Section 5 Experiments` is much less clear than other parts. For example, the explanation on ABC was not comprehensible for me, who first learned the approach from this paper, based on the reading of this paper. The low score is mostly due to the lack of clarity on these parts.\n\nQUESTIONS:\n- In general, one can consider a family of divergences defined as a Schatten-$p$ norm for any $p\\ge 1$, where $p=1$ gives the KT distance and $p=2$ gives the MMD distance with the kernel $k^2$. Is there any reason that $p=1$ is particularly special from this family? For example, I wonder if the robustness result in Proposition 3.6 would hold for any $p\\ge 1$. Would the distance with larger $p$ have less discriminative power in general? It'd be very instructive if the authors can discuss this general family for more insights. \n- In Proposition 3.5, what is the meaning of the first sentence? Do the authors mean to consider the mixtures for distributions that satisfy orthogonality between covariances?\n- It is claimed in the last paragraph that `[The KT distance] is the greatest in a family of kernel-based IPM including MMD`, but how can I see this directly? Maybe I am missing something, but it seems not obvious to me.\n- In line 328, is `For the proposed distance $d\\_{KT}$` a typo?\n\n## Typos\n- Line 104: it ~is~ could also be...\n- Line 147, put , after `expressive`\n- Line 217, the ``internal energy'' $\\|\\|\\Sigma\\_\\mu\\|\\|_2$ ...\n- After line 274, $\\phi$ should be $\\varphi$, which is the feature map of the considered RKHS?\n- The subsection Section 4.1 seems to be not necessary as this is the only subsection in Section 4.\n\n## Suggestions\n- Reading Eq. (5), I had to try to find the definition of $F(A,B)$ again. It'd be helpful if the definition of fidelity can be recalled in the revision.\n- Please consider revising and expanding the section on experiments in the camera-ready, if accepted. \n\n\nOverall, I think the manuscript has an interesting contribution, and I will be happy to increase the score if the authors promise to revise the manuscript and address the comments.\n\nLIMITATIONS:\nThe limitations of this method are properly mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}