Title: Statistical and Geometrical properties of the Kernel Kullback-Leibler divergence
Venue: NeurIPS 2024
Accepted: yes
Keywords: kernels; optimisation; optimal transport
Abstract: In this paper, we study the statistical and geometrical properties of the Kullback-Leibler divergence with kernel covariance operators (KKL) introduced by [Bach, 2022, Information Theory with Kernel Methods]. Unlike the classical Kullback-Leibler (KL) divergence that involves density ratios, the KKL compares probability distributions through covariance operators (embeddings) in a reproducible kernel Hilbert space (RKHS), and compute the Kullback-Leibler quantum divergence. 
This novel divergence hence shares parallel but different aspects with both the standard Kullback-Leibler between probability distributions and kernel embeddings metrics such as the maximum mean discrepancy. 
A limitation faced with the original KKL divergence is its inability to be defined for distributions with disjoint supports. To solve this problem, we propose in this paper a regularised variant that guarantees that divergence is well defined for all distributions. We derive bounds that quantify the deviation of the regularised KKL to the original one, as well as concentration bounds. 
In addition, we provide a closed-form expression for the regularised KKL, specifically applicable when the distributions consist of finite sets of points, which makes it implementable. 
Furthermore, we derive a Wasserstein gradient descent scheme of the KKL divergence in the case of discrete distributions, and study empirically its properties to transport a set of points to a target distribution.

Statistical and Geometrical Properties of Regularized Kernel Kullback-Leibler Divergence

Abstract

In this paper, we study the statistical and geometrical properties of the KullbackLeibler divergence with kernel covariance operators ( KKL ) introduced by Bach [2022]. Unlike the classical Kullback-Leibler (KL) divergence that involves density ratios, the KKL compares probability distributions through covariance operators (embeddings) in a reproducible kernel Hilbert space (RKHS), and compute the Kullback-Leibler quantum divergence. This novel divergence hence shares parallel but different aspects with both the standard Kullback-Leibler between probability distributions and kernel embeddings metrics such as the maximum mean discrepancy. A limitation faced with the original KKL divergence is its inability to be defined for distributions with disjoint supports. To solve this problem, we propose in this paper a regularized variant that guarantees that the divergence is well defined for all distributions. We derive bounds that quantify the deviation of the regularized KKL to the original one, as well as finite-sample bounds. In addition, we provide a closed-form expression for the regularized KKL , specifically applicable when the distributions consist of finite sets of points, which makes it implementable. Furthermore, we derive a Wasserstein gradient descent scheme of the KKL divergence in the case of discrete distributions, and study empirically its properties to transport a set of points to a target distribution.

1 Introduction

A fundamental task in machine learning is to approximate a target distribution q . For example, in Bayesian inference [Gelman et al., 1995], it is of interest to approximate posterior distributions of the parameters of a statistical model for predictive inference. This has led to the vast development of parametric methods from variational inference [Blei et al., 2017], or non-parametric ones such as Markov Chain Monte Carlo (MCMC) [Roberts and Rosenthal, 2004], and more recently particlebased optimization [Liu and Wang, 2016, Korba et al., 2021]. In generative modelling [Brock et al., 2019, Ho et al., 2020, Song et al., 2020, Franceschi et al., 2023] only samples from q are available and the goal is to generate data whose distribution is similar to the training set distribution. Generally, this problem can be cast as an optimization problem over P ( R d ) , the space of probability distributions over R d , where the optimization objective is chosen as a dissimilarity function D ( ·| q ) (a distance or divergence) between probability distributions, that only vanishes at q . Starting from an initial distribution p 0 , a descent scheme can then be applied such that the trajectory ( p t ) t ≥ 0 approaches q . In particular, on the space of probability distributions with bounded second moment P 2 ( R d ) , one

Anna Korba

CREST, ENSAE, IP Paris

can consider the Wasserstein gradient flow of the functional F ( p ) = D ( p || q ) . The latter defines a path of distributions, ruled by a velocity field, that is of steepest descent for F with respect to the Wasserstein-2 distance from optimal transport.

This approach has led to a large variety of algorithms based on the choice of a specific dissimilarity functional F , often determined by the information available on the target q . For example, in Bayesian or variational inference, where the target's density is known up to an intractable normalizing constant, a common choice for the cost is the Kullback-Leibler (KL) divergence, whose optimization is tractable in that setting [Wibisono, 2018, Ranganath et al., 2014]. When only samples of q are available, it is not convenient to choose the optimization cost as the KL, as it is only defined for probability distributions p that are absolutely continuous with respect to q . In contrast, it is more convenient to choose an F that can be written as integrals against q , for instance, maximum mean discrepancy (MMD) [Arbel et al., 2019, Hertrich et al., 2024b], sliced-Wasserstein distance [Liutkus et al., 2019] or Sinkhorn divergence [Genevay et al., 2018]. However, sliced-Wasserstein distances, that average optimal transport distances of 1-dimensional projections of probability distributions (slices) over an infinite number of directions, have to be approximated by a finite number of directions in practice [Tanguy et al., 2023]; and Sinkhorn divergences involve solving a relaxed optimal transport problem. In contrast, MMD can be written in closed-form for discrete measures thanks to the reproducing property of positive definite kernels. The MMD represents probability distributions through their kernel mean embeddings in a reproducing kernel Hilbert space (RKHS), and compute the RKHS norm of the difference of embeddings (namely, the witness function). Moreover, the MMD flow with a smooth kernel (e.g., Gaussian) as in Arbel et al. [2019] is easy to implement, as the velocity field is expressed as the gradient of the witness function, and preserve discrete measures. However, due to the non-convexity of the MMD in the underlying Wasserstein geometry [Arbel et al., 2019], its gradient flow is often stuck in local minimas in practice even for simple target as Gaussian q , calling for adaptive schemes tuning the level of noise or kernel hyperparameters [Xu et al., 2022, Galashov et al., 2024], or regularizing the kernel [Chen et al., 2024]. MMD with non-smooth kernels, e.g., based on negative distances [Sejdinovic et al., 2013], have also attracted attention recently, as their gradient flow enjoys better empirical convergence properties than the previous ones [Hertrich et al., 2024a,b]. However, their gradient flow does not preserve discrete measures; and their practical simulation rely on implicit time discretizations [Hertrich et al., 2024a] or slicing [Hertrich et al., 2024b].

In contrast to the MMD with smooth kernels, the KL divergence is displacement convex [Villani, 2009, Definition 16.5] when the target is log-concave (i.e., q has a density q ∝ e -V with V convex), and its gradient flow enjoys fast convergence when q satisfies a log-Sobolev inequality [Bakry et al., 2014]. In this regard, it enjoys better geometrical properties than the MMD. Moreover, the KL divergence is equal to infinity for singular p and q , which makes its gradient flow extremely sensitive to mismatch of support, so that the flow enforces the concentration on the support of q as desired. On the downside, while the Wasserstein gradient flow of KL divergences is well-defined [Chewi et al., 2020], its associated particle-based discretization is difficult to simulate when only samples of q are available, and a surrogate optimization problem usually needs to be introduced [Gao et al., 2019, Ansari et al., 2020, Simons et al., 2022, Birrell et al., 2022a, Liu et al., 2022]. However, it is unclear whether this surrogate optimization problem preserves the geometry of the KL flow.

Recently, Bach [2022] introduced alternative divergences based on quantum divergences evaluated through kernel covariance operators, that we call here a kernel Kullback-Leibler ( KKL ) divergence. The latter can be seen as second-order embeddings of probability distributions, in contrast with first-order kernel mean embeddings (as used in MMD). In Bach [2022], it was shown that the KKL enjoys nice properties such as separation of measures, and that it is framed between a standard KL divergence (from above) and a smoothed KL divergence (from below), i.e., a KL divergence between smoothed versions of the measures with respect to a specific smoothing kernel. Hence, it cannot directly be identified to a KL divergence and corresponds to a novel and distinct divergence. However, many of its properties remained unexplored, including a complete analysis of the KKL for empirical measures, a tractable closed-form expression and its optimization properties. In this paper, we tackle the previous questions. We propose a regularized version of the KKL that is well-defined for any discrete measures, in contrast with the original KKL . We establish upper bounds that quantify the deviation of the regularized KKL to its unregularized counterpart, and convergence for empirical distributions. Moreover, we derive a tractable closed-form for the regularized KKL and its derivatives that writes with respect to kernel Gram matrices, leading to a practical optimization algorithm. Finally,

we investigate empirically the statistical properties of the regularized KKL , as well as its geometrical properties when using it as an objective to target a probability distribution q .

This paper is organized as follows. Section 2 introduces the necessary background and the regularized KKL . Section 3 presents our theoretical results on the deviation and finite-sample properties of the latter. Section 4 provides the closed-form of regularized KKL for discrete measures as well as the practical optimization scheme based on an explicit time-discretisation of its Wasserstein gradient flow. Section 5 discusses closely related work including distances or divergences between distributions based on reproducing kernels. Finally, Section 6 illustrates the statistical and optimization properties of the KKL on a variety of experiments.

2 Regularized kernel Kullback-Leibler ( KKL ) divergence

In this section, we state our notations and previous results on the (original) kernel Kullback-Leibler ( KKL ) divergence introduced by Bach [2022], before introducing our proposed regularized version.

Notations. Let P ( R d ) the set of probability measures on R d . Let P 2 ( R d ) the set of probability measures on R d with finite second moment, which becomes a metric space when equipped with Wasserstein-2 ( W 2 ) distance [Villani, 2009].

For p ∈ P ( R d ) , we denote that p is absolutely continuous w.r.t. q using p ≪ q , and we use dp/dq to denote the Radon-Nikodym derivative. We recall the standard definition of the Kullback-Leibler divergence, KL( p || q ) = ∫ log( dp/dq ) dp if p ≪ q , + ∞ else.

If g : R d → R r is differentiable, we denote by Jg : R d → R r × d its Jacobian. If r = 1 , we denote by ∇ g the gradient of g and H g its Hessian. If r = d , ∇· g denotes the divergence of g , i.e., the trace of the Jacobian. We also denote by ∆ g the Laplacian of g , where ∆ g = ∇· ∇ g . We also denote I the identity matrix or operator.

For a positive semi-definite kernel k : R d × R d → R , its RKHS H is a Hilbert space with inner product ⟨· , ·⟩ H and norm ∥ · ∥ H . For q ∈ P 2 ( R d ) such that ∫ k ( x, x ) dq ( x ) < ∞ , the inclusion operator ι q : H → L 2 ( q ) , f ↦→ f is a bounded operator with its adjoint being ι ∗ q : L 2 ( q ) →H , f ↦→ ∫ k ( x, · ) f ( x ) dq ( x ) [Steinwart and Christmann, 2008, Theorem 4.26 and 4.27]. The covariance operator w.r.t. q is defined as Σ q = ∫ k ( · , x ) ⊗ k ( · , x ) dq ( x ) = ι ∗ q ι q , where ( a ⊗ b ) c = ⟨ b, c ⟩ H a for a, b, c ∈ H . It can also be written Σ q = ∫ R d φ ( x ) φ ( x ) ∗ dq ( x ) where ∗ denotes the transposition in H (recall that for u ∈ H , uu ∗ : H → H denotes the operator uu ∗ ( f ) = ⟨ f, u ⟩ H u for any f ∈ H ).

Kernel Kullback-Leibler divergence ( KKL ). For p, q ∈ P ( R d ) , the kernel Kullback-Leibler divergence ( KKL ) is defined in Bach [2022] as:

where Λ p and Λ q are the set of eigenvalues of the covariance operators Σ p and Σ q , with associated eigenvectors ( f λ ) λ ∈ Λ p and ( g γ ) γ ∈ Λ q . The KKL (1) evaluates the Kullback-Leibler divergence between operators on Hilbert Spaces, that is well-defined for any couple of positive Hermitian operators with finite trace, at the operators Σ p and Σ q . From Bach [2022, Proposition 4], if p and q are supported on compact subset of R d , and if k is a continuous positive definite kernel with k ( x, x ) = 1 for all x ∈ R d , and if k 2 is universal [Steinwart and Christmann, 2008, Definition 4.52], then KKL( p || q ) = 0 if and only if p = q . In Bach [2022], it also was proven that the KKL is upper bounded by the (standard) KL-divergence between probability distributions (see Proposition 4 therein) and lower bounded by the same KL but evaluated at smoothed versions of the distributions, where the smoothing is a convolution with respect to a specific kernel (see Section 4 therein). Thus, the KKL defines a novel divergence between probability measures. It defines then an interesting candidate as to compare probability distributions, for instance when used as an optimization objective over P ( R d ) , in order to approximate a target distribution q .

Definition of the regularized KKL. Amajor issue that the KKL shares with the standard KullbackLeibler divergence between probability distributions, is that it diverges if the support of p is not

included in the one of q (1). Indeed, for the KKL( p || q ) to be finite, we need Ker(Σ q ) ⊂ Ker(Σ p ) . This condition is satisfied when the support of p is included in the support of q . Indeed, if f ∈ Ker(Σ q ) , then ⟨ f, Σ q f ⟩ H = ∫ R d f ( x ) 2 dq ( x ) = 0 , and so f is zero on the support of q , then also on the support of p . Hence, the KKL is not a convenient discrepancy when q is a discrete measure (in particular, if p is also discrete with different support than q ). A simple fix that we propose in this paper is to consider a regularized version of KKL which is, for α ∈ ]0 , 1[ ,

The advantage of this definition is that KKL α is finite for any distribution p, q . It smoothes the distribution q by mixing it with the distribution p , to a degree determined by the parameter α . This divergence approximates the original KKL divergence without requiring the distribution p to be absolutely continuous with respect to q for finiteness. Moreover, for any α ∈ ]0 , 1[ , KKL α ( p || q ) = 0 if and only if p = q . As α → 0 , we recover the original KKL (1), and as α → 1 , this quantity converges pointwise to zero.

Remark 1. The regularization we consider in (2) has also been considered for the standard KL divergence [Lee, 2000]. These objects, as well as their symmetrized version, were also referred to in the literature as skewed divergences [Kimura and Hino, 2021]. The most famous one is JensenShannon divergence, recovered as a symmetrized skewed KL divergence for α = 1 2 , that is defined as JS( p || q ) = KL( p || 1 2 p + 1 2 q ) + KL( q || 1 2 p + 1 2 q ) .

3 Skewness and concentration of the regularized KKL

In this section we study the skewness of the regularized KKL due to the introduction of the parameter α , as well as its concentration properties for empirical measures.

Skewness. We will first analyze how the regularized KKL behaves with respect to the regularization parameter α . First, we show it is monotone with respect to α in the following Proposition.

Proposition 2. Let p ≪ q . The function α ↦→ KKL α ( p || q ) is decreasing on [0 , 1] .

Proposition 2 shows that the regularized KKL shares a similar monotony behavior than the regularized, or skewed, (standard) KL between probability distributions, as recalled in Appendix A.1. The proof of Proposition 2 can be found in Appendix B.1. It relies on the positivity of the KKL divergence, and the use of the identity [Ando, 1979]

where I is the identity operator, that is used in all our proofs. We now fix α ∈ ]0 , 1[ and provide a quantitative result about the deviation of the regularized (or skewed) KKL to its original counterpart.

Proposition 3. Let p, q ∈ P ( R d ) . Assume that p ≪ q and that dp dq ⩽ 1 µ for some µ > 0 . Then,

Proposition 3 recovers a similar quantitative bound than the one we can obtain for the standard KL between probability distributions, see Appendix A.2; and state that the skewness of the regularized KKL can be controlled by the regularization parameter α , especially when the latter is small. However, the tools used to derive this inequality are completely different by nature than for the KL case. Its complete proof can be found in Appendix B.2, but we provide here a sketch.

Sketch of proof. Let Γ = α Σ p +(1 -α )Σ q . We write KKL( p || q ) α -KKL( p || q ) = Tr Σ p log Σ q -TrΣ p log Γ that we write as (3). In order to upper bound this integral we use the operator equalities, for two operators A and B , A -1 -B -1 = A -1 ( B -A ) B -1 = A -1 ( B -A ) A -1 -A -1 ( B -A ) B -1 ( B -A ) A -1 which we apply to A = Γ+ βI and B = Σ q + βI . We then use the assumption µ Σ p ≼ Σ q and carefully apply upper bounds on positive semi-definite operators, using the matrix inequality results from Appendix A.3, to conclude the proof.

Statistical properties. We now focus on the regularized KKL for empirical measures and derive finite-sample guarantees.

Proposition 4. Let p, q ∈ P ( R d ) . Assume that p ≪ q with dp dq ⩽ 1 µ for some 0 < µ ⩽ 1 and let α ⩽ 1 2 . We remind that φ ( x ) is the feature map of x ∈ R d in the RKHS H . Assume also that c = ∫ + ∞ 0 sup x ∈ R d ⟨ φ ( x ) , (Σ p + βI ) -1 φ ( x ) ⟩ 2 H dβ is finite. Let ˆ p , ˆ q supported on n, m i.i.d. samples from p and q respectively. We have:

Remark 5. It is possible to calculate a similar bound for the above proposition which does not require the condition p ≪ q . This bound, which can be found at the end of Appendix B.3.3, worsens as α approaches 0 because it scales in O ( 1 α 2 ) instead of O ( 1 α ) above.

The latter proposition extends significantly Bach [2022, Proposition 7] that provided an upper bound on the entropy term only, i.e., the first term in (1):

Our bound (5) is explicit in the number of samples n, m for ˆ p, ˆ q , and for n = m we recover similar terms as (6). Our contribution is to upper bound the cross term, i.e., the second term in (1), involving both p and q . We do so by closely follow the proof of [Bach, 2022, Proposition 7] in order to bound the cross terms difference. In consequence, our proof involves technical intermediate results, among which concentration of sums of random self-adjoint operators, and estimation of degrees of freedom. The proof of Proposition 4 can be found in Appendix B.3, but we provide here a sketch.

Sketch of proof. We denote ˆ Γ = α Σ ˆ p + (1 -α )Σ ˆ q and Γ its population counterpart. In order to bound the cross term we write TrΣ ˆ p log ˆ Γ -TrΣ p log Γ using (3). We split the integrals in three terms, with respect to two parameters 0 < β 0 < β 1 that we introduce: (a) one for β between 0 and β 0 , (b) one for β between β 1 and infinity and (c) an intermediate one. The β 0 quantity is chosen to be dependent of m and n , so that it converge to zero as n and m go to infinity. This way, for (a) the integral between 0 and β 0 we simply have to bound TrΣ p (Γ + βI ) -1 and TrΣ ˆ p ( ˆ Γ + βI ) -1 by constant or integrable quantities close to 0. Then, for (b), β 1 is chosen so that it goes to infinity when n and m go to infinity and (b) is bounded by 1 /β 1 . Finally we upper bound finely enough (c) to compensate for the fact that the bounds of the integrals tend towards 0 and infinity.

4 Time-discretized regularized KKL gradient flow

In this section, we show that the regularized KKL can be implemented in closed-form for discrete measures, as well as its Wasserstein gradient, making its optimization tractable.

regularized KKL closed-form. We first describe how to compute the regularized KKL for (any, not necessarily empirical) discrete measures in practice. This will be useful for the practical implementation of regularized KKL optimization coming next. We provide a closed-form for the latter, involving kernel Gram matrices between supports of the discrete measures.

Proposition 6. Let ˆ p = 1 n ∑ n i =1 δ x i and ˆ q = 1 m ∑ m j =1 δ y j two discrete distributions. Define K ˆ p = ( k ( x i , x j )) n i,j =1 ∈ R n × n , K ˆ q = ( k ( y i , y j )) m i,j =1 ∈ R m × m , K ˆ p, ˆ q = ( k ( x i , y j )) n,m i,j =1 ∈ R n × m . Then, for any α ∈ ]0 , 1[ , we have:

Proposition 6 extends non-trivially the result of Bach [2022, Proposition 6] that only provided a closed-form for the entropy term Tr(Σ ˆ p log(Σ ˆ p )) , that corresponds to our first term in Equation (7). Its complete proof can be found in Appendix B.4 but we provide here a sketch.

Sketch of proof. Our goal there is to derive a closed-form for the cross-term in ˆ p, ˆ q of the KKL , that is Tr(Σ ˆ p log( α Σ ˆ p +(1 -α )Σ ˆ q )) . It is based on the observation that if we define ϕ x = ( φ ( x 1 ) , . . . , φ ( x n )) ∗ , ϕ y = ( φ ( y 1 ) , . . . , φ ( y m )) ∗ and ψ the concatenation of √ α n ϕ x and √ 1 -α m ϕ y , then the covariance operators write Σ ˆ p = ψ T I α ψ and α Σ ˆ p +(1 -α )Σ ˆ q = ψ T ψ . Then, the matrices K ˆ p and K write K ˆ p = ψI α ψ T and ψψ T = K . Finally, we apply an intermediate result (Lemma 12) to obtain the expression of log( α Σ ˆ p +(1 -α )Σ ˆ q ) as a function of log K .

Gradient flow and closed-form for the derivatives. We now discuss how to optimize p ↦→ KKL α ( p || q ) for a given target distribution q . For a given functional F : P 2 ( R d ) → R + , a Wasserstein gradient flow of F can be thought as an analog object to a Euclidean gradient flow in the metric space ( P 2 ( R d ) , W 2 ) [Santambrogio, 2017], which defines a trajectory ( p t ) t ≥ 0 in P 2 ( R d ) following the steepest descent for F with respect to the W 2 distance. It can be characterized by a continuity equation :

where F ′ ( p ) : R d → R is the first variation of F at p ∈ P 2 ( R d ) . We recall that the first variation at p ∈ P 2 ( R d ) as defined in Ambrosio et al. [2005, Lemma 10.4.1] is defined, if it exists, as the function F ′ : R d → R such that

for any ξ = q -p , q ∈ P 2 ( R d ) . To optimize KKL α , it is then natural to consider its Wasserstein gradient flow and discretize it in time and space. Since KKL α is well-defined for discrete measures ˆ p, ˆ q , we directly derive its first variation for this setting. Our next result yields a closed-form for the first variation of the regularized KKL .

Proposition 7. Consider ˆ p, ˆ q and the matrices K ˆ p , K as defined in Proposition 6. Let g ( x ) = log x x . Then, the first variation of F = KKL α ( ·|| ˆ q ) at ˆ p is, for any x ∈ R d :

where

̸

where ( c j ) j are the eigenvectors of K , and ( a j ) j the vectors of first n terms of ( c j ) j .

The proof of Proposition 7 can be found in Appendix B.5, we provide a sketch below.

Sketch of proof. Our proof deals separately with the entropy and the cross term, writing F = F 1 + F 2 . Starting from the definition (9), we denote ∆ = ε Σ ξ . For F 1 , we write F 1 (ˆ p + εξ ) - F 1 (ˆ p ) = ∑ n i =1 f ( λ i (Σ ˆ p +∆)) -f ( λ i (Σ ˆ p )) . To write this term, we use the residual formula, which can be used to differentiate eigenvalues of functions. Indeed, we can write, for an operator A with finite number of positive eigenvalues, ∑ λ ∈ Λ( A ) f ( λ ) = ∮ γ f ( z ) Tr ( ( zI -A ) -1 ) dz where γ is a loop in C surrounding all the positive eigenvalues of A . Applying this to our case, if we choose γ such that it surrounds both the eigenvalues of Σ ˆ p and of Σ ˆ p +∆ , we obtain ∑ n i =1 f ( λ i (Σ ˆ p +∆)) -f ( λ i (Σ ˆ p )) = 1 2 iπ ∮ γ f ( z ) Tr ( ( zI -Σ ˆ p -∆) -1 ) -f ( z ) Tr ( ( zI -Σ ˆ p ) -1 ) dz . Using the identity A -1 -B -1 = A -1 ( B -A ) A -1+ o ( B -A ) , the previous quantity becomes ∑ n i =1 f ( λ i (Σ ˆ p +∆)) -f ( λ i (Σ ˆ p )) = 1 2 iπ ∮ γ ∑ n k =1 f ( z ) ( z -λ k ) 2 dz Tr( f ∗ k f k ∆)+ o ( ε ) . Under the integral we recognise a holomorphic function with isolated singularities and we can therefore apply the residue formula again. Concerning F 2 , we proceed in the same way, with the difference that as we have a cross term, eigenvectors will appear in the calculation and in the final result.

Leveraging the analytical form for the first variation given by Proposition 7, the Wasserstein gradient of F = KKL α ( ·|| ˆ q ) at p is given by ∇F ′ ( p ) : R d → R d by taking the gradient with respect to x in Equation (10). Notice that the latter only involves derivatives with respect to the kernel k , and can be computed in O (( n + m ) 3 ) due to the singular value decomposition of the matrix K defined in Proposition 6.

Starting from some initial distribution p 0 , and for some given step-size γ > 0 , a forward (or explicit) time-discretization of (8) corresponds to the Wasserstein gradient descent algorithm, and can be written at each discrete time iteration l ≥ 1 as:

where Id is the identity map in L 2 ( p l ) and # denotes the pushforward operation. For discrete measures µ n = 1 / n ∑ n i =1 δ x i , we can define F ( X n ) := F ( p n ) where X n = ( x 1 , . . . , x n ) , since the functional F is well defined for discrete measures. The Wasserstein gradient flow of F (8) becomes the standard Euclidean gradient flow of the particle based function F . Furthermore, Wasserstein gradient descent (11) writes as Euclidean gradient descent on the position of the particles.

5 Related work

Divergences based on kernels embeddings. Kernels have been used extensively to design useful distances or divergences between probability distributions, as they provide several ways to represent probability distributions, e.g., through their kernel mean or covariance embeddings. The Maximum Mean Discrepancy (MMD) [Gretton et al., 2012] is maybe the most famous one. It is defined as the RKHS norm of the difference between the mean embeddings m p := ∫ k ( x, · ) dp ( x ) and m q := ∫ k ( x, · ) dq ( x ) , i.e., MMD( p ∥ q ) = ∥ m p -m q ∥ H . When k is characteristic, MMD( p ∥ q ) = 0 if and only if p = q [Sriperumbudur et al., 2010]. MMD belongs to the family of integral probability metrics [Müller, 1997] as it can be written as MMD( p ∥ q ) = sup f ∈H , ∥ f ∥ H ≤ 1 E p [ f ( X )] -E q [ f ( X )] . Alternatively, it can be seen as an L 2 -distance between kernel density estimators. It became popular in statistics and machine learning through its applications in two-sample test [Gretton et al., 2012], or more recently in generative modeling [Bi´ nkowski et al., 2018].

However, kernel mean embeddings are not the only way (and maybe not the most expressive) to represent probability distributions. For instance, MMD may not be discriminative enough when the distributions differ only in their higher-order moments but have the same mean embedding. For this reason, several works have resorted to test statistics that incorporate the kernel covariance operator of the probability distributions. For instance, Harchaoui et al. [2007] construct a test statistic that resembles and regularizes the MMD( p ∥ q ) by incorporating covariance operators (more precisely, ∥ (Σ p + q 2 + βI ) -1 ( m p -m q ) ∥ H ) yielding in some sense a chi-square divergence between the two distributions. This work has been recently generalized in Hagrass et al. [2022] to more general spectral regularizations, and in Chen et al. [2024] with a different covariance operator. A similar regularized MMD statistic is employed by Balasubramanian et al. [2021], Hagrass et al. [2023] in the context of the goodness-of-fit test.

Kernel variational approximation of the KL. An alternative use of kernels to compute probability divergences is through approximation of variational formulations of f -divergences [Nguyen et al., 2010, Birrell et al., 2022b] of which KL -divergence is an example. Indeed, the KL divergence between p and q writes sup g ∈ M b ∫ gdp -∫ e g dq where M b denotes the set of all bounded measurable functions on R d . For instance, Glaser et al. [2021] consider a variational formulation of the KL divergence restricted to RKHS functions, namely the KALE divergence:

Recently, Neumayer et al. [2024] extended the latter work and studied kernelized variational formulation of general f -divergences, referred to as Moreau envelopes of f-divergences in RKHS, including the KALE as a particular case. They prove that these functionals are lower semi-continuous, and that their Wasserstein gradient flows are well defined for smooth kernels (i.e., the functionals are λ -convex, and the subdifferential contains a single element). However, the KALE does not have a closed form expression (in constrast to the kernelized variational formulation of chi-square, which writes as a

regularized MMD, see [Chen et al., 2024]). For discrete distributions p and q supported on n atoms, the KALE divergence can be written a strongly convex n -dimensional problem, and can be solved using standard Euclidean optimization methods. Still, this makes the simulation of KALE Wasserstein gradient flow (e.g., gradient descent on the positions of particles) computationally demanding, as it requires solving an inner optimization problem at each iteration. This inner optimization problem is solved calling another optimization algorithm. Glaser et al. [2021] use various methods in their experiments, including Newton's method (that scales as O ( n 3 ) due to the matrix inversion), or less computationally demanding ones such as gradient descent (GD) or coordinate descent. For large values of the regularization parameter λ, using plain GD works reasonably well, but for small values of λ , the problem becomes quite ill-conditioned and GD needs to be run with smaller step-sizes. Moreover, as KALE (and its gradient) are not available in closed-form, they cannot be used with fast and hyperparameter-free methods, such as L-BFGS [Liu and Nocedal, 1989] which requires exact gradients. This contrasts with our regularized KKL divergence and its gradient, which are available in closed-form. In our experiments, we will investigate further the relative performance of KALE and KKL .

6 Experiments

In this section, we illustrate the validity of our theoretical results and the performance of gradient descent for the regularized KKL . In all our experiments, we consider Gaussian kernels k ( x, y ) = exp ( -∥ x -y ∥ 2 σ 2 ) where σ denotes the bandwith. Our code is available on the github repository https://github.com/clementinechazal/KKL-divergence-gradient-flows.git .

Illustrations of skewness and concentration of

the KKL . We first illustrate our results of Proposition 3 and Proposition 4, i.e. the skewness and concentration properties of KKL α . Weinvestigate these properties for various settings of p, q two fixed probability distributions on R d , varying the choice of α , dimension d , and distributions p, q . We consider empirical measures ˆ p , ˆ q supported on n i.i.d. samples of p, q (in this section we take the same number of samples for both distributions, i.e., n = m in the notations of Section 3), and we observe the concentration of KKL α (ˆ p, ˆ q ) around its population limit as the number of samples n (particles) go to infinity.

Each time, we plot the results obtained over 50 runs, randomizing the samples drawn from each distribution. Thick lines represent the average value over these multiple runs. We represent the dependence

0

250

500

750

1000

number of particules

0

2

4

6

8

KKL (

p

||

q

)

= 1e-06

= 1e-05

= 0.0001

= 0.001

= 0.01

= 0.1

= 0.5

in α and n in dimension 10 in Figure 1, for p, q anisotropic Gaussian distributions with different means and variances. Alternative settings and additional results are deferred to the Appendix C, such as different distributions (e.g. a Gaussian p versus an Exponential q ) , as well as the dimension dependence for a fixed α . We can clearly see in Figure 1 the monotony of KKL α with respect to α (as stated in Proposition 2) and the concentration of the empirical KKL α around its population version, which happens faster for a larger value of α , as predicted by our finite-sample bounds in Proposition 4.

Sampling with KKL gradient descent. Finally, we study the performance of KKL gradient descent in practice, as described in Section 4. We consider two settings already used by Glaser et al. [2021] for KALE gradient flow, reflecting different topological properties for the source-target pair: a pair with a target supported on a hypersurface (zero volume support) and a pair with disjoint supports of positive volume. Alternative settings, e.g. Gaussians source and mixture of Gaussians target that are pairs of distributions with a positive density supported on R d , are deferred to Appendix C. We also report there additional plots related to the experiments of this section.

We have treated α as a hyperparameter here, and in this section for simplicity of notations we refer to KKL as the objective functional. As both KKL and its gradient can be explicitly computed, one can

implement descent either using a constant step-size, or through a quasi-Newton algorithm such as L-BFGS [Liu and Nocedal, 1989]. The latter is often faster and more robust than the conventional gradient descent and does not require choosing critical hyper-parameters, such as a learning rate, since L-BFGS performs a line-search to find suitable step-sizes. It only requires a tolerance parameter on the norm of the gradient, which is in practice set to machine precision. In contrast, as said in the previous section, the KALE and its gradient are not available in closed-form.

The first example is a target distribution q supported (and uniformly distributed) on a lowerdimensional surface that defines three non-overlapping rings, see Figure 2. The initial source is a Gaussian distribution with a mean in the vicinity of the target q . We compare Wasserstein gradient descent of KKL , Maximum Mean Discrepancy [Arbel et al., 2019] and KALE [Glaser et al., 2021], using the code provided in these references. For each method, we choose a bandwith σ = 0 . 1 , and we optimize the step-size for each method, and sample n = 100 points from the source and target distribution. Our method is robust to the choice of α and generally performs very well on this example, as shown in Figure 2. We can notice that since MMD is not sensitive to the difference of support between p and q , the particles may leave the rings; while for the regularized KKL flow, as for KALE flow, the particles follow closely the support of the target distribution.

The second example consists of a source and target pair p, q that are supported on disjoint subsets, each with a finite, positive volume, in contrast with the previous example. The source and the target are uniform supported on a heart and a spiral respectively. We again run MMD, KALE and KKL gradient descent. In this example, both KKL and KALE recover the spiral shape, much before the MMDflow trajectory; but both have a harder time recovering outliers, disconnected from the main support of the spiral.

KALE

T=0

T=2

T=30

T=60

MMD

KKL

kale_0001

T=0

T=10

T=30

T=60

T=99

MMD

target

KKL

7 Conclusion

In this work, we investigated the properties of the recently introduced Kernel Kullback-Leibler ( KKL ) divergence as a tool for comparing probability distributions. We provided several theoretical results, among which quantitative bounds on the deviation from the regularized KKL to the original one, and finite-sample guarantees for empirical measures, that are validated by our numerical experiments. We also derived a closed-form and computable expression for the regularized KKL as well as its derivatives, enabling to implement (Wasserstein) gradient descent for this objective. Our experiments validate the use of KKL as a tool to compare discrete measures, as its gradient flow is much better behaved than the one of Maximum Mean Discrepancy which relies only on mean (first moments) embeddings of probability distributions. It can also be computed in closed-form, in contrast to the KALE divergence introduced recently in the literature, and can benefit from fast and hyperparameterfree methods such as L-BFGS.

While our study has advanced our understanding of the KKL divergence, several limitations must be acknowledged. Firstly, theoretical guarantees for the convergence of the KKL flow remain unestablished. Secondly, reducing the computational cost is crucial for practical applications. Investigating the use of random features presents a promising avenue for making the computations more efficient.

8 Acknowledgments

A. Korba and C. Chazal acknowledge the support of ANR PEPR PDE-AI.

References

Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient Flows: In Metric Spaces and in the Space of Probability Measures . Springer Science & Business Media, 2005.

Tsuyoshi Ando. Concavity of certain maps on positive definite matrices and applications to hadamard products. Linear Algebra and its Applications , 26:203-241, 1979.

Abdul Fatir Ansari, Ming Liang Ang, and Harold Soh. Refining deep generative models via discriminator gradient flow. arXiv preprint arXiv:2012.00780 , 2020.

Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. Advances in Neural Information Processing Systems , 32, 2019.

Francis Bach. Information theory with kernel methods. IEEE Transactions on Information Theory , 69(2):752-775, 2022.

Dominique Bakry, Ivan Gentil, and Michel Ledoux. Analysis and Geometry of Markov Diffusion Operators , volume 103. Springer, 2014.

Krishnakumar Balasubramanian, Tong Li, and Ming Yuan. On the optimality of kernel-embedding based goodness-of-fit tests. Journal of Machine Learning Research , 22(1):1-45, 2021.

Rajendra Bhatia. Positive Definite Matrices . Princeton University Press, 2009.

Mikołaj Bi´ nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD gans. International Conference on Learning Representations (ICLR) , 2018.

Jeremiah Birrell, Paul Dupuis, Markos A Katsoulakis, Yannis Pantazis, and Luc Rey-Bellet. (f,gamma)-divergences: Interpolating between f-divergences and integral probability metrics. Journal of Machine Learning Research , 23(39):1-70, 2022a.

Jeremiah Birrell, Markos A Katsoulakis, and Yannis Pantazis. Optimizing variational representations of divergences and accelerating their statistical estimation. IEEE Transactions on Information Theory , 68(7):4553-4572, 2022b.

David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational Inference: A Review for Statisticians. Journal of the American statistical Association , 112(518):859-877, 2017.

Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations , 2019.

Zonghao Chen, Aratrika Mustafi, Pierre Glaser, Anna Korba, Arthur Gretton, and Bharath K Sriperumbudur. (De)-regularized maximum mean discrepancy gradient flow. arXiv preprint arXiv:2409.14980 , 2024.

Sinho Chewi, Thibaut Le Gouic, Chen Lu, Tyler Maunu, and Philippe Rigollet. SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence. Advances in Neural Information Processing Systems , 33:2098-2109, 2020.

Jean-Yves Franceschi, Mike Gartrell, Ludovic Dos Santos, Thibaut Issenhuth, Emmanuel de Bézenac, Mickaël Chen, and Alain Rakotomamonjy. Unifying GANs and score-based diffusion as generative particle models. arXiv preprint arXiv:2305.16150 , 2023.

Alexandre Galashov, Valentin de Bortoli, and Arthur Gretton. Deep MMD gradient flow without adversarial training. arXiv preprint arXiv:2405.06780 , 2024.

Yuan Gao, Yuling Jiao, Yang Wang, Yao Wang, Can Yang, and Shunkang Zhang. Deep generative learning via variational gradient flow. In International Conference on Machine Learning , pages 2093-2101. PMLR, 2019.

Andrew Gelman, John B. Carlin, Hal S. Stern, and Donald B. Rubin. Bayesian data analysis . Chapman and Hall/CRC, 1995.

Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning generative models with Sinkhorn divergences. In International Conference on Artificial Intelligence and Statistics , pages 1608-1617, 2018.

Pierre Glaser, Michael Arbel, and Arthur Gretton. KALE flow: A relaxed KL gradient flow for probabilities with disjoint support. Advances in Neural Information Processing Systems , 34: 8018-8031, 2021.

Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research , 13(1):723-773, 2012.

Omar Hagrass, Bharath K. Sriperumbudur, and Bing Li. Spectral regularized kernel two-sample tests. arXiv preprint arXiv:2212.09201 , 2022.

Omar Hagrass, Bharath K. Sriperumbudur, and Bing Li. Spectral regularized kernel goodness-of-fit tests, 2023.

Zaïd Harchaoui, Francis Bach, and Eric Moulines. Testing for homogeneity with kernel Fisher discriminant analysis. Advances in Neural Information Processing Systems , 20, 2007.

Johannes Hertrich, Manuel Gräf, Robert Beinert, and Gabriele Steidl. Wasserstein steepest descent flows of discrepancies with Riesz kernels. Journal of Mathematical Analysis and Applications , 531(1):127829, 2024a.

Johannes Hertrich, Christian Wald, Fabian Altekrüger, and Paul Hagemann. Generative sliced MMD flows with Riesz kernels. International Conference on Learning Representations , 2024b.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems , volume 33, pages 6840-6851, 2020.

Masanari Kimura and Hideitsu Hino. α -geodesical skew divergence. Entropy , 23(5):528, 2021.

Anna Korba, Pierre-Cyril Aubin-Frankowski, Szymon Majewski, and Pierre Ablin. Kernel Stein discrepancy descent. In International Conference on Machine Learning , pages 5719-5730, 2021.

Lillian Lee. Measures of distributional similarity. arXiv preprint cs/0001012 , 2000.

Dong C Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical programming , 45(1-3):503-528, 1989.

Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose Bayesian inference algorithm. Advances in Neural Information Processing Systems , 29, 2016.

Song Liu, Jiahao Yu, Jack Simons, Mingxuan Yi, and Mark Beaumont. Minimizing f-divergences by interpolating velocity fields. arXiv preprint arXiv:2305.15577 , 2022.

Antoine Liutkus, Umut Simsekli, Szymon Majewski, Alain Durmus, and Fabian-Robert Stöter. Sliced-Wasserstein flows: Nonparametric generative modeling via optimal transport and diffusions. In International Conference on Machine Learning , pages 4104-4113, 2019.

Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in Applied Probability , 29(2):429-443, 1997.

Sebastian Neumayer, Viktor Stein, and Gabriele Steidl. Wasserstein gradient flows for Moreau envelopes of f-divergences in reproducing kernel hilbert spaces. arXiv preprint arXiv:2402.04613 , 2024.

XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory , 56(11):5847-5861, 2010.

Rajesh Ranganath, Sean Gerrish, and David Blei. Black box variational inference. In Artificial Intelligence and Statistics , pages 814-822. PMLR, 2014.

Gareth O. Roberts and Jeffrey S Rosenthal. General state space Markov chains and MCMC algorithms. Probability Surveys , 1:20-71, 2004.

Alessandro Rudi and Lorenzo Rosasco. Generalization properties of learning with random features. Advances in Neural Information Processing Systems , 30, 2017.

Filippo Santambrogio. { Euclidean, metric, and Wasserstein } gradient flows: An overview. Bulletin of Mathematical Sciences , 7:87-154, 2017.

Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, and Kenji Fukumizu. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics , 41(5): 2263 - 2291, 2013.

Jack Simons, Song Liu, and Mark Beaumont. Variational likelihood-free gradient descent. In Fourth Symposium on Advances in Approximate Bayesian Inference , 2022.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 , 2020.

Bharath K Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, and Gert RG Lanckriet. Hilbert space embeddings and metrics on probability measures. The Journal of Machine Learning Research , 11:1517-1561, 2010.

Ingo Steinwart and Andreas Christmann. Support Vector Machines . Springer Science & Business Media, 2008.

Eloi Tanguy, Rémi Flamary, and Julie Delon. Properties of discrete Sliced Wasserstein losses. arXiv preprint arXiv:2307.10352 , 2023.

Cédric Villani. Optimal Transport: Old and New , volume 338. Springer, 2009.

Andre Wibisono. Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem. In Conference on Learning Theory , pages 2093-3027. PMLR, 2018.

Lantian Xu, Anna Korba, and Dejan Slepcev. Accurate quantization of measures via interacting particle-based optimization. In International Conference on Machine Learning , pages 2457624595, 2022.

A Additional Background

A.1 Monotonicity of the KL α divergence

Proposition 8. The function α ↦→ KL( p || αp +(1 -α ) q ) is decreasing on [0 , 1] .

Proof. Let 0 < α ′ < α < 1 ,

We have ∫ (log( q + α ′ ( p -q )) -log( q + α ( p -q )))( αdp + (1 -α ) dq ) = -KL( αp + (1 -α ) q ) || α ′ p + (1 -α ′ ) q ) ⩽ 0 . For the second term, note that because of the increasing nature of the log , for the points for which p -q > 0 , log( q + α ′ ( p -q )) ⩽ log( q + α ( p -q )) and vice versa. Hence

This concludes the proof.

A.2 Skewness of the KL α divergence

Proposition 9. Suppose that dp/dq ⩽ 1 µ ,

Proof. First, we have

Now, we remind the following identity which is the real-valued analog of Equation (3). Let x, y > 0 ,

Hence,

The first term in (13) can be bounded as

where the penultimate inequality uses q ⩾ µp for the first term. The second term in (13) can be bounded similarly as:

Finally,

A.3 Background operator monotony

We recall here results about matrix and operator monotony, that we extensively use in all our proofs. These are set out in the blog post https://francisbach.com/ matrix-monotony-and-convexity/ , see [Bhatia, 2009] for a more complete reference. For 2 operators A and B in H , we denote A ≼ B the operators inequality in the sense : ∀ x ∈ H , x ∗ Ax ⩽ x ∗ Bx . Let S being the set of symmetric operators and S + the set of symmetric positive operators. We have

If A,B ∈ S , X another operator in H , A ≼ B ⇒ X ∗ AX ≼ X ∗ BX .

ii) If A,B ∈ S , M ≽ 0 , A ≼ B ⇒ Tr( AM ) ⩽ Tr( BM ) .

iii) If B is invertible, A ≼ B ⇒ B -1 / 2 AB -1 / 2 ≼ I .

iv) If B ∈ S , B ∗ B ≼ I ⇒ BB ∗ ≼ I .

If A,B ∈ S + , A ≼ B ⇒ A 1 / 2 ≼ B 1 / 2 .

vi) If A,B ∈ S + and are invertible, A ≼ B ⇒ B -1 ≼ A -1 .

B Proofs

B.1 Proof of Proposition 2

Let 0 < α ′ < α . We have:

For the first term in (14), we recognize

For the second term in (14), we write

where the last equality uses A -1 -B -1 = A -1 ( B -A ) B -1 . The term under the integral writes as ( AX ) T AX , where A = (Σ q + βI + α (Σ p -Σ q )) -1 and X = Σ p -Σ q , hence it is positive. Then, knowing that (1 -α )( α ′ -α ) ⩽ 0 , we conclude that the second term in (14) is negative. Finally,

B.2 Proof of Proposition 3

This proof makes repeated use of the results about matrix monotony, some of which we recall in Appendix A.3. The reader may refer to Appendix A.2 for analog computations in the KL case. We denote Γ = α Σ p +(1 -α )Σ q = Σ q + α (Σ p -Σ q ) . We write using direct integration [Ando, 1979]

We will first upper bound ( a ) in absolute value, since it is not necessarily positive. We first bound

where we used for the second term the matrix inequalities Σ p ≼ 1 µ Σ q . Hence, we have

We can then upper bound | ( a ) | as:

We now turn to ( b ) which we can upper bound without absolute value since it is a positive term. Since Γ ≽ (1 -α )Σ q , α ∈ [0 , 1] and we are dealing with p.s.d. operators, we can bound the inverse as (Γ + βI ) -1 ≼ 1 1 -α (Σ q + β 1 -α I ) -1 ≼ 1 1 -α (Σ q + βI ) -1 and so, using Tr( AM ) ≤ Tr( BM ) for A ≼ B and M ≽ 0 , we have:

We will split the r.h.s. of the previous inequality in four terms, involving twice Σ q , two cross terms (bounded similarly) with Σ p , Σ q and twice Σ p . We have for the first one:

For the cross-term we have:

and for the last term we have:

Finally, combining our bounds for the terms in ( b ) and integrating with respect to β , we get

Adding the bounds on (a) and (b) concludes the proof.

B.3 Proof of Proposition 4

B.3.1 Intermediate result 1: Concentration of sums of random self-adjoint operators

Lemma 10. Assume the conditions of Proposition 4 hold. Denote ˆ Γ = α Σ ˆ p + (1 -α )Σ ˆ q and Γ its population counterpart. Let β > 0 , D = (Γ + βI ) -1 2 ( ˆ Γ -Γ)(Γ + βI ) -1 2 and C ( β ) = sup x ∈ R d ⟨ φ ( x ) , (Σ p + βI ) -1 φ ( x ) ⟩ H . Then, for 0 < u < 1 ,

where λ max ( D ) is the maximal eigenvalue of ( D 2 ) 1 / 2 .

Proof. Denoting

we have (Γ + βI ) -1 / 2 Σ p (Γ + βI ) -1 / 2 = E [ X ] , (Γ + βI ) -1 / 2 Σ q (Γ + βI ) -1 / 2 = E [ Y ] and so (Γ + βI ) -1 / 2 Γ(Γ + βI ) -1 / 2 = α E [ X ] + (1 -α ) E [ Y ] . We can thus write

However, for two operators A and B we have ∥ A + B ∥ op ⩽ ∥ A ∥ op + ∥ B ∥ op which means that λ max ( A + B ) ⩽ λ max ( A ) + λ max ( B ) . Then,

which is equivalent to

The quantities λ max ( D p ) := λ max ( α (Γ + βI ) -1 2 (Σ ˆ p -Σ p )(Γ + βI ) -1 2 ) and λ max ( D q ) λ max ( (1 -α )(Γ + βI ) -1 2 (Σ ˆ q -Σ q )(Γ + βI ) -1 2 ) are positive so

Then Bach [2022, Lemma 2] can be applied twice to ˜ φ ( x ) = √ α (Γ + βI ) -1 / 2 φ ( x ) and to ˜ ˜ φ ( y ) = √ (1 -α )(Γ + βI ) -1 / 2 φ ( y ) .

For the term with D p ,

where we used Γ ≽ α Σ p and define C ( β ) = sup x ∈ R d ⟨ φ ( x ) , (Σ p + βI ) -1 φ ( x ) ⟩ H . Using the same inequality, we also obtain

and

Then,

For the term with D q , using the matrix inequalities Γ ≽ (1 -α )Σ q and Σ q ≽ µ Σ p , with similar computations we get

Then,

We can combine both results on D p and D q and use µ ≤ 1 to get

B.3.2 Intermediate result 2: Degrees of freedom estimation

The Proposition below adapts the proof of Bach [2022, Proposition 15] to bound the cross terms between the empirical covariance operators of p and αp +(1 -α ) q .

Proposition 11 (Estimation of skewed degrees of freedom) . Assume the conditions of Proposition 4 hold. Denote ˆ Γ = α Σ ˆ p +(1 -α )Σ ˆ q and Γ its population counterpart. We have:

where C ( β ) = sup x ∈ R d ⟨ φ ( x ) , (Σ p + βI ) -1 φ ( x ) ⟩ H is supposed to be inferior to µ ( m ∧ n ) 24 .

Proof. We will denote

In expectation, the first term in (16) can be upper-bounded as follows, using that Tr ( (Σ p -Σ ˆ p )(Γ + βI ) -1 ) is the sum of zero-mean random variables:

where the third inequality uses Γ ≽ (1 -α )Σ q ≽ µ (1 -α )Σ p and the fourth uses the definition of C ( β ) for β > 0 . The last inequality is due to the facts that α ⩽ 1 2 and so 1 1 -α ⩽ 2 , and also that µ ⩽ 1 so β µ (1 -α ) ⩾ β and because β ↦→ C is non increasing on ]0 , + ∞ [ , we have C ( β µ (1 -α ) ) ⩽ C ( β ) . These simplifications are used many times in this proof.

The second term in (16) can be written as follows. Consider D = (Γ + βI ) -1 2 (Γ -ˆ Γ)(Γ + βI ) -1 2 defined in Lemma 10 and consider the case where λ max ( D ) ⩽ u < 1 . Then D ≺ I . Using the identity D ( I -D ) -1 = D ( I -D ) -1 ( D + I -D ) , as in the proof of [Rudi and Rosasco, 2017], we can write

We have for the first term in (18), using ˆ Γ ≽ α Σ ˆ p :

and, by the definition of D above,

where λ max ( D ) is the absolute value of the maximal eigenvalue of ( D 2 ) 1 2 . So

In this case,

Still considering that λ max ( D ) < 1 we have for the second term in (18):

where in the second inequality we used Σ ˆ p ≼ 1 α ˆ Γ and (19). Let 0 < u < 1 . Combining (17), (20) and (21), we have:

We now bound the first term of (22) by upperbounding, going back to the formula given in (16). Using Γ ≽ µ (1 -α )Σ p , the term TrΣ p (Γ+ βI ) -1 is bounded by 1 µ (1 -α ) C ( β µ (1 -α ) ) ⩽ 2 µ C ( β ) ⩽ n ∧ m 12 by hypothesis. And with ˆ Γ ≽ 1 α Σ ˆ p , we have both TrΣ ˆ p ( ˆ Γ + βI ) -1 ⩽ 1 α TrΣ ˆ p (Σ ˆ p + β α I ) -1 ⩽ n α and TrΣ p (Γ+ βI ) -1 ⩽ 1 (1 -α ) µ C ( β ) ⩽ 2 µ C ( β ) ⩽ n ∧ m 12 . Then, almost surely, | A | ⩽ max { n α , n ∧ m 12 } ⩽ 2 n α . Then, (22) becomes:

With Lemma 10 we have

Using the hypothesis that C ( β ) ⩽ µ ( n ∧ m ) 24 ,

We now turn to bounding E [Tr D ] . We have

and denoting we have

The variables X 1 , ..., X n , Y 1 , ..., Y m are independent so we get

The last equality is due to E [ αφ ( x ) φ ( x ) ∗ +(1 -α ) φ ( y ) φ ( y ) ∗ ] = 0 . We have

and the equivalent inequality is also verified for Y . Then,

so we (24) becomes

Finally, (23) becomes

Taking u = 3 4 we get the final bound

Using similar calculations, we obtain

B.3.3 Final proof of Proposition 4

From [Bach, 2022, Proposition 7] we already have a bound on the entropy term:

In the following we will closely follow the proof of [Bach, 2022, Proposition 7] in order to bound the cross terms difference. We can write with the integral representation in Bach [2022] (Eq 5),

We will treat separately the integral part close to infinity, the one close to zero and the central part. Let β 1 > β 0 > 0 . From β 1 to infinity we have

From 0 to β 0 we have

By Proposition 11 we have

We now take β 0 such that C ( β 0 ) = µ n ∧ m 24 log( n ) . The function β ↦→ C ( β ) being non-increasing on ]0 , ∞ [ , the condition of Proposition 11, which is C ( β ) ⩽ µ ( m ∧ n ) 24 , is well satisfied between β 0 and β 1 for this choice of β 0 . We then have

and also, ∫ β 1 β 0 C ( β ) 2 dβ ⩽ c . Then,

The function β ↦→ C ( β ) is decreasing so, C ( β ) 2 β 2 ⩽ ∫ β β/ 2 C ( β ′ ) 2 dβ ′ ⩽ c and so, C ( β ) ⩽ √ 2 c β . We also deduce from that

Hence

We now take β 1 = β 0 + n . We have

Then, plugging the bound on β 0 given by (26),

Concatenating with (25), we get

Using increments such that 1 ⩽ log n ⩽ (log n ) 2 , 1 n ⩽ 1 n ∧ m ⩽ 1 √ m ∧ n ⩽ 1 and α, µ ⩽ 1 , we can upperbound (28) by a simpler bound, while still retaining the main convergence rates in log n √ m ∧ n and in (log n ) 2 m ∧ n . This bound is

As mentioned in Remark 5, it is possible to re-write this proof without considering the assumptions that p ≪ q and dp dq ≤ 1 µ and to derive a bound similar to this one but which scales in O ( 1 α 2 ) instead of O ( 1 α ) . This bound is

To get this new upper bound, the operator inequality Γ ≽ (1 -α )Σ q ≽ (1 -α ) µ Σ p must be replaced, each time it is used, by Γ ≽ α Σ . This way, the operator inequality (Γ+ βI ) -1 ≼ 1 µ (1 -α ) (Σ p + βI ) -1 becomes (Γ + βI ) -1 ≼ 1 α (Σ p + βI ) -1 which explains the additional factor 1 α in the final bound.

B.4 Proof of Proposition 6

According to [Bach, 2022, Proposition 6] we have that the eigenvalues of Σ ˆ p (resp Σ ˆ q ) are the same than the ones of 1 / n K ˆ p ; and we also have that for an eigenvalue λ of 1 / n K ˆ p with associated eigenvector α , the function f = ∑ n i =1 α i φ ( x i ) is an eigenvector of Σ ˆ p associated to the same eigenvalue. Hence, denoting ( λ i ) n i =1 the eigenvalues of 1 / n K ˆ p and a s the associated normalized eigenvectors, the first term in (2) writes:

We now turn to the second term in (2). Let ϕ x = ( φ ( x 1 ) , . . . , φ ( x n )) ∗ , ϕ y = ( φ ( y 1 ) , . . . , φ ( y m )) ∗

also remark that ψψ T = K . Knowing that the operator ψ T ψ and the matrix ψψ T have the same spectrum, we will replace α Σ ˆ p +(1 -α )Σ ˆ q by K in the expression of KKL α , which we can do with the following lemma.

Lemma 12. If ψ ∈ R d × r with d > r , g : R + → R and ψψ T is invertible, then g ( ψ T ψ ) = ψ T ( ψψ T ) -1 2 g ( ψψ T )( ψψ T ) -1 2 ψ

Proof. Let ψ = UDiag ( S ) V T the singular value decomposition of ψ with U ∈ R r × r and V ∈ R d × d orthonormal matrices. We have : ψψ T = UDiag ( S 2 ) U T and ψ T ψ = V Diag ( S 2 ) V T , so, g ( ψψ T ) = UDiag ( g ( S 2 )) U T and g ( ψ T ψ ) = V Diag ( g ( S 2 )) V T . And so, g ( ψ T ψ ) = V U T g ( ψψ T ) UV T . Noticing that ( ψψ T ) -1 2 ψ = UV T concludes the proof.

By Lemma 12 we can write the second term in (2) as:

where the last equality results from the fact that K 1 2 and log K commute because they have the same eigenbasis.

B.5 Proof of Proposition 7

We write F = F 1 + F 2 and derive the first variation of each functional in the next two lemmas. Then, we conclude on the first variation of F .

Lemma 13. Let ˆ p as defined in Proposition 6. The first variation of F 1 : ˆ p → Tr(Σ ˆ p log Σ ˆ p ) at ˆ p is, for x ∈ supp(ˆ p ) :

and + ∞ else.

Proof. In this proof we use residual formula which is useful to derive spectral functions 1 . Indeed, we can write Tr(Σ ˆ p log Σ ˆ p ) = ∑ n i =1 f ( λ i (Σ ˆ p )) with f : C \ R -→ C , z → z log z . Consider a perturbation ξ ∈ P ( R d ) , ε > 0 and let ∆ = ε Σ ξ . Let z ∈ C . Using the linearity of p ↦→ Σ p , we have

Let γ be a closed directed contour in C \ R -which surrounds all the positive eigenvalues of Σ ˆ p and Σ ˆ p +∆ . We have

where

The residue of h ( z ) = f ( z ) ( z -λ k ) 2 = z log z ( z -λ k ) 2 at λ k is 2 Res ( h, λ k ) = 1 + log λ k . Applying again the residue formula we have

This concludes the proof by dividing the later quantity by ϵ and taking the limit as ϵ → 0 .

Lemma 14. Let ˆ p, ˆ q as defined in Proposition 6. The first variation of F 2 : ˆ p → Tr(Σ ˆ p log( α Σ ˆ p +(1 -α )Σ ˆ q )) at ˆ p is, for any x ∈ supp(ˆ p ) :

̸

where ( η j , h j ) n + m i =1 are the eigenvalues and eigenvectors of α Σ ˆ p +(1 -α )Σ ˆ q .

The second term on the r.h.s. is already linear in ∆ as desired, hence we focus on the first one. Using a singular value decomposition of ˆ Γ + α ∆ and ˆ Γ we write: :

where ( h ′ j ) j are the eigenvectors of positive eigenvalues of ˆ Γ + α ∆ . Let γ a loop surrounding all the eigenvalues η j ( ˆ Γ + α ∆) and η j ( ˆ Γ) , then,

Moreover, we have ( zI -ˆ Γ -α ∆) -1 -( zI -ˆ Γ) -1 = ( zI -ˆ Γ) -1 α ∆( zI -ˆ Γ) -1 + o ( ε ) . Hence,

̸

With the residue theorem, for j = k , ∮ γ log( z ) ( z -η j )( z -η k ) dz = 2 iπ ( log η j ( η j -η k ) + log η k ( η k -η j ) ) = 2 iπ log η j -log η k η j -η k , and for k = j , ∮ γ log( z ) ( z -η j ) 2 dz = 2 iπ η j . We then have:

̸

We note that if Σ ˆ p an Σ ˆ q were diagonalizable in the same eigenbasis, then the previous quantity would be equal to α ∑ n + m j =1 λ j η j Tr ( h j h ∗ j ∆ ) = TrΣ p Γ † ∆ where Γ † is the pseudo inverse of Γ . We conclude again dividing the latter quantity by ϵ and considering its limit as ϵ → 0 .

We can now write the matrix expression for the first variation of F using Lemma 12. We remind that ϕ x = ( φ ( x 1 ) ∗ .. φ ( x n ) ∗ ) (resp ϕ y ) and ψ = ( √ α n ϕ x √ 1 -α m ϕ y . ) , and that ψ T ψ = α Σ ˆ p +(1 -α )Σ ˆ q and ψψ T = K . We remark that Σ ˆ p = 1 √ n ϕ T x 1 √ n ϕ x and ϕ x ϕ T x = 1 n K ˆ p . By Lemma 12, we have

since S ( x ) = ϕ x φ ( x ) . We show the same way that Trlog( α Σ ˆ p +(1 -α )Σ ˆ q ) φ ( x ) φ ( x ) ∗ = T ( x ) T g ( K ) T ( x ) .

For the third term in the first variation of F = F 1 + F 2 , i.e. the second one in Equation (31), our goal is to rewrite Tr ( h k h ∗ k Σ ˆ p h j h ∗ j ∆ ) in terms of matrices. We have h k = ψ T c k / ∥ ψ T c k ∥ (idem j)

where c k is an eigenvector of K of eigenvalue η k , and ∥ ψ T c k ∥ 2 = K c k = η k . Hence,

We have ψφ ( x ) φ ( x ) ∗ ψ T = V ( x ) V ( x ) T where V ( x ) = ( α n k ( x, x 1 ) , . . . , 1 -α m k ( x, y 1 ) ) T , and if we note c j = ( a j , b j ) T :

̸

Finally,

C Additional Experiments

Skewness and concentration of the KKL . In these examples, we plot KKL α (ˆ p || ˆ q ) as the number of n = m samples of two distributions p, q increases. In Figure 4 we plot the KKL for two Gaussians by varying the dimension d . It can be seen that the larger d is, the less KKL oscillates. We can also remark that the value of KKL increases with the dimension, reflecting the effect of the constants of Proposition 4. Figure 5 is the same experiment as in the main text, except that the dimension is 2. We can also see here that KKL α is monotone in α . We can also see that convergence to the value of KKL in population is faster in this case than for d = 10 . The third experiment in Figure 6 is in dimension 1 and the distribution of q is an exponential distribution with parameter λ = 1 , while p is a Gaussian distribution. We can notice a few points, such as for example that the values taken by KKL are smaller and that it varies less with α than in the case of Figure 5.

0

500

1000

number of particules

3.0

3.5

4.0

KKL (

p

||

q

)

d

= 2

d

= 3

d

= 4

d

= 5

d

= 10

d

= 20

0

500

1000

number of particules

0.0

2.5

5.0

7.5

10.0

KKL (

p

||

q

)

= 1e-06

= 1e-05

= 0.0001

= 0.001

= 0.01

= 0.1

= 0.5

0

500

1000

number of particules

0.0

0.5

1.0

1.5

2.0

KKL (

p

||

q

)

= 0.0001

= 0.001

= 0.01

= 0.1

Sampling with KKL gradient descent on Gaussians and mixtures of Gaussians. We are interested in sampling on Gaussians or mixtures of 2 Gaussians by varying the dimension. Figure 7 and Figure 8 show the evolution of the KKL value during the gradient descent of different dimensions d , starting with a Gaussian p and taking q to be a mixture of 2 Gaussians for Figure 7 and p and q Gaussians distributions for Figure 8. In Figure 7 and Figure 8, the stepsize

h = 1 n ( ∑ i,j ∥ x i -y j ∥ 2 ) 1 / 2 n -1 / ( d +4) . For each d , we report the average and error bars of our results for 20 runs, varying the samples drawn from the initialization and the thick lines represents the average value. We can see that in both cases the convergence is faster for small values of d . On Figure 9 we observe the evolution of W 2 (ˆ p || ˆ q ) , the 2 Wasserstein distance, during gradient descent in dimension d = 10 for various parameters α . The distribution p and q are respectively a Gaussian and a mixture of 2 Gaussians. The values of W 2 at each iteration t is computed as the mean of W 2 (ˆ p, ˆ q ) on 10 runs of the gradient descent where for each the mean of p is drawn at random. We can see that if the α value is too high, then convergence in 2-Wasserstein is slower, whereas if it is too small, convergence is faster at the beginning, but does not lead to an optimal value in Wasserstein distance at the end of the algorithm.

0

25

50

75

100

iterations

0.0

0.2

0.4

0.6

0.8

1.0

KKL (

pT

||

q

)

d = 2

d = 3

d = 5

d = 10

d = 20

0

25

50

75

100

iterations

0.0

0.2

0.4

0.6

KKL (

pT

||

q

)

d = 2

d = 3

d = 5

d = 10

d = 20

0

100

200

300

iterations

0

2

4

6

8

10

W

2(

pT

||

q

)

=  = 0.0001

=  = 0.001

=  = 0.01

=  = 0.1

=  = 0.5

3 rings. Appendix C compares the evolution of the gradient flows of MMD, Kale and KKL α in terms of Wassertsein distance and Energy distance in the case where optimisation of KKL is done with L-BFGS linesearch in Figure 2. We observe that both Kale and KKL seem to converge towards 0 in terms of energy distance and Wasserstein distance but KKL α is faster to converge, in term of number of iterations than Kale. The MMD flow decreases the energy distance but does not converge to 0 in 2-Wasserstein distance, unlike Kale and KKL, reflecting the fact that some particles are not supported on the target support. The bandwidth of k is fixed at σ = 0 . 1 for Kale and MMD and at σ = 0 . 3 for KKL . In Figure 13 this time we repeat the experiment but for a simple gradient descent for KKL with constant step h = 0 . 01 . We see that in this case the speed of convergence in terms of iterations for KKL is slower than for Kale (there are only about 100 iterations necessary for Kale and MMDand 300 for KKL ) but it ends up obtaining (see Figure 12), in terms of Wassertsein distance, a similar limit. On the other hand, the execution time of the gradient descent for 300 iterations of KKL is about the same as for Kale and MMD for 100 iterations.

0

25

50

75

100

iterations

0.0

0.2

0.4

energy distance

KKL

MMD

KALE

0

25

50

75

100

iterations

0.0

0.2

0.4

0.6

0.8

W2 distance

KKL

MMD

KALE

0

100

200

300

iterations

0.0

0.2

0.4

0.6

0.8

W2 distance

KKL

MMD

KALE

KALE

T=0

T=2

T=99

T=300

MMD

KKL

D NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: the abstract and introduction list our contributions, i.e. introduction of a novel divergence, theoretical and empirical novel results..

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: we clearly state the limitations of our work, among which the lack of theoretical guarantees for the convergence of the Regularized KKL gradient flow, and more efficient numerical implementations resorting on state-of-the-art techniques for kernels matrices operations.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: For each theoretical result, we worked with minimal and justified assumptions, making all the dependencies of the problem clear. We provide a clear and detailed proof in the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: For each experimental result, we provide all the detailed setting including choice of hyperparameters to reproduce it. We took several example datasets used in papers that are close to our work.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We will provide a link to a public github repository with a Python code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: we specify all details for all our experiments, report the variability of our experiments over a set of different runs.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For our experiments we provide averages and error bars for the results over a set of runs.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Our experiments run on a standard laptop.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: In our opinion, this papper does not address societal impact directly, and consider the generic problem of optimization over measures and sampling.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: In our opinion the paper does not have direct positive or negative social impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper does not present such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: all original owners of assets are credited, for instance regarding our experi977 ments, we cite the public papers and code that were used as benchmarks for comparison.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: we do not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: our experiments do not involve crowdsourcing.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: our study do not involve risk for participants.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper proposes the regularized kernel KL divergences. With the kernelization, the authors derive closed form formula and makes it computationally cheap to optimize using gradient methods. Theoretically, a new finite sample estimation convergence bound is derived. Numerical simulations suggest this proposed divergences are not only easy to implement, but also suitable for sampling complicated distributions.

STRENGTHS:
1. The paper is well-written and easy to follow.

2. Finite sample approximation bound is derived.

3. Numerical simulations demonstrate good performance over other related divergences.

WEAKNESSES:
1. Proposition 1 still assumes $p$ is absolutely continuous with respect to $q$.

2. The bound in Proposition 3 becomes trivial since it approaches $\infty$ as $\alpha\to 1$.

QUESTIONS:
1. Typo at line 301: ''For small values of the regularization parameter'' $\to$ ''For large values of the regularization parameter'.

2. Can the authors comment on how the performance of KKL compares with that of skewed KL divergence which is kernel-free and not restricted by a kernel, though the later one can be computationally more expensive? 

3. You mention your algorithm is robust to the choice of $\alpha$. But I was wondering when $\alpha$ is close to 1, as suggested in Figure 1 even for $\alpha=0.5$, the value of the divergence is always small, so it will be hard to distinguish between different distributions. Can you comment on this?

4. Why does Figure 2(a) begin with different particles across different methods at $T=0$?

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-06-19

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors generalize and analyze the kernel Kullback-Leibler divergence introduced by Francis Bach a few years ago. The main contributions are 1) use a skew divergence (like in JS) tinstead of KL  consider non-absolutely continuous divergences, 2) provide an explicit formula of the divergence for discrete measure which allows a direct implementation for data, 3) provide refined finite sample guarantees. The divergence is used then to build a Wasserstein gradient flow and generative modeling. Some low-dimensional numerical examples are provided.\n\nSTRENGTHS:\nThe work extend the original work by F. Bach in several aspects. In particular the explicit formula of the divergence for discrete measures is a nice and very useful results which wiil come handy  in applications. The paper is very clearly and nicely written and the proof looked sound to the reviewer.\n\nWEAKNESSES:\nThe reviewer find the applications somewhat underwhelming. The gradient flows built via kernel methods do not seem to perform particularly well, perhaps due to the well-known lack of expressivity. One might have hope that the covariance embeddings in these new divergence would have helped compared to MMD based gradient flow but the authors experiments do not make that case.  Kernel methods have the advantages that provide explicit formula for the velocity vector field and one would hope that this allow to consider high-dimensional problem but no such experiment have been provided so it is not clear if the proposed model scales up.\n\nQUESTIONS:\n1) Is there a reasonable expectation that their flows to scale up to high dimension? Or what is the obstacle?\n\n2) It seems to the reviewer that Wasserstein gradient flow based on kernel methods do not seem to perform well compared to gradient flows using neural network architecture.  This occurs in the case of SVGD but also for plain Wasserstein gradient flows. For example the gradient flows in Gu et al (https://arxiv.org/abs/2210.17230) based on Lipschitz regularization of KL-divergence are comparable in spirit to the MMD of Kernel-KL flows considered here.  Is the reviewer mistaken? Can the authors provide some insights on these issues.\n\nLIMITATIONS:\nThe limitations of the work are properly addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nComparing probability measures is a fundamental task in many statistical and machine learning problems. One of the metrics that has been ubiquitously used is Kullback-Leibler (KL) divergence. KL contrasts the information contained in two probability distributions. Statistical learning using KL divergence involves density ratios and needs the fact that the supports of probability measures in question should be not disjoint. \n\nA kernel variant of KL called kernel KL (KLL) divergence was proposed in Bach (2022, IEEE Trans. Info.). KLL compares probability measures through covariance operators in an RKHS space.  As the standard KL, the KLL shares the limit of inability to compare measures with disjoint supports. This paper proposes a regularized variant of KLL that guarantees calculating divergence for all distributions. The authors derive bounds that quantify the deviation of regularized KLL to the original KLL.  they further propose a closed form for KLL for empirical measures. Numerical experiments are conducted on simulated data to corroborate the theoretical results.\n\nSTRENGTHS:\n- Proposing $KKL_\\alpha$ a regularized version of standard KLL, that allows calculating divergence for all distributions.\n- Presenting statistical properties of $KKL_\\alpha$.\n- Providing a closed for KLL between empirical measures.\n- Plugging $KKL_\\alpha$ in a Wasserstein gradient flow learning.\n\nWEAKNESSES:\n**Weakness and Questions**\n\n- The decreasing property  of $KLL_\\alpha$ in Proposition 2 needs the condition that $p$ is absolutely continuous with respect to $q$. However, in the definition of $KLL_\\alpha$  (L144-L145) there is no need for this absolutely continuous. My question does this property still valid without $p<<q$ ? The same remark for the result in Proposition 4.\n- In Proposition 4, the definition of $\\varphi(x)$ needs to be recalled. \n- In Proposition 4, could you please motivate the condition On the Radon-Nikodym derivative to be bound above by $1/\\mu$.\n- To facilitate the lecture on the bound given in Equation (5), I think it will be better to write its order. \n- In Proposition 5, I didn't understand the definition of the gram kernels $K_x, K_y, K_{xy}$. Do these operators design the covariance operator or other more general kernels?\n\nQUESTIONS:\n**Minor Typos**\n\n- L108: the notation $p$ is used for denoting probability measure and the dimension $\\mathbb{R}^p$.\n- L368: there is no caption of the figure and for (b) Shape transfer, the iteration $T=99$ is the same.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-14"}

comments: SUMMARY:
### Summary:

 

In this paper, the authors study kernel KL divergences (Equation 1). First, they extend the definition of kernel KL to a new setting via regularization that works for distributions with disjoint support (Equation 2). 
In Propositions 2,3, they prove the approximation results for the regularized kernel KL divergence compared to the unregularized version of it. 
Second, they derive convergence rates for finite sample estimation of kernel KL divergences (Proposition 3). Moreover, they obtain a closed-form expression for the new divergence (Proposition 5). They also study Wasserstein GF of the new divergence in the paper.

STRENGTHS:
### Pros:


- very well-written paper
- well-motivated setting with comprehensive results

WEAKNESSES:
## Cons:

- the title does not reflect the contributions of the paper

QUESTIONS:
### Questions/Comments:

I recommend changing the title of the paper. The whole draft is devoted to the study of "regularized kernel KL divergence" but in the title there is no reference to being regularized. 


- line 124 -- what do you mean by "if $\mathbb{R}^d$ is compact"

- line 131 -- typo "an interesting"

- how tight Proposition 4 is?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-24