Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysis
Authors: Christophe Vauthier
Anna Korba
Quentin Mérigot
Venue: ICML 2025 poster (2025)
Keywords: optimal transport
optimisation
sliced-wasserstein distance
Primary area: optimization

Abstract: In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when employed as an objective functional. The SW metric has gained significant interest in the optimal transport and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining  computationally tractable, making it a valuable tool for various applications, including generative modeling and domain adaptation.
Our study aims to provide a rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concentrate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objective. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.

Towards Understanding Gradient Dynamics of the Sliced-Wassersee Wasserstein Distance via Critical Point Analysis
Christophe Vauthier'
Anna Korba Quentin Mérigot
Abstract
such tasks has been to consider gradient flows, i.e., optimiza-
In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when em- ployed as an objective functional. The SW metric has gained significant interest in the optimal trans- port and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining compu- tationally tractable, making it a valuable tool for various applications, including generative mod- eling and domain adaptation. Our study aims to provide rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concen- trate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objec- tive. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.
tion dynamics on the space of probability measures, to min- imize an objective functional of the form F(u) = D(ulp), where D is a discrepancy (c.g. a distance, or a divergence) between measures, In practice, these can be simulated by considering an initial distribution that is a discrete mea- uniformly supported on set of particles. The parti- cle positions then evolve according to a system of ODEs X = VF((). which corresponds to the gradient flow of functional F (IR + R, where d is the dimension of the space and N the number of particles. Then, a practical scheme is derived by discretizing in time this flow, e.g. with gradient descent. Reversely, gradient descent on particles can be seen as a discretized flow described by this system of ODEs.
Many divergences or distances can be considered the dis- crepancy D, each offering different tradeoff's between attrac- tive geometrical properties and computational burden the associated training dynamies. Generally the objective func- tion chosen so that the dynamic tractable given the avail- able information on p. When the density of p is known up to normalization constant, often the case in Bayesian infer- ence, standard choices include the Kullback-Leibler diver- gence (Salim et al., 2020), Kernel Stein Discrepancy (Fisher et al., 2021; Korba et al., 2021) or (eventually weighted) Fisher Divergences (Cai et al., 2024a;b). On the other hand, when samples of the target distribution are available, Integral Probability Metrics (IPM) or Optimal Transport distances are preferred, since they are well-defined for discrete mea- sures. For instance in generative modeling, while original Generative Adversarial Networks are known to optimize a Jensen-Shannon divergence to the distribution of the sam- ples (Goodfellow et al., 2020) and can be understood via the perspective of Wasserstein flows (Yi et al., 2023), wide range of these metrics have been used for the training of GAN variants, e.g. Wasserstein-1 (Arjovsky al., 2017), Sinkhorn divergences (Genevay et al., 2018), Maximum Mean Discrepancies (Li et al., 2017) or novel metrics in- terpolating between IPM and f-divergences (Birrell et al., 2022). Alternatively, recent work directly tackled gener- ative modeling tasks through simulating Wasserstein gra- dient flows of such discrepancies, e.g. distances (Liutkus al., 2019; Dai & Seljak, 2021; Du
1. Introduction
An important problem in statistical learning is approxi- mate an intractable target probability measure p Rd with probability measure supported on finite set of points. Such problems arise in various contexts, such as sampling from Bayesian posterior distributions (Blei et al., 2017; Wibisono, 2018), generative modeling (Bond-Taylor et al., 2021) and training neural networks (Chizat& Bach, 2018; Mei al., 2018). Recently, a popular framework address
'Laboratoire de Mathématiques d'Orsay, Université Paris- Saclay, Gif-sur-Yvette France "Centre de recherche en économie et statistique, ENSAE, Palaiseau, France. Correspondence to: Christophe Vauthier xchristophe.vauthiin wauttherruuiivessseeeppep. universite-paris- saclay.fr>, Anna Korba <aanaa wew.ae.fr.. Quentin Mérigot gqquenin.merigott. universite- saclay-faa........
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Towards Understanding Gradient Dynamics of the Sliced- Wassestessnrttiinn Distance
al., 2023), Energy distances (Hertrich al., 2024), or f-divergences (Fan et al., 2022; Choi et al., 2024). For all these methods, the choice of the discrepancy objective is crucial for their empirical success'.
of critical points:
We will call an Eulerian critical point if it satisfies (1) for all perturbations of μ of the form f4 = (1-t)u+ttv for v E (R((Rd). This coincides with the standard notion of critical point on the "flat" space Pa (2((K)) (i.e.. not equipped with W2).
For instance, Wasserstein distances themselves appear to be suitable objectives, in the sense that they preserve the geometry of probability distributions, e.g. when comput- ing barycenters (Rabin et al., 2012). However, for discrete measures, such distances are known to suffer from a large computational cost and poor statistical efficiency (Peyré et al., 2019). To alleviate this issue, several alternatives to the Wasserstein distance were proposed. Among these, the Sliced-Wassersee Wasserstein distance (SW) (Bonneel et al., 2015) a computationally attractive proxy. It involves averages of Wasserstein distances in dimension 1 (each of which can be computed in closed-form) with respect to infinite number of directions. It has gained popularity in machine learning applications, such as computing barycenters of dis- tributions (Bonneel et al., 2015), variational inference (Yi & Liu, 2023) or recently generative modeling (Kolouri et al., 2018; Liutkus et al., 2019; Dai & Seljak, 2021; Du et al., 2023). While its statistical and computational properties have been studied extensively in the literature (Nadjahi et al., 2020; Manole et al., 2022; Nietert et al., 2022), the behavior of its optimization dynamics remain largely unknown. In this paper, we consider the objective functional F to be a SW distance to a fixed measure p. We consider a gradient descent scheme on particles, as well as its continuous time and space counterpart, as an optimization scheme pushing particles from source y to approximate the target p. As this latter optimization problem is non-convex, it is natural to study the critical points that may be encountered during minimization. Our main objective is not only to understand the discretized problem, but also its continuous time and space analog, which motivates us to propose a notion of crit- ical point for the continuous functional F that is compatible with the critical points for the discretized problem.
We will call μ a Wasserstein critical point if it satisfies (1) for all Wa-geodesics emanating from μ. If p is a probability density, we know from Brenier's theorem that geodesics are all curves of the form f = ((1 #1 with T the gradient of a convex function. t) ld ++T") #
Finally, we will call μ a Lagrangian critical point if satisfies (1) for all curves of the form 14 = (Id +11) for any vector field & E L (1,,,,
We now discuss the case where G = 9p := 2 W (p) is the squared Wasserstein distance to a probability density p to fix ideas. First, we note that the only Eulerian critical point of this functional is p. a non-obvious fact, which follows from strong convexity of this Gp (Santambrogio, 2015, Proposi- tion 7. 7.19). However, such critical points are not meaningful when considering continuous time limits of gradient descent schemes (the ODE dynamics obeyed by the particles), as we do in this paper. Second, if μ # p and if (4))tefo,11 is the W2-geodesic between μl and p. one can verify that Go (1)) S 9p(p)) - cl for some e >> 0, thus implying that μ is not critical. Therefore, the only Wasserstein critical point of G, is, again, μ = p. In this case, every Wasserstein criti- cal point is therefore also Lagrangian critical point. The converse holds when is absolutely continuous, because one can take & = T - Id, but not in general. As explained in (Mérigot et al., 2021) and studied in detail in (Sarrazin, 2022, Chapter 4). the functional Gp admits many Lagrangian critical points. First and foremost, any local or global min- imizer of X (ZI,...,,N') IN) E ((R))N + Gp N Σ, dn,)) induces a Lagrangian critical point μux N Σ, 6.. (show- ing the practical relevance of this notion), but moreover any Wz-limit of Lagrangian critical points are Lagrangian critical (with the caveat that the definition of Lagrangian critical points in (Sarrazin, 2022) is restricted to compactly supported measures and continuous perturbations &). This notion of critical point translates difficulty that comes from the discretization, but that persists in the continuous limit.
We note at this point that there exists many natural notions of critical points for a functional G defined on the space of probability measures over Idd A measure je is called a critical point of G if for any curve (J4) in the space of measures such that pt0 = μ belonging to a certain family of allowed perturbations, one has
d at
((m)) 1=--
O.
Contributions and outline. Regarding the theoretical guarantees of optimization schemes applied to SW, a natu- ral question is the following: given a sequence of discrete measures (ux) supported on N atoms, and constructed us- ing a first-order algorithm applied on a SW objective, can we expect this sequence to converge to the target measure
Our aim at this point is not to discuss the differentiability as- sumptions on G, and we will therefore remain at an informal level. Depending on the set of allowed perturbations, we will recover several distinct and arguably interesting notions
Note though that GANs use parametric setting, that is, we optimize 0 + D(uo. p) where @ is a parameter vector for neural network.
(1, R = {f TR. + R². {(( < oo).
2

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstinn Distance
p N + oo? This question is difficult because of the non-convexity of the discretized SW objective. However, we could hope that the non-convexity becomes milder as N + +o0, in the spirit of (Chizat & Bach, 2018; Mérigot et al., 2021).
is defined
WD(u inf / xell(u,,) JR
|| y/l (2, (2)
where || denotes the Euclidean norm, and [[, v) is the set of probability measures on Id R with marginals y and v.
Our paper is first step towards answering this question and organized follows. In Section 2, we introduce the neces- sary background on optimal transport and distances. In Section 3, we discuss properties of gradient descent of the functional F over discrete measures and of its critical points, showing in particular that trajectories of gra- dient descent avoid the non-differemiabiiliyy locus of F. In Section 4, we give an explicit characterization of Lagrangian critical points of the SW objective F = & Sw p), and we prove that our notion of critical points passes to weak limits under mild assumptions. This implies that the limit of discrete critical points (e.g., obtained numerically), is Lagrangian critical point. In Section 5 we construct explicit examples of Lagrangian critical points of F supported on diverddimessooal subsets of IR This shows in particular that there exists "bad" Lagrangian critical points points of the SW objective which are distinct from the target p. A natural question is then whether these "bad" Lagrangian critical points can actually occur as the limit of discrete measures obtained by an optimization algorithm. Since we expect that gradient descent will converge to stable criti- cal points (Panageas et al., 2019), it is tempting to rule out these bad critical points by showing that they are unsta- ble. We establish in Section 5 in dimension Lagrangian critical point that contains segment must be unstable. Since our proof relies on delicate explicit compu- tations, the extension to lower dimensional critical points in higher dimension is left as future work. Finally Section 6 presents illustrations of our theoretical results on numerical experiments.
ID optimal transport Consider probability measures H, 1/ E PP)(R), and let F-1 and F be their quantile func- tions, i.e. (t) = inf E IR Fx))) 2 f where Fμ. is the cumulative distribution function (cdf). By (Rachev & Rüschendorf, 1998. Theorem 3.). the ID Wasserstein distance is the L distance between the quantile functions,
WH(L)) = J
- F dt)
(3)
If = (#1,...,,NN) 2N) C RN is finite set in R. μ N Σ, dr. is the associated empirical measure, and o is a permutation such that i + (x) is non-decreasing (sim- ilarly, we define ,yy,ay) Equation (3) becomes more explicit:
1 N Σ N Fox(()) (o
W5(
(4)
showing the complexity of ID optimal transport is the same as sorting, i.e. O(N log N). However, in dimension higher than one, there is no explicit expression for WT v) and despite the progress made in the last decade, the compu- tational cost remains superlinear in the number of atoms (Peyré et al., 2019).
2 that any
Wesedd-Wssrrstein distance The Sliced- Wasserstein (SW) distance (Rabin et al., 2012) defines an alternative metric by leveraging the computational efficiency of W for univariate distributions. For O E sd. Po IR + R denotes the linear form x + (0|x). Then, the SW distance of order p between μ, v E (Rd) is
2. Background
Measures and optimal transport We first give some background on optimal transport distances. We denote P(IR)) the set of probability measures on RRd and (I the set of probability measures with finite pth moment (p 2 1). The d-dimensional Lebesgue and h-dimensional Hausdorff measures are denoted respectively by Ld and H* In our setting, probability density p on Idd is a probabil- ity measure which is absolutely continuous with respect to the Lebesgue measure; for simplicity we will often use the same notation for p and its density. Given a measurable map T from Rd to itself and μ E P P(R"), Ty denotes the pushforward measure of j by T. The Wasserstein distance of order p between any probability measures μ. (R:)
SW Jgu { WP Pay
(5)
where Sd is the (d - 1) -)ddimensional unit sphere and d0 is the uniform distribution on sá-1, Since Pozut Pogu are univariate distributions, the Wasserstein distances in (5) are conveniently computed using (3). The sliced-Wai -asserstennr distance SW, is always smaller than the original Wasser- stein distance (Bonnotte, 2013, Proposition 5.1.3), and is even bi-Hölder equivalent to this distance on the subset P(B((0, R)) C (Rd).. The computational and statistical aspects of Wassee--esss distances are by now well studied, we refer to (Nadjahi et al., 2020) and references therein.
3

---

Towards Understanding Gradient Dynamics of the Sliced-Wassereee Wasserstinn Distance
3. Discrete Sliced- Waiser-Wasserstein distance dynamics
In the particular case where p 2, this expression can be farther simplified by introducing the barycenters of the Power cells Vo i.e. b -N Svv,, =dPo)p(():
Before investigating the convergence of the gradient flow of the Sliced- Wasserstein distance to its critical points and the characterization of the latter, we first study in this sec- tion the optimization of the Sliced- Wasserstein distance in practice, where the optimized (source) measure is discrete. Our first subsection studies the differentiability properties of the Wasserseens objective when the first argument is a discrete measure, while the second provides a descent lemma for this objective. Finally, we show quantitatively that for a suitable stepsize, gradient descent does not col- lapse particles and is thus well-defined for all times.
1 [xx N
1
F(x
SSL. b,, (eox)(1)000) (9)
The proof of Proposition 3.1 is deferred to Appendix B. This proposition is valid in the semi-discrete setting, where the source measure is finitely supported and p has a density, while similar results in the literature tackle different settings, e.g. fully-discrete (Tanguy et al., 2024a) or where both measures densities (Manole et al., 2022).
Descent lemma. While our previous result provides a gen- eral formula for gradients of SW distances of order 2 2, we focus on the particular case p = 2 where the compu- tations are the most simple. We then have the following "descent lemma", which gives guarantees that a gradient step decreases the loss, for the gradient descent on F. Proposition 3.2. For every X E (RRd) \ AN and every X > O, denoting Y == X - XVF(X), we have
Differentiability of the SW functional. We consider target probability density E PP(())), and we define the function
I SWE(ex,)) p), (6) F: x (X1. ... XN) E (BO)N H p
where μxx N Ei= ōx; is the uniform empirical measure associated to the set of points X. As p has finite p-moment, F(X) < +oo for every point cloud X. As seen in Sec- tion 2, the SW distance involves sorting the projections of X over directions. However, the sorting operation, seen as a function of E to RN, is piecewise linear and non- differentiable when two of the coordinates agree. We may therefore expect our functional F to be non non-differentiable at any point cloud X which belongs to the generalized diag- onal AN= == {(;
F(Y) - F(X) - -1-1111111 2Nd |VF(X)I (10)
The proof of Proposition 3.2 is provided in Appendix B.2 and relies on the semiconcavity of F. This proposition implies that if X is not a critical point of F and if the step- size X belongs to (0, 2Nd). one gradient descent step from X strictly decreases the value of F. In particular, the r.h.s. of the inequality (10) is minimal for a step-size X = Nd, and we may expect the convergence speed of the gradient descent to be the fastest for step-sizes around this value. Considering the expression of VF(X) given by (9), one iteration of the gradient descent with such step writes:
E (RdyN Ii & j, X; = X₁). The next proposition shows differentiability of F on the complement of this generalized diagonal.
As usual, we denote GN the group of permutations of N}.. We will use the notation Vo,; for the i-th Power cell associated to Pe Peup, i.c.
f(e
Vo, Froup ([ (+)) N' N
x + -Nd,, = s 10,0 (1) 0d0.
(7)
(11)
Interestingly, choosing a step of N for the SW? objective is reminiscent of the results obtained by (Mérigot et al., 2021).
Moreover, given a point cloud X = (Xi,..., XN) E (R.)), we denote σx,0 E 6N a permutation such that the map i {L,..., N} + (()) ()) is non-decreasing. Proposition 3.1. p 2 2, then F is differentiable at any point cloud X = (X,,...,Xo. NN) E (Rd)) which does not belong to the generalized diagonal AN. The gradient of F is continuous on (Re \ AA, and its component with respect to the i-th vector X; is then
They study a variant of Lloyd's algorithm, which optimizes X + W p) by assigning to x+1 the barycenters of the Power cells (also referred to as Laguerre cells) associated to X and which was proven, under certain conditions, to approximate p closely after a single step (see Theorem 3 and Corollary 4 in (Mérigot et al., 2021)).
Another consequence of Proposition 3.2 is that the sum of squared gradients of F at X is bounded. Indeed, for X = Nd, we have
Vx.
sgn( (g(((X))0) - x))
Vo. (1)
((x,10) - x2p-1 OdPP (8)
2 |FF(*1)|
Nd (d(((") F(x+t))), (12)

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
which implies that any converging subsequence of (X*) converges critical point X* of the energy. The conver- gence of the whole sequence (X*) to a critical point is open in general. It can be proven if one assumes that that the energy level F- (FXx)) only contains a finite number of critical points, as in (Bourne et al., 2020, Appendix), but this hypothesis cannot be checked in practice. (Portales et al., 2024) proves convergence of the whole sequence of iterates of Lloyd-type algorithms in several settings, but they acknowledge that their techniques do not extend to the case of F = SW2( p) when p is probability density.
F= = 3 S (. p). Unlike in the previous section, where we worked in the semi-discrete setting (i.e. with f dis- crete and p density), our framework will hold for general H. p E (RR)..
4.1. Barycentric characterization
As in the introduction, we first define Lagrangian critical points using derivatives of F along perturbations of the measure.
Definition 4.1. A measure j E P2(RR is Lagrangian critical point for SW2(s,) p) if for every & E L2 (2( 1R,,
well-behavedness of gradient descent In the gradient descent scheme described above, it is a priori possible that the iterates will get close to the generalized diagonal AN. This is a problem, as F is only known to be differentiable on (Rd)a AN- The following property ensures that, if the densities of the projections of p are bounded, the iterates will remain away from AN.
d SW3((d++15 p) di
=0.
(14)
1=
The right derivative is always well-defined, since a convex function always has left and right directional derivatives, as will be justified in Proposition 4.7(a).
Proposition 3.3. Let X E (R \. AN and X > O, and define Y := X - XVF(X). Then, i (0, Nd), we have Y & AN.
As Definition 4.1 is difficult to verify in practice, will now define second notion of Lagrangian criticality, which we will prove to be equivalent to the first under mild as- sumptions on and which will be very similar in spirit to the concept of Lagrangian critical measures for the standard Wasserstein distance developed in (Sarrazin, 2022).
Furthermore, if there exists 3 > which bounds from above the density of Pou p for every 0 E Sd-1, then there exists dC C' C(d) such that for every i # j. l| - Xyll= < NS In particular. if X is a critical then |Y--Y > Xx,l- x111.11
We assume that μ E (2( fixed, and for every direction 6, we denote To the unique 1D optimal transport plan be- tween 148 P and Pa = Pa+p. We finally consider the barycentric projection To of this transport plan (Ambrosio et al., 2005, Definition 5.4.2), which we can define using conditional expectations:
point of F, then
dc
min X - X3
2
(13)
NB
The proof of Proposition 3.3 is provided in Appendix B.3. Interestingly, the proof strategy we use also implies that the continuous flow X- - F(X) is defined for all times when initialized from a point cloud (0) not in A.. as discussed in the same appendix.
70 : R - IR, u + EQU.Y) V U = u].
(15)
We are now ready to state our second definition of La- grangian critical points.
Remark 3.4. Note that Proposition 3.1, Proposition 3.2, and the first part of Proposition 3.3 actually admit straight- forward extensions, with the same statements, to the case where p is only assumed have no atoms (note that this includes for instance densities supported on a lower dimen- sional manifold of Rd (which are not absolutely continuous w.r.t. Lebesgue in Idd but are without atoms). Indeed, it turns out that for such for almost every 0 E gd-1, its projection Po#p has no atoms, and we can thus define the Power cells Vo,; and the barycenters by.i, which requires minimal changes in the proof. For further discussion, we refer to Appendix B.4, which also examines extensions of these results to more general target measures p E (2
Definition 4.2. A measure μ E P2(R)) is barycentric Lagrangian critical point for SW p) if U/ = 0 p-a.e. where Up is the vector field defined by
1 Up : x x d -=-
- (o((x)6))8dd)
(16)
Note that this integral is well-defined by the selection result (Villani, 2008, Corollary 5.22). Remark 4.3. This notion of barycentric Lagrangian critical point appears in (Li & Moosmueller, 2025) (although is not explicitly named), where it plays role in the study of the convergence of stochastic iterative approximation schemes for the -asseerttnn distance. Indeed, Assumption (A3) therein rewrites in our framework as "n is a barycentric Lagrangian critical point for SW2(., 11)" (see also (Li & Moosmueller, 2025, Remark 8)).
4. Characterization of critical points
The goal of this this section is to derive a rigorous charac- terization of Lagrangian critical points of the SW objective
5

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
Our notions of Lagrangian critical points compatible with the notion of critical points of the discretized problem defined in the previous section, as stated in the following Proposition.
on B(0, R), then μ p if and only if y is barycentric Lagrangian critical for SW2(
Now, we will see that Definition 4.1 and 4.2 coincide if 14, p are compactly supported and without atoms. For E P P(R*), we denote || [|Λم(() and (s)))100)) the norm and the inner product on (2,
Proposition 4.4. Assume that p is a probability density and let X E (R4)N \ AN. Then (X)) Of and only if px is a Lagrangian critical point for SW2(-,)) p) if and only if px barycentric Lagrangian critical point for SW2(.))
Proposition 4.7. Let μ E P2(RR), then
(a) The function Fμ (1, Rd) + R defined as follows is convex:
The proof of Proposition 4.4 is deferred to Appendix B.5. A natural (non trivial) follow-up question is then whether the limit of sequence of discrete critical points JA 1
1 F : 2161li - +ld (18)
N EE= ox, (e.g. obtained numerically) is also a critical point (as defined either in Definition 4.1 or in Definition 4.2). The following theorem provides an answer to this question.
(b) The vector field up belongs to (1, R2).. Furthermore, -20, belongs to the subdifferential of F, at O, that is, for every & E L2, 1.).
Theorem 4.5 (Limits critical points are critical), Assume that p is without atoms and supported on compact S2 C Rd. If sequence (1N)N21 of barycentric Lagrangian critical points for SW2( (((p)) p) converges weakly an atomles measure μ, then J is barycentric Lagrangian crilical for p)).
(0) 2v (E) K) S FE((E
(19)
(c) If p. and p have compact support and are without atoms, then for every vector field & E L2 Rd), ) the function g(t) = SW2((Id +tE) H.P p) is differentiable at t 0, with
The proof of Theorem 4.5 can be found in Appendix B.8. Crucially, it relies on the study of the intricate relationship between the two definitions of Langrangian critical points we have defined. This study is detailed in the next section.
<0(0) 2(ppll)
(20)
Corollary 4.8. If μa is Lagrangian critical point for SW2(.p) p). then it is also barycentric Lagrangian critical point for SW2( p). furthermee and p have compact support and are without atoms, then the converse statement is also true.
4.2. Technical tools for Theorem 4.5
We have already shown in Proposition 4.4 that the two no- tions of critical points agree for discrete measures. Here, we discuss why Definition 4.2 is also natural in more gen- eral setting, such as those of Wasserstein gradient flows, i.e., curves (μ4)sso of steepest descent with respect to the Wasserstein-2 (W2) metric of the objective F. Indeed, by (Bonnotte, 2013, Section 5.7.1), when P is absolutely con- tinuous, the absolutely continuous stationary points μ of the gradient flow dynamics of JF are characterized by
The proof of Proposition 4.7 and Corollary 4.8 can be found in Appendix B.6 and Appendix B.7 respectively. Our Proposition 4.7(c) extends the result (Bonnotte, 2013, 5.1.7. Proposition) the differentiability of SW. In particular, Bonnotte's 's results holds under the strong assumption that M4 P are absolutely continuous, whereas Proposition 4.7(c) makes the much milder assumption that they are without atoms.?
J (0(((())))))) = 0, a.c. I Rd
5. Lower-dimensional critical points: existence and instability
(17)
where is the Kantorovitch potential from μ4 to Pe for the cost csstt) = 1 (s - t)2. But since we have 466 = Id -To where To is the unique optimal transport map from μ4 to Po (Santambrogio, 2015, Section 1.3.1), and 5 To (as na (Id, + we see that (17) rewrites as o, = μ-aae and thus absolutely continuous measure μ is stationary point of the Wasserstein gradient flow of F iff it is a barycentric Lagrangian critical point. Furthermore, (Bonnotte, 2013, Lemma 5.7.2) immediately rewrites as the following result:
5.1. Leveraging symmetry to find critical points
Now that we have characterized Lagrangian critical points, it is natural to ask ourselves whether there can exist La- grangian critical measures μ different than the target distri- bution p. An effective approach construct such critical points is to look for measures that are supported on a sym- metry axis of a well-chosen measure p. Our next result provides several examples.
For instance, distributions on lower dimensional manifolds do not have density with respect the Lebesgue measure but be without atoms.
Proposition 4.6. (Bonnotte) If p. E P(B(O, R)) are ab- solutely continuous and both have a strictly positive density
6

---

Towards Understanding Gradient Dynamics of the Slecd--asserstein Distance
Proposition S.1. The following are barycentric Lagrangian critical points:
We will focus on a particular case of unstable behavior. We will restrict ourselves to the case d = 2, and we will show that when the target measure p is absolutely continuous, measures μ that contain part supported on segment are not stable for SW? when perturbed in a certain way.
(a) In dimension d = 2, the measure F = 8 1 +.=1 is a barycentric Lagrangian critical point for the measure p with density p(z) 2 Vi-lzja 1 which we will hereafter call the (io-diimessoona) sliced -niffo- measure.
Proposition 5.2. Let p E P2( be an absolutely continu- ous measure, such that the densities of its projections pa are uniformly bounded from above by b > O. Let μ. E (R)) be any measure such that there exists segment S _ R and > O such that a'H's s S p. Then, if pt is the perturbation
(b) In dimension d > 1, the measure Js defined by μ ==
(Id, 0d-1) #1o with pi = .N(0, a2) is a barycen- tric Lagrangian critical point for the standard Gaus- sian P N(O,,a) Ia), where aa is defined by ad d [ga_ (()ea)) d00 and Ca) is the canonical ba- sis of R
n² 2 (T + T2##1)
(22)
where Ta is the translation by ā E R2 and ńt Stt is orthogo- nal to S. then the perturbation μr is unstable for SW2 p): that is, for any C > O, there exists a neighborhood (-E,E) of 0 in which
We refer to in Proposition (a) as the sliced-uniform measure, as for every 0 E gd-1, its projection Po is the normalized restriction of the Lebesgue measure to [-1, 1]. Proposition 5. (a) provides an example of target measure P on a disk in d = 2 that is symmetric with respect to any line, and which admits in this case critical point supported on segment, hence of strietly lower dimension. Proposi- tion 5. (b) provides a similar result for isotropic Gaussians. The proof of Proposition 5.1 is deferred to Appendix B.9.
Sw3('' S SSWSSS,2.... Ct
(23)
The proof of Proposition 5.2 is deferred Appendix B.10. Our Proposition 5.2 proves that critical points as described therein, are highly unstable. Indeed, we do not have a Taylor expansion SW2 p) = SW3(u, p) + at + 6 + o(t²) with a = 0 and b < O. Instead, the inequality SW3 .) S 2 p) - Ct2 is true for any C >> O provided that t is close enough to 0. In particular, this implies that SW3 pp) is not twice differentiable at = O. Hence, while the SW flow may exhibit critical points that are not global minimiz- ers, they may be unstable general. Our result proves this in the case where the target contains a segment.
We now discuss informally why we expect to find critical points of this type. Assume that there exists a subspace H of Rd such that the target p is symmetric with respect to H, i.e. SHmP= p where Sir is the reflection H. Then, if spt(u) S H. then for every f E gd-1, we have PSy(0) = Pv and (o) = Pa, thus T3 = (m)- Thus, for every E ssp(()) S H. we have by straightforward computations 4:
- To((0 e H. (21)
x
uu( =
On the other hand, the perturbation j² used in Proposition 5.2 is not of the form (Id +tt) f4, and thus does not fit in our previously defined framework of Lagrangian critical points. However, this result suggests that by taking a T.2 vector field & which alternates rapidly between and -77 on the segment S, then ((Id +tE) p) will also have
d
where PH is the projection on H. This means that the iterates of the gradient descent j + (Id + will remain supported on H. Therefore, taking the limit of the trajectory (for an infinite number of iterations) should be critical point of F. still supported on H.
local maximum at = 0 (see for example the numerical experiments in Figure below).
5.2. Some explicit unstable critical points
Note that the proof of Proposition 5.2 makes heavy use of the properties of the segment, among which that the existence of a relatively simple closed form of the quantile functions of the projections are available. In general, is difficult to describe how the quantile functions of the projections behave when considering general measures and perturbations.
Previously, we highlighted critical points that are supported on a subset of Rd, for target distribution that is full- dimensional. This is problematic because our gradient al- gorithm may be stuck at these critical points, which are typically at a high level in the energy landscape. We now investigate their stability, as gradient descent is unlikely to get stuck at unstable critical points, with the aim of showing that such points do not appear in practice.
6. Experiments
(x) = -S To((0)=)) 10 (15 H (0)1)))8))
d8
This section presents the results of our experiments, de- signed to examine the extent to which the theoretical find-
I ((0(())))) (0 do as x H.
7

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
[a]
(b)
(c)
(d)
0.5
0.5
0.5
0.5
0.4
0.4
0.4
0.4
0.3
0.3
0.3
0.3
SW3
SWR
SWR
SW3
0.2
0.2
0.2
0.2
0.1
0.1
0.1
0.1
0.0
0.0
0.0
0.0
1
-1
0
1
0
-1
t
(a)
(b)
(c)
3
2
2
2
1.0
1
1
1
0.5
0-
0.0
-05
-2
-1.0
-3
1 -2
-2
-2
2
-1 0
0
0
x
x
x
Figure 1. Instability of measures containing an horizontal segment. On the top line are plotted the value SW2(n', p)) for different measures p. and perturbations ç. On the bottom line are depictions of the different ju (black points), P (approximated by the blue points) and g (red arrows). Columns (a) and (b): μu is point cloud of N 100 points uniformly distributed the segment 1-4/x,44 4/m] x {0). alternates between 62 and -e2, and p is the normal (a) and -icee-uniffrm distribution (see Proposition 5.1) ) (b). Column (c): Same and and this time p the uniform measure the shell C(O, 1, 1). Column (d) p is again the shell, and y is point cloud with "dumbbell-like" shape, whose central segment is perturbed similarly as in (a).(b)(cc)
ings from the previous sections hold in practice".
exhibit an instability. This suggests that, if we approximate the perturbation in Proposition 5.2 closely enough with a vector field that alternates between řī and -77, we could ob- tain a unstable perturbation of the form (Id (Id+))) # nt, which would fit in our framework of Lagrangian critical points.
In the experiments, F(X) is approximated by taking the av- erage of ID Wasserstein distances over L = 100 directions, and by approximating with a point cloud Y containing
M = 10000 points.
Gradient descent. We also investigated the convergence speed of the gradient descent for Sw? for diffcrent choices of step-sizes, as shown in Figure 2. We observe that choos- ing step-sizes close to x dNN (here d 2), as justified in Section 3 does indeed yield important decrease of the loss at the first few iterations, while lower step-sizes result in slower convergence of the descent, and step-sizes larger than 2dN (the threshold above which Proposition 3.2 stops applying) result in divergence of the descent.
Instability of critical points. First, we considered a point 8 i-1
cloud X = (X1,..,,%,, with X; - क N-T. with N = 100, that approximates the measure μ 11 that was studied in Section 5. We considered a perturba- tion& & that alternates between 62 and -02 and we plotted I + F(X- + (E) SW2 p) in Figure for different choices of p. We see that the numerical results are consistent with our theoretical findings: indeed, we have local maxi- mum for all three considered target measures. Furthermore, when X is a point cloud with a more complex shape but which includes an horizontal segment, we still observe an instability by perturbing the segment and leaving the other points of the point cloud unchanged. Moreover, while the perturbation considered in Proposition 5.2 is not induced by vector field ç, those in these experiments are, and they do
7. Conclusion
In this work, we have studied critical points of SW objec- tives with respect to a probability measure p. by leveraging the notion of Lagrangian critical points in the space of mea- sures. We provided a detailed analysis of the critical points of a flow associated with a non-convex objective distance, in contrast with most of the literature that primarily deals
$Code available https: / / thub.uu.o. com. hatmm//aathier/ Criti cal Maaal-Pooinn -Points-ot ff sed- Wasserstein
8

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
p=gaussian
p=sliced-unifom
10
2.0
1.0
3.pp=3.
10"
step-3.0
10a
2tpp-300
stg-5.0
0.5
stepm4.0
steg=40
SW?
10-1
#.ep-0.5
Sep-10
SW3
10-2
step-05
steg-10
0.0
5top=0.11
stop=0.1
2.
0tep=0.01
step=0.01
-2
10 4
10
-1.0
0
100 Iterations
200
-2
0
2
100
200
-1
1
x
Iterations
Figure 2. Gradient descent of SW3. On a point cloud of N = 1000 points for different choices of step-size and p. Left: convergence speed of gradient descent, where p is the normal distribution, for different step-sizes (given in multiples N in legend). Center left Initial point cloud (in green), sampled uniformly in [-1, ")" and final point cloud (in red) after 200 iterations with step-size À = 2N. Center right and right: same as respectively the left and center left images, but with p the sliced-uniform measure (see Proposition 5.1).
with convex ones or that uses functional inequalities.
points (e.g. the global minimizers of the discretized energy) and we have shown in Proposition 5.2 that stable critical points cannot contain segment. More generally, one could hope to show that any stable critical point μ of F which is atomless must be equal to p. Third, we note that there exists other proxies of the Wasserstein-p distances based on 1 1-dimensional projections, such as Max-sliced Wasser- stein (Deshpande et al., 2019), SW distances with respect to other probability measures on the unit sphere (Nguyen Ho, 2024; Rowland et al., 2019; Mahey al., 2024). Ex- tending our study to these variants of SW is the topic of future research.
One limitation of our study is that, while we have defined our framework of Lagrangian critical points for all measures u, E (2 most of our results require the target p to be without atoms (in Section 3, is assumed to be density, but, as pointed out in Remark 3.4, most of its results can be extended to without atoms). This can limit the applica- bility of our results to machine learning applications where one often has work with discrete targets p. However, our assumptions are sufficient to allow to tackle many types of singular measures which arise in machine learning and generative modeling, such as densities supported on lower dimensional manifold of Rd (which are not absolutely con- tinuous but are without atoms). Furthermore, the fact that our numerical experiments, in which the target measures were discretized, exhibit the behaviors of convergence and instability that theoretical analysis highlighted, suggests that results should still be relevant in the cases where the target measure is approximated by discrete measure. Another limitation is that our main instability result, Propo- sition 5.2, only holds in dimension d = 2 and involves a perturbation which is technically outside our framework of Lagrangian critical points. Generalizing this result to higher dimensions or exhibiting more general unstable Lagrangian critical points could be an avenue for future work.
Acknowlegements
QM and AK acknowledge the support of the Agence na- tionale de la recherche, through the PEPR PDE-AÍ project AANR-23-PEIA-00N0 CV acknowledges the support of Région Ce-de-FFaance through the DIM AI4IDF project.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Finally, many important open questions about critical points of SW remain. First, is it possible to prove that any Wasser- stein or Lagrangian critical point 1u of F = 2 (.)) which is absolutely continuous must be equal to p ? The- orem 4.1 in (Cozzi& Santambrogio, 2024) gives (very) partial answer to this question: it implies in particular that if p is standard Gaussian and if μ has finite entropy, then p. Second, can we get better understanding of stable critical points? There exists finitely supported stable critical
References
Ambrosio, L.., Gigli, N., and Savaré, G. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2005.
Arjovsky, M., Chintala, S., and Bottou, L. Wasserstein gen- erative adversarial networks. In Precup. D. and Tch, Y. W. (eds.), Proceedings of the 34th International Conference

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
on Machine Learning, volume 70 of Proceedings of Ma- chine Learning Research, pp. 223.223. PMLR, 06-11 Aug 2017.
Cozzi, G. and Santambrogio, F. Long-time asymp- totics of the sliced-wasserstein flow. SIAM J. Im. Sciences, 2024. URL http:/ /tt////vt////////////////// sns it/ paper/ 6495// cvgmt preprint.
Birrell, J., Dupuis, P., Katsoulakis, M. A., Pantazis, Y., and Interpolating
Rey-Bellet, I.. (f, gamma)-divergenee between I-divergences and integral probability metrics. Journal of machine learning research, 23(39):1-70, 2022.
Dai, B. and Seljak, U. Sliced iterative normalizing flows. Proceedings of Machine Learning Research, 2021.
Deshpande, I., Hu, Y.-T., Sun, R., Pyrros, A., Siddiqui, N., Koyejo, S., Zhao, Z., Forsyth. D., and Schwing, A. G. Max-sliced wasserstein distance and its use for gans. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 10648-10656, 2019.
Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Varia- tional Inference: Review for Statisticians. Journal of the American statistical Association, 112(518):55-887,, 2017.
Bond-Taylor, S., Leach, A., Long, Y., and Willcocks, C. G. Deep Generative Modelling: A Comparative Review of VAŻs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models. IEEE transactions on pattern analysis and machine intelligence, 44(11):7327-7347, 2021.
Du, C.,, Li, T., Pang, T., Yan, S., and Lin, M. Nonparametric generative modeling with conditional sliced wisser-wassrsttein flows. In International Conference on Machine Learning (ICML), 2023.
Fan, J., Zhang, Q., Taghvaei, A., and Chen, Variational wasserstein gradient flow. In International Conference Machine Learning. 2022.
Bonneel, N., Rabin, J., Peyré, G., and Pfister, H. Sliced and
radon wasserstein barycenters of measures. Journal of Mathematical Imaging and Vision, 51:22-45, 2015.
Fisher, M., Nolan, T., Graham, M., Prangle, D., and Oates, C. Measure transport with kernel stein discrepancy. In International Conference on Artificial Intelligence and Statistics, pp. 1054-1062. PMLR, 2021.
Bonnotte, N. Unidimensional and evolution methods for optimal transportation. PhD thesis, Université Paris Sud- Paris XI; Scuola normale superiore (Pise, Italie), 2013.
Genevay, A., Peyré, G., and Cuturi, M. Learning genera- tive models with sinkhorn divergences. In International Conference on Artificial Intelligence and Statistics, pp. 1608-1617. PMLR, 2018.
Bourne, D. P., Kok, P. J., Roper, S. M., and Spanjer, W. D. Laguerre tessellations and polycrystalline microstruc- tures: fast algorithm for generating grains of given volumes. Philosophical Magazine, 100(21):677-27077, 2020.
Goodfellow, I., Pouget-Abadie, J., Mirza, M.. Xu, B. Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial networks. Communications of the ACM, 63(1 (3(11):139-144 2020.
Cai, D., Modi, C., Margossian, C., Gower, R., Blei, D., and Saul, L. Eigen score-based variational inference with orthogonal function expansions. Advances in Neu- ral Information Processing Systems, 37:132691-132721, 2024a.
Hertrich, J., Wald, C., Altekrüger, F,, and Hagemann, P. Generative sliced MMD flows with Riesz kernels. In International Conference of Learning Representations, 2024.
Cai, D., Modi, C., Pillaud- Vivien, L., Margossian, C. C., Gower, R. M., Blei, M., and Saul, Batch and match: black-box variational inference with score-based divergence. In International Conference Machine Learning, 2024b.
Kolouri, S., Pope, P. E., Martin, C. E., and Rohde, G. K. Sliced wasserstein auto-encoders. In International Con- ference on Learning Representations, 2018.
Chizat, L. and Bach, F. On the Global Convergence of Gradient Descent for ever-parameterized Models using Optimal Transport. Advances in neural information pro- cessing systems, 31, 2018.
Korba, A., Aubin-Frankowski. P.-C.. Majewski, S., and
Ablin, P. Kernel stein discrepancy descent. In Interna- tional Conference on Machine Learning, pp. 5719-5730. PMLR, 2021.
Choi, J., Choi, J., and Kang, M. Scalable wasserstein gra- dient flow for generative modeling through unbalanced optimal transport. In International Conference on Ma- chine Learning, 2024.
Li, C.-L., Chang, W.-C., Cheng, Y., Yang, Y., and Póczos, B. Mmd gan: Towards deeper understanding of moment matching network. Advances in neural information pro- cessing systems, 30, 2017.
10

---

Towards Understanding Gradient Dynamics of the Biccd-Wassersteine Distance
Li, S. and Moosmueller, C. Measure transfer via stochastic slicing and matching, 2025. URL https::/
Peyré, G., Cuturi, M., al. Computational optimal trans- port: With applications data science. Foundations and Trends& in Machine Learning, 11(5-6):3556077, 2019.
org/abs abs/2307 /207 05705.
Liutkus, A., Simsekli, U., Majewski, S., Durmus, A., and Stöter, F.-R. -aasee--wssrrttinn flows: Nonparametric generative modeling via optimal transport and diffusions. In International Conference on Machine Learning, pp- 4104-4113. PMLR, 2019.
Portales, L.,, Cazelles, E., and Pauwels, E. On the sequential convergence of Lloyd's algorithms. working paper or preprint, May 2024. URL https sciencellsieecee/////eeee ha1-04593962.
Rabin, J., Peyré, G., Delon, J., and Bernot, M. Wasser stein barycenter and its application to texture mixing. In Scale Space and Variational Methods in Computer Vision: Third International Conference, SSVM 2011, Ein-Gedi, Israel, May 29-June 2, 201 I, Revised Selected Pupers 3, pp. 435 446. Springer, 2012.
Mahey, G., Chapel, L., Gasso, G., Bonet, C., and Courty, N. Fast optimal transport through sliced generalized wasserstein geodesies. Advances Neural Information Processing Systems, 36, 2024.
Manole, T., Balakrishnan, S., and Wasserman, L.. Minimax confidence intervals for the sliced wasserstein distance.
Rachev, S. and Rüschendorf, L. Mass transportation problems: Volume I: theory, volume Springer Science & Business Media, 1998.
Electronic Journal of Statistics, 16(1):2252-2344 2022.
Mei. S., Montanari, A., and Nguyen, P.-M. A Mcan Field View of the Landscape of Two-Layer Neural Networks. Proceedings of the National Academy of Sciences, 115 (33):17665-E7777,, 2018.
Rowland, M., Hron, J., Tang, Y., Choromanski, K., Sarlos, T., and Weller, A. Orthogonal estimation of wasserstein distances. In The 22nd International Conference on Ar- tificial Intelligence and Statistics, pp. 186-195. PMLR, 2019.
Mérigot, Q. Santambrogio, F,, and Sarrazin, C. Non- asymptotic convergence bounds for wasserstein approximation using point clouds. In Neural Infor- mation Processing Systems, 2021. URL https: //api. www.eeeiinnnnlllaeee. org/CorpusI:: 235436063.
Salim, A., Korba, A., and Luise, The Wasserstein Proxi- mal Gradient Algorithm. Advances in Neural Information Processing Systems, 33: 12356-123666, 2020.
Santambrogio, F. Optimal transport for applied mathemati- cians. Birkäuser, NY, 55(58-63):94, 2015.
Nadjahi, K., Durmus. A., Chizat, L., Kolouri, S., Shahram- pour, S., and Simsekli, U. Statistical and topological properties of sliced probability divergences. Advances in Neural Information Processing Systems, 33:20802- 20812, 2020.
Sarrazin, C. Lagrangian discretization of variational prob- lems in Wasserstein spaces. Theses, Université Paris- Saclay, January 2022. URL http:::///ttteeeeeee hal.
Nguyen, K. and Ho, N. Energy-based sliced wasserstein distance. Advances in Neural Information Processing Systems, 36, 2024.
Tanguy, E., Flamary, R., and Delon, J. Properties of discrete sliced Wasserstein losses. Math-
Nietert, S., Goldfeld, Z., Sadhu, R., and Kato, K. Statisti- cal, robustness, and computational guarantees for sliced wasserstein distances. Advances in Neural Information Processing Systems, 35:28179-2881 2022.
ematics of Computation, June 2024a. URL
https://wee /tt:://wwwww. ams org/ http://wwww.....ooooaaaaaaaaaaoa http://ooonaanamommmmo 10000000-100 10000000000000000111100000000001 00t////002555511--10
Tanguy, E., Flamary, R., and Delon, J. Reconstruct- ing discrete measures from projections. conse- quences on the empirical sliced Wasserstein distance. Comptes Rendus. Mathématique, 362:1121-1129, 2024b. doi: 10.5802/ermath.60) URL https: // /ccomptes-enduse www.emie--cienneeeeeee.. Fr hht.aaheeaattieeeetttileeeeeeeee crmath. 601/.
Panageas, I., Piliouras, G., and Wang, X. First-order methods almost always avoid saddle points: The case of vanishing step-sizes. In Wallach, H. Larochelle, H., Beygelzimer, A., d'Alché-Buc, F, Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL bttps::/ neurips. cc sc/paper__m / ppaper__filessappprr/////////// 3 $
Paper.
Villani, C. Optimal transport - Old and new, volume 338.
pdf.
Springer Berlin, Heidelberg, 01 2008

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
Wibisono, A. Sampling optimization in the space of measures: The Langevin dynamics composite opti- mization problem. In Conference on Learning Theory, pp. 2093-3027. PMLR, 2018.
Yi, M. and Liu, S. Sliced wasserstein variational inference. In Asian Conference Machine Learning, pp. 1213- 1228. PMLR, 2023.
Yi, M., Zhu, Z., and Liu, S. Monoflow: Rethinking diver- gence gans via the perspective of wasserstein gradient flows. In International Conference on Machine Learning, pp. 39984-40000 PMLR, 2023.
12

---

Towards Understanding Gradient Dynamics of the
Distance
A. Some useful results
A.I. Projections of measures without atoms
In this subsection, we prove an useful lemma on measures without atoms. If μ is a measure on I2 we say that j is with atomless projections, which we abbreviate WAP, if its projection pe is without atoms for almost every 0 E g It is straightforward that if je is WAP, then is without atoms. It turns out that for finite measures, the converse is also true: Proposition A.1. Let μ be a finite measure R then μ is atomless if and only if it is WAP.
Proof. We have already seen that if y has atoms, then it can't 't be WAP. Now, for every k E {O..... d - 1), let (B") be the k-th affine Grassmannian of R that is the set of affine subspaces of Ed of dimension k, and for every k E {o.....e d - 1} and measure μ on Rd, we note
AR = (V E AGI (VV) > 0}
(24)
(in particular, A is the set of atoms of μΛ) Let μ be fixed finite measure on IR without atoms. We construct by induction sequence of finite measures Mo μ. Mi......, Mdd1 such that for every k, AGk, 0, and if k >> 0, then t is WAP = 1 is WAP. Our first term pa0 = f satisfies by assumption Ao = 0. Now assume that we have built Po...., 1k-1-
If V Vi...., Vi A are distinct, then
Pk- PK-I(KU...
Σ (K)
(25)
as the intersection of any subset of these has null Pk-1 measure since Ag-1, = 0. In particular, the family (V)= is summable, with sum I 1, and AE, is at most countable. Define
Pk = 1k-1 1k-1IU ANPrE1
(26)
Then, by construction, AR = 0. Now, let E gd-1 be such that (mę=1) has an atom : there exists u E IR such
that (Mx-1)o(fu)) > 0. Assume that (Hx)o(fu)) = 0. then this implies that there exists V E A such that
(PR-IIV) (o(=))))(((u)) > 0, that is PV-1(V n Po (u)) > O. Since A 0, this implies that V Po (u) is an affine
subspace of dimension k that is V C Pol (w), and 0 V+. This argument thus proves
(w),
E E sd
(m la has an atom} C (0 [ (μa) )o has an atom) U (0 E s 3V E AR. ,,0 E VV
(27)
Since the second set in the RHS is of null measure (as an at most countable union of sets of null measure), this inclusion implies that if μu is WAP, then p is also WAP. This finishes our induction.
Now, we have built our sequence (4o.-..; Pd-1- But A4-1, = 0 implies that (a-11 is WAP (and that in fact (Ma-1)o is without atoms for every 0). Thus, all the measures of the sequence are WAP, and in particular pi = y is WAP.
A.2. Disintegration of measures
We state here the so-called disintegration theorem, which we will need in the proofs of our results. Let X, Y be two separable metric spaces. We say that a family (Hæ)zex of probability measures on P(Y) is a Borel family of measures if for every Borel set B C Y, the map x E X >> uz (B) is Borel measurable. We say that a separable metric space X is a Radon space if for every μ P(X). every >> 0 and every Borel set B C X, there exists compact Ke C X such that K. C B and p(B \ K2) I In particular, it is known that Polish spaces (i.e. complete metric separable spaces) are Radon spaces (see (Ambrosio et al., 2005, Section 5.1)), so Rd is Radon space.
Theorem A.2. Let X, Y be two Radon separable metric spaces, E P(X) and π X + Y be a measurable map. Let V. = J4 E P.Y). Then there exists a Borel family of measures (My)veY S P(X), which is v-a.e. uniquely defined, such that
My \ (()) =0 for y EY
(28)
and. for every measurable map f X + 20).
f(z)dp(x) = J (())dp((()dr(()
(29)
1(x)
13

---

Towards Understanding Gradient Dynamics of the Sliced-Wassersice Distance
The statement of this theorem is taken from (Ambrosio et al., 2005, Theorem 5.3.1). In the where X is of the form the second coordinate, may identify each ==) with Z, and the theorem reformulates as there exists a Borel family of measures (uy)eeY C P(Z), which is v-a.e. uniquely defined, such that for every measurable map f Z x Y + [0, oo), Szxx (2.,))d((z,)) = fy ((2,,))dp(())dd())
X 7 Y and is the projection
B. Proofs
B.1. Proof of Proposition 3.1
First, consider probability density & E (R),,, with cumulative distribution function F2 R + [0: 1]. Let μ.
- 8z, the uniform empirical measure associated to X = (II....,IN) E IR For every i E NJ,, we define V F I) the i-th Power cell associated to p. Then the properties of one-dimensional optimal transport imply that, for every X ([I,,,,))) E E with to S ... S Io(N). a GN we have
N
N Σ
G(X)= = W) =
Σ
(o)) - xx = Σ / V4
14 - ddp(xx)
(30)
P 1=1 V-1
We can then easily see that when 1, G C* on the complement of the generalized diagonal AN = ((x1,...,23
R Bi j.m; = 2,)) and its partial derivatives are given by
-
O,G( EN)
sgn(2; - 20)22 - dd(=),
(31)
JV
where a E n is such that To(1) < ... < Zo(N). In the particular case where p = 2, the partial derivatives take the simpler form
0,G(I1,..,,NN) 1--100 (x: (t5-x)dte x))dp(()) VV-1(0
1 (2x - ()))
(32)
with b, N Sv; zdp(x)) the barycenter of the i-th Power cell Vi-
With these considerations on one-dimensional measures in mind, we can now move on to prove Proposition 3.1. For this, we will need the following lemma.
Lemma B.1 P 2 2, p (R) is probability density and X = (21, IN) E AN with Fo(1) < < ToiN) E GN and H = (h1, .., ht) E R is perlurbation such that X + H has the same ordering a as X, then wriling R&G(X, H) = G( + H) - G(X)= (VG(X))H) we have
N
|A + 1hi / |0; 1x/--/-- -xpp-ppppx)
R G S 2 1)
Σ
(33)
(this is a finite quantity since p has finite order p moments).
Proof.
Consider the function f(x) = |x|". Since p 2 2, we see that ʃ is C2 and that f'(x) = pr|p//-2 (x
p(p - 1) As a consequence, applying Taylor's 's thcorem, for every x, E R,
r4t
f(x hc+ (x))
(t) - t)dt
(34)
1Dth
[f(z + f())(()) -()))
(f" t)ldt
(35)
14th
p(p 1) max Jx + A1)" //ldt
(36)
$ p(p ())b|(((x) + 1/1)=-2
(37)
< 2 p 1)]6 + 2
(38)
< 2 22-2ppp ppp (1b + |||3
(39)
14

---

Towards Understanding Gradient Dynamics of the Gleed-Wasserstenn Distance
Therefore, since X + H and X have the same ordering a,
N
R H) Σ
(1x; + h44 - x - z0" psgn(a' - 2))2 - x x (ddp(()))
(40)
1=1
N < R
ppp 1)( + 1A, 2dp((()((x)
(41)
1m1
< 2 (p 1)
halp + |h,1 2018-22p(
(42)
S 2p 1) Σ
|A,1 + |1:1? |m; xx1 =pp-2dp(1))
(43)
Now we can prove Proposition 3.1.
Proof (Proposition 3.1). First, let's introduce the following definitions: for every c > let
O := (0 s
3i i,,, - Xx < c}
(44)
and for every 0 S define the function G. X E R + 1 W Po+p) For every point cloud X E (RR) and every 0 E Sd-1, let 0z,x E GN be a (not necessarily unique) permutation such that (Xoo,x (1) s... S (X (X 10),
and
J J
vx,
xpp sgn (g, (000==))([X(0 x)) - x p- 1--PPooppp))d
(45)
0,
We want prove that if X & AN. F is differentiable X and VF(X) FF(X). Let >> be fixed. We see that if |||l|| S then for every 0 O2 08,*+ 08,x Furthermore we know that there exists Co Co(X) > 0 such that
1 (9.) Cue
(46)
where Uga-= is the uniform distribution (i.e. the normalized volume measure) on 5d-1 We now consider perturbation H such that ||||| < e/2. We have
F(X H) - F(X) (VF( B(H) C(H)
(47)
with
A(H) / (Go(Po(X + H)) - (2C Je; B(H)= / (GolPo(X + H)) (X) je.
(48)
(49)
C(H) (P((4))VG((P((X)))))) e.
(50)
When 0 E O; we have 08,*+ H Tp,x and we can apply lemma B. to Ga to obtain that
IGo H)) (o(P)(X) (P)(H)) (G. & C $
(51)
with a constant C' that is uniform on and depends only on X, p. and p (indeed, the moments of Po#p p are bounded by those of p). Therefore deduce that
A(H) = a()) HT
(52)
15

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
Now, notice that
/
(0.G((P((X)){
(x,10) PP=##(#(
(53)
V. 1.00,0000
so
N S
Σ (P)))
Σ i=1
(X,10) = dd)))(())= W Po#P) S W (p-1(ux,))
(54)
therefore we deduce that
C(H) & Cucl Cuc|#FIIWP
(55)
Finally, for a generic 0, using the mean value inequality on f(x) xp with f'(x) pzp-1, and using the shorthand
notations W (X))) = (x) Po+p), we have
H)) Go H)) (x(P(()) WW,(X + H)P - W2(X)'
(56)
W + H) Wx(X)
sup 15
(57)
s p + H) - WW,()) (X), W,,,, + H))
(58)
Now, by the triangle inequality, we have
[ H) WX(X) - Po*ux) S Wx) S |11I1|1
(59)
And similarly (x(X) S (,(xx,p) and
Wp(X + H) < (x)))+ Wy.CooRx-HMPPouNs < Wx())++ ||]|| S Wx(xx,p)+ +
(60)
Therefore, we have
Go - C|
(61)
with constant C' which is uniform in 0 and depends only p. and W(xx, p). Therefore
[B(HI S |
(62)
Thus, we have proven that
( - F(X) (ŌF(X)(H)) = HI)
(63)
which shows that VF(X)= = (X)). To show the continuity of VF, let xx e (Rd) be a sequence converging to X, with X* x & Ay. Recall that for every E ( .N. we have
form 1 - (00
Vx,F(x*)
sgn sgn(((110) x))({(100) - -d o (odPo+p(x))dl p(z))(
(64)
Let 0 E gd-1 be such that (X, * (X,10) for every + j. then there exists hi such that for every k 2 ko. 00,x* = 09,x. In particular, this implies that for every i, n (i) = - (i), and thus
sgn((X+10) x)) - PdPPoddPoPPGe = /
sgn - x){ (000))=)((([) 0) x: xrp-10d
Po 8 p(x)
(.
(65)
sgn((X,10) x))(x) -x1" 2d
Pd a p(x)
(66)
16

---

Proof. First, let 0 gd-1 be fixed, such that (X,10) + (X₃10) for every * j. Then, since the map which sends Vo, to Towards Understanding Gradient Dynamics of the Sliced-Wassereee Wasserstein Distance
where the limit is obtained by dominated convergence, using the fact that the sequence X is bounded and that Powp has finite moments of order 1. Moreover, since for every le and we have
V Vuee Ve (3)
$ /|(x510) =1p=''d dPo# p(z)
10 =))(( =1l~''0dPppp OdPo#p(()
(67)
S 11X / Plp-'dPooPlaa
(68)
s 22- + 1 |x(x=)
(69)
and since the sequence X is bounded and has finite moments of order p this implies by dominated convergence that lime Vx, Fxx) Vx, F(X) for every i. This proves the continuity of V.. Finally, in the case p = 2, the expression
of Vxx x.F F. simplifies as
Vx, F(X) = S /
sgn(((;)0) x) x: o
(70)
J ((x,10) z))dPo=p(z)d8
(71)
0,00,2
ff J 1
J N (X,1))dd
zdLo=p(22000
(72)
Vo. Vo.
1
à x; /
(. 0d0
(73)
N
where we used the definition of the bo.; in the last line.
As a side note, remark that F is actually twice differentiable almost everywhere, as a consequence of the following semi-concavity property for F:
Proposition B.2. Na -essmiicone (i.e. F- zNall 1|2 is concave).
Proof. Indeed, F(X) INa 2 w W3( Po+p) 2N (P) for every X RR and we the fact that the projection Po linear and that Y E ERN, + 2 ( semmiconcave (see for example Proposition (Mérigot et al., 2021))
B.2. Proof of Proposition 3.2
To prove the descent lemma Proposition 3.2, we first need to prove that F is smooth.
Proposition B.3. For every X,Y E (IR))N N \ AN. we have
F(Y) +(X))+ x) 2Nd X 1
(74)
17

---

Towards Understanding Gradient Dynamics of the Glecd--assersteiie Distance
(0xxo(0)00) is (not necessarily optimal) transport map from pe (PP(Y)) we have
N
W3( S
[V-10) - (x))
(75)
KY100) (X,10) + (X,10) x)d)p(xx)
(76)
(i (4)
N
N (Y, -x,003 + Σ
2(Y; - (1))) -
+ Wa(mp((x):
(77)
N
i=1
i=1
0,0 x,01)
N
2
Σ
(Y,
x,10)²
+ Σ
(Y, (x10)((,10) - -b00))000 + W3(MP((X):
(78)
N
N
1=1
i=1
Integrating over the sphere we have
N
N
SWy
(V,-
+
Σ
(Y,-X,00)( - )0,0)(0()) )d) SW2(Ux-P) (79)
N
1=1 g4
38
1=1
N Y, X;
I x,112
+ 1=1
/ gd
((X,10) 1 (1) )o00 + Sw (80)
Nd
N
In the RHS of the last inequality, we recognize the expression of the gradient of F which we recall is Vx,
N Jss ((x,10) bo,oz xoxx000 (1) Odd. Therefore, substituting it gives the intended result
1 2 X - YII X(VX(X)) F(X).
F(Y)
(81)
2Nd
Now, we can prove Proposition 3.2. Equation (10) is obtained directly from Equation (74) by taking Y = X AVF(X).
B.3. Proof of Proposition 3.3
We will first need to prove the following lemmas
Lemma B.4. Let E P(a,, b)) be an absolutely continuous probability measure, with density (which we will also denote p)
bounded from above by 0. Then the barycenter xo J xdp(x) of p satisfies aa50 al, [r0
b) 2 20'
Proof. Since p S 3, integrating p on [a, b). we note that 1/- < b Let po E P((a, b)) be the probability with density / on (a, a + 1/3) and 0 Ia 1/3, bj. Its cumulative distribution function is thus
(x) -11 B(x [((x a) x E (a, a + 1/3 {
(82)
if 2 a 1/
and, since S 8, we have Fp S Foo on (a, b}. Thus, the quantile functions of Po satisfy Fz1 2 F-1 (this follows directly from their definition), and we have
xo (x - a)dp(a)= f (F))((0) - a)dr
(83)
2 (F))(() -ddr = '
(a)ppp)p)))) ( a)dpo()) =
patt f.utt 3(x - o)de
(84)
2
where we used the fact that FTT 1o.11 for any probability measure l on the real line (see (Santambrogio, 2015, Proposition 2.2)). Similarly, we can show that b To 2 28
18

---

Towards Understanding Gradient Dynamics of the Sliced- Wassertennr Distance
Lemma B.S. For every X E (R4) \ AN. we have for every + j.
X1) s -11l
(85)
If we further assume that there exists B > 0 bounding from above the density of po for every 8 E Sd-1, then there exists
C C(d) such that for every # j.
C
Na -XI
X,1l
(86)
Proof. Using the notations of Proposition 3.1 and Equation (9), we have
N(Vx, F(X) V F(X)XX X,) 1 21,, x,11² [ (0002100)) - ( - x;) (87)
By symmetry, we have in fact
N(Vx, F(X) Vx F(x), F(X)X; Xj) |X x511²- -
1(01X1-X,)20) -1100xx-x))1) (00 b 10 00)0 (01x; Xz)d0 (88)
Indeed, for every E gd-1, we can check that we have xx (k) N +1 N x and b_ b for every k 1, N. However, if 0 sd-1 is such that (0) Xj) > 0, then have xx) xx (i). and thus
00,0%/100 bg 00,03/00) 0. Therefore the integrand in the right-hand side of (88) is nonnegative, and we deduce from this
1 -1xx X,112 N(Vx,F(X) V F(X)XX x,) s
(89)
This proves (85). Now, if we assume that there exists such that po S 3 for every O 8d-1, then we have
1
b 0,00 0,002000 b x0,0
(90)
N
Indeed, for every l = 1,... N, the distance separating the barycenter bø, from the boundary of its corresponding Power cell Vo, at least 2 which we see by applying Lemma B.4 to the probability measure N Neivvo. In particular, since (0|X,- - Xj) is also positive, we have
(0|X: X1)(0,,(z10)
)a))))))
N8 10|X, X₁)
(91)
Injecting this into Equation (88), we obtain the inequality
1
N(Vx - Vx, F(X)X - -x)) < |x: |X,--x11 -2
-((01x- N3 (0 - -X,)d8
(92)
2
< |X - Xyl12
|1: -X,1|
(0100)
(93)
NB
[10100)>00
C
< 2 |X: - X,112
||
Xxyll
(94)
NB
where 0 |x, x and where C : 2 and depends only on d. This proves (86).
X, -x,
(B)0u) > 0. Note that, by symmetry, does not depend on 0g E gd-1
We can now prove the proposition.
19

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
Proof (Proposition 3.3). if a j, then we have
N - Xj) = (X; - -
- -(
(95)
= ||x; - -x,ll² -xx) -X,)
(96)
> | -x,1I² N ( -xxI=x
-(1 (1
A
(1-Na)
|x;-
|x; - Xj11²
(97)
Nd
where we used (85) from Lemma B.5 to obtain the last line. In particular, if À E (0, Nd), then (Y, - Yg X3) 0 and
thus Y; # y,, for every i + j. Therefore, Y $ AN. This proves the first part of the proposition. Now, assuming that there exists B > 0 such that the density of pa is bounded from above by 8 for every A E Sd-1 we then have for every? i *j. j.
|Y, Y5ll²= = 11(X; X,) - XVxx - F(X)(X
(98)
||X, - -X,11² F(X) Vx, F(X) - XX) VX Vx,, F(X)
(99)
2 |X, - -x,11² - -2-22VVXX X, F(X) - [( -xx)
(100)
A
C
2 | X, - -2 5 X2 - x,1l= -x,1
X; -xyl1
(101)
N3
where we used (86) from Lemma B.5 in the last line. Thus, we have proved
X |x
(
C
1 - - X111
| 11l 2 XX x,l1 + N
- X,1|
(102)
NB
Now: :
If |X - X,yl
S
4C No we have directly | - Y,バ| > || - X,y || from Equation (102).
If X is critical point, we have FF(X)= = 0 and thus Y X. Therefore, Equation (102) yields
C 2 |x, X,1]
0 0 N || - X,ı1
(103)
N3
which implies
C
_
x,, N
(104)
NB
As side note, observe that if we consider the continuous time gradient flow
X(t = 0) = Xo ((()) = -VF(X(t))
with Xu E (RR)) \ An for > 0
(105)
then Lemma B.5 implies that for every t > 0 at which the flow is well-defined, for every i # j.
d 1
X, x,バ1² = (Vx Vx, F(X))X, -X,)
(106)
dt 2
C
2 Nd
Xx x,1
N28 [, - -Xyl
(107)
XX X11l
NB z |X - X,1|
(108)
N
where we used (86) to obtain the second line, and, in particular,
d |x, - x,113> > 0
(109)
dt
whenever |X, - -Xll S
dc This implies that: :
N3.
20

---

Towards Understanding Gradient Dynamics of the Sliced- Bliced-Wsseerteiin Distance
If [f| Xyll
0, then this inequality must stay true at every >>> 0.
If ||x; - X,l - Np at f = 0, then |X Xyll increases until is greater or equal than 4C lower than this threshold afterwards.
N3 and does not become
Thus, we see that the continuous time gradient flow is also well-behaved, in that will tend to stay far away from the generalized diagonal AN-
B.4. Extensions of the results of Section 3
If, instead of assuming that p is absolutely continuous with respect to the Lebesgue measure, we simply assume that p is without atoms, then, by Proposition 1. the projection Po is without atoms for every 0 E @ N, where N, _ gd-1 is some set of directions of measure zero. In particular, for every 0 & No. the cumulative distribution function Fp, is continuous, and we can define the Power cells Ve,: := F ([')) every i E fi.....Ny Then Proposition 3.1, Proposition 3.2 and the first part of Proposition 3.3 extend to p. with the exact same statement. Indeed, their proofs stated in Appendix B.1, Appendix B.2 and Appendix B.3 work exactly the same (with the difference that we only consider directions 0 that not in Np).
N' x)) N and the barycenters bø, := N Sva., zdpe(x) for
N3.
In fact, we can find extensions of Proposition 3.1 and Proposition 3.2 when we only assume that p P2(RR)) The difficulty is that, since the projections Po can no longer be assumed to be without atoms, we can't I characterize the optimal transport between Pe and Poylix terms of Power cells. For this, we first recall the following well-known results on optimal transport in ID:
Theorem B.6. Let μ.v E P2(E).. then the so-called monotone transport plan between iu and v, given by Tmon == (Fr F. *1o... 1). is the unique optimal transport plan between μ and v for any cost of the form c(x,, v) h(x y) with IR + [0, os) strictly convex. Furthermore, "mon is the unique trunsport plan 7 E I[(,, v) which satisfies
V(ry),, (a', (.',y) spt(T), < - v
(110)
We refer to (Santambrogio, 2015, Chapter 2) for the detailed statement and proof of these results. In the special case where one of the two mesures is a point cloud, we then have the following lemma:
Lemma B.7. Let E (2( and N > 0. Then there exists an unique family of probability measures PI...., PN E P2((R) such that
Ei=1 Ps N
N
For every <.j. j. Iy E spt( and vy E sp)( we have x4 S 3i
For every X = (21,..., IN) E R with æ1 <.. < xN, the unique optimal transport plan beiween px and p (for any cost of the form c(x, v) = h(x - y) with h: R + [0, strictly convex) is given by
N
8z. 20
(111)
Proof. First, we fix point cloud X = (x1,.... In) R such that < <... < UNN, and we let 7x be the unique optimal transport plan between μх and p. By the disintegration theorem Theorem A.2, there exists an unique family Px.1,...,Px.AA E (R)(R) such that 7x = 1 E E2, @ px.i- In particular, we have = T2#7x = N E Px,x- Furthermore, if i < j. y1 E spx() and yJ E spt(pxx)), then we have (i), (2r Mj) E spt(7x). and Theorem B.6 directly implies that y/1 S Vj. Now, all that is left do is to show that the family (Px,);; does not actually depend on X. This is the case since, if X' (z' .e......22 xxy) E Rg-V is another point cloud with x ... a'y, see that the transport plan 7 = 1 E.N dzg @ a also satisfies (110), so that = 7x by Theorem B.6. But then we must have px.: = PX for every by unicity of the family (Px(,)). This finishes the proof.
N
N
Remark B.8. In the case where p E P2((R) has no atoms, it is not difficult to see that p. = NPI, where Vú is the i-th Power F-1((i ([(i 1//N,
cell V; FØ
21

---

Towards Understanding Gradient Dynamics of the Glecdd-Wsssesteiie Distance
Then, fixing P Pa(RR), for every 0 gd-1, we denote P8.13 Pe.N (2( the measures given by applying Lemma B.7 po, and define bo. = f (x)) the corresponding barycenters (by Remark B.8, if p is without atoms, this is indeed the barycenter of pa the Power cell Ve,i, and there is no conflict of notation). We then have the following extensions of Proposition 3.1, Proposition 3.2 and Proposition 3.3:
Proposition B.9. If 2 2, then F X E (R4) + & ( is differentiable at any point cloud X (XI...., .N) (IRd which does not belong to the generalized diagonal AN. The gradient of F continuous on (Rd)N AN and the expression of its component with respect to the i-th vector X; is then
J J sgal (x - -0)==))(Xa0 x)((0 - x/.- -0doo (000))00x000)0000000 (2)d00
Vx,
(112)
In the particular case where p = this expression can be further simplified:
1 Vx,F(x) = N
( X, fxx-_fe
- b (1)
(113)
Still in the case = 2. for every X E (R AN and every X > O, denoting Y = AVF(X), we have
F(Y) F(X) S -> -2(1- (1-2Nd) |VF((X)
A
(114)
2Nd
and, provided À E Nd), we have Y & AN.
Proof. The optimal transport plan between Po and pa is given by
1 N
7o
6(x100 & (o
(115)
We then prove that F is differentiable at X with the given expression, and that F is continuous, the same way as in the proof Proposition 3.1 (in Appendix B. I).,, where we replace every integration on a Power cell (of the form Jvo.. ... (x)()))
by an integration on Po.: (of the form 1. f f... (x).
Similarly, we prove the upper bound ((Y) F(X) the same way as in the proof of Proposition 3.2 (in Appendix B.2), by noting that (not necessarily optimal) transport plan between Po fy and pe is given by
%
Soalo @ (o
(116)
N
Finally, if X E (0, Nd), we prove that Y AN the same way as in the proof of Proposition 3.3 (in Appendix B.3), as we will still have x( Vx, F(X) X,) s 1l X1 by the same reasoning.
Furthermore, for a fixed N 0, provided p Pa (R. satisfies technical assumptions on its barycenters ba then we have the following extension of Proposition 3.3:
Proposition B.10. Assume that there exists
0 and θ S with Uga- (())aaa1))))) 0, such that for every O E θ and
i E (I,..., N 1), bo.i+1 bo.5 2 m. Then there exists some constant C = C(θ) > 0 such that for every X E (Rdy and X > 0, setting Y = X - XVXX((()) ), for every i # j. if [x XX,Il < dCm, then | Y > ||X Xxyl- II. In particular,
if X is a critical point of F, then
min X itj
X,9 > dCm
(117)
Proof. By the same argument as in the proof of Proposition 3.3 in Appendix B.3, we have
|| Y, 1 2 |X x,11² 2A(V - Vx, F(X)xX -X))
(118)
with
F(V) - Vx,
- - xx,12 2-12-2 - [(01Re)20) (be,z22000 - -X,)d0 (119)
22

---

Towards Understanding Gradient Dynamics of the Sliced- waaserrssserttiiin Distance
and 0o |x, XII every k- 1, ,,...,,.. N. In particular, we may assume that è ー Now, if A sd-1 is such that (010)) 2 0, again
X1-X Indeed, we can again check that we have a ok) N ++1-xxx
x (k) and b_o,x = -b. for
have (0)00))(()) (a) (01x; - Xz) > 0, and if furthermore 0 E θ we have (1p.0z)a(0) b 00,0
X; - Xj) 2
m - -Xy), so that
(0 O002 11006>>200000
00.76 - Xz)d0
m( -X3)d0
(120)
(0((0))))0))
2 m |x - Xxyl 1
( (0
(121)
Now, let a E (0, 1] be the unique value such that
1
Uga-1 (fa > (810)) > 0}) = =Uga-i > >0] = -- -
(122)
(by symmetry, a does not depend on do. and only depends on (e). Then, we have
Jenttore)220) (0100) = / (on(10vo)20) (010)) + / (00(2220100)20) (0)
(123)
2 ollsa- (0 2 a)))+ 1 (en(a>(010))00) (010))d0
(124)
2 ol n (a > (010a) > 0))+ + 1 | (0)0))
(125)
d0
(126)
(0160)))))
where the third line is obtained by noticing that (122) implies that Ug (On 2 a}) = Ugg I na > (010))
>
0}). Thus, denoting C C(O)= :- 2 [aa> (016)) 003 (0 (016))d0 d0, we have, combining (118), (119), (121) and (126),
2A N
1
I - Y211² 2 X; xz1
2
|x
- x,1 Cml - - x,1l
(127)
from which we deduce ||Y; - -Y,l > 1X x,, if ||,, X,ll < Cmd. In particular, if X is a critical point of F, then VF(X) 0 and Y X, so we must have ||x: X,バ| 2 Cmd.
Remark 11 In particular, if there exists > 0 and θ C g with Uga ()) 0 such that for every O, pe is absolutely continuous with density bounded from above by 3, then, by the same reasoning as in the proof of Proposition 3.3 in Appendix B.3, for every 0 E θ and i = 1,..., N - 1, we have hg. - b,, 2 NB. Thus satisfies the assumption of Proposition 1.10 for every N > 0 with è and m == 3N- of the Po for a non negligible set of directions 0 (instead of all of them) for the gradient descent to be well-behaved (i.e. to guarantee that the iterates do not get too close the generalized diagonal and are repelled by it).
1 Therefore, we only need to have an upper bound on the densities
B.5. Proof of Proposition 4.4
First, it will be helpful to introduce the following family of transport plans between the projected measures: : for a given 0 e gd-1, we use Theorem A.2 to disintegrate μ and p with respect P& to get families of probabilities (pe.w)ueR and (Po..)EER such that spt( C Py (u), spt(Pa.:) C (s) and for every test function & E (Q),, J ((z)du(()) ((())pp..<)))dpe
f (z)dee and ʃ (())dp(v) whose integral over a test function ((.. v) E C 22 is
We then define no the probability measure
f I / / l p(a,
(128)
We can see then that to is a transport plan (not necessarily optimal) between μ and v and that (Po. Po) = 7o (in other words, to is optimal for the cost function (z.y) + (y 20))2). We also disintegrate to with respect to the first
23

---

Towards Understanding Gradient Dynamics of the Sliced- Bleed-Wasserstein Distance
variable, giving family of probabilities (uER such that for every function ş E (R R), p(u, o)dro(u,)) JJ J ((,, vi) (r)ddpo))) Notice that these give an alternative definition of to inddedd To(u) odyo,a())
We can now proceed to the proof of Proposition 4.4.
Proof (Proposition 4.4). First, if E ("x, R²). then, defining H (RR)) N by : E(X;) for every I,..., N, we have for every > 0 (small enough so that + tH & AN).
F(X tH) sW
(1d)
(129)
from which we deduce, by taking the right derivative at t 0,
d SW2 +t)
(VF()))))
(130)
dt
1==++
In particular, we immediately see from Definition 4.1 that VF(X) = 0 if and only if pax a Lagrangian critical point.
Second, the condition Unx px-ace. from Definition 4.2 writes as
ā - - (o(( = O, E (I....,NN
(131)
Fix 8 E gd-1 such that the (X₂ (). ..., (Xx10) are distinct. Using the notations from Appendix B.4, we know that Yo = N Em 0(x100) @ Po.oxta0 (1) where pe, P9.N N is the family given by applying Lemma B.7 to pa. In particular, we deduce that for every i, (0.(x.10 P8.0 (())) and thus
50 ((x,100) = / (d)0,(01;;(0))= udPp
= (0,0
(132)
and, using (113), (131) rewrites as
NV F(X) = 0, i f1....,NN N}
(133)
Thus, VF(X)= = 0 iff is barycentric Lagrangian critical point.
Remark 1. Notice that this proof works in fact for general p E (2R(R (with the expression of gradient V given in Proposition B.9).
B.6. Proof Proposition 4.7
First, we prove Proposition 4.7(a). Let Eo, & E L2(,, Rd), we denote 3==1d Id= t) + 1 and μr S/ For any fixed t E [], == ((Po. Pe) (So, Si)) u is transport plan between ji and μul such that
p ((1 - t) + tT22#7
(134)
where n. is the projection on the i-th coordinate. Furthermore, by Proposition 7.3.1 of (Ambrosio et al., 2005). there exists plan E P(R x R x R) such that (TI, T2)#7) and ((1 t)= + ÊTT, T3) is an optimal transport plan between 1ú and Pa- Then, according to Theorem 7.3.2 of (Ambrosio et al., 2005), asserting the semi-concavity the squared Wasserstein distance, we have
Wa(ng, pa) > (1 - 1) w3(ng. po) + W3 po) - <(1 - () W?, (b))).
(135)
where W,, defined in (7.3.2) of (Ambrosio et al., 2005) by
( - t) + (T.)) Tk =
[11- t) + tx, xx xxl2 (dn(nn Ix)
(136)
JRRXR xR
24

---

Towards Understanding Gradient Dynamics of the Sliced-Wassereee Wasserstein Distance
for every j, E 31, and 1 [0, 1]. In this case, have
(1gg = JR3 (a) - d1)) 12, = / (o - dd;)
(137)
/(So(e) -
(x y1- S1) =
(i)))0))d((2))
(138)
RR2
(we take i 0, 2, and f 0 in (136)). Integrating the inequality (135) over @ gd-1, get
SW3(d p) 2 (1 1) sw Sw3 .) + + - -t(1 -t) / | (51(z) - Eo(x))0)²
(139)
-
2 (1 - 1) s) + .p) - +)) Eoll2a(H)
(140)
This rewrites as
Pu((1-t)6 + t - (1- ()F + (FF((E)
(141)
which proves the convexity of Fp-
Now, we prove Proposition 4.7(b). First, we show that U/ E (H, This is the case because Id E (,, Rd) as E (RR)) and
f 165 (0((210))0d)) d0dp(z) S
12
( /
((((1))))dddll))
(142)
RR
184-1
1o (v)dOdp(z)
(143)
84-1
v²
(144)
JRd
(w)
(145)
(d, v)d0
(146)
R2
pd)po(o)d0
(147)
= 1 2/2 11yl2dp(y)) < 00
-
(148)
where we used Jensen's 's inequality in the first lines, and E (2(R)). This proves that up is in L (1, R.).
Fix now & E (2( Denote Se 1d +E and ue Se#M, then for every 0 E gd-1, the plan 35 (S Id) th is transport plan between u and such that (Po, Po) E TT po) is not necessarily optimal. Then, we have
(R-)2 / (11))0))
W po) -
/ (8e(x) y10)2
(ROP2
< (Reys (x + ((x) y10;2
< / (x (5- -00000)ddill + / / (xr (00 + / (((=)))))dda((,,, v) (2aj2 (x (1000000((()))))000r...... + (E(x))0))
(Rag²
< w2(Ho;po) po) +2
(149)
(Rdy2
25

---

Towards Understanding Gradient Dynamics of the Sliced- Wassersesnrt Distance
The second term in the right hand side of the last inequality is
J.ayp (ar 010):001())) (u- (
/ / - /
[ (0((())) / ((x))) oo
(=)))) - -(o(())))(
(150)
Therefore, integrating (149) using (150), we get
SW2 s SW3(u, p) +
g4-1 Jec (1x10) (() + 11l11/pw
(151)
but since
Isan / ((x10) - ((())
/ (5(x)) [ ((x10) - (2((
(152)
3dd-1
= [ (5(x))=0= J
(153)
= (ElU))
(154)
equation (151) rewrites as
SW2(nS,)) .p) SW?(u, + 2( + jleliem
(155)
that is
O2 - (())))=(( PA(5)
(156)
and this finishes proving Proposition 4.7(b).
Finally, we prove Proposition 4.7(c). Assume that μt, p are supported in some compact set C Rd and are without atoms. Let E (p, be fixed and define ct() = SW2(p', p) where u² (Id field to gives
Equation (155) applied the vector # +
((t) S x00) + (t(ealE) + 11811P6
1
(157)
Therefore, we immediately have the inequalities
lim sup (sp(1) (<()) s 2((1)))
(158)
b-0+
lim inf (p(() =p()) 2 2(Elu))
(159)
60 t
Let's derive the other inequalities: : let (ye, va) be a pair of c-concave Kantorovich potentials for (Ho, po) (for the cost c(u, v) [u 2) For every > we then have
2 /
(W3(ub. Po) Wa(no,
po(w)(dpb(() - -qp((u)))
(160)
f. ga
2
pol(o =E()))0)) spo (2(()))))
(161)
ga
26

---

Towards Understanding Gradient Dynamics of the Bliced-Wsseesteei Distance
(the factor 2 comes from the factor 2 in the cost c). By c-concavity, (Py is Lipschitz on Po( (it has the same modulus of continuity
that we here the fact that j and p have compact support). Thus, + 1 ((po((ar ({(=))0)) (o(((r(0)) (0))) is bounded from below by (z) which is integrable as Ç E (M, Rd), where L is the Lipschitz constant of fa, which depends only on diam Since for the cost c-concavity means that 21 | ie is convex and Isc (see (Santambrogio, 2015, Proposition 1.21)), sp has at every point right and left derivatives P0 and Pg therefore, applying Fatou's is lemma and integrating on gd-1
1 lim inf 7 (W3 pa) - (2 po)) 2 2 0=0+
(162)
24-1 Jra
However, since is without atoms, by Proposition A.I, for almost every 0 E g Me is without atoms, and for ua-almost every u, yp is differentiable at u with (2) = 98 (u) = PP (u) Furthermore, we have (/((u) = (u - Tu())), where To the optimal transport map from p40 to po. and 50 To (as "Yo (Id, To) (o), and therefore
= ((x10) T(((=
(163)
Rd
((x10) - -0((()))))))((()))
(164)
80
lim / (W2(né po) W 2 { [ ((x:00 (000000000()000000000000000000
18a-1 Jzz"
(165)
++0++
Integrating this latter inequality, we obtain
lim inf [(0(() - -0)))=000)>)))))))))))))) 2 2(() b-0++
((0(())(()))
(166)
Using similar argument, we show that
lim sup (1(1) ( -(()) S 2(( L7()) 6-00
(167)
This proves that to is differentiable at / 0, with
& (0) 2(p)))
(168)
This finishes the proof.
B.7. Proof of Corollary 4.8
First, if is a Lagrangian critical point for SW3(,,)) p). then for every & E I.2 (1, it satisfies (14)
d Sw2( +1E)
=0
(169)
1=0+
But applying Proposition 4.7(b) to the vector field iε, we have for every >>
((d < SW3(u,)) 20(u + 1 11ll
(170)
Combined with the previous equation, this yields
d
0 = dt
SWI +t5) 1==+
S 2((vplE)>>00)
(171)
Therefore, we have (c)ll) 2 0 for every & E Rd), and this implics Un = 0 in RH, Thus, μ is a barycentric Lagrangian critical point.
sPo is convex, it is differentiable almost everywhere with nondecreasing differential. Furthermore its of nondifferen- tiability is at most countable, so it has zero poo mo-muasure as no is without atoms.
Since 2
27

---

Towards Understanding Gradient Dynamics of the
Distance
Now, assume that μr, p are compactly supported and without atoms. Then, by proposition 4.7(c), for every & (1 have
d (Wd #11
=2(v
(172)
dt
1-0+
Therefore y satisfies Definition 4.1 if and only if (e,l))) 0 for every E (u, R2), which is equivalent to ep = 0 j.a.c. Thus μ is Lagrangian critical if and only if is barycentric Lagrangian critical.
B.8. Proof of Theorem 4.5
First, up to extending Qt we may assume that the Mn, i are supported in Ω. Indeed, if R >> 0 is such that Ω S B(0, R), then if I E spt(pn) is such that Upa (x) = O, we have
1 / g=
0 Uun (2)
_ga-1 Fn 7 (z)0))00)
(173)
where for every 0 E gd-1, 7,,0 is the optimal transport plan between Mn,0 and po. so that
|a|| s a 1snn 30 70((x)0))0dd S dl
f _ga-
|50((= < dR
(174)
Since UP, f,, -amost everywhere, this implics that Un is supported in Ω = [(,, dR), and so is μ..
Now consider { Ω +> Rd continuous vector field. For every n and R, applying Proposition 4.7(b) ig, we have (ld p) S SW2(um, p) + 21(u (2))))))) + 1 1 (1:3
(175)
S SW3(mm p) + d 1El
(176)
since Up.. 0. Letting - 0o, thus find
+T SW3(n,)) 11
(177)
(Recall that SW2 S W2 and that on compact spaces, weak convergence coincide with convergence in the Wa topology). But since y by assumption without atoms, by Proposition 4.7(c), t + (Id- p) differentiable at 0 with derivative 2(v,l() so this inequality implies (val() 0. Since this holds for every continuous vector field : > Rdd by a density argument we conclude that U/ = in [,, RRd), and μ indeed a barycentric Lagrangian critical point for SW2(. p). This finishes the proof.
B.9. Proof Proposition 5.1
First, let μ = -
$ and let p be the sliced-uniform measure, which we recall the definition below.
Definition B.13. The probability measure P E P(TR2) supported on the unit open ball B(0, 1) of the plane with the density
1 f(x) 25 -ー is such that in every direction 0 E gd-1, its projection Po#p is the normalized restriction of the Lebesgue measure [-1: We'll call p the (two-dimensional) sliced -nifform measure on [-1: 1].
As explained in Definition B.13, each projection Poup is the normalized restriction of the Lebesgue measure to [-1, 1]. Indeed, the density of Pei#par at [-1;1] is given by
1
1 dy x² y2
1
1
1
3
Pa Pa)
dl
d0
(178)
2T J-
2T
VI
2
with the changes of variables
VI z2t, t- = sin 0. By symmetry, the same result holds for all 0.
Then, identifying 81 ~ (- ,, π| ~ (0,2 we have for every direction 6, Pe = _ and when 0 # + we have 140 sicol L col slcgl with the notation Co cos(0) and sy = sin(f) (in the vertical direction, P+g 8y). The optimal
28

---

Towards Understanding Gradient Dynamics of the Sliced Blece--asserstein Distance
transport map from Hig to pe then (1x)) 4|cal xr If (I1 0) #iel E spt(f)) [-,, (0), where (e1 62) is
the canonical basis of IR? we have (noting 0 0= (co, 80)² with the vector notation to differentiate with the scalar angle 0),
d {
To To(
y000 2π
=2
(o(1))))
To
Cg
d0
(179)
-
89
2
π
Co
(180)
2
cel
80
2m
Ico d0e1
(181)
(We that the integral on the second coordinate cancels by antisymmetry). Since
1
ccol0 ldf = 2 % lcold0 |d0 =
le² cod0 1
x//2
(182)
4
we thus have
T1 S To( 00
(183)
2T
that is (x) = 0. This proves that j satisfies Definition 4.2 and therefore a barycentric Lagrangian critical point for pW)(.
Now, consider the where d 1, p NN(0,, Ia) and μ (Id, 02-1) az) with oa d fga- ((lea) d0.
For every 8 E gd-1, we have pa (0 1). Noting (e1 ... ed) the canonical basis of R when (0le1) & 0, have 140 Po#f (O, (aal and when (0|c)) 0, P00 6g- Therefore, the optimal transport map from sso po is given by Ta (0le()))))) -1x. Let Iι6ι spt R x {0)d-1, then we have
d af Jgs To(())0)))dd d ( To (0les) )000
(184)
= dd { I oca (0
(0le))
0d0
(185)
(186)
By symmetry that the components of this integral along 62, ..., Cdt zero, and thus
d 1 To (o(())0)) dri ---1
(0le))²
dAc1
(187)
Jga- arl (fle1)
d [(le))
(188)
ad 3
2161 by definition of ora
(189)
This proves that μ satisfies Definition 4.2 and is therefore a barycentric Lagrangian critical point for SW3(.pp.
B.10. Proof of Proposition 5.2
Sketch of proof. Up translating, rotating, and rescaling, may decompose as μ (1 A)po + A where in H For every 0 e S!, let fo E II(H, p) be such that (Po. Po) # to is optimal between and Pa- Then we can decompose fa and p into
40 -X)oo.0 + Xño.1
(190)
and
p = (1 A)po.a Apo,1,
(191)
29

---

Towards Understanding Gradient Dynamics of the Sliced-Waseeras Distance
where i0,, couples br and pa.s (2(())) Denoting P0,1,0 the projection of Pa.: 0 for 0, 1, these decompositions verify
SW p) = (1 (1- / W ( f W Pu.1,))
(192)
with equality at t = 0. We bound separately the two terms of the right hand side. The first term can be easily bounded by
(1 (1- / W2(Ho,0.P.000)d Po.0,))00 (tt))
(193)
All that is left is then show that the second term can be bounded for any C 0, neighborhood of 0, by
Jss- W2(e1,0 Pe,1,,,,,,,1,,00 1,,1,9)d0 Ct
(194)
We obtain such bound by writing W ( 1,0 F -Frel.. (1.2(().)) and by making use of an explicit expression of F-1 and its symmetry to compute a Taylor expansion of 1i
Mi.
fgr. o P0,10))
(195)
and bound it from above in the desired way.
Up to translating, rotating and rescaling, we may assume that = [-1, x {0,0) and = e2. Since aHts S μ,, we write
# = (1 - A)Ho + Ap1
(196)
where E [0, 1] and No, in are probability measures such that la H fl and X 2a. For every 0 St let 70 E [T p) be such that (Po, Po)#fo to is an optimal transport plan between μs and pa. Using Theorem A.2 we can disintegrate to with respect to μ., thus writing dfi( y) dio (v)a)d((()) and we define two probability measures Pe.0: P8.1 E P2( by
I ((u)pmalu) = / (ls)drollle)ddooo) iE (0, E C&(R")
(197)
and two transport plans fo. E I((s;, by (1r v) dio(vl=)ddux(() By (Villani, 2008, Theorem 4.6), the (Po, Pe) i are actually optimal between their margins. In fact, we have
Walmh. p0) S - 2) wā(ri.o p0.00) W2(Hi, po.1.)
(198)
where v² = 1/2 (Treg # + V) for any measure v, with equality at l O. We will establish bounds separately on W2(pb P0.0,0) and W3(M), p0.1.0). First, we notice that
[ Wa(t WollhooPooo0d00 Po <
W + 2t² 1
(199)
Indeed, if consider the transport plan 96,0 TTpb po.0) defined by
1 ((Tez) Id) 90,0 + (T- Id) f0.0))
%0.0 = 2
(200)
we have
30

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
Wa(Hb,o Po.u,0) - / (x 1l- (2.
(201)
/ (1x + lez - 11 0) + - le2 2l) v)
(202)
/ (o - v1))² (ez) 03 "))))))))).)....
(203)
W P0.0,0) (e
(204)
and by integrating on the sphere we get (199).
Now, all we need to prove is that for every C > O, there exists neighborhood of / = 0 in which
[
<
P0.1.0) C12
(205)
By summing it with (199), we obtain the proposition's 's statement. To derive this bound, we look the quantile functions for every $1, we have
Walpf (0,1,)d
[F=1
Fro.1. 11 [[
(206)
Mi.
||=-1
-F + F-1 - -Fm 11 (1.2((0.1))
(207)
Mi.
||=11
Fm-1
F (22.01) + +(F 1i,0 -Frm |F
F=1
(208)
Pi.0
11 F=1 (1
(209)
W2 P0..0) + 2(F W (16,0 m1
ki
F
Fm
Fm
po.
(2((0.1)
(210)
We easily see that W3 (pi, 10.ii.111111 (1,0) I W2 (ff. p1) S 8 Therefore, we simply need to show that for every C > 0, there
exists neighborhood of = 0 which
[
(F F Fio F F (3(10,1))00 LP -Ct? g1 ni.0
(211)
Since #1 H [o] we have, for every
A = 7 (H -1- + (1-1.11(0))
(212)
Now let 0 E ES (+5) (we make again the identification 31 ~ RR/2z2). The projections of μ4 and pa RO are
14.0 Alcol (AA.. + Axt,),
A = |stl |col =|sel + Ical
(213)
and
11.8
Aoo
Ag = [-1o
(214)
2c0
Therefore the quantile function of f1,9 is simply
M (x) = -Icel 2/coo/2 x 20
(215)
In the following, since for any and any measures V1, 122 E P(R?), (11 = (v1.0. V2,0), we can restrict ourselves ( n )). To compute the quantile function of ph_0- we then need to consider two cases.
31

---

Towards Understanding Gradient Dynamics of the Sliced- wassessensrrttnnnn Distance
First, when | E [0, arc4an(1/|44)], the segments A overlap. Their union can then be decomposed into three
segments where the density pf,e is constant
B_ U Bo U
B
| col [tsol: -col | [ssll]
(216)
col
[sool,
|col
[Aso]]
(217)
U [[cel
[tsel,
|cyl
[tse]]
(218)
On B2, the density is Teal while on Bo. it is 2 (using the shorthand notation tan(
One can check that the quantile function of pf,0 and μ is then
col
[ss
4 4|co /x:
for x E
0,
[t]
(o
2
F7 (x) Pi.
-cal
[ssll
2|cal
(x 1 [10l]
for
2 [t): 1
Lo
(219)
cce
|tsøl
+ 4/ce|
(x - 1 + 1t to
for
1 [tll
Second, when |0] e (arctan(1/|t]) x/2). the two segments Ab,, do not overlap, in which case the quantile function of "9,0 is
cel
Isel + 4|co/1
for [0, g for x E (1.1]
F=1 (1)
(220)
11.00
Col + |t
+ 4|col (x - 4
Denoting m4,0 2 min(1, [tto),, actually condense the two previous expressions of F-1 into single valid for every 0 E (-n/2,m/2))
Mi.o
ccol |sa + 4|ce|æ
for x E me,e
F.1. (x)
col + 2/cg |x
for x E (m,,0: - my, for x E (1 -,,,,, 1]
(221)
cco + Ise
4/col + (xr
We see in particular that
(x) F-11 a1, (x) for every x (m,,0, m
For every E R and x E [0, 1], F=1 (1 x) = 1- FT (x) (in fact, we only needed to use the symmetry of μi, to see this)
Mi
Therefore, have
(F-1 F [F-1 Fm110 (oo.o)2200 L2((0,1)) = -f (F (F -x) (z)) - Fm (x))
11.0 11.0
Mi,
(F=1 (x) Pz. (z)) (x) o (x))
Mi
(F (x) Fm (z) (z) F (x))
1-m
mi,a
(F-1 (x) psie (x)) M1 (x) -F-1
F (x)))xx
Mi.8
m0,0
1 (1 - x) - F-1 (1 - x)) 1 (1 - x) - =F -Fol (1 - x))
M1.9
J0
m,
(F-1
1io0 (x) F (z)) / (x) - F - (F (1 - -x) Fm (1 - xd
32

---

Towards Understanding Gradient Dynamics of the Sliced- Saede--assersttinn Distance
We have
1
F F. Mi.8"
(x) - Fi (a) 2|co|2 - -tsol = 2/col(x -
2 \tol))
(222)
M1.9 (1:) F (1 - x) = -col + 2/coll (-lcel 2/c0|(1 x)) = 4/co (rr 2
2
(223)
for x E m..) If ffror: x E [0, 1] ( we note
Ga(a)= = p0.0 (x) F=1
(1 x)
po.3.
(224)
then we have
mi [/ (x (44coo)
1 Itttl)ddz
(F 11.8
Fm1 mi.
F o L2(00.1)
Go(x))2)ool( -G=
(225)
However, our hypothesis that for every the density of Pe is bounded from above by b allows us derive lower bound for Gø. Indeed, since p (1 - A)po.o + Apo,1, have p8,1 S 3 and thus p0,1.0 S b with b X- Then, using the shorthand notations Fe Fpo...o and Fo Fo for almost every x E [0, 1].
FØ (Fo()) = x
(226)
Let a h with h > 0. Since
Fo(x)= = Fa(a)+ + poalaaaa < Fo(a) + bh
(227)
we have
a+ - FT bh)
(228)
Similarly, i = a - h with h > 0, we have
Fo(x) Fo(a) - p0.1, h,) 2 Fo(a) bhh
(229)
thus
a h = (Fala - -))) 2 FØ (Fo(o)) ih)
(230)
and thus we have
2h 2 8 (Fo(a) bh) - F (Fe(a) + bh)
(231)
Now, pick a such that (a 1 Let x E [0. 1/2], and let h > 0 be such that
1 - bh. Then, substituting the value of
2'
x in the previous equation, we get
2
(x) (x) Fr (1 - x) < -2
- x)
(232)
Go(z)= 2 0
(233)
for almost every x [0, 1/2). Thus, since by definition of m,,0, (x {x - [ 2 0 for x E [0, m. this means that for every 0 E (-T/2, x/2),
- } mmme0 (xx
2, mi
1 ) - eeto
(F F 1
11.0 po, "p c2 S 2/co|(4|co)
(234)
Let's compute the integral on the right-hand side
mLe
1 )x - 1ttta)dxx
mis
1
(x
1)le =
x²
-1 (1+ [ttee) 2 + tt dd
(235)
2
4
m
1
1
(1
|
)om + |1al 10,0
(236)
33

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
If [0] S arctan(1/14). then m4,8 [gg1 and
m
1
[tta 3
(x (xx
(1 + [ttol [ttel) ltol 1
(237)
2
24
16
|u 1
|u4a 3
(238)
16
48
ttg (1
3 lttel)
(239)
16
and in fact, since [tto| < 1 when |0| S arctan(1/|d)), we have
ms.6
1
1
(x
2 7) [tte)dxx
|tt 21 (1-
lttel)
ltyl > 0
(240)
16
24
Let 01 (/2) be such that 4cp,
S - and let be small enough so that a, = arctan (1/la)())))))))))))))))))))) > 0,. Then
If [0] (ar, x/2). then we can simply bound (234) from above by 0
14.0 - F Mi,0
F L((000))( S
(241)
as 4|col 3 << 0 and the integral is positive. Thus
(FT1 Mi.
F
( s
(242)
If [0] E (0) then, combining (234) and (240) we have
2
m4,
1
(77. -Fr1 F F=1 P L2(1011) s 2/ce|(4|cel
(I
7)
(243)
b
2
0
2
2
s 2/co| (4co((4/co)
-
b
16
Ht
z g(ttol)
(244)
- (2 + 26 (1 +
|tto,
(245)
Therefore, conclude that there exists constant Co > such that
[-m--ojjues.on V - Fm Mi.0 - Fmol L S Cotz + o(t?)
(246)
Finally, if |0| [01, arl then, again combining (234) and (240), we have
2
me,0
1
(F-1 Mi.0 M1. F - F=1 2 r2((0.1)) F - 2/co|(44ca))
(x
2 [x
lttl)ddr
(247)
0
2
s 3lcol(4|ca)
ttg
|tto|
|tto l)
(248)
b
16
I 12b
Ico collito|?
(249)
|so,
12
(250)
12b
[ce
12b
|cel
However, the integral fat d0 Jcol diverges to infinity when f + 0. Indeed, using the development
n -|t| [||| + o(()
a == arctan (1/|t)
aarctnn(tt)
(251)
2
2
34

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstein Distance
have
d0
sin(ae)
du
(252)
81
sin(O1)
u²
[In(1 + u) In(1 u) ()))))())))
(253)
(In(1 1 + sin((a)) In(1 - sin(a:))) +C
(254)
2
[In (1 sin _ - + o((1)) In (1 (1-sin sin ( -
TT
- 14 14 o(())) +C
(255)
(In(1 cos([t] + (({ In(1 cos( + o(f)))) +C
(256)
(In(1 1 cos((t]| + o(t2)) - 1n(1 cos( + o(t²)))) +
(257)
In 2
1 12 + ol(3) In
[
ol1F) +C
(258)
2
2
= (In(2) + o(1) 2 ln(t) + ln(2) + o()) + C
(259)
2
In(1) + C + o(1) 2- +o0
(260)
Therefore, for any C' > 0, there exists a neighborhood of t = 0 in which,
{ J-a.--)-1100.
(F-1
1 Fre.1 ) [= S -Ct²
(261)
Mi.
11.0
11,9
Thus, we can prove (211) by bounding the integral of (F-71 Pirt 11.6 -F on (-x/2, /2, x x/2) separately on the wi.6
three regions (-n//2, -oy] [a+, T 1/2). [-01, 01] and [-at,- -01] U (0,, ar.] using (246), (242) and (261), taking in (261) a constant C' big enough compensate the constant Co in (246). This concludes the proof.
C. Stability and numerical approximation
In this section, we will discuss briefly the regularity properties of (practical) Monte Carlo approximations of the SW objective and what they entail for applying our theoretical understanding of F to practical applications involving Fz. The discussion will be similar to the one found in (Tanguy et al., 2024a), although they focus on the discrete setting, where p also a point cloud, whereas we focus on the semi-discrets one.
In practice, the Sliced-Wart Wasserstein distance objective (6) discussed in Section 3 is usually computed through Monte Carlo estimator to approximate the integral. In the semi-discrete setting, this amounts to approximating the function F(X) discussed in Section 3 with the function Ft 2 Σ/= ( (X) (e.) where 01, ..., 0r 5d-1 are chosen directions. The latter may vary: for example, they may be uniformly sampled on gd-1 at every step of stochastic gradient descent (or some other optimization algorithm), or fixed once and for all.
In fact, the local behavior of Fr, is quite different from that of F, and exhibits a cell structure. Indeed, for every o E 6 let C. (X e (Rd) N 1l E {1, D}, Te.. is uniquely defined and is 0.). Then, for every X E Co. we have
L
N
FL(X) = 2L
((xa (0)) - x xPdpo, (x)
(262)
1=1 i=1
Voyua
which simplifies
Fx(X) (X) Co
(263)
with the quadratic function
N Σ
g (X) = 2NL L
(X bo..el
(264)
I=1 1=1
35

---

Towards Understanding Gradient Dynamics of the Sliced- Wasserstiin Distance
and the constant
N
Co 2L
|x-
(265)
Vo,
In fact, Co can also be written as Co = (X E (R4)* 'Yo' E G 1or (X) > (x)) from which we can deduce that C. an open polyhedral cone, obtained as the intersection L(N! - 1) half-open planes. Furthermore, Fz is actually the infimum of the Co + 9r=
FL(X)
inf qo (X) + Co OE6%
(266)
As consequence of these considerations, inside every cell Co. Fr. will be C is equal to quadratic function, and its gradient and Hessian at X E Co are respectively
1 Vx,FL(X)) NL
Σ ((X,101)
b 06)0
(267)
and
1 NL 815 Σ 0 2
Vx,
(268)
Thus, Ft is convex inside every cell Co. In fact, we know by (Tanguy et al., 2024b, Theorem 2) that when L >> d, for almost every family ....... Og E S n (RO)) = {0} and 1. E/=n 0:67 is definite positive, which makes Fz strictly convex inside every cell. In these conditions, any critical point contained in a cell will be a local minimum.
This is of significance when optimizing Fz. Indeed, even it were possible to derive theoretical guarantees that high energy critical points of F unstable, a numerical scheme optimizing Fr. could end up converging high energy critical point of Fz because of its local convexity. Consequently, must be chose a number of directions L and of points N large enough to make sure the size of the cells Co is small enough prevent this behavior.
Experiments In another experiment, based on the discussion of Section C, we considered again the point cloud X (Xi,.., XN) with X; = - - N- with N = 100, the perturbation & that alternates between and -é2, and we plotted the estimator 1 + (x (E) in Figure C, where p is the sliced-uniform measure, for different sets of test directions {0,.... }. We tested different values of L, and, for each of these values, we considered two cases:
one set of test directions {0,,...,0r) including e2, with 0; = _ + 2m(i-1)
L for i E (1,..., L}
one set of test directions (O.,...,0 0r) excluding e2, with 0: + E 2m(i-11 for i E (1...., 7.}
We observe that, as expected from the discussion in Section C, when the test directions exclude e2 (so that the points X have distinct projections for every test direction), the estimator + (x + { is locally smooth, and we distinctively see its cell structure for the smaller values of L. On the other hand, when the test directions include 62, we see that the estimator smooth at O. This again conforms to what we theoretically expect, as
1 W Poz) W 1 (6 + δ1l N ( 1/ 1))-
1 3
14
(269)
so Fr.(X 18) s(() 1 where f(t) some smooth term.
36

---

Towards Understanding Gradient Dynamics of the Sliced-Weid Wasserstein Distance
(a) L=10, with e2
(c) L=20, with e2
(e) L=40, with e2
(g) L=100, with e2
0.06340
0.00125
0.08100
0.0676
0.06335
0.0642
0.06320
0,08050
0.0641
0.08025
0.0672
4 0.06325
0.0640
0.0670
0.06320
0,07975
0.0639
0.07950
0.0668
0.06315
0.07925
0.0638
0.06310
-0. 00.010-00 -0.000.005
0,000
0.005
0.010
-0 00010-0.005 0.000
0.005 0.010
-0 -0.010-0.000 005
0.000
0.005 0.010
-0.010-0.005 0.000
0.005 0,010
(b) L=10, without
(d) 20, without e2
L=40, without e2
(h) L=100, without
0.06096
0.06312
0.05413
0.06095
0.06264
0.05417
0.06094
0.06262
0.06308
0.06093
0.06306
0.06260
0.05411
0.06092
0.06304
0.06258
0.06302
0.05410
0.06090
0.02566
0.06300
0.05409
0.06089
0.06298
0.005 0.010
0.010-0..05
Figure 3. Behavior of Fz for different sets of test directions. Depicts the value of Fx + tε), where X is point cloud of N 100 points uniformly distributed on the segment [-4/Tr, i, 4/m] x {0). & alternates between e2 and -e2, and p is the sliced-unifora distribution. Each column corresponds different number L {10, 20, 40, 100) of fixed test directions the top line is included in the test directions while on the bottom line is excluded
37


Reviews:
[{"review_id": "3DUELnxQXf", "reviewer": "Reviewer_BmZL", "summary": "This paper investigates the properties of gradient flows for the Sliced-Wasserstein (SW) distance when used as an objective functional. It rigorously develops different notions of critical points—Eulerian, Wasserstein, and Lagrangian (including a barycentric variant)—and studies the convergence and stability properties of discrete gradient descent schemes that approximate the continuous Wasserstein gradient flow. The theoretical contributions include proving that, under suitable assumptions, the discrete (particle-based) critical points converge to a continuous critical point and that “bad” critical points (e.g., those supported on lower-dimensional structures) are unstable. Numerical experiments are provided to validate the theoretical findings and illustrate the behavior of gradient descent dynamics with various step sizes.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive remarks and relevant questions. Even though we restricted our theoretical analysis of the $SW$ distance to absolutely continuous target measures (and in Section 3 to discrete approximating candidates), this was mostly for the sake of simplicity, and it is indeed possible to extend many of our results to larger classes of measures, often (but not always) without requiring significant adaptation of the proofs. For example :\n\n- Direct extensions :\n\n    - In section 4, the notion of barycentric Lagrangian critical point for $SW^2_2(\\cdot,\\rho)$ can be defined for arbitrary measures $\\mu, \\rho \\in P_2(R^d)$, as the 1D optimal transport plans $\\gamma_\\theta$ are always uniquely defined.\n\n    - If we replace the assumption that $\\rho$ is absolutely continuous by the assumption that it has no atoms, then Propositions 3.1, 3.2, 4.3, 4.6, Theorem 4.4 and Corollary 4.7 remain true. \n\n- Minor extensions : \n\n    - When $\\rho$ is an uniform point cloud $\\rho = \\frac 1N \\sum_{i=1}^N \\delta_{Y_i}$ (with the same $N$ as in $\\mu$), it should be possible to prove analogues of Propositions 3.1 and 3.2, replacing the barycenters $b_{\\theta,i}$ by the reordered projections of the points $Y_i$ in the statements of the propositions (in fact, Theorem 1 in Bonneel et al. 2015 is the analogue of our Proposition 3.1 with $p=2$).\n\n    - Proposition 3.3 remains true under the weaker assumption (A1) on $\\rho$ that there exists $\\beta > 0$ and $\\Theta \\subset \\mathbb{S}^{d-1}$ with $\\mathcal{H}^{d-1}(\\Theta) > 0$ such that $\\rho_\\theta$ is a density bounded from above by $\\beta$ for every $\\theta \\in \\Theta$\n\n    - The proof of Proposition 5.2 should be adaptable to the case where we only assume of $\\rho$ that it has no atoms, and $\\exists b > 0$ and a neighborhood $U \\subset \\mathbb{S}^1$ of $\\vec{n}$ such that $\\forall \\theta \\in U$, $\\rho_\\theta$ is a density bounded from above by $b$ - let's call this assumption (A2).\n\n- Significant extensions :\n\n     - The proofs of Propositions 3.1 and 3.2 should be adaptable to arbitrary $\\rho$. The main difficulty would be that the Power cells $V_{\\theta,i}$ are not well-defined. We would instead have to work with a decomposition $\\rho_\\theta = \\sum_i \\rho_{\\theta,i}$ where $\\rho_{\\theta,i}$ is coupled to $\\langle X_{\\sigma_\\theta(i)}, \\theta \\rangle$ by the optimal 1D transport plan (when $\\rho_\\theta$ has no atoms, we have $\\rho_{\\theta,i} = \\rho_{\\theta|V_{\\theta,i}}$)\n\n    - Theorem 4.4 also holds under the assumption that $\\rho$ has no atoms and that the set of atoms of $\\mu$ is closed, but proving this requires more sophisticated methods. The sketch of the proof is roughly the following : the beginning of the proof is the same as in the article. We cannot use Proposition 4.6(c) as $\\mu$ may have atoms, but we can prove that Equation (152) still holds is $\\xi$ is assumed to cancel at the atoms of $\\mu$, and this allows to prove $v_\\mu = 0$ $\\mu$-ae on the complementary of the set of atoms of $\\mu$. Then, by considering perturbations of each individual atom of $\\mu$, we show that $v_\\mu$ also cancels at the atoms. Since this assumption on $\\mu$ seems unnatural and its proof is more complex, we chose to state the theorem in the article under the stronger but more natural assumption that $\\mu$ is without atoms.\n\nMoreover, regarding the limitations of our analysis :\n\n- While our original assumption of absolute continuity of $\\rho$ does limit the applicability of our results, the extensions we discussed to weaker assumptions on $\\rho$ such as atomlessness, (A1) or (A2), allow us to cover a much wider range of target measures, including many types of singular measures which arise in machine learning, such as densities supported on a lower dimensional manifold (for example, $\\rho = \\frac 12 \\mathcal{H}^1_{[-1,1]\\times\\{0\\}} \\in \\mathcal{P}_2(\\mathbb{R}^2)$ has no atoms, and satisfies (A1) and (A2) for $\\vec{n} = \\vec{e}_2$) \n\n- The assumption of compact support of $\\mu, \\rho$, which we some of our results require, seem harder to relax. Indeed, it is needed in Proposition 4.6(c) to obtain regularity properties of the Kantorovich potentials, which we use to prove the differentiability, while the proof of Theorem 4.4 makes use of the fact that in a compact space, $SW_2$ and $W_2$ are equivalent distances metrizing the topology of weak convergence.\n\n- Finally, the fact that our numerical experiments, in which the target measures were discretized, exhibit the behaviors of convergence and instability that our theoretical analysis highlighted, suggests that our results should still be relevant in the cases where the target measure $\\rho$ is approximated by a discrete measure.\n\nWe will add in a revised version of our article a discussion of the generalizability of our results and of their limitations.", "claims_and_evidence": "The paper’s claims are well supported by a combination of rigorous theoretical analysis and simple synthetic numerical experiments. The main claims -- such as the equivalence of discrete and continuous notions of criticality (Proposition 4.3 and Theorem 4.4) and the instability of lower-dimensional critical points (Proposition 5.2) -- are backed by detailed proofs in the supplementary material and supported by illustrative experiments. However, some claims rely on technical assumptions (e.g., compact support, absence of atoms) that may limit generality, and additional discussion on these assumptions would be beneficial.", "methods_and_evaluation_criteria": "The methodology is sound: the paper formulates the SW objective within the Wasserstein space $P_2(\\mathbb R^d)$  and develops discrete gradient descent dynamics on an empirical measure. This approach is appropriate for bridging continuous optimal transport theory with practical particle methods. Empirical evaluation is conducted through of numerical experiments that give the readers a better sense of validation of the theoretical analysis.", "theoretical_claims": "I carefully checked several key proofs (notably those for Proposition 3.1, Proposition 4.3, and Theorem 4.4). The derivations appear to be mathematically rigorous, and the use of techniques from optimal transport theory (e.g., Wasserstein geometry, barycentric projections) is correct.", "experimental_designs_or_analyses": "The experiments are designed to illustrate both the convergence behavior of gradient descent (with respect to various step sizes) and the instability of undesired critical points. The numerical analyses (e.g., the plots in Figures 1 and 2) support the theoretical insights. One concern is the limited range of experiments only on synthetic data, but given the theoretical nature of this paper, this is not a major issue.", "relation_to_broader_scientific_literature": "The paper is well situated within the literature on optimal transport, Wasserstein gradient flows, and generative modeling using SW distances. It builds on foundational works in the field (e.g., by Ambrosio et al., Villani, and Bonnotte) and connects with recent studies on particle methods and generative models (e.g., Merigot et al., 2021; Liutkus et al., 2019).", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "The paper provides a deep and rigorous theoretical analysis of the SW gradient flow, filling a gap in the literature regarding the convergence properties and stability of critical points.", "other_comments_or_suggestions": "n/a", "questions_for_authors": "- How do the results extend (or fail to extend) if the target measure $\\rho$ or the approximating measures $\\mu$ are not atomless? Could you comment on potential generalizations or necessary modifications of your framework in such cases?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rdtAQ6Bnfk", "reviewer": "Reviewer_F449", "summary": "The paper under consideration presents theoretical results regarding (Section 3) properties of discrete gradient descent w.r.t. SW functional (with absolute continuous target measure) and (Section 4) properties of (lagrangian) critical points of SW functional; (Section 5) provides some examples of lower-then-data dimension critical points different from the target functional and states that they (seems to be) unstable. The paper has an experimental section which illustrates some of the theoretical findings.\n\n### Update after the rebuttal\n\nI thank the authors for their response. I think, my current score is solid", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive remarks and comments. Regarding the questions you raised :\n\n- We meant by \"descent lemma\" a result that guarantees that, provided some conditions on the initial point and the step size are satisfied, one step of the gradient descent will decrease the loss. In particular, it gives the maximal step-size for which descent is maximal and still guaranteed - this size is related to the inverse of the smoothness constant of the functional. Above this limit, descent is not guaranteed for an explicit time-discretization, as corroborated by our experiments (Figure 2) where step-size below $2Nd$  yields descent while step-size above $2Nd$ diverges. Hence, while this is a theoretical result, it gives practical hints for optimization of the $SW$ distance in practice since below $2Nd$ (which is a known quantity) the descent is guaranteed. The term \"descent lemma\" is a standard term in optimization, cf for instance [1]. We will revise the article to clarify this point.\n\n[1] Bauschke, H. H., Bolte, J., & Teboulle, M. (2017). A descent lemma beyond Lipschitz gradient continuity: first-order methods revisited and applications. Mathematics of Operations Research. \n\n- Regarding the meaning of \"a suitable alternating vector field $\\xi$\" at lines 369-371 (second column), we meant was that (using the notations of Proposition 5.2) by approximating the perturbation $\\mu^t$ by $(Id+t\\xi)\\mu$ where $\\xi$ rapidly alternates between $\\vec{n}$ and $-\\vec{n}$ on the segment $S$, we may hope that $SW^2_2((Id+t\\xi)\\mu,\\rho)$ will also have a maximum at $t = 0$. For example, for $S = [-1,1] \\times \\{0\\}$, we may consider $\\xi$ such that $\\xi(x,0) = \\vec{e_2}$ for $x \\in [i/n,(i+1)/n)$ with $i$ even, and $\\xi(x,0) = -\\vec{e_2}$ for $x \\in [i/n, (i+1)/n)$ with $i$ odd, with $n$ large. In the experiments shown in Figure 1, we use such alternating perturbations. We will make our formulation clearer in a revised version of the article.\n\nWe thank you for the additional references, which we will accordingly cite in a revised version of our article.", "claims_and_evidence": "Ok. \nBut I have a question: (lines 369-371): “We can find $\\xi$ such that …”. I do not understand how to find such $\\xi$? May you give some examples of such $\\xi$?", "methods_and_evaluation_criteria": "N/A: The paper is theoretical", "theoretical_claims": "I checked only the general flow of the statements of the theorems/propositions in the main text. I didn’t check the proofs.", "experimental_designs_or_analyses": "N/A: The paper is theoretical", "relation_to_broader_scientific_literature": "Some related but not cited papers devoted to modelling Wasserstein gradient flows w.r.t KL-divergence:\n\n[1] Large-scale Wasserstein Gradient Flows, NeurIPS’21\n\n[2] Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks, TMLR\n\n[3] Proximal Optimal Transport Modeling of Population Dynamics, AISTATS’22\n\nAlso, not that directly related, but also a good reference. This work study gradient flows in Sliced-Wasserstein space:\n\n[1] Efficient Gradient Flows in Sliced-Wasserstein Space, TMLR", "essential_references_not_discussed": "Everything seems to be ok.", "other_strengths_and_weaknesses": "On the one hand (without delving into proofs), I find the manuscript more-or-less understandable and easy-to-read. It is definitely a strength for such a theoretical work with a lot of statements. \n\nOn the other hand, the main weakness of this paper is that it is almost fully theoretical. There is an experimental section, but it just presents some illustrations of some theoretical claims. \n\nSo, my opinion is as follows: as the theoretical work, the paper is good, but it has rather limited practicality. Maybe Proposition 3.2. is interesting to some extent.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. Line 154, second line: “descent lemma” for this objective. What is “descent lemma”?\n\n2. Minor, lines 367 an 371 - conflict of notations, $a$ used for different properties.\n\n3. What does it mean: “alternating vector field $\\xi$”? - lines 369-370 (second column)", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3Yx40CZJye", "reviewer": "Reviewer_cZYb", "summary": "This paper proposes a systematic study of the Sliced-Wasserstein functional in an optimization context where the target measure is continuous and the measure to optimize is discrete. The authors use the notion of ‘Lagrangian critical point’ of a functional defined over probability measures, since it aligns well with a particle based discretization of the objective functional. The authors then show a number of properties of this functional related to its well-behavedness when a gradient descent is used to optimize it. They also provide a characterization of the critical points of the objective (namely good behavior when the number of points in the optimized measure grows), and show that unstable critical points may exist both theoretically and experimentally. They finally show experimentally the convergence behavior of gradient descent on this functional.\n\n## Update after rebuttal\n\nThanks to the authors for the answers to some of my questions. I would have appreciated discussion about the other few points raised in the rest of my review. Nevertheless, my opinion on the paper is largely positive and I maintain my score to 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your relevant remarks and questions.\n\n- Regarding the possible generalizations of our results to other types of optimized or target measures (notably discrete ones), we refer to our answer to reviewer BmZL which addresses these points extensively. Note in particular that even though we assumed throughout the article that the target measure $\\rho$ is a density, this is mostly because we started our investigation from the study of the semi-discrete setting, and most of our results can actually be proven when $\\rho$ is simply assumed to be without atoms. This covers many types of measures often encountered in machine learning, such as measures supported on lower dimensional manifolds of $\\mathbb{R}^d$.\n\n- We did make some attempts at a second order analysis of the $SW$ distance, but it turned out to be a significantly difficult problem. Most of what we know concerns cases such as the ones in Proposition 5.2, where we have explicit expressions of the quantile functions and we can compute explicit Taylor expansions of the $SW$. Even the discrete case is difficult : while formally differentiating under the integral sign does indeed give the expression of $\\nabla F$ (Proposition 3.1), we cannot obtain the Hessian of $F$ by this method, as formally differentiating again $\\nabla F$ under the integral sign gives $\\frac{1}{Nd} I_d$, which cannot be the actual Hessian of $F$ as $F$ is not convex (it is semiconcave by Proposition A.2). Furthermore, attempts to numerically approximate the Hessian were inconclusive, as the computed Hessian would converge to $\\frac{1}{Nd} I_d$ when we increased precision or added more directions (which is to be expected from the analysis of Appendix B, which shows that the approximation of $SW$ using a fixed number $L$ of directions $\\theta_1,\\ldots,\\theta_L$ has Hessian $\\frac{1}{NL} \\sum_l \\theta_l \\theta_l^T$ everywhere where it is defined, and this converges to $\\frac{1}{Nd} I_d$ when $L \\to \\infty$ and the $\\theta_l$ are choosed randomly).", "claims_and_evidence": "The paper is mostly theoretical. In terms of technical contributions, the authors characterize the points of non-differentiability of the objective, compute the gradient explicitly for Lp-norm ground costs on the subset where the functional is actually differentiable. They also show that a standard gradient descent typically remain away from the problematic points. They proceed to characterize critical points and show practical examples of unstable critical points in relatively simple settings. The experiments support the theoretical analysis of the critical points and are consistent with the expected behavior of gradient descent from the theoretical analysis.", "methods_and_evaluation_criteria": "This point does not really apply since the paper is mostly theoretical and the experiments are mostly confirming qualitatively some of the theoretical developments.", "theoretical_claims": "I did not check thoroughly the proofs or the supplementary material. I find that a few claims in the introduction about the analysis of W_2^21 as the objective are a bit too depth in the cited works and would benefit a bit more exposure and recontextualization (I couldn’t find some of them in the related references), maybe as an additional appendix to avoid that the reader has to check many references to find the relevant theorems , or rederive things on their own.", "experimental_designs_or_analyses": "I did check the soudness of the experiments, and have no particular complaints about them. It would have been interesting, experimentally at least, to give an idea about the behavior of other algorithms that are more likely to be used in practice, e.g. SGD or ADAM.", "relation_to_broader_scientific_literature": "The paper raises interesting points about the optimization of optimal transport distances to a reference measure. A choice that is deliberate is that the authors choose to consider cases where the target measure is absolutely continuous but the measure to optimise is discrete, and discrete in the sense that it corresponds to a Lagrangian view on the measure, i.e. a collection of particles. This brings the paper close to the setting of semi-discrete optimal transport. This choice is different from what has already been treated in the literature, which adressed the cases where both the target and the variable admit densities, or are both discrete. The case chosen by the authors is indeed of practical interest, but corresponds to one possibility among several to optimize the SW distance. The choice of an absolutely continuous target is not suited to generative modeling (where the target is only known by discrete samples) but is suited for variational inference, which is of broad interest. The choice of a Lagrangian discretization of the measure is also of interest, but other choices such as parameterizing the density of the objective via a generative model could also have been discussed (which is something that is actually done in practice). More generally, this positioning within the existing literature could have been made more explicit. However, the findings in the considered case are interesting and insightful.", "essential_references_not_discussed": "I am not aware of such missing references for this paper.", "other_strengths_and_weaknesses": "Though one could find that the setting of the paper could be restrictive, the approach is original and the results interesting. Namely it shows the theoretical difficulties of the optimization of SW distance, and offers a few guarantees the GD is likely to be relatively well behaved and to avoid non differentiabilities, while converging in practice to a stable critical point. The theoretical results generally provide new insight on important questions about the optimization dynamics of SW.  \nHowever, I found the rest of the introcution to be particularly well written and provides a good introduction to optimization over measures in ML.", "other_comments_or_suggestions": "It is a bit awkward to have the introductory material about OT and SW distances presented after an introduction that is clearly not understandable if one is not already familiar with that material. I suggest moving the last part of the introdution which already gets quite technical and discusses results that are not broadly known about critical points and the behavior of W2 in a specific section after the introduction. Maybe details could be provided in an additional appendix (citing relevant theorems in the literature for instance) to make the introductory material more self contained.", "questions_for_authors": "1) Which parts of the theoretical analyses also hold in more general settings in terms of discretization choices for the measure to be optimized, or with other types of target measures (atomless, discrete, mixed…) ? This would make the contributions more impactful and more broadly applicable in practical scenarios.\n2) Is there any hope in a second order analysis of the critical points, or somehow accessing the quality of the (stable) critical points towards which GD would converge ? An experiment showing possible bad behavior in practice would be interesting.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NV6xDOTw7U", "reviewer": "Reviewer_9JEB", "summary": "The paper studies the existence and stability of critical points for semi-discrete sliced Wasserstein loss functions. In particular, they prove that there exist critical points which do not coincide with the global minimum, but also that any of those critical points is unstable under small perturbations. The authors include some numerical experiments for validating the theoretical results.\n\n## After rebuttal\n\nThe authors addressed my comments and questions adequately. I keep my rating at \"accept\".", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback and relevant questions.\n\n- Concerning the limitations of the analysis in Section 5 : it is indeed true that most of the examples we discuss are in 2D, with the exception of our Proposition 5.1(b) which gives some examples of critical points in arbitrary dimension $d > 1$. We will make the introduction clearer about the limitations of this section.\n\n- Regarding the limitations of our assumption of absolute continuity of the target measure $\\rho$, we refer to the discussion in our answer to reviewer BmZL. We will make sure to discuss this more extensively in a revised version of our article. In particular, note that one of the advantages of the Sliced-Wasserstein distance is precisely that the Laguerre tesselations are easy to compute in 1D. Indeed, if we assume that $\\rho$ is discretized using $M$ points on the real line, then the Laguerre cells are computed by sorting the $M$ points, and the $i$-th Laguerre cell consists of the $i$-th chunk of $\\frac{M}{N}$ points in the sorted list. \n\n- Thank you for bringing these additional references to our attention. In particular, the paper by Li and Moosmueller appears to be extremely relevant to our analysis as their assumption (A3) essentially means that the target $\\mu$ is the only barycentric Lagrangian critical point in the compact set $K_{\\sigma_0,\\mu}$ in which the iterates of their scheme remain. This indeed ties our theoretical analysis of Lagrangian critical points to considerations of convergence of practical schemes optimizing $SW$ objectives. On the other hand, as we understand it, the paper by Altekrüger at al considers perturbations by velocity plans, which is a larger class of perturbations than in our Lagrangian framework (as a velocity plan allows to split atoms, while a perturbation by a vector field $\\xi \\in L^2(\\mu,\\mathbb{R}^d)$ cannot)\n\n- We agree that it would be fruitful to establish clear links between the different notions of critical points and the different types of tangent spaces investigated by Gigli et al. We will include a discussion on this in a revised version of the article, maybe in an appendix, since for the sake of clarity we have avoided to make use of the theory of gradient flows developed by Ambrosio, Gigli, Savaré in the main body of the article.\n\n- Whether the Wasserstein gradient flow starting at a discrete measure with an absolutely continuous target measure remains discrete is an excellent question. For $W_2$ objectives, it is known that the gradient flow diffuses (this is necessary as the flow converges to $\\rho$ exponentially in the $W_2$ metric, see for instance Proposition 3.1 in [1]. For the $SW$ objective, although we have not formally shown that the flow of the $SW$ \"diffuses\" discrete measures, we expect it to do so. This is in fact why we chose to work with the \"Lagrangian\" framework and Lagrangian critical points. Indeed, what we study is the behavior of the system $dX_t^i/dt = - \\nabla F(X_t^i)$ where $F$ is defined in Section 3, which corresponds to the continuous time limit of the gradient descent algorithm implemented in practice. Therefore, considering perturbations by vector fields $\\xi$, which can't \"split\" atoms, allows for a theory better suited to analyze how algorithms relying on particle dynamics work.\n\n[1] Huang, Y. J., & Malik, Z. (2024). Generative Modeling by Minimizing the Wasserstein-2 Loss. arXiv preprint arXiv:2406.13619.\n\n- We have not investigated specifically what happens when the gradient descent is performed using a very small number of directions, such as $L = 1$. However, in our numerical experiments, we did observe that it is extremely easy to escape critical points such as those described in Proposition 5.1 or Proposition 5.2, as long as some stochasticity is introduced in the choice of directions.", "claims_and_evidence": "The results are clear and underpinned by formal proofs.\n\nIn particular, the authors provide in Section 5 explicit examples of (Lagrangian) critical points located on lower-dimensional subspaces.\nAs an elementary tool, they characterize critical points of the semi-discrete sliced Wasserstein distance by the barycentric projection and prove that limits of discrete measures which are Lagrangian critical points are again Lagrangian critical point.\n\nWhile I believe that the claims are interesting and the proofs are correct (even though I hadn't the opportunity to check them in detail), some limitations of the analysis could be emphasized more clearly in the introduction (please correct me if I got one of those limitations wrong):\n- the examples and instability results from Section 5 are only in 2D\n- the instability result only considers critical points of this specific form", "methods_and_evaluation_criteria": "The proof techniques are feasible.", "theoretical_claims": "Due to the high review load, I was not able to check the proofs in detail. Based on my intuition, the claims of the statements are realistic.", "experimental_designs_or_analyses": "Not applicable (the major claims are purely theoretical).", "relation_to_broader_scientific_literature": "The geometry and properties of sliced Wasserstein losses was studied in several papers recently. As far as I know, the results on the non-existence of stable critical points are new and I consider them as a significant contribution.", "essential_references_not_discussed": "Generally, the literature part is comprehensive and well-organized. Some additional papers should/could be discussed:\n\nLi and Moosmueller consider in the paper \"Measure transfer via stochastic slicing and matching\" a stochastic gradient descent on the sliced Wasserstein distance in the continuous case (where both measures have densities). It is stochastic in the sense that in each iteration one random direction is chosen. The authors show global convergence of this scheme, which also implies that there don't exist stable critical points.\n\nThere is a paper, that proposes a numerical scheme (Altekrueger et al. \"Neural Wasserstein Gradient Flows for Discrepancies with Riesz Kernels\" ICML 2023) to escape critical points introduced by searching for Lagrangian instead of Wasserstein critical points (i.e. in the regular tangent space instead of the geometric one see the field \"strenghts and weaknesses\" below).", "other_strengths_and_weaknesses": "The existence and characterization of critical points of the sliced Wasserstein distance is a problem which is of very high interest for the optimal transport community. Given that the paper makes significant progress in this direction, I definitely vote for acceptance. However, I have a couple of comments (not ordered by importance):\n\n- From my perspective, and also in the view of machine learning applications, the most fundamental limitation of the paper is the absolute continuity assumption for the target measure. In particular, for most machine learning applications, the target measure is given by a dataset and therefore discrete. From a computational viewpoint it is usually impossible to compute the Laguerre tesselations which are required to compute the gradient in the semi-discrete case. Given the difficult nature of the problem, I would not expect the authors to consider a more general case as they have done. But I would expect them to discuss this limitation.\n\n- Similarly as in the previous content, I would ask the authors to clarify in the abstract, that they study mostly the semi-discrete case of the SW objective.\n\n- The authors extensively work with the different notions of critical points (Lagrangian and Wasserstein). These notions are directly related to the notions of geometric and regular tangent spaces of the Wasserstein space, which are contained in the book of Ambrosio, Gigli, Savare and were studied in more detail in the PhD thesis of Gigli. In these notations, Wasserstein critical points coincide with critical points with respect to the geometric tangent space and Lagrangian critical points coincide with critical points in the regular tangent space. For absolutely continuous measures both tangent spaces coincide. For the final version I would highly recommend the authors to work out this relations in a clean way. \n\n- In particular, I would guess that the claim of Prop 4.3 is somehow related to the statement, that the barycentric projection is always contained in the regular tangent space (see the PhD thesis of Gigli Thm 4.15).\n\nReference for the PhD thesis of Gigli:\n\"On the geometry of the space of probability measures endowed with the quadratic optimal transport distance\"", "other_comments_or_suggestions": "see other fields", "questions_for_authors": "- Is it clear that the Wasserstein gradient flow starting at a discrete measure with a absolutely continuous target measure remains discrete? If not, can we relate the discrete setting to the continuous one? Maybe in the sense of mean-field limits?\n\n- Maybe some \"outlook-question\": Is it sufficient to choose a random direction in a gradient descent scheme of the loss to escape the critical points (like in the paper of Li, Moosmueller mentioned above)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors study the properties of gradient flows with Sliced-Wasserstein (SW) distance as an objective functional. In particular, the authors consider the semi-discrete SW loss functions, and study the existence and stability of critical points via the notion of Lagrangian critical point. The authors derive important theoretical results including the existence (in lower-dimensional subspaces) and instability of critical points (which are not global minimum) but in relatively simple settings; properties of SW functional for well-behavedness of gradient descent. The authors also characterize critical points by the barycentric projection and prove the limits of critical points are critical. The authors provide simple empirical simulations to illustrate their theoretical findings. Additionally, during the rebuttal, the authors also discuss extensions for the semi-discrete SW. We urge the authors to incorporate these extensions and also discuss the limitations (e.g., compact support assumption, relatively simple settings for theoretical instability analysis) in the update version. Overall, we think that it is a good submission, and provides interesting theoretical results for gradient flows with SW as an objective functional.
Decision: Accept (poster)