Spherical Sliced-Wasserstein
Authors: Clément Bonet
Paul Berg
Nicolas Courty
François Septier
Lucas Drumetz
Minh Tan Pham
Venue: ICLR 2023 poster (2023)
Keywords: Optimal Transport
Sliced-Wasserstein
Sphere
Radon Transform

Abstract: Many variants of the Wasserstein distance have been introduced to reduce its original computational burden. In particular the Sliced-Wasserstein distance (SW), which leverages one-dimensional projections for which a closed-form solution of the Wasserstein distance is available, has received a lot of interest. Yet, it is restricted to data living in Euclidean spaces, while the Wasserstein distance has been studied and used recently on manifolds. We focus more specifically on the sphere, for which we define a novel SW discrepancy, which we call spherical Sliced-Wasserstein, making a first step towards defining SW discrepancies on manifolds. Our construction is notably based on closed-form solutions of the Wasserstein distance on the circle, together with a new spherical Radon transform. Along with efficient algorithms and the corresponding implementations, we illustrate its properties in several machine learning use cases where spherical representations of data are at stake: sampling on the sphere, density estimation on real eath data or hyperspherical auto-encoders.

Published as a conference paper at ICLR 2023
SPHERICAL SLICED-WASSENE
Clément Bonet', Paul Berg', Nicolas Courty?, François Septier', Lucas Drumetz', Minh-Tan Pham? Université Bretagne Sud, LMBA', IRISA?; IMT Atlantique, Lab-STICC* {clement bonet, paul meul.berg, francois sept septier Junivvvveenvuubs fr (nicolas, courty,, minh-tare phamppphjppp. Girisa. fr; lucas. drumet imt aatttaallae atlanttquuueee fr
ABSTRACT
Many variants of the Wasserstein distance have been introduced to reduce its original computational burden. In particular the which leverages one-dimensional projections for which a closed-form solution of the Wasserstein distance is available, has received lot of interest. Yet, it is restricted to data living in Euclidean spaces, while the Wasserstein distance has been studied and used recently sphere, for which we define a novel SW discrepancy, which we call spherical Sliced- Wasserstein, making a first step towards defining SW discrepancies on manifolds. Our construction is notably based on closed-forn solutions of the Wasserstein distance on the circle, together with a new spherical Radon transform. Along with efficient algorithms and the corresponding implementations, we illustrate its properties in several machine learning use cases where spherical representations of data are at stake: sampling on the sphere, density estimation on real earth data or hyperspherical auto-encoders.
distance (SW),
manifolds. We focus more specifically the
INTRODUCTION
Optimal transport (OT) (Villani, 2009) has received a lot of attention in machine learning in the past few years. As it allows to compare distributions with metrics, it has been used for different tasks such as domain adaptation (Courty et al., 2016) generative models (Arjovsky et al., 2017), to name a few. The most classical distance used in OT is the Wasserstein distance. However, calculating can be computationally expensive. Hence, several variants were proposed to alleviate the computational burden, such as the entropic regularization (Cuturi, 2013; Scetbon et al., 2021), minibatch OT (Fatras et al., 2020) or the sliced- Wasserstein distance (SW) for distributions supported on Euclidean spaces (Rabin et al., 201 lb).
Although embedded in larger dimensional Euclidean spaces, data generally lie in practice manifolds (Fefferman et al., 2016). A simple manifold, but with lots of practical applications, the hypersphere sd-1 Several types of data are by essence spherical: good example is found in directional data Garcia-Portugués, 2021) for which dedicated machine learning
(Mardia et al., 2000; Pewsey solutions are heing developed (Sra, 2018), but other applications concern for instance geophysical data (Di Marzio et al., 2014), meteorology (Besombes et al., 2021), cosmology (Perraudin et al., 2019) 9) or extreme value theory for the estimation of spectral measures (Guillou al., 2015). Remarkably, in a more abstract setting, considering hyperspherical latent representations of data is becoming more and more common (e.g. (Liu al., 2017: Xu & Durrett, 2018: Davidson et al., 2018)). For example, in the context of variational autoencoders (Kingma & Welling, 2013), using priors on the sphere has been demonstrated to be beneficial (Davidson al., 2018). Also, in the context self-superrvsed learning (SSL), where one wants to learn discriminative representations in an unsupervised way, the hypersphere is usually considered for the latent representation (Wu et al., 2018; Chen et al., 2020a; Wang & Isola, 2020; Grill et al., 2020; Caron et al., 2020). It is thus of primary importance develop machine learning tools that accommodate well with this specific geometry.
The OT theory on manifolds is well developed (Villani, 2009; Figalli & Villani, 2011; 1; McCann, 2001) and several works started to use in practice, with a focus mainly on the approximation of OT maps. For example, Cohen et al. (2021); Rezende & Racanière (2021) approximate the OT map define normalizing flows on Riemannian manifolds, Hamfeldt & Turnquist (2021a;b); Cui et al. (2019) derive algorithms to approximate the OT map the sphere, Alvarez-Melis al. (2020); Hoyos-

---

Published as a conference paper at ICLR 2023
Idrobo (2020) learn the transport map hyperbolic spaces. However, the computational bottleneck to compute the Wasserstein distance on such spaces remains, and, as underlined in the conclusion of (Nadjahi, 2021). defining SW distances manifolds would be of much interest. Notably. Rustamov & Majumdar (2020) proposed a variant of SW, based on the spectral decomposition of the Laplace- Beltrami operator, which generalizes manifolds given the availability of the eigenvalues and cigenfunctions. However, it is not directly related to the original SW on Euclidean spaces.
Contributions. Therefore, by leveraging properties of the Wasserstein distance on the circle (Rabin et al., 201 la), we define the first, to the best of our knowledge, natural generalization of the original SW discrepancy on a non trivial manifold, namely the sphere 5d and hence we make first step towards defining SW distances on Riemannian manifolds. We make connections with new spherical Radon transform and analyze some its properties. We discuss the underlying algorithmic procedure, and notably provide efficient implementation when computing the discrepancy against a uniform distribution. Then, we show that we can use this discrepancy on different tasks such as sampling, density estimation generative modeling.
BACKGROUND
The aim of this paper is define Sliced Wasserreen discrepancy on the hypersphere Sd {ar Rd, ||ж||| = 1}. Therefore, in this section, we introduce the Wasserstein distance on manifolds and the classical SW distance on Rd
2.1 WASSERSTEIN DISTANCE
Since we interested in defining SW discrepancy the sphere, start by introducing the Wasserstein distance on a Riemannian manifold M endowed with the Riemannian distance d. We refer (Villani, 2009; Figalli Villani, 2011) for more details.
Let 2 and μ,, E (M) {a E P(M), SM (x mo) du(x)) < oo for some xo E MJ. Then, the p- Wasserstein distance between μ and v defined as
[ JAKXAs dz( dr(x. v), JEll(aM)
inf
(1)
where (), (> P(M M), AA M, xM A) v(A) and 7(A M) (())) denotes the set of couplings.
For discrete probability measures, the Wasserstein distance can be computed using linear programs (Peyré et al., 2019). However, these algorithms have a On n) complexity w.rl. the number of samples n which is computationally intensive. Therefore, whole literature consists of defining alternative discrepancies which are cheaper to compute. On Euclidean spaces, one of them is the Sliced-Wasserseee Wasserstein distance.
2.2 SLICED- WASSERSTEIN DISTANCE
On M 2 with d(., y) ||< yllp more attractive distance is the Siised-Wassersteinn (SW) distance. This distance relies the appealing fact that for dimensional measures μl, E P(R), we have the following closed-form (Peyré et al., 2019, Remark 2.30)
L/
WP v)
(u) du,
(2)
where Fi-1 (resp. F-!) is the quantile function of f (resp. v). From this property, Rabin al. (2011b); Bonnotte (2013) defined the SW distance as
Vp,VE P,(R). SWP(p,
J WP(P P4.) dx(().
(3)
where (x)) = (z, 8). À is the uniform distribution on S and for any Borel set A E B(R").
pd (a ))= (((p))-1(4))
2

---

Published as a conference paper at ICLR 2023
This distance can be approximated efficiently by using Monte-Carlo approximation (Nadjahi et al., 2019), and amounts to a complexity of (( n)) where L denotes the number of projections used for the Monte-Carle approximation and n the number of samples.
SW can also be written through the Radon transform (Bonneel et al., 2015). Let f E L (RR).. then the Radon transform R: I (R + (R 10-1) is defined as (Helgason al., 2011)
AO E VER, Rf(t, - f ((.))
(4)
Its dual Rº : CR 1 Co(RR)) (also known back-projection operator), where Co denotes the set continuous functions that vanish at infinity, satisfies for all f. (Rf, g) (f, RI" g) a and can be defined as (Boman & Lindskog, 2009; Bonneel et al., 2015)
SSe g (zx 0),0) d0.
Vg E Co x 1t Vo RA, R g()
(5)
Therefore, by duality, we can define the Radon transform of measure .M(R)) as the measure Ru E M(R x s such that for all g E Co x 5t-1). (Ru, 9)Rx = (H, Rt" g) Since Ru a measure on the product space
5d-1, we can disintegrate w.rt. X, the uniform measure on Sd-1 (Ambrosio et al., 2005), as Ry À 6 K with K a probability kernel on 5 x B(R), i.e. for
all E Sd-1, K(0,-) :) is probubility on I, for any Borel set A E B(R)) K K(,, (,, A) is measurable, and
JENSH- (t, (t, = fSS-. g @t, ( dt)da(0),
Vo E CCC x S
with C(R x 5d-1) the set of continuous functions on R x 5 By Proposition in (Bonneel et al., 2015), we have that for A-almost every 0 Sd-11 (Ry)² P4 where we denote ((0,-) (Ry)².
Therefore, we have
VU,UEP, SWP(KD)) = /
ga-1 ( (Ri) d)(0).
Variants of SW have been defined in recent works, either by integrating w.r.f different distributions (Deshpande et al., 2019; Nguyen et al., 2021; 2020), by projecting on using different projections (Nguyen & Ho. 2022a;b; Rustamov & Majumdar, 2020) Radon transforms (Kolouri et al., 2019; Chen et al., 2020b), or by projecting on subspaces of higher dimensions (Paty & Cuturi, 2019; Lin et al., 2020; 2021; Huang et al., 2021).
3 A SLICED-WASSERSS WASSERSTEIN DISCREPANCY ON THE SPHERE
Our goal here is to define a sliced- Wasserstein distance on the sphere 5d-1, To that aim, we proceed analogously to the classical Euclidean space. We first rely on the nice properties of the Wasserstein distance on the circle (Rabin et al., 201 la) and then propose to project distributions lying on the sphere to great circles. Hence, circles play the role of the real line for the hypersphere. this section, first describe the OT problem the circle, then we define the sphere and discuss some of its properties. Notably, we derive a new spherical Radon transform which is linked to our newly defined spherical SW. We refer to Appendix A for the proofs.
Wasseestenn discrepancy
3.1 OPTIMAL TRANSPORT ON THE CIRCLE
On the circle 5 = R// equipped with the geodesic distance ds,, an appealing formulation of the Wasserstein distance is available (Delon et al., 2010). First, let us parametrize St by [0. 1f, then the geodesic distance can be written as (Rabin et al., 201 for all x, y E [0, 1[, dg (I, = min ( y/, I - v/). Then, for the cost function c(x, y) (ddsa (I,9) y)) with h R Rt an increasing convex function, the Wasserstein distance between μ E P(S and v E P(S!) can be written
W Wc(,,v) = inf S. aeR
dr,
(8)
where F2 : 1[- [0, denotes the cumulative distribution function (cdf) of μ, Fy1 its quantile function and a is shift parameter. The optimization problem over the shifted edf F, can be
3

---

Published as a conference paper at ICLR 2023
as looking for the best "cut" (or origin) of the circle into the real line because of the 1-periodicity. Indeed, the proof of this result for discrete distributions in (Rabin et al., 2011a) consists in cutting the circle the optimal point and wrapping around the real line, for which the optimal transport map is the increasing rearrangement F Fp which can be obtained for discrete distributions by sorting the points (Peyré et al., 2019).
Rabin et al. (2011a) la) showed that the minimization problem is convex and coercive in the shift parameter and Delon et al. (2010) derived a binary search algorithm to find it. For the particular case of h = Id, it can further be shown (Werman et al., 1985; Cabrelli & Molter, 1995) that
(14,, p) = -inf / oER
(Fx(t) - F2(t) al dt.
(9)
In this case, we know exactly the minimum which is attained at the level median (Hundrieser et al., 2021). For f [0,1[-+0] R.
S. (f((t) -ajat) aldt) = inf { EE 8(fx E [0. f(x) $t) 2
1
LevMed(f) = min argmin
2
aeR
(10)
where 3 is the Lebesgue measure. Therefore, we also have
W wi(,, f(
(F.(() [[)F.(()
Fu(() - LevMed(F,, - Fu)l dt.
(11)
Since we know the minimum, we do not need the binary search and can approximate the integral very efficiently as we only need to sort the samples to compute the level median and the edfs.
Another interesting setting in practice is compute W2. i.e. with h(x) x², w.r.t. a uniform distribution on the circle. We derive here the optimal shift á for the Wasserstein distance between an arbitrary distribution on S and v. We also provide a closed-form when şu is discrete distribution. Proposition 1. Let μ E P2(S and v Uni Then,
/'
[FT -- al2 à dt with à à= d 1
W2(,v
(12)
Jo
In particular, if .... < In and Un > S2 then
1 12
(2 v)
Tz
1 10++1-i)a;+ +
(13)
n
1=1
This proposition offers intuitive interpretation: the optimal eut point between an empirical and uniform distributions is the antipodal point of the circular mean of the discrete samples. Moreover, a very efficient algorithm can be derived from this property, as it solely requires sorting operation to compute the order statistics of the samples.
3.2 DEFINITION OF SW ON THE SPHERE
On the hypersphere, the counterpart of straight lines are the great circles, which are circles with the same diameter as the sphere, and which correspond to the geodesics. Moreover, Wasserstein distance on the circle fairly efficiently. Hence, to define a sliced- Wasserstein discrepancy on this manifold, we propose, analogously the classical SW distance, to project measures great circles. The most natural way to project points from Sd-1 to great circle C' is to use the geodesic projection (Jung, 2021; Fletcher et al., 2004) defined
compute the
Vx E 5d-1 (C(x) argmin dga (xx
(14)
yEC
where dga-1 (v. y) arccos((2,, y)) is the geodesic distance. See Figure 1 for illustration of the geodesic projection on a great circle. Note that the projection is unique for almost every x (see (Bardelli & Mennucci, 2017, Proposition 4.2) and Appendix B.I) and hence the pushforward PC a of E (S where P, (Sd-1) denotes the set of absolutely continuous measures w.t. the Lebesgue measure and with moments of order p, is well defined.
4

---

Published as a conference paper at ICLR 2023
Great circles can be obtained by intersecting Stt-1 with a 2-dimensional plane (Jung al., 2012). Therefore, to average over all great circles, we propose to integrate over the Grassmann manifold Gd,2 {E Rd, dim(E) 2} (Absil et al., 2004; Bendokat et al., 2020) and then to project the distribution onto the intersection with the hypersphere. Since the Grassmannian is not very practical, consider the identification using the of rank 2 projectors:
G,.2 {P E Rdxd pT P. = Tr(P) 2} {UU", U Va2)
(15)
where V, IU E Ráx2, U I2) is the Stiefel manifold (Bendokat et al.. 2020).
Finally, we can define the Spherical Wasssertenn distance (SSW) for p 2 between locally absolutely continuous measures w.r.f. the Lebesgue measure (Bardelli & Mennucci, 2017) μ4, E Pracc (5 as
SSWP(u,)) = - W H. P do(U),
(16)
JVa.2
where a is the uniform distribution over the Stiefel manifold V4,2, pU the geodesic projection on the great circle generated by U and then projected on Sh,ie..
YU E Vd,2, Vx E 5 p²(x)) = UT
argmin dga-1 (I, y) = argmin dga- (xx, U2), V'span(UU? jos
#ES1
(17)
and the Wasserstein distance is defined with the geodesic distance dga
Moreover, can derive closed form expression which will be very useful in practice:
Lemma 1. Let U E Va. then for a.e. E Sd-1
UTx
pd (z)
(18)
UTzl|e
Hence, we notice from this expression of the projection that we recover almost the same formula as Lin et al. (2020) but with an additional 8² normalization which projects the data on the circle. As in (Lin al., 2020), we could project on higher dimensional subsphere by integrating over Va, with 2. However, we would lose the computational efficiency provided by the properties of the Wasserstein distance on the circle.
Illustration of the geodesic projections on a great circle (in black). In red, random points sampled the sphere. In green the projections and in blue the trajectories.
Figure 1:
3.3 SPHERICAL RADON TRANSFORM
As for the classical SW distance, can derive second formulation using a Radon transform. Let f L' (Sd-1), ))
we define spherical Radon transform R (Sdd-1)
LS x V4,2)
S! YU e Va2. Rf(2,U)= Js. (())
(19)
This is basically the same formulation the classical Radon transform (Natterer, 2001; Helgason et al., 201 I) where we replaced the real line coordinate ł by the coordinate on the circle z and the projection is the geodesic one which is well suited to the sphere. This transform is actually new since we integrate over different sets compared to existing works on spherical Radon transforms.
Then, analogously to the classical Radon transform, we can define the back-projection operator pe CS Va,2) + Co(S (s being the space of continuous bounded functions, for g CS" Va.2) for a.e. E 5d11
K(g(s) [ g(P" (x), do(U).
(20)
Proposition it" is the dual operator of R. i.e for all f ('( g E CulSt x V42).
(f.(.g) (J, R' g) ga-1.
(21)
5

---

Published as a conference paper at ICLR 2023
Now that we have dual operator, also define the Radon transform of an absolutely continuous measure ja E Mac(St-1) by duality (Boman & Lindskog, 2009; Bonneel et al., 2015) as the measure Rp satisfying
Vg Cols x Va.2), /
g(2, ((Rp)(z,U) - g(z) du(x).
(22)
Since Ru is a measure on the product space S x Va,2, Ru can be disintegrated (Ambrosio et al., 2005, Theorem 5.3.1) w.r.t a as Rp = a & K where K is a probability kernel on Va,2 x S! with S the Borel o o-field of St. We will denote for o-almost every U E Va,., (Ru)) = K (U, :) the conditional probability.
Proposition 3. Let μa E Maa(S then for a-almost every U E Va,. (Rp) PV n
Finally, we can write SSW (16) using this Radon transform:
Vu, ( SSW3(n, = J. WP((R))) (((R)) do)
(23)
Note that natural way to define SW distances can be through already known Radon transforms using the formulation (23). is for example what was done in (Kolouri al., 2019) using generalized Radon transforms (Ehrenpreis, 2003; Homan & Zhou, 2017) to define generalized SW distances, or in (Chen al., 2020b) with the spatial Radon transform. However, for known spherical Radon transforms (Abouelaz & Daher, 1993; Antipov et al., 2011) such as the Minkowski-Funk transform (Dann, 2010) or more generally the geodesic Radon transform (Rubin, 2002), there is no natural way that we know of to integrate over some product space and allowing to define SW distance using disintegration.
As observed by Kolouri et al. (2019) for the generalized SW distances (GSW), studying the injectivity of the related Radon transforms allows to study the set on which SW actually a distance. While the classical Radon transform integrates over hyperplanes of Rd the generalized Radon transform over hypersurfaces (Kolouri et al., 2019) and the Minkowski Funk transform over "big circles", i.e. the intersection between hyperplane and 5 (Rubin, 2003), the set of integration here is half of big circle. Hence, R is related to the hemispherical transform (Rubin, 1999) 56-2 We refer to Appendix A.6 for more details on the links with the hemispherical transform. Using these connections, we can derive the kernel of it as the set of even measures which are null over all hyperplanes intersected with 5d-1
Proposition 4. ker(R) {u E (S AH Gdd--1 P( n Sd-1) 0} where μ Meven for all f C(Sd-1), (). (M, f4) with
all x.
We leave for future works checking whether this set null or not. Hence, we conclude here that SSW a distan-dsttanee, but distance on the sets of injectivity of R (Agranovskyt Quintott, 1996).
Proposition 5. Letp 2 1, SSW, p is pseudo-distanee on P (Sp
4 IMPLEMENTATION
In practice, we approximate the distributions with empirical approximations and, as for the classical SW distance, we rely on the Monte-Carlo approximation of the integral on Va.2. We first need to sample from the uniform distribution a E P(V..2). This can be done by first constructing Z E Rdxx by drawing each of its component from the standard normal distribution (0, 1) and then applying we project the samples on
the QR decomposition (Lin et al., 2021). Once we have (U,) the circle 5 by applying Lemma I and we compute the coordinates on the circle using the atan2 function. Finally, we can compute the Wasserstein distance the circle by either applying the binary search algorithm of (Delon et al., 2010) or the level median formulation (11) for SSW, In the particular case in which we want to compute SSW2 between measure μı and the uniform measure on the sphere v/ = (S we can use the appealing fact that the projection of y on the circle is uniform, i.e. pK v SS") (particular case of Theorem 3.1 in (Jung, 2021), see Appendix B.3). Hence, can the Proposition 1 to compute W2, which allows very efficient implementation either by the closed-form (13) or approximation by rectangle method of (12). This will be of particular interest for applications in Section 5 such autoencoders. We sum up the procedure in Algorithm 1.

---

Published as a conference paper at ICLR 2023
Algorithm 1 SSW
Input: (z:);=1 M4 (v,){im L the number of projections, p the order
for = 1 to L do
Draw random matrix Z Rdx2 with for all i, Zi.j 10)
U = (R) ~
Project on S the points: vi,j. &f
u . D3 1² y
v o.
Compute the coordinates on the circle S Vi, J. K ( + atan2 - -0,, -2.1))) 83
(i + (0tan((-U332 (2x))
Compute W Σ 6gg m Ej= 6g;) by binary search or (11) for p =
end for
Return SSW? p
(1, v) Σ W
8zç1 m -3 075)
Complexity. Let us note n (resp. m) the number of samples of μ (resp. 1/), and L the number of projections. First, we need to compute the QR factorization of L matrices of size d x 2. This can be done in O(Ld) by using e.g. Householder reflections (Golub Van Loan, 2013, Chapter 5.2) or the Schharz-Rutissausee algorithm (Gander, 1980). Projecting the points on 5 by Lemma 1 is in O((n + m))T) since we need to compute L(n + m) products between U. E R2xd and x E Rd. For the binary search or particular case formula (11) and (13), we need first to sort the points. But the binary search also adds a cost of ((n + m) log( ()) to approximate the solution with precision e (Delon et al., 2010) and the computation of the level median requires to sort (n + m) points. Hence, for the general SSW, p. the complexity is O(L(n + m) (d log(? ( + Ln log 72 + Lm logm) versus OLL(n md log(n + m))) for SSW, with the level median and O ((Ln(d log n)) for SSW2 against a uniform with the particular advantage that we do not need uniform samples in this case.
Runtime Comparison. We perform here some runtime comparisons. Using Pytorch (Paszke et al., 2019), we implemented the binary search algorithm of (Delon al., 2010) and used it with 10 10-6 We also implemented SSW, using the level median formula (11) and SSW, against a uniform measure (12). All experiments are conducted on GPU.
On Figure 2, we compare the runtime between two distributions on S2 between SSW. SW. the Wasserstein distance and the entropic approximation using the Sinkhorn algorithm (Cuturi, 2013) with the geodesic distance as cost function. The distributions were approximated using E (10?, 103, 10", 5- 10a, 10 5. 103)
SSW, L- 200
SW2 L- 200
S5W, BS, ( = 200
Wasserstein
SSW,, Unif, L - 200
Sinkhorn
10P
Seconds
samples of each distribution and we report the mean over 20 computations. We use the Python Optimal Transport (POT) library (Flamary et al., 2021) to compute the Wasserstein distance and the entropic approximation. For large enough batches, we observe that SSW is much faster than its Wasserstein counterpart, and it also scales better in term of memory because of the need to store the n X n cost matrix. For small batches, the computation of SSW actually takes longer because of the computation of the QR factorizations and of the projections. For bigger batches, it is bounded by the sorting operation and we recover the quasi-linear slope. Furthermore, as expected, the fastest algorithms are SSW, with the level median and SSW, against a uniform as they have quasilinear complexity. We report in Appendix C.2 other runtimes experiments w.r.t. to e.g. the number of projections or the dimension.
10
10-4
10-1
10a
10*
Number of samples each distribution
Figure 2: Runtime comparison in log-log scale between W. Sinkhorn with the geodesic distance, SW2, SSW, with the binary search (BS) and uni- form distribution (12) and SSW, with formula (11) between two distributions on S The time includes the calculation of the distance matrices.
Additionally, study both theoretically and empirically the projection and sample complexities in Appendices A.9 and C. We obtain similar results as (Nadjahi et al., 2020) derived for the SW distance. Notably, the sample complexity is independent w.r.t. the dimension.
7

---

Published as a conference paper at ICLR 2023
(a) SWAE
(b) SSWAE
(a) Target
(b) KDE estimate
Figure 4: Latent space of SWAE and SSWAE MNIST for uniform prior on S
Figure 3: Minimization of SSW with respect to a mixture of vMF.
EXPERIMENTS
Apart from showing that SSW is an effective discrepancy for learning problems defined over the sphere, the objectives of this experimental Section is to show that it behaves better than using the more immediate SW in the embedding space. We first illustrate the ability to approximate different distributions by minimizing SSW w.t.. some target distributions on S and by performing density estimation experiments on real earth data. Then, we apply SSW for generative modeling tasks using the framework of Wassersein autoencoder and we show that we obtain competitive results with other Wasserstein autoencoder based methods using a prior on higher dimensional hyperspheres. Complete details about the experimental settings and optimization strategies are given in Appendix C. We also report in Appendices C.5 or C.7 complementary experiments on variational inference on the sphere or self supervised learning with uniformity prior on the embedding hypersphere that further assess the effectiveness of SSW in wide range of learning tasks. The code is available online'.
5.1 SSW AS LOSS
Gradient flow on toy data. We verify on the first experiments that we can learn some tar- get distribution // E P( by minimizing SSW, iee we consider the minimization problem argmin, SSWB (1,, v). We suppose that we have access to the target distribution through samples, i.e. through pm m E)= 6y, where (y,)g" are i.i.d samples of v. We add in Appendix C.5 the case where we know the density up to some constant which can be dealt with the iasserseess variational inference framework introduced in (Yi & Liu, 2021). We choose as target distribution a mixture of 6 well separated von Mises-Fisher distributions (Mardia, 1975). This is a fairly challenging distribution since there are 6 modes which are not connected. We show on Figure 3 the Mollweide projection of the density approximated by kernel density estimator for a distribution with 500 particles. To optimize directly over particles, we perform a Riemannian gradient descent on the sphere (Absil et al., 2009).
Density estimation on earth data. We perform density estimation on datasets first gathered by Mathieu & Nickel (2020) which contain locations of wild fires (EOSDIS, 2020), floods (Brakenridge, 2017) eathquakes (NOAA, 2022). We use exponential map normalizing flows intro- duced in (Rezende et al., 2020) (see Appendix B.4) which are invertible transformations mapping the data to some prior that need to enforce. Here, we choose prior uniform distribution on S and we learn the model using SSW. These transformations allow to evaluate exactly the density at any point. More precisely, let T' be such transformation, let pz be a prior distribution on S and μ the measure of interest, which we know from samples, i.e. through fr E dz.- Then, we solve the following optimization problem minr SSW: (T p1, PZ). Once it is fitted, then the learned density f, can be obtained by
Table 1: Negative test log likelihood.
Earthquake
Flood
Fire
SSW SW
0.84
1.26_a..s
0.23
0.94_oo
1.3610.0s
0,54
Sterco
1.91
2.00
1.27
Yr S (z) = pz (T(x)) det (z)),
(24)
where we used the change of variable formula.
We show on Figure 5 the density of test data learned. We observe on this figure that the normalizing flows (NFs) put mass where most data points lic, and hence are able to somewhat recover the principle
'https://so thub. com/ http:///oooo/tttt/ee /Speeripprrrree cal S1 ced- Wasser sn
8

---

Published as a conference paper at ICLR 2023
16
(a) Fire
(b) Earthquake
Flood
Figure 5: Density estimation of models trained on earth data. We plot the density on the test data.
modes of the data. We also compare on Table the negative test log likelihood, averaged over trainings with different split of the data, between different OT metrics, namely SSW, SW and the stereographic projection model (Gemici et al., 2016) which first projects the data on R2 and use regular NF in the projected space. We observe that SSW allows to better fit the data compared to the other OT based methods which are less suited the sphere.
5.2 SSW AUTOENCODERS
Table 2: FID (Lower is better).
In this section, space autoencoders (AE). We rely on the SWAE framework introduced by Kolouri al. (2018). Let f be some encoder and be some decoder, denote pz prior distribution, then the loss minimized in SWAE is
SSW to learn the latent
Method Dataset
MNIST
Fashion
CIFARIO
SSWAE
14.91+0.22
43 43.944
98. coss
SWAE
15.1810.32
44.78
98.54...45
WAE-MMB IMQ
18. 18.1222
68.51 68.51
100.14 10
WAE-MMD RBF
20. 20.09
70. 1.77
100.27 20.74
SAE
19.39+a.50
56.754 75+1.
99.344..00
Circular GSWAE
15.01
44.65
98.820.8s
L(S.9) = o(
+ XSW
(25)
where yu is the distribution of the data for which have access samples. One advantage of this framework over more classical VAEs (Kingma & Welling. 2013) that no parametrization trick is needed here and therefore the choice of the prior is more free.
In several concomitant works, was shown that using a prior on the hypersphere can improve the results (Davidson et al., 2018; Xu & Durrett, 2018). Hence, propose in the same fashion as (Kolouri et al., 2018 8; 2019; Patrini et al., 2020) to replace SW by SSW, which we denote SSWAE, and enforce prior on the sphere. In the following, we use the MNIST (LeCun & Cortes, 2010), FashionMNIIST (Xiao et al., 2017) and CIFAR10 (Krizhevsky, 2009) datasets. and we put an E normalization at the output of the encoder. As a prior, we use the uniform distribution on 5 for MNIST and Fashion, and on S64 for CIFAR1O. We compare in Table 2 the Fréchet Inception Distance (FID) (Heusel et al., 20, for 10000 samples and averaged over 5 trainings, obtained with the Wasserstein Autoencoder (WAH) (Tolstikhin et al., 2018) 8), the classical SWAE (Kolouri et al., 2018), the Sinkhorn Autoencoder (SAE) (Patrini et al., 2020) and circular GSWAE (Kolouri et al., 2019). We observe that we obtain fairly competitive results on the different datasets. We add on Figure 4 the latent space obtained with a uniform prior on 5 on MNIST. We notably observe a better separation between classes for SSWAE.
CONCLUSION AND DISCUSSION
In this work, we derive a new practical advantages when computing optimal transport distances on hyperspherical data. We notably showed that it is competitive or even sometimes better than other metrics defined directly on IR on variety of machine learning tasks, including density estimation or generative models. Our work is, up to our knowledge, the first to adapt the classical sliced Wasserstein framework to non-trivial manifolds. The three main ingredients are: closed-form for Wasserstein on the circle, ii) closed-form solution to the projection onto great circles, and íii) novel Radon transform on the Sphere. An immediate extension of this work would be to consider sliced-Vo Wasserstein discrepancy in hyperbolic spaces, where geodesics are circular arcs in the Poincaré disk. Beyond the generalization other, possibly well behaved, manifolds, asymptotic properties as well as statistical and topological aspects need to be examined. While postulate that results comparable to the Euclidean case might be reached. the fact that the manifold is closed might bring interesting differences and justify further use of this type of discrepancies rather than their Fuclidean counterparts.
discrepancy on the hypersphere, that comes with

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
Clément Bonet thanks Benoît Malézieux for fruitful discussions. This work was performed partly using HPC resources from GENCI-IDRIS (Grant
This research was funded by project DynaLeam from Labex CominLabs and Région Bretagne ARED DLeamMe, and by the project OTTOPIA ANR-20-CHIA-0030 of the French National Research Agency (ANR).
REFERENCES
Ahmed Abouelaz and Radouan Daher. Sur la transformation de radon de la sphère S Bulletin de la Société Mathématique de France, 121(3)333--32,, 1993. (Cited on 6)
P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Riemannian geometry of grassmann manifolds with a view on algorithmic computation. Acta Applicandae Mathematica, 80(2):199-220, 2004.
(Cited on p. 5)
P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization algorithms on matrix manifolds. In Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2009. (Cited on p.
25, 29, 32)
Mark I. Agranovskyt and Eric Todd Quintott. Injectivity of the spherical mean operator and related problems. Complex analysis, harmonic analysis and applications, 347:12, 1996. (Cited on 6)
David Alvarez-Melis Youssef Mroueh, and Tommi Jaakkola. Unsupervised hierarchy matching with optimal transport over hyperbolic spaces. In International Conference on Artificial Intelligence and Statistics, pp. 1606-1617. PMLR, 2020. (Cited p. 1)
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2005. (Cited on p. 3, 6)
Yuri A Antipov, Ricardo Estrada, and Boris Rubin. Inversion formulas for the spherical means in constant curvature spaces. arXiv preprint arXiv:1107.5992, 2011. (Cited on 6)
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks.
In International conference machine learning, pp. 214-223. PMLR, 2017. (Cited on p.
Eleonora Bardelli and Andrea Carlo Giuseppe Mennucci. Probability measures on infinite- dimensional stiefel manifolds. Journal of Geometric Mechanics, 9(3):291, 2017. (Cited on p. 4, 5, 18, 24, 25)
Thomas Bendokat, Ralf Zimmermann, and P-A Absil. A grassmann manifold handbook: Basic geometry and computational aspects. arXiv preprint 1rXiv:2011..3666 2020. (Cited on p. 5)
Camille Besombes, Olivier Pannekoucke, Corentin Lapeyre, Benjamin Sanderson, and Olivier Thual. Producing realistic climate data with generative adversarial networks. Nonlinear Processes in Geophysics, 28(3):347-370, 2021. (Cited on p. 1)
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association. 112(518):859 2017. (Cited on 31)
Jan Boman and Filip Lindskog. Support theorems for the radon transform and cramér-wold theorems. Journal of theoretical probability, 22(3):683-710, 2009. (Cited on p. 3, 6)
Silvere Bonnabel. Stochastic gradient descent riemannian manifolds. IEEE Transactions on Automatic Control, 58(9):2117-2229, 2013. (Cited on p. 25)
Nicolas Bonneel, Julien Rabin, Gabriel Peyré, and Hanspeter Pfister. Sliced and radon wasserstein barycenters measures. Journal of Mathematical Imaging and Vision, 51(1):22-45, 2015. (Cited p. 3, 6)
Nicolas Bonnotte. Unidimensional and evolution methods for optimal transportation. PhD thesis, Paris 11, 2013. (Cited p.
10

---

Published as a conference paper at ICLR 2023
Nicolas Boumal. An introduction to optimization smooth manifolds. To appear with Cambridge University Press, Apr 2022. URL http: //tp://wwwww/ www..nnnnolaabaaaaoooommmmm het.maan..et/tooooooo (Cited on p. 25, 29)
Brakenridge.
active https//wwwwwwrooooooooooddddddee
Global
archive of large
flood events.
2017. (Cited on p. 8)
Simon Byme and Mark Girolami. Geodesic monte carlo on embedded manifolds. Scandinavian
Journal of Statistics, 40(4))825-88) 2013. (Cited p. 31)
Carlos A Cabrelli and Ursula Molter. The kantorovich metric for probability measures on the circle. Journal of Computational and Applied Mathematics, 57(3):345-361. 1995. (Cited p. 4)
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.). Advances in Neural Information Processing Systems, volume 33, pp. 9912 9924. Curran Asso- ciates, Inc., 2020. URL pttttps://prooeedinng neurips cc poaper/202/// file/ påf. (Cited on p. 1,)
70 f0 6e..2822222222 (tteee441aaa22fff222eeee
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 1 119 of Proceedings of Machine Learning Research, pp. 1597-1607. PMLR, 13-18 Jul 2020a. URL https: /tts::/pproee /ptp/eeeccceee.... malr.press// v h1e1//hee2111 chen20eenn00 html. (Cited on p. 1, 36, 37, 38)
Xiongjie Chen, Yongxin Yang. and Yunpeng Li. Augmented sliced wasserstein distances. arXiv preprint urXiv:2006.088722 2020b. (Cited on p. 3, 6)
Samuel Cohen, Brandon Amos, and Yaron Lipman. Riemannian convex potential maps. In In- ternational Conference on Machine Learning, pp. 2028-2038. PMLR, 2021. (Cited on p. 1, 26)
Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamoniy. Optimal transport for domain adaptation. IEEE transactions on pattern analysis and machine intelligence, 39(9):1853-188) 1865, 2016. (Cited on 1)
Li Cui, Xin Qi, Chengfeng Wen, Na Lei, Xinyuan Li, Min Zhang, and Xianfeng Gu. Spherical optimal transportation. Computer-Aided Design, 1 115:181-192, 2019. (Cited on p. I)
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013. (Cited on p. 7)
Susanna Dann. On the minkowski- transform. arXiv preprint arXiv: 2010. (Cited on p. 6)
Tim R. Davidson, Luca Falorsi, Nicola De Cao. Thomas Kipf, and Jakub M. Tomczak. Hyperspherical variational auto-encoders. In Amir Globerson and Ricardo Silva (eds.), Proceedings of the Thirty- Fourth Conference Uncertainty in Artificial Intelligence, UAI 2018, Monterey, California, USA, August 6-10, 2018, pp. 856-865. AUAI Press, 2018. URL http:/ www.i.oog//aaa01111111 proceedi ngs hwtpppappassa/eee pdf. (Cited on p. 1, 9, 27)
Nicola De Cao and Wilker Aziz. The power spherical distribution. arXiv preprint arXiv:2006.04437 2020. (Cited p. 32)
Julie Delon, Julien Salomon, and Andrei Sobolevski. Fast transport optimization for monge costs on the circle. SIAM Journal Applied Mathematics, (0X7):2239-2255 2010. (Cited p. 4, 6, 7)
Ishan Deshpande, Yuan- Slan-Ting Hu, Ruoyu Sun, Ayis Pyrros, Nasir Siddiqui, Sanmi Koyejo, Zhizhen Zhao, David Forsyth and Alexander G Schwing. Max-sliced wasserstein distance and its use for gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10648-10653 2019. (Cited 3)
11

---

Published as a conference paper at ICLR 2023
Marco Di Marzio, Agnese Panzera, and Charles C Taylor. Nonparametric regression for spherical data. Journal of the American Statistical Association, 109(506):748- 763, 2014. (Cited on p. 1)
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv
preprint arXiv:1605.08800 2016. (Cited on p. 27, 30)
Arnaud Doucet, Nando De Freitas, Neil James Gordon, et al. Sequential Monte Carlo methods in practice, volume 1. Springer, 2001. (Cited on p. 33)
Leon Ehrenpreis. The universality of the Radon transform. OUP Oxford, 2003. (Cited on p. 6)
EOSDIS. Land, atmosphere near real-time capability for eos (lance) system operated by nasa's earth science data and information system (esdis). https:/earrthdat.nnsss http:///www..a...gooo////eeettae real-timeefimm//ativve fireddaata, 2020. (Cited on p. 8)
Kilian Fatras, Younes Zine, Rémi Flamary, Remi Gribonval, and Nicolas Courty, Learning with minibatch wasserstein asymptotic and gradient properties. In Silvia Chiappa and Roberto Calandra (eds.). Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statisties, volume 108 of Proceedings of Machine Learning Research, pp. 2131-2141, PMLR, 26-28 Aug 2020. https:s:: http:///ooeeeeeeeee/ee/ee ww..m..prrerr. press/pes/seee (Cited on p. 1)
www.mm....s//vl1/.m.m 08 httpe/ssvvoo//aaaaaaaaaaa Fatras20a html.
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis.
Journal of the American Mathematical Society, 2944)9983-104, 2016. (Cited on p.
Jean Feydy, Thibault Séjourné, Saaçoii-Xaviee Vialard, Shun-ichi Amari, Alain Trouve, and Gabriel Peyré. Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd International Conference on Artificial Intellignnce and Statistics, pp. 2681 16900, 2019. (Cited on p. 35)
Alessio Figalli and Cédric Villani. Optimal transport and curvature. In Nonlinear PDE's and Applications, pp. 171-217. Springer, 2011. (Cited on p. 1, 2)
Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78):1-8, 2021. URL http: fimrr. org/ papers/v2222 20- 451 html. (Cited on p. 35)
P Thomas Fletcher, Conglin Lu, Stephen M Pizer, and Sarang Joshi. Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE transactions on medical imaging, 2398)99955 2004. (Cited on p. 4)
Walter Gander. Algorithms for the qr decomposition. Res. Rep, 80(02): 1980. (Cited on
p. 7)
Mevlana C Gemici, Danilo Rezende, and Shakir Mohamed. Normalizing flows on riemannian manifolds. arXiv preprint arXiv:1611.02304, 2016. (Cited on p. 26, 27, 30)
Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013. (Cited on p. 7)
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent a new approach self-supervised learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 21271 www.ips....... cc /aper/ 2020/ 1020/File// f3ada (a0ada800ca 80d5c4ee 701 42b1 7b81 922.992b22.. Paperr fd (Cited on p. 1)
Curran Associates, Inc., 2020. URL ht htwps:ttt/tttct https::/ppooceeiincccesooo/o///e
Armelle Guillou, Philippe Naveau, and Alexandre You. A folding methodology for multivariate extremes: estimation the spectral probability measure and actuarial applications. Scandinavian Actuarial Journal, 2015(():549-5777 2015. (Cited on p.
12

---

Published as a conference paper at ICLR 2023
Brittany Froese Hamfeldt and Axel GR Turqquis.. A convergence framework for optimal transport on the sphere. arXiv preprint arXiv:2103.05739 2021a. (Cited on p. 1)
Brittany Froese Hamfeldt and Axel GR Turnquist. A convergent finite difference method for optimal transport on the sphere. arXiv preprint arXiv:2105. 2xxx:2205..0500,, 2021b. (Cited on p. 1)
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. (Cited on p. 36)
Sigurdur Helgason et al. Integral geometry and Radon transforms. Springer, 2011. (Cited on 3, 5)
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. (Cited p. 9)
Andrew Homan and Hanming Zhou. Injectivity and stability for generic class of generalized radon transforms. The Journal of Geometric Analysis, 27(2):1515-1529 2017. (Cited on 6)
Andrés Hoyos-Idrobo. Aligning hyperbolic representations: an optimal transport-based approach. arXiv preprint urXiv:2012..00083 2020. (Cited on p.
Minhui Huang, Shiqian Ma, and Lifeng Lai. A riemannian block coordinate descent method for computing the projection robust wasserstein distance. In International Conference on Machine Learning, pp.
PMLR, 2021. (Cited on p. 3)
Shayan Hundrieser, Marcel Klatt, and Axel Munk. The statistics of circular optimal transport. arXiv preprint arXiv:2103. 15426 2021. (Cited on p. 4)
Michael Jordan, Zoubin Ghahramani, Tommi Jaakkola, and Lawrence Saul. An introduction to variational methods for graphical models. Machine learning, 37(2):183-233, 1999. (Cited on p. 31)
Sungkyu Jung. Geodesic projection of the von mises-fisher distribution for projection pursuit of directional data. Electronic Journal of Statistics, 15(1):984-1033, 2021. (Cited on p. 4, 6, 26)
Sungkyu Jung, Ian I. Dryden, and James Stephen Marron. Analysis of principal nested spheres. Biometrika, 99(3)555--568, 2012. (Cited on p. 5, 20)
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv: arXiv:1412.6901 1412.6980, 2014. (Cited on 29, 33, 34,36)
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1312611 14, 2013. (Cited on 1,
Soheil Kolouri, Phillip E Pope, Charles E Martin, and Gustavo K Rohde. Sliced wasserstein auto- encoders. International Conference on Learning Representations, 2018. (Cited on p. 9, 34)
Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, and Gustavo Rohde. Generalized sliced wasserstein distances. Advances in Neural Information Processing Systems, 32, 2019. (Cited on p. 3, 6, 9, 19, 34)
Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009. (Cited on p. 9,
Gerhard Kurz and Uwe D Hanebeck. Stochastic sampling of the hyperspherical mises-fisher fisher dis- tribution without rejection methods. In 2015 Sensor Data Fusion: Trends, Solutions, Applications (SDF), pp. 1-6. IEEE, 2015. (Cited on p. 25)
Shiwei Lan, Bo Zhou, and Babak Shahbaba. Spherical hamiltonian monte carlo for constrained target distributions. In International Conference on Machine Learning, pp. 629-637. PMLR, 2014. (Cited on p. 31)
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URL http: htthtt/p/yy.....
lecun. com/ eeeeex//mmee i st.. (Cited p.
13

---

Published as a conference paper at ICLR 2023
Mufan Bill Li and Murat A Erdogdu. Riemannian langevin algorithm for solving semidefinite programs. arXiv preprint arXiv:2010.11111 2020. (Cited on p. 31)
Tianyi Lin, Chenyou Fan, Nhat Ho, Marco Cuturi, and Michael Jordan. Projection robust wasserstein distance and riemannian optimization. Advances in neural information processing systems, 33: 9393-9397, 2020. (Cited on p. 3,
Tianyi Lin, Zeyu Zheng, Elynn Chen, Marco Cuturi, and Michael Jordan. On projection robust optimal transport: Sample complexity and model misspecification. In International Conference on Artificial Intelligence and Statistics, pp. 262-270. PMLR, 2021. (Cited p. 3, 6)
Chang Liu, Jun Zhu, and Yang Song. Stochastic gradient geodesic meme methods. Advances in neural information processing systems, 29, 2016. (Cited on p. 32)
Jun S Liu and Rong Chen. Blind deconvolution via sequential imputations. Journal of the american statistical association, 90(430):567-5777 1995. (Cited on p. 33)
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj. and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2017. (Cited on p. 1)
Kanti V Mardia, Peter E Jupp, and KV Mardia. Directional statistics, volume 2. Library, 2000. (Cited on p. 25)
Wiley Online
Kantilal Varichand Mardia. Statistics of directional data. Journal of the Royal Statistical Society: Series B (Methadological) 37(3):349-371, 1975. (Cited on p. 8)
Emile Mathieu and Maximilian Nickel. Riemannian continuous normalizing flows. Advances in
Neural Information Processing Systems, 33:2503-2515 2020. (Cited on 27)
Robert J McCann. Polar factorization of maps on riemannian manifolds. Geometric & Functional
Analysis GAFA, 11(3):589-6000 2001. (Cited on p. 1)
Kimia Nadjahi. Sliced- Wasserstein distance for large-scale machine learning: theory, methodology and extensions. PhD thesis, Institut polytechnique de Paris, 2021. (Cited p. 2)
Kimia Nadjahi, Alain Durmus, Umut Simsekli, and Roland Badeau. Asymptotic guarantees for learning generative models with the sliced- wasserseennr distance. Advances in Neural Information Processing Systems, 32, 2019. (Cited on p. 3)
Kimia Nadjahi, Alain Durmus, Lénaic Chizat, Soheil Kolouri, Shahin Shahrampour, and Umut Simsekli. Statistical and topological properties of sliced probability divergences. Advances in Neural Information Processing Systems, 33:20802-2081 2020. (Cited on p. 7, 23, 24, 28)
Frank Natterer. The mathematics of computerized tomography. SIAM, 2001. (Cited on p. 5)
Khai Nguyen and Nhat Ho. Amortized projection optimization for sliced wasserstein generative models. arXiv preprint arXiv:2203. 13417, 2022a. (Cited on p.
Khai Nguyen and Nhat Ho. Revisiting sliced wasserstein on images: From vectorization to convolu- tion. arXiv preprint arXiv:2204.0f /88, 2022b. (Cited on p.
Khai Nguyen, Son Nguyen, Nhat Ho, Tung Pham, and Hung Bui. Improving relational regularized autoencoders with spherical sliced fused gromov wasserstein. arXiv preprint arXiv:2010.0/787, 2020. (Cited p.
Khai Nguyen, Nhat Ho, Tung Pham, and Hung Bui. Distributional sliced- wasserstein and applications to generative modeling. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https:://ppnnt..et
net. net/forum? forum
(Cited on 3)
NOAA.
global https//www..ncei.ooz. wtt.eeiwww...g/acccc//atataaaaaa htw.:///ww/....oonnnoooooioiioon
Neei/wds
significant
earthquake htw.://www.aaa.oogooooaoooooooo. ha2a1d:
database.
2022. (Cited p.
14

---

Published as a conference paper at ICLR 2023
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1-64 2021. (Cited on p. 26)
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. A'lché-Bucc E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019. (Cited on p. 7)
Giorgio Patrini, Rianne van den Berg, Patrick Forre, Marcello Carioni, Samarth Bhargav, Max Welling, Tim Genewein, and Frank Nielsen. Sinkhorn autoencoders. In Uncertainty in Artificial Intelligence, pp. 733-743. PMLR, 2020. (Cited on p. 9, 34)
François-Piers Paty and Marco Cuturi. Subspace robust wasserstein distances. In International conference on machine learning, pp. 5072-5081. PMLR, 2019. (Cited on p. 3)
Nathanaël Perraudin, Michaël Defferaard Tomasz Kacprzak, and Raphael Sgier. Deepsphere: Effi- cient spherical convolutional neural network with healpix sampling for cosmological applications. Astronomy and Computing, 27:130-146, 2019. (Cited p.
Arthur Pewsey and Eduardo Gareía- Partua-Portugués. Recent advances in directional statistics. Test, 30(1):
1-58, 2021. (Cited on p. 1)
Gabriel Peyré, Marco Cuturi, et Computational optimal transport: With applications data science. Foundations and Trends@ in Machine Learning, 11(5-6):355-677, 2019. (Cited on p. 2, 4)
Julien Rabin, Julie Delon, and Yann Gousseau. Transportation distances on the circle. Journal of Mathematical Imaging and Vision, 41(1):147-167, 201 (Cited on p. 2, 3, 4, 17)
Julien Rabin, Gabriel Peyré, Julie Delon, and Marc Bernot, Wasserstein barycenter and its application to texture mixing. In International Conference on Scale Spuce and Variational Methods in Computer Vision, pp. 435-446. Springer. 2011b. (Cited p. 1,
Danilo J Rezende and Sébastien Racanière. Implicit riemannian concave potential maps. arXiv preprint arXiv:2110.012880 2021. (Cited on p. 26)
Danilo Jimenez Rezende, George Papamakarios, Sébastien Racaniere, Michael Albergo, Gurtej Kan- war, Phiala Shanahan, and Kyle Cranmer. Normalizing flows tori and spheres. In International Conference on Machine Learning, pp. 8083-8092. PMLR, 2020. (Cited on p. 8, 26, 30, 32, 33)
Boris Rubin. Inversion and characterization of the hemispherical transform. Journal d' d'Analyse Mathématique, 77(1):00--228, 1999. (Cited on p. 6, 21)
Boris Rubin. Inversion formulas for the spherical radon transform and the generalized cosine transform. Advances in Applied Mathematics, 29(3):471 19974474499 2002. (Cited on p. 6)
Boris Rubin. Notes radon transforms in integral geometry. Fractional Calculus and Applied Analysis, 6(1):25-72, 2003. (Cited on p. 6, 19, 20)
Raif M Rustamov and Subhabrata Majumdar. Intrinsic sliced wasserstein distances for comparing collections probability distributions on manifolds and graphs. arXiv preprint arXiv=2010.15285, 2020. (Cited p. 3)
Meyer Scetbon, Marco Cuturi, and Gabriel Peyré. Low-rank sinkhorn factorization. International Conference Machine Learning, pp. 9344 9354. PMLR, 2021. (Cited p. 1)
Suvrit Sra. Directional statistics in machine learning: a brief review. Applied Directional Statistics: Modern Methods and Case Studies, 225:6, 2018. (Cited on p. 1)
15

---

Published as a conference paper at ICLR 2023
Iyy O. Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schölkopf. Wasserstein auto- encoders. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL httpps: /ttas///ppereeee wwwppepevvvveeeeeeeee ew net/ / forum? d 7 Ob. (Cited on p. 9, 34)
Gary Ulrich. Computer generation of distributions on the m-sphere. Journal of the Royal Statistical Society: Series C (Applied Statistics), 33(2):158-163, 1984. (Cited on p. 25)
Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009. (Cited on p. 1, 2, 23)
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Coumapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Ker,, Eric Larson, C Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020. doi:
(Cited on p. 33)
Dilin Wang and Qiang Liu. Learning to draw samples: With application to amortized mle for generative adversarial learning. arXiv preprint arXiv: 1011111112222 2016. (Cited on p. 31)
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through align- ment and uniformity on the hypersphere. In International Conference on Machine Learning, pp. 9929-9939. PMLR, 2020. (Cited on p. 1, 36, 37, 38)
Xiao Wang, Qi Lei, and Ioannis Panageas. Fast convergence of langevin dynamics on manifold: Geodesics meet log-sobolev. Advances in Neural Information Processing Systems, 33:18894 18904, 2020. (Cited on p. 31)
Michael Werman, Shmuel Peleg, and Azriel Rosenfeld. A distance metric for multidimensional histograms. Computer Vision, Graphics, and Image Processing, 32(3):328-336, 1985. (Cited on p.
4)
Andrew TA Wood. Simulation of the von mises fisher distribution. Communications in statistics- simulation and computation, 23(1):157-164, 1994. (Cited on p. 25)
Zhirong Wu, Yuanjun Xiong, Stella X. Yu, and Dahua Lin. Unsupervised feature learning via non-pare con-parametric instance discrimination. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (C (CVPR), June 2018. (Cited on p. 1,
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint 17XY::7708.07477 2017. (Cited p. 9)
Jiacheng Xu and Greg Durrett. Spherical latent spaces for stable variational autoencoders. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun Jun'ichi Tsujii (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 November 4, 2018, pp. 4503 4513. Association for Computational Linguistics, 2018. (Cited on p. 1, 9, 27)
Mingxuan and Song Liu. Sliced wasserstein variational inference. In Fourth Symposium on Advances in Approximate Bayesian Inference, 2021. (Cited on p. 8, 31)
16

---

Published as a conference paper at ICLR 2023
A PROOFS
A. PROOF OF PROPOSITION 1
Optimal a. Let Jt E P2(SS). )ni Since v is the uniform distribution on S its cdf is
the identity on [0, 1] (where we identified S and [o. 11.. We can extend the cdf F on the real line
as in (Rabin et al., 201 1a) with the convention Fly + +) Fly) +1. Therefore, F, Id on R.
Moreover, we know that for all x E (F, - (x) (x aa) = x + a and
oeR /'
W2( v) inf
[FT - (Fy - -a) (t) dt.
(26)
For all E R, let f(a)= -o (1) (F, - -a)- 't. Then, we have:
Va R, f(a)= = S/¹ (FT (t) -t a)² dt
/ (F) -)²² dt + a² 2a [/ (F) t) dt
(27)
L/ (FI (t) - t)² dt + +a² 2a (f, x ddu()) - 2)
where we used that (F Unif( (10, 1]) = μ.
Hence, f'(a)= = a= fo x du(xz) 2'
Closed-form for empirical distributions. Let (z;) E [0, 1[" such that ø1 <... < In and let
(4n - -m 8z a discrete distribution.
To compute the closed-form of W2 between H4, and v Unif(S'), we first have that the optimal a is an = Ei 24 z.2 Moreover, we also have:
(t W2(mn, v) = //' (1- amm))² dt
(28)
//' (1) di- - 2 S(¹ (1) 2â, [ (D)( + ân + G..
Then, by noticing that (t) (t)
I, for all [F(x,), F(1:11) we have
pr.
! tF (t)dt Σ
n 5
tx Σ
n
22 1);
(29)
Pre
2n
i=1
[ (1)(1)
1 Σ
x
F-1 (I)d) = Σ Ti
n
(30)
n
and we also have:
ān + 2
Σ
12
x
1
+
1
11
34
2
1
n
72
2
1
(31)
n
2
71
71
Then, by plugging these results into (28), we obtain
(2,, v)
2
(2i - 1)2, - 2 Σ 24
2
+
1
+
1
+
1
n²
71
3
4
1=1
1=1
1=1
1=1
i=1
1
7
1
2
+ n²
(n +
20)x; +
71
12'
(32)
17

---

Published as a conference paper at ICLR 2023
A.2 PROOF OF EQUATION 17
Let U E Va.. Then the great circle generated by U E Va. is defined as the intersection between span(UU?)) and 5d-1, And we have the following characterization:
span(UU")) n
By E Rd UU y and | =1
By E R 2 UU y and |UU y12 yUUTT ||UT
3 E St 2 U2.
And we deduce that
VU E V2 E p (x) argmin dga-1 U2).
(33)
=E8
A.3 PROOF OF LEMMA 1
Let U E Va and E gd-1 such that U * 0. Denote U = (u) U2), i.e. the 2-plane E is E span(UU") span(u u2) and (#1: u2) is an orthonormal basis of E. Then, for all Sd-1, the projection on is (=(x) = (u1,2) 2)u1 + (u2, 2/u2 = UU
Now, let compute the geodesic distance between E Sd-1 and (f(2) (p>
E Ens-
p pxx) arccos
p (x)
dga-1
8.
1p (1)]12
(1r
" (x)(1))12
arccos((112 (x)]l2)
(34)
using that = px() + F (z).
Let y E 5dd--1 another point on the great circle. By the Cauchy-Schwand incquality, we have
(x, v) = (p (x), & 1p (x) (2) = ||p (x)
(35)
Therefore, using that arceos decreasing on (-1,11 1),
dga-1 (I, y) = arccos((z, y\) 2 (()c = dga- (z,
(36)
(x))l
Moreover, we have equality if and only if Xx And since E Sd-1 |||
">))" Using
again that arecos is decreasing, we deduce that the minimum well attained in
T>
=
Finally, using that |UUTx|l 2012 = .TUUTUUT x = = ||UTx|] 21/2. we deduce that
UT/
p
(37)
1UTx
Finally, by noticing that the projection is unique if and only if U 0, and using (Bardelli & Mennucci, 2017, Proposition 4.2) which states that there is a unique projection for a.e. x, we deduce that {x 8d-1, U x 0) is of measure null and hence, for 5 we have the result.
18

---

Published as a conference paper at ICLR 2023
A.4 PROOF OF PROPOSITION 2
Let E ( E Co(Sa x Va2), then by Fubini's theorem,
(RI.)) - [ / Rf(2, U)g(z, U) dzdo(U)
Vo.2
J
Va. Js Jsa f(x) (=))) U) dzdzdo(U)
-sg ((T) I / g(z, U) dzdo(U)dr
Vas
(38)
JSm.
gu-1 f(x) / Va.
(()") do(U)dr
[ M(x)(( g(a) dx
(S, RC ga-1-
A.5 PROOF OF PROPOSITION 3
Let g E Co(S Va.2).
Son S U12,U) (Rx) do(U)=
g(2, U) dzRp)(z,UU)
Stx
[ Sd
it" (()()) du(x)
9(P" Uz),U) (o(U))p(zz)
(39)
ga-- N4
g(Pu (x).U) du(z)dd((U)
Sa-1
g(2 d( 1 (=)do(()
Va
Hence, for a-almost every U e V4 (Rn) = pl n.
A.6 STUDY OF THE SPHERICAL RADON TRANSFORM R
In this Section, we first discuss the of integration of the spherical Radon transform R (19). We further show that it is related to the hemispherical Radon transform and we derive its kernel.
Set of integration. While the classical Radon transform integrates over hyperplanes of Rd and the generalized Radon transform integrates over hypersurfaces (Kolouri et al., 2019), the set of integration of the spherical Radon transform (19) is half of "big circle", i.e. half of the intersection between hyperplane and gd-1 (Rubin, 2003). We illustrate this on S in Figure On S², the intersection between hyperplane and 5 is great circle.
19

---

Published as a conference paper at ICLR 2023
Figure 6: Set of integration of the spherical Radon transform (19). The great circle is in black and the set of integration in blue. The point Uz E span(UU") n Sdd blue.
Proposition 6. Let U Va S The set of integration of (19) is
{x E Su-1, p (x) = 2) fo E U2) > 0},
(40)
where F span @ sUa))
Proof. Let U E V4.2, 2 E St. Denote E' = Span(UU/)) the 2-plane generating the great circle, and E- its orthogonal complementary. Hence, E $ E Rd and dim E-) = d - 2. Now, let F = E+ span(U=). Since Uz UU E E, have that dim(F) = d 1. Hence, F is hyperplane and F n Sii-1 is a "big circle" (Rubin, 2003), i.e. (d ((d--)-dimng)) dimeensioaa subsphere of sd-1
Now, for the first inclusion, let r E fo E S 1ºx
(2(())) = 2}. First, we show that Fn5t-11 By
Lemma and hypothesis, know that PU (I) 100x011 z. By denoting by p" the projection on E, we have:
p ((x)=UU*) = U( 01122) = ||U æ1|2U22 E (U2).
(41)
Hence, x = (x) "|0">'11UU>> + Fg. E F. Moreover,
(r, U2) = IUT (U,, U2) - x112 > 0.
(42)
we deduce that x E {F s (z. U2) > 0).
For the other inclusion, let {F n sd-1 (z, U2) 0}. Since F, have Fn + XU2, e R. Hence, using Lemma 1,
UTx x
X
P (x)
sign(A)
(43)
| x [|2 | |
But, we also have (r. U2) = >1|22|1 = > 0. Therefore, sign(A) = 1 and PU (x) = 2. Finally, conclude that {ar E sd-1, PU (x) 2) fa Fnns-l (x, U2) > 0}.
Link with Hemispherical transform. Since the intersection between a hyperplane and S is isometric to 8d-2 (Jung et al., 2012), we can relate R to the hemispherical transform H. (Rubin, 2003) on gul22 First, the hemispherical transform of a function f (Sdd11 is defined as
Ya E sd-1 Hff()) = -1 (1
(44)
From Proposition 6, we can write the spherical Radon transform (19) as a hemispherical transform on Sd-2
20

---

Published as a conference paper at ICLR 2023
Proposition 7. Let f E (s U E Yaz and 2 S', then
Rf(2,U) 1 f(z)) -H((()) H.f(U2),
(45)
ga-2
where for all E S0-2, (1x) 5 Jæ) with O the rotation matrix such that for all E F, Or span(e),..... Ed-1) where (eI....,edd ea) denotes the canonical basis, and J (1,d-1) = R(d-1)x2.
Td-1
and
Proof. Let f E (S4 E S1, U Va,2, then by Proposition 6,
RS(2,U)= fS inF (1a
(46)
F is a hyperplane. Let O Rdxd be the rotation such that for all E F, Oz E span(ei,..... , = F where (e1....,ed) is the canonical basis. By applying the change of variable Or = y, and since O-1 0², det 1, we obtain
Rf(z,U)= JoLFnSt-., f(0" ()) J f(0" ())0)))00))))) (47)
Now, we have that OU E Va since (OU)?(OU) = I2, and since Uz E F, OU> E F. For all l--1 E Rdx(d-1), then for all y E Enst
y E F, we have (g) = Yd = 0. Let J = (1,a-11)
= Jÿ where ÿ E 5d-2 is composed of the d - first coordinates of
Let's define, for all j E S f(ū) = ʃ(0 JØ). 0 = S"OU.
Then, since 5 5d-2, we can write:
ISa sa- (0 -if((((
Rf(2zU))
(48)
Kernel of Rt By exploiting the expression using the hemispherical transform in Proposition can derive its kernel in Appendix A.7.
A.7 PROOF OF PROPOSITION 4
First, we recall Lemma 2.3 of (Rubin, 1999) on Sd-2
Lemma 2 (Lemma 2.3 (Rubin, 1999)). ker(H) = {u E Meven (Sd-2), (SS0-2) 0} where Meven the set of even measures, i.e. measures such that for all f E (Sdd-2), (1,5) = (n, 8-) where (xx f(-x) x) for all z E Sd-22
Let E (dt-111 First, we notice that the density of Ru w.r.t. A is, for all 2 all E U E Va.2.
(Rx)(2, U) / |s-- (1P0
f (((.)))))))p(()))
(49)
Indeed, using Proposition 2, and Proposition 6, we have for all E Co(S x V42),
( (M, R. Ri { ga-1 R'g(x)du(z))
SA-1 Iaa g(z, g(2,U))1==PP))) (
(50)
- g(2,U) I sa-1 (==-"))) dzda(U)
- Jraaxsi 0(2,U) /
Pnsu 11t.00))0odf du)
21

---

Published as a conference paper at ICLR 2023
Hence, using Proposition 7, write (Rp))z, (Hz))U2) where j ==
Now, let l E ker( R)), then for all 2 E 8 U E Va... RT((2 = Hf((22) = and hence ji E ker(H) = {īr Meven(S ((SS = 0}.
First, let's show that μ MMven (Sven(Sut1)). Let f C(Sa-1) and U Va2,, then, by using the same notation as Propositions and 7, we have
(1, () 1 [())d((x)=
f(x) (1) de du(x)
54-1
S4-1 (=
I Jm f(z) ((0))
by Prop. 6
( { fly) (1n
ga
(51)
(H.js, f) dz
[
Ss (js, H.f ga-2 dz
(ū, (HLf) da since jt E Meven
f (x))d(()) = g4
using for the last line all the opposite transformations. Therefore, f E Meven (Stt1). Now, we need to find on which set the measure is null. We have
Vz E Sa U E Va.. ( =0
(52)
V2 E St U E V4, 1/0 ((0) =
Hence, we deduce that
ker(R) {n E (S VU E V4 V2 S', F= span(UTTT)) span(U2),
(53)
gd-) 0}.
Moreover, we have that Uv. Fu., n Sd-1 = ( (H c Rd, dim H d 1}.
Indeed, on the one hand, let H an hyperplane, E H n Sd-1, U e Va and note pاù (z). Then, E F S by Proposition and И S c Fu.
On the other hand, let E V4 2 81, F is a hyperplane since dim(F) = and therefore F F C (H, dim(H) = - 1}.
Finally, deduce that
ker( R (u (St-)) 4H E Ia,a-1, (( ((H St
(54)
A.8 PROOF OF PROPOSITION 5
Let p 2 First, it is straightforward to see that for all j,, E (sttt))), SSW (W, 2 0. SSW, (1,,(1,,)) SSW, (w, P = SSW,(n,u) and that have the triangular
22

---

Published as a conference paper at ICLR 2023
inequality since
VU,Ka Pp(Stt1), V) = (
p u p do(l))) 3
(Wy(PH p a) + W wp 1) do()))) ;
Jran
WB(P n, o) do(v))
WP a, p do(t))
Va.2 SSW,( a.) + SSW (a,
(55)
using the triangular inequality for Wp and the Minkowski inequality. Therefore, it is at least pseudo-distance.
To be distance, we also need SSW, (M, v) 0 μ v. Suppose that SS,,(,,)) = 0. Since, for all U E Va.2, WB (P μ, P v) 2 0, SSWP (U, implies that for o-ae U E Va,2 Wp ni, p p) and hence P P or (Rqu) = (Rw)) for o-ae U E Va since W, is distance the circle. Therefore, it a distance the sets of injectivity of ft.
A.9 ADDITIONAL PROPERTIES
In this Section, we derive additional properties of SSW. First, we will show that the weak convergence implies the convergence w.r./ SSW. Then, we will show that the sample complexity is independent of the dimension. Finally, we will derive the projection complexity of SSW.
Convergence Properties. Proposition 8. Let (Hx), (Pp(5= such that Pik
ns, then
k-500
SSWp ())
O.
(56)
k
Proof. Since the Wasserstein distance metrizes the weak convergence (Corollary 6.11 (Villani, 2009)),
we have pe Mi k0000
PV μ (by continuity) >
0 and hence by the WP(P # Pk, PV μ1) k-+00
dominated convergence theorem, SSWP p) k-000
0.
Sample Complexity. We show here that the sample complexity is independent of the dimension. Actually, this is a well known properties of sliced-based distances and it was studied first in (Nadjahi et al., 2020). To the best of our knowledge, the sample complexity of the Wasserstein distance on the circle has not been previously derived. We suppose in the next proposition that it is known as we mainly want show that the sample complexity of SSW does not depend on the dimension.
Proposition 9. Let > 1. Suppose that for p.v E P(S'), with empirical measures pan independent samples,
Em= 8z, and in 1 Ei= dy. where (x:)s μ,, (yi)s have
n
EllWp (n) (p x)]l S 8(p.)).
(57)
Then, for any μ., V. P (Sd-1) with empirical measures pmn and Dno we have
FIISSWP((m Pn.) (S,F(u, n)
(58)
23

---

Published as a conference paper at ICLR 2023
Proof. By using the triangle inequality, Fubini-Tonelli, and the hypothesis the sample complexity of W on S', we obtain:
FIISSWT(An Pn)) 2/, =F
(WP(P) Pn: P.) Wp p # uu, p v)) do(U)
Va.2
WP + Nn: vn) W μu, P do(())
Va.2
E Wl Pn.
P wp(P M, 1 do(U)
((p.n) do)
n()
(59)
Projection Complexity. We derive in the next proposition the projection complexity, which refers to the convergence rate of the Monte Carlo approximate w.rf of the number of projections L towards the true integral. Note that we find the typical rate of Monte Carlo estimates, and that it has already been derive for sliced-based distances in (Nadjahi et al., 2020).
Proposition 10. Letp 2 1, μ, v E P PSpac(Sd-1) Then, the error made with the Monte Carlo estimate of SSW, can be bounded as
Eu [SSW" P (, SSW, (S, 1 .aa. (W p.PPP SSWF(H.)) du((U)
(
I Vary (WF(P p. P v))
(60)
where SSW" (, = 1 ELe WP p. Pf v) with (U.) ~ a independent samples.
Proof. Let (U:) be iid samples of Then, by first using Jensen inequality and then remembering that Eu ( μ, PU] = SSW? (H, we have
Ev
SSWW P. (1, SSW3(u, mil
< Evr
SSW p. (u, D) SSWp(u,
Eu
1
WP (Pup u. PDiv) SSWP ()WP),,D)
L
i=1
1
Varu
WP u. pK'
L2
I Varv (WB(P* p4 pH
I. I (WP(P pW - SSWP v))² do(U).
(61)
-
B BACKGROUND ON THE SPHERE
B. UNIQUENESS OF THE PROJECTION
Here, we discuss the uniqueness of the projection Pt for almost every x. For that, we recall some results of (Bardelli & Mennucei, 2017).
24

---

Published as a conference paper at ICLR 2023
Let M be closed subset of a complete finieeddieeniioal Riemannian manifold N. Let d be the Riemannian distance on N.. Then, the distance from the set M is defined as
d
inf d(x, v). yEM
(62)
The infimum is minimum since M is closed and N locally compact, but the minimum might not be unique. When it is unique, let's denote the point which attains the minimum as <(x), i.e. d(z, [()))= (x).
Proposition 11 (Proposition 4.2 in (Bardelli & Mennucci, 2017)). Let M be a closed set in complete m-dimensional Riemannian manifold N. Then, for almost every x, there exists a unique point πI (z) E M that realizes the minimum of the distance from I.
From this Proposition, they further deduce that the measure # is well defined on M with y a locally absolutely continuous measure w.r.f. the Lebesgue measure.
In our setting, for all Va,2, we want to project a measure E P(SS-11 on the great circle span(UU?)) n 5 Hence, we have N 5 which is complete finte-admensional Riemannian manifold and M = span(U'" (UT") 5 1 closed set in N. Therefore, we can apply Proposition and the push-forward measures are well defined for absolutely continuous measures.
B.2 OPTIMIZATION ON THE SPHERE
Let F: = -> R be some functional the sphere. Then, we can perform gradient descent Riemannian manifold by following the geodesics, which are the counterpart straight lines in Id. Hence, the gradient descent algorithm (Absil et al., 2009; Bonnabel, 2013) reads
0, Ik+1 = expx2 - -xgradf((:))
(63)
where for all a E Sd-1 exp, T + 5dl1 is map from the tangent space T,S = {o E Rd, (z,, v) 0} to Sd-i such that for all Td5d-1 expx (t) = (1(1) with Yo the unique geodesic
starting from a with speed v, i.e. r(0) = x and /(0)) = =.
For Sd-1, the exponential map is known and is
Vx E Sd-1 Vu T exp,, = cos((l))22) + sin([[u|l2)
(64)
Moreover, the Riemannian gradient s is known (Absil et al., 2009, Eq. 3.37)
(x) = Proj,, (Vf(z)) = -V
(65)
Proj,, denoting the orthogonal projection on 1
For more details, we refer to (Absil et al., 2009; Boumal, 2022).
B.3 VON MISES-FISHER DISTRIBUTION
The von Mises-Fisher (yMF) distribution is distribution S characterized by concentration parameter x > 0 and a location parameter μ E 5d-1 through the density
dd/2-1
v0 Sd-1, f.MF 14,
exp(xy
(66)
where (K)) = 2n f cos(f) cos(v the modified Bessel function the first kind.
Several algorithms allow to sample from it, e.g. (Wood, 1994; Ulrich, 1984) for algorithms using rejection sampling or (Kurz & Hanebeck, 2015) without rejection sampling.
For d 1, the vMF coincides with the von Mises (vM) distribution, which has for density
vB
n, T/. fv (0: [o( cxp(x cos(f (1)),
(67)
with μ E [0, 2T the mean direction and > its concentration parameter. We refer to (Mardia et al., 2000, Section 3.5 and Chapter 9) for more details these distributions.
25

---

Published as a conference paper at ICLR 2023
In particular, for O, the vMF (resp. vM) distribution coincides with the uniform distribution on the sphere (resp. the circle).
Jung (2021) studied the law of the projection of vMF on a great circle. In particular, they showed that, while the vMF plays the role of the normal distributions for directional data, the projection actually does not follow von Mises distribution. More precisely, they showed the following theorem:
Theorem 1 (Theorem 3.1 in (Jung, 2021)). Let d 2 3, X ~ vMF k) E S U E aa and T = (X) the projection on the great circle generated by U. Then, the density function of T is
S: fulr) 0, cos(6)r) dr,
Vt E |-,,mb| f(1) 5(t)= (
(68)
where 5 the deviation of the great circle (geodesic) μ and the mixing density is
vr 16 fn(() = (K) To To((:cos())))))) -
(69)
with v = (d - 2)/2 and 15(2) = (z) > 0, 15 = 1/T(++1)
Hence, noticed by Jung (2021), in the particular case = i, X ~ Unif(S then
s(t)= = S( ((ll))ss((tt000)) dr Svmicss.0) S( fn(()dr= = fvsa(t;,,
f(1)
(70)
and hence ~ Unif(S²).
B.4 NORMALIZING FLOWS ON THE SPHERE
Normalizing flows (Papamakarios et al., 2021) are invertible transformations. There has been recent interest in defining such transformations on manifolds, and in particular on the sphere (Rezende et al., 2020; Cohen et al., 2021; Rezende & Racanière, 2021).
Exponential map normalizing flows. Here, we implemented the Exponential map normalizing flows introduced in (Rezende et al., 2020). The transformation T is
VI E 5d-1 2 = T(x) = exp2 (V)()))V)
(71)
where o(x) = ΣKn &e "a""" 1)-1), 1), a₁ 2 0. E, a, S 1, pb E sul and 3, > 0 for all i. (a:)s. (B,), and (14); are the learnable parameters.
The density of can be obtained
pz(2) px(x) det (E(a) Jr(2)" (T())(()))
(72)
where Jy is the Jacobian in the embedded space and E(x) it the matrix whose columns form an orthonormal basis of Sd-1
The common way of training normalizing flows is to use either the reverse or forward KL divergence. Here, we use them with different loss, namely SSW.
Stereographic projection. The stereographic projection p S + Rd-1 maps the sphere Sd-1 the Euclidean space. A strategy first introduced in (Gemici et al., 2016) is to use it before applying a normalizing flows in the Euclidean space in order to map some prior, and which allows to perform density estimation.
More precisely, the stereographic projection is defined as
Va E S p(x))
32.d
(73)
+ 211
and its inverse is
2 Tull +I
Yu E 1
p-((w)- =
(74)
k +1
26

---

Published as a conference paper at ICLR 2023
Gemici et al. (2016) derived the change of variable formula for this transformation, which comes from the theory of probability between manifolds. If we have a transformation f p. where is normalizing flows Rd e.g. RealNVP (Dinh et al., 2016), then the log density of the target distribution can be obtained as
1
p(r) = log Pz + log det Jy())
log det J
2
(75)
log (2) log det Jy(2) - ld [1p(z)]1 +18+11 1) 11)
2
where used the formula of (Gemici al., 2016) for the change of variable formula of p. and where Pz is the density of some prior on Rd-1, typically of a standard Gaussian. We refer to (Gemici et al., 2016; Mathieu & Nickel, 2020) for more details about these transformations.
C ADDITIONAL EXPERIMENTS
C.I EVOLUTION OF SSW BETWEEN VON DISES-FISHER DISTRIBUTIONS
The KI. divergence between the von Mises-Fisher distribution and the uniform distribution has been derived analytically in (Davidson et al., 2018; Xu & Durrett, 2018) as
Ta/2(k) 1k/2
( d
)1) log log(2x) log 14/2-1(1))
d
+ 2
log π + log log r
2
(76)
We plot on Figure the evolution of KL and SSW w.r.t for different dimensions. We observe different trend. SSW seems to get lower with the dimension contrary to KL.
SSW
d= 25
0.06
100
0.04
25
10
100
0.00
102
200
250
(a) (TM(v ()) (MMFF( 0))
(b) SW3 (vMF x)])MMF(.
Figure 7: Evolution w.r.f 6 between .MK(H, k) and vMF(.. 0). For SW, we used 100 projections (for memory reasons for 100), and computed it for {1,5, 10, 20, 30, 40, 50, 75, 100, 150, 200, 250}, 10 times by dimension and s, and with 500 samples of both distributions.
As sanity check, compare on Figure the evolution of SSW between vMF distributions where we fix vMF(H4, 10) and we rotate the first vMF along a great circle. More precisely, we plot SW3 (vMF((1,, 0, 0, ...). 10), expected, we obtain a bell shape which is maximal when the second vMF distribution has for location parameter -p. We observe a similar behavior between SSW,, SSW, and SW2 with different scales.
(Cos()))
...), 10)) for 0 E ( (.E 12)- As
27

---

Published as a conference paper at ICLR 2023
5SW
SSW
SW3
0.150
d=3
0-3
0.8
0.125
d
0.40
5
d'25
0.000
0.2
0.000
0.0
Figure 8: Evolution of SW between vMF samples in S (mean over 100 batch).
On Figure 9, we plot the evolution of SSW w.r. the number of projections for different dimensions. We observe that for around 100 projections, the variance seems to be low enough.
K
K 50
3
d = 1000
10
=
100
d= 2000
== 20
== 200
500
10-2
SSW3
0.04
S5W3
10-3
0.02
0.00
10
10a
10a
10a
10a
10a
10
10a
10a
Number of projections
Number of projections
Figure 9: Influence of the number of projections. We compute SW3(vMFF k)][ 0)) 20 times, for 500 samples in dimension d 3.
Nadjahi et al. (2020) proved that, contrary to the Wasserstein distance, the classical sliced- Wasserstein distance has sample complexity independent of the dimension d. As shown in Propositon 9, we have similar results for SSW. We show it empirically on Figure 10 by plotting SSW and the Wasserstein distance (with geodesic distance) between samples of the uniform distribution the number of samples. We observe indeed that the convergence rate of SSW is independent of the dimension.
the sphere w.r.f.
SSW3, d=3
10-1
Wg. d=3
SSW3, d-50 W3. d=50
10-4
10
10:4
10-4
Number of Samples
Figure 10: Spherical Sliced- Slised-Wasserstein Wasserstein and Wasserstein distance (with geodesic distance) between samples of the uniform distribution on the sphere. Results are averaged over 20 runs and the shaded correponds the standard deviation.
28

---

Published as a conference paper at ICLR 2023
C.2 RUNTIME COMPARISONS
We study here the evolution of the runtime w.r. different parameters. On Figure 11, we plot for several dimensions the runtime compute SSW2 w.E. the number of projections and the number of samples. We observe the linearity w.rl. the number of projections and the quasi-linearity w.r.l. the number samples.
0.8
100
0.6
d00000
Serords
500
Computational
=500
0.6
1000 d d'
d= 1000
644
0.2
0.2
250
1250 Number of projections
750
1000
1500
1750
2000
1,000
1500
2000
2500
30.00
Number of samples
Figure 11: Computation time w.t. the number of projections or samples, taken for = 10 and 500 samples for the left figure, and 10 and 200 projections for the right figure, and for 20 times.
C.3 GRADIENT FLOWS
Mixture of vMF distributions. For the experiment in Section 5.1, we use as target distribution of mixture of vMF distributions from which we have access to samples. We refer to Appendix B.3 for background on vMF distributions.
The 6 vMF distributions have weights 1/6, concentration parameter k = 10 and location parameters 11 = (L, 0), 112 (0, 1, μs (0, 1). μ1 (-1, 0). μs (0,-1,0) and μl (0, -1).
We use two different approximation of the distribution. First, we approximate it using the empirical distribution, i.e. j Σ 8z, and we optimize over the particles (x:))=1. To optimize over particles, we can either use projected gradient descent:
(b+1) (k) V (n SSW? (Pu,
(((++)=
(+++1)
(77)
||z(++1]||2> ||2
or a Riemannian gradient descent on the sphere (Absil et al., 2009) (see Appendix B.2 for more details). Note that the projected gradient descent is Riemannian gradient descent with retraction (Boumal, 2022).
We also neural networks such as multilayer perceptron (MLP). We used MLP composed of 5 layers of 100 units with leaky relu activation functions. The output of the MLP is normalized on the sphere using l normalization. We perform gradient descent using Adam (Kingma & Ba, 2014) as the optimizer with a learning rate of 10 for 2000 epochs. We approximate SSW with L 1000 projections and batch size of 500. The base distribution choose the uniform distribution on the sphere.
We report Figure 12 comparison of the 2 approximations where the density is estimated with Gaussian kernel density estimator.
(a) Target: Mixture vMF
(b) KDE estimate of MLP
(c) KDE estimate of 500 particles
Figure 12: Minimization of SSW with respect mixture of vME.
29

---

Published as a conference paper at ICLR 2023
vMF distribution. A simpler experiment, choose simple vMF distribution with 10. We report on Figure 13 the evolution of the density approximated using a KDE, and on Figure 14 the evolution of particles.
(a) Target distribution
(b) Initial samples
(c) Approximated dis- tribution at the epoch 100
(d) Approximated dis- tribution at the epoch 1000
Figure 13: Gradient Flows on SW with a vMF target and Mollweide projections. The distributions approximated using KDE.
Target
Target
Target
i
i
10
10
0.5
05
0.5
-100
0000
05
0 00
0
10 05 00 1.0.10
10 0. 0.0
10 05 0.0
y
-0.5.10
10
-05.10
10
y
(a) Initial samples
(b) Approximated distribution at the epoch 100
(c) Approximated distribution at the epoch 1000
Figure 14: Gradient Flows on SW with vMF target and Mollweide projections.
C.4 EARTH DATA ESTIMATION
Let be a normalizing flow (NF). For a density estimation task, we have access to a distribution through samples (x,) i.e. through the empirical measure Ar, Σn 8z, And the goal is to find an invertible transformation such that T = pz. where pz is prior distribution for which know the density. In that case, indeed, the density of μr, denoted as fa be obtained
Va, fa(x) P2))=(=)(TT)) det (x))-
(78)
For the invertible transform, we propose to use normalizing flows on the sphere (see Appendix B.4). We use two different normalizing flows, exponential map normalizing flows (Rezende et al., 2020) and Real NVP (Dinh et al., 2016) + stereographic projection (Gemici et al., 2016) which we call "Stereo" in Table
To fit TH Pz, use either SSW, SW on the sphere, or SW on Rdd1 for the stereographic projection based NF. For the exponential map normalizing flow, we compose 48 blocks, each one with 100 components. These transformations have 24000 parameters. For Real NVP, we compose 10 blocks of Real NVPs, with shifting and scaling as multilayer perceptron, composed of 10 layers, 25 hidden units and with leaky relu of parameters 0.2 for the activation function. The number of parameters of these networks are 27520.
For the training process, perform 20000 epochs with full batch size. We use Adam optimizer with learning rate of 10-1 For the sterographic NE, we use learning rate of 10-3
We report in Table 3 details of the datasets.
30

---

Published as a conference paper at ICLR 2023
Algorithm 2 SWVI (Yi & Liu, 2021)
Input: V potential, K the number of iterations of SWVI, N the batch size, e the number of MCMC steps
Initialization: Choose do sampler
for ===1 1 to K do
Sample (2 Ji=1
qe
Run E MCMC steps starting from (zf) to get (z))j=1
// Denote Ae = N شز= 8ag and fie = N E3=1 84g
Compute SW2 (jo: it))
Backpropagate through J w.r.. 0
Perform a gradient step
end for
Table 3: Details of Earth datasets.
Earthquake
Flood
Fire
Train set size
4284
3412
8966
Test set size
1836
1463
3843
Data size
6120
4875
12809
C.5 SLICED- WASSERSTEIN VARIATIONAL. INFERENCE
C.5.1 VARIATIONAL. INFERENCE
In variational inference (VT) (Jordan et al., 1999; Blei et al., 2017), we have some observed data (z,)ā=1 and some latent data (24) The goal of variational inference is to approximate the posterior distribution p(- (z) by some distribution E Q where Q is a family of probabilities. The usual way of doing that is minimize the Kullback-Leibler divergence among this family, Lee.
9(Z) p(Z(z)
min (L(glp((-1)) 4E
loog
(79)
But the KI. divergence suffers from some drawbacks, as it only divergence (i.e. it does not satisfy the triangular inequality, and it is non symmetric), but it also suffers from under estimating the target distribution (or over estimating it for the reverse KL).
Yi & Liu (2021) propose use optimal transport distance instead, namely the SW distance which gives the Sriced-Wasserster variational inference method. Basically, given some unnormalized probability p(-|æ) that we want approximate with some variational distribution gg. we can first apply a MCMC algorithm and then learn is using a gradient descent on SW with the target being the empirical distributions of the samples given by the MCMC. But running long MCMC chain is time consuming and it might be difficult to diagnose burn-in period. Therefore, they propose to only each iteration some number of steps / of MCMC chain, and then learn by gradient descent the variational distribution. Therefore, the variational distribution is guided at each step by the MCMC samples toward the stationary distribution which is the target. This is called an amortized sampler (see Problem 1 in (Wang& & Liu, 2016)). We sum up the procedure in Algorithm 2.
We propose here to substitute SW by SSW in order to perform SSWVI on the sphere. To do that, we first need a MCMC method on the sphere.
C.5.2 MCMC ON THE SPIIERE
Several MCMC methods the sphere have been proposed. For example, Hamiltonian Monte-Carlo (HMC) methods were proposed in (Byrne & Girolami, 2013; Lan et al., 2014; Liu et al., 2016), and Riemannian Langevin algorithms were proposed in (Li & Erdogdu, 2020; Wang al., 2020).
In our experiments, we use the Geodesic Langevin algorithm (GLA) introduced by Wang et al. (2020). This algorithm is natural generalization of the Unadjusted Langevin Algorithm (ULA) and
31

---

Published as a conference paper at ICLR 2023
it consists at simply following the geodesics of the regular ULA step, i.e.
Vk O, 2k+1 cxPax (Projay (~>VV + /2vZ)), Z N10,1). where for the sphere,
(80)
Ve E T, expx(v) cos([1 - | sin(llul)),
(81)
Projr is the projection the tangent space Ty,Sa-1 for E R (x, 03 (which is the orthogonal space) and is defined as
(v) = (0, u)T.
(82)
For more details, we refer to (Absil et al., 2009).
We use GLA here for simplicity and as a proof of concept. But note that GLA, as ULA, is biased and therefore the distribution learned will not be the exact stationary distribution. However, Cetropolis-Hastinne step at each iteration could be used to enforce the reversibility w.z.. the target distribution or could use other MCMC with more appealing convergence properties (see egg. Liu et al. (2016)).
C.5.3 APPLICATIONS
Target: Power spherical distribution. First, as a simple example on S², we use the power spherical distribution introduced by De Cao & Aziz (2020). This distribution has the advantage the vMF distribution to allow for the direct use of the reparameterization trick since it does not require rejection sampling. The pdf is obtained as.
Yar S Px (2; K) x (1 x²)'")"
(83)
with E S and K > O. We can sample from drawing first ~ Beta( d-1 + K, 1=1), 2d then constructing T 27 1 and Y = [T, xT -1 Finally, apply Householder reflection about fu to Y. All the operations are well differentiable and allow to apply the reparametrization trick. For the algorithm, see Algorithm 1 in (De Cao & Aziz, 2020). Hence, in this case, we denote Ge the map which takes samples from uniform distribution on Sdd2 and from a Beta distribution input and outputs samples of power spherical distribution with parameters 9 (K, p)), we can use as the sampler. We test the algorithm with a target being a power spherical distribution of parameter y (0, 1, and 10, starting from μ (1, 1, and 0.1. Perform- ing 2000 optimization steps with a gradient descent (Riemannian gradient descent on to stay on the sphere), and 20 steps of the GLA algorithm, we are getting close enough to the distribution as we can see on Figure 15.
For the hyperparameters, SSW, a Riemannian gradient descent on the sphere (Absil et al., 2009) to learn the location parameter μ with learning rate of 2, and learning of 200 for k. We performed K 2000 steps and used
used a step size of 10-3 for GLA, 1000 projections approximate
N 500 particles.
000
(a) Target distribution (b) Initial sampler
(c) Approximated dis- tribution at the epoch 1000
(d) Approximated dis- tribution at the epoch 2000
Figure 15: SWVI on Power Spherical Distributions with Mollweide projections.
Target: mixture of vsFs. In Section 5.1, we perform amortized variational inference with mixture of vMF distributions as target. For this, we train exponential map normalizing flows (see (Rezende et al., 2020) and Appendix B.4). Moreover, use the same target as Rezende et al. (2020),
32

---

Published as a conference paper at ICLR 2023
k= e000' (00'TT-e(mn)) with μ4 (0.7, μ2 (-1, 1). As = (0.6,0.5 and μ사 = (-0.7,4). These are spherical coordinates which are converted to euclidean using T (0,
Lee. the target has density p(x) E
(sin Φ sin sin 0, cos d).
The exponential map normalizing flow is composed N = blocks with = 5 components. We the algorithm for 10000 iterations, with each iteration 20 steps of GLA with ? 10-1 as learning rate, and one step of backpropagation through SSW using the Adam (Kingma & Ba, 2014)
optimizer with learning rate of 10
We report on Figure 16 the Mollweide projection of the learned density. Since we learn to samples from noise distribution, here the uniform distribution the sphere, do not have directly access to the density and we report kernel density estimate with Gaussian kernel using the implementation of Seipy (Virtanen al., 2020).
Mixture of vMF Target distribution
(b) Density learned with NE
Figure 16: SSWVI on mixture of vMF
70
60
ESS
40
SSWVI
30
SWVI
2000
4000
6000
8000
10000
Iterations
Figure 17: Comparison the ESS between SWVI et SSWVI with the mixture target (mean over 10 runs).
also report in Figure 17 the effective sample size (ESS) (Doucet et al., 2001; Liu Chen, 1995) over the iterations. The ESS is estimated by (Rezende et al., 2020)
2
Vatunis (atumiy(e-Bu(X)) Bu(X))
s=1 Wi
ESS
(84)
Var,
w2
(x(X)
mm
where w, = (0 The ESS is reported as a percentage of the sample size. Higher ESS indicates that the flow matches the target better (Rezende et al., 2020).
33

---

Published as a conference paper at ICLR 2023
C.6 SLICED-WARD WASSERSTEIN AUTOENCODER
We recall that in the WAE framework, want to minimize
- cl.gtftt))))) + AD(f#H.p2)
((f.g) -
(85)
where / is encoder, 9 decoder, pz prior distribution, e some cost function and D is divergence in the latent space. Several D were proposed. For example, Tolstikhin et al. (2018) proposed to use the MMD, Kolouri al. (2018) used the SW distance, Patrini et al. (2020) used the Sinkhorn divergence, Kolouri et al. (2019) used the generalized SW distance. Here, we use D = SSW?.
Architecture and procedure. We first detail the hyperparameters and architectures of neural networks for MNIST and Fashion MNIST. For the encoder f and the decoder g, use the same architecture as Kolouri et al. (2018).
For both the encoder and the decoder architecture, use fully convolutional architectures with 3x3 convolutional filters. More precisely, the architecture of the encoder is
T' E 28 + Conv2d + LeakyReL.Up
Conv2d LeakyReL.Up 0.2 AvgPoolp
+ Conv2d + LeakyRel.Up..
Conv2dg2 LeakyRel.Up2. A AvgPoolp
+ Conv2de4 + LeakyReLU
Conv2des Leaky AvgPool2
+ Flatten - FC128 + ReLU
FCdz + l² normalization
where dz is the dimension of the latent space (either 11 for 510 3 for $2).
The architecture the decoder
R2
FC128 + FC1024 ReLU
Reshape(64x4x4) + Upsample, -> Conves + LeakyRel.U0.
Convas LeakyReLU
Upsample, + Conv64 - LeakyReL.U,.
Conv32 LeakyRel.U0.
Upsamplez + Conv32 - Leaky Rel
Conv, Sigmoid
To compare the different autoencoders. we used as the reconstruction loss the binary cross entropy, 10, Adam (Kingma Ba, 2014) as optimizer with a learning rate of 10-3 and Pytorch's 's default momentum parameters for 800 epochs with batch of size 500. Moreover, when using SW type of distance, we approximated it with 1000 projections.
For the experiment CIFARIO, we the same architecture as Tolstikhin et al. (2018), More precisely, the architecture the encoder
I E R X32x332 + Conv2d + BatchNorm + ReLU
Conv2d2s6 BatchNorm ReLU
+ Conv2d512 + BatchNorm + ReLU
Conv2d1024 BatchNorm ReLU
+ FCa, l normalization
where d_ 65.
34

---

Published as a conference paper at ICLR 2023
The architecture of the decoder is
Rd FC4096 Reshape(1024 2 x 2)
+ Conv2dT + BatchNorm + ReLU
Conv2d + BatchNorm - ReLU
+ Conv2dT 128 - BatchNorm - ReLU
C Sigmoid
We use here a batch size of n = 128, = 0.1, the binary cross entropy as reconstruction loss and Adam optimizer with learning rate of 10-3
report in Table the FID obtained using 10000 samples and we report the mean over trainings.
For SSW, we used the formulation using the uniform distribution (12). To compute SW, we used the POT library (Flamary et al., 2021). To compute the Sinkhorn divergence, we used the GeomLoss package (Feydy et al., 2019).
Additional experiments. on S00
We report on Figure 18 samples obtained with SSW for uniform prior
2
4
a
R
7
9
7
1
3
4
9
4
4
0 2
2 3
9
3
4
7
4
3
5
1
4
0
7
S
6
2
3
8
0
3
8
6
2
C
3
9
9
0
5
2
/
9
9
6
7
7
8
9
A
2
4
5
7
6
9
5
3
4
7 2
4 4 8 8 0 6
8
4
9
2
8
8
3
0
8
7
3
1
7
6
7 1 0 1 4
7 4
Y
4
9
9
6
3
9
8
7
-
6
$
8
7
0
w
3
S
3 4
5
0 3
0
7
3
0
3 6 /
2
9
3
f
/
$
9
3
d
9
6
0
6
8
4
8
0
6
2
7
1
/
3
7
8
9
8 7
8 8
/
7
4
9
5
0 2
4
7
8
4
0
/
6
8
2
7
Q 3
9
3
6
4
(a) SSWAE
(b) SWAE
(c) SAE
Figure 18: Samples generated with Sliced- Wasserstein Autoencoders with a uniform prior on 100
On Figure 19, we add the evolution over epochs of the Wasserstein distance between generated images and samples from the test set.
log:o(W3(gefau, 1I)
log (gg)o 111)
1.6
1.65
SSWAE SWAE
1.5
SSWAE
SWAE
1.4
1.3
1.55
1.2
1.50
1.1
1.45
0.9
1.40
200
Epochs
Epochs
(a) With uniform prior on 2
(b) With prior on $10
Figure 19: Comparison of the evolution of the Wasserstein distance over epochs between SWAE and SSWAE MNIST (averaged over 5 trainings).
35

---

Published as a conference paper at ICLR 2023
C.7 SELF-SUPERVISED LEARNING
Table 4: Linear evaluation on CIFARIO. The features are taken either on the encoder output or directly on the sphere S
We conduct experiments using SSW to
prevent collapsing representations in con- trastive self-supervised learning (SSL) models. Such contrastive losses on the hy- persphere have exhibited great representa- tive capacity (Wu al., 2018; Chen et al.,
Method
Encoder output
g
Supervised
82.26
81.43
2020a; Caron et al., 2020) unlabelled datasets by learning robust image represen- tations invariantly to augmentations. As proposed in (Wang& & Isola, 2020), the contrastive objective can be decomposed into an alignment loss which forces posi- tive representations coming from the same image to be similar and a uniformity loss which preserves maximal information of the feature distri- bution and hence avoids collapsing representations. Without the uniformity loss, the representations tend to converge towards constant representation which yields the best alignment loss possible but also contains no information about original images. Wang & Isola (2020) propose to enforce uniformicity by leveraging the Gaussian potential kernel which is bound to the uniform distribution on the sphere. This formulation is also related to the denominator of the contrastive loss as specified in Chen et al. (2020a). We propose to replace the Gaussian kernel uniformity loss with SSW for which the complexity is more linear w.r.t. the number of batch samples. A simple choice of the align- ment loss is to minimize the mean squared euclidean distance between pairs of different augmented versions of the same image. A self-supervised learning network is pre-trained using this alignment loss added with uniformity term. Our overall self-supervised loss can be defined as:
Chen al. (2020a)
66.55
59.09
Wang & Isola (2020)
60.53
55.86
SW-SSL, 1, 10
62.65
57.77
SW-SSL, = 1, = 3
62,46
57.64
SSW-SSL, A 20, L
64.89
58.91
SSW-SSL, = 20,
63.75
59.75
Lssw-ssL =
ΣΣ
1
+
()) ,)),
(86)
Uniformity loss
Alignment
where 24,2 E Raxd are the representations from the network projected on the hypersphere of two augmented versions of the same images, v Unif(5d-11) is the uniform distribution on the hypersphere and is used to balance the two terms.
We pretrain ResNet1 8 (He et al., 2016) model on the CIFAR10 (Krizhevsky, 2009) data with projections projected onto the sphere Sa. This feature dimension allow us to visualize the entire validation set of CIFARIO 10 and its distribution on 5 are visible on Figure 20. We then evaluate the performance each contrastive objective by fitting linear classifier top of the output of the layer before the projection the sphere the training dataset as is common for SSL methods. For comparison, we also report the results when the features taken directly the sphere. As baseline, also train predictive supervised encoder by training jointly the linear classifier and the image encoder in a supervised manner using cross entropy.
the sphere. The visualization of the projections
We use ResNet18 (He et al., 2016) encoder which outputs 1024 features that are then projected onto the sphere S using last fully connected layer followed by e normalization. We pretrain the model for 200 epochs using minibatch stochastic gradient descent (SGD) with momentum of 0.9, a weight decay of 0.001 and an initial learning rate of 0.05. We use a batch size of 512 samples. The images are augmented using standard set of random augmentations for SSL: random crops, horizontal flipping, color jittering and gray scale transformation as done Wang & Isola (2020). For the trade-off parameter X, we X = 20 for SSW and À for SW.
To evaluate the performance representations, we use the common linear evaluation protocol where a linear classifier is fitted top of the pre-trained representations and the best validation accuracy is reported. The linear classifiers are trained for 100 epochs using the Adam (Kingma & Ba, 2014) optimizer with learning rate of 0.001 with decay of 0.2 at epoch 60 and 80. We compare methods with two other contrastive objectives, Chen et al. (2020a) with the normalized temperature- scaled cross-entropy (NT--eett) loss and Wang & Isola (2020) which proposes decompose the
36

---

Published as a conference paper at ICLR 2023
20.0-
SWA-1,1-10
oigeane
SFi ane
sutomebbilc
Sutomeblle
lrdd
brdd
cad
doct
deet
doos
Frog
Frog
horse
NOTTE
ENo
ATUD
DFUCE
truck
(a) SSW-SSL, À = 20, L = 10 (Ours)
(b) SW-SSL, = 1, L = 10
Vanc
SimCLR
Aiplane
Wothmablee
wiltmvblee
(c) Wang & Isola (2020)
(d) Chen al. (2020a)
Supervised
arglane
deer
dog
Frog
Frog
horse
horse
chip
chip
buck
Buck
(e) Supervised prediction
(f) Random initialization
Figure 20: The CIFARIO validation set on 2 after pre-training.
37

---

Published as a conference paper at ICLR 2023
Table 5: Comparison of contrastive methods and their respective uniformity objective where 21, =B Raxd are representations from augmented versions of the same set of images and v Unif(Sa-)) 1) the uniform distribution on the hypersphere.
Method
Cuniform (2 + C Cuniform (z
Complexity
Chen et al. (2020a)
1= log Ejzt exp( (A. ) cat(2 2P)
O
Wang & Isola (2020)
2e log n(n-1) Liss exp(-1||2 =
O
SSW-SSL (Ours)
2 (SSW3 (2 v) + (S
O(Ln(d log
objective in two distinct terms Lalign and Luniform- We recall the respective uniformity loss of each method in Table As one can see Table our method achieves here comparable performances to two state-of-the-art approaches, yet slightly meder-performing compared to (Chen et al., 2020a). We suspect that finer validation of the balancing parameter needed. Especially since the representations on Figure 20a are not completely uniformly distributed around the sphere after pre-training compared to other contrastive methods. Nevertheless, these preliminary results show that SSW-SSL is a promising contrastive learning approach without explicit distances between negative samples, especially compared to SW on the sphere. To this end, further works should be devoted to finding good balance between the alignment and uniformity objectives.
38


Reviews:
[{"review_id": "aOSWkwz3pb", "reviewer": "Reviewer_7Kyx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a slice-based approach to efficiently compute the Wasserstein distance between two distributions $\\nu$ and $\\mu$ defined on the sphere. The method termed SSW (Spherical  Sliced Wasserstein) projects the distributions on great spheres and then, hinges on the efficient computation of Wasserstein distance on the circle to propose the sliced-variant of Wasserstein distance on the sphere. Along with SSW, the related spherical Radon Transform and its dual operator are formulated. Finally it is established that the proposed SSW defines a pseudo-distance. A numerical algorithm that summarizes the design of the random projection matrices, the computation of the samples' coordinate on the circle as well as the Wasserstein distance on the circle. Empirical evaluations on simulation dataset or on learning latent representations via auto-encoders highlight the potential of the proposed method.", "summary_of_the_review": "The paper is well written and proposes a new sliced optimal transport on a sphere. It also proposes the related spherical Radon Transform. The new discrepancy measure of distributions over sphere is supported by its pseudo-metric property and empirical experiments showing its effectiveness. Topological properties of the proposed approach and some few details on the empirical  evaluations remain to be clarified. ", "strength_and_weaknesses": "**Strengths**\n- The rationale behind the proposed Spherical Sliced-Wasserstein (SSW) method is justified.\n-  The approach aims to design an appropriate geodesic projection on great circles of distributions defined over sphere. That projection acts as the projection on straight line used in Sliced Wasserstein (SW). Overall the projection operators are formulated as matrices over Stiefel manifold. Using known results on Wasserstein distance on circle, a sliced version of Wasserstein distance on sphere is derived.\n- As a contribution, the paper proposes a closed-form projection of the training samples from the sphere to the great circle using matrix on Stiefel manifold.\n- Another interesting contribution is the proposal of the spherical Radon transform and related dual operator that allow to rephrase SSW as a function of the new Radon transform.  The pseudo-distance property of SSW is established in Proposition 4.\n- Intensive empirical evaluations are conducted. They show that SSW exhibit a favorable runtime in a large scale setting. Used as a loss function, SSW allows for density estimation of simulated and real datasets with advantageous empirical results. Also SSW proves effective in generative modeling applied to classical Mnist, Fashion Mnist and Cifar datasets.\n\n**Weaknesses**\n- The plain metric property of SSW is not guaranteed.\n- Topological properties of SSW compared to plain Wasserstein distance on sphere are not discussed. Can one theoretically or empirically relate SSW to its plain counterpart?\n- For readability purpose, it would be interesting to formally introduce the notion of great circle in the paper. \n\n*Minor comments*\n\n- In Figure 4 illustration, which dataset is shown?\n- On page 8, the sentence \"We see on this figure that the normalizing flows fitted are able to recover the principles modes of the data\" is not clear. Where exactly on Figure 5 the recovery of the modes are highlighted?\n- The paper should better introduce (at least in the supplementary material) some preliminary notions of Wasserstein distance on the circle. Especially, the parametrization by the interval [0, 1[ on the real line should be better explained.\n- Some notations or abbreviations have to be clarified or formally defined within the main text. For instance, in equation (6), $C(\\mathbb{R} \\times S^{d-1})$ is not stated. Page 4, what $\\mathcal{P}_{p, ac}$ stands for? Page 9, NF is for normalizing flows?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OyjAfSivdMi", "reviewer": "Reviewer_NCpB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Following the well-established Sliced-Wasserstein (SW) used as a Wasserstein-based metric to compare measures supported on the Euclidean space $\\mathbb{R}^d$, this paper introduces a natural Spherical-Sliced-Wasserstein (SSW) distance to compare measures supported on an hypersphere. \n\nThe construction technique follows the SW-one, namely : \n- First, observe that comparing measures supported on the circle $S^1$ can be done fairly efficiently (akin to the comparison of measures supported on $\\mathbb{R}$). \n- In dimension $d$, intersect the hypersphere $S^{d-1}$ with 2d-planes (parametrized by the Stiefel manifold $V_{d,2}$) to get a (great) circle, project measures of interest $\\mu,\\nu$ on these circles, compute (efficiently) the Wasserstein distance between those measures. Eventually, integrate over all the circles. \n\nThe authors show that this construction defines a pseudo-metric on $P(S^{d-1})$ (probability measures supported on the hypersphere) ; provide implementation details and showcase their approach in a variety of experiments.", "summary_of_the_review": "I think this is a well-written paper that present a natural, well-motivated and easy-to-understand approach.\n\nOn the downsides, I would say that the paper does not bring any brand new idea from the theoretical side that would shed a new light on \"numerical OT on manifolds\" ; and while the experiments are interesting, I cannot assess the potential impact for numerical OT practitioners (though it clearly is non-zero).", "strength_and_weaknesses": "# Strengths : \n- The paper is overall very well written, with appropriate level of mathematical rigor (though this may be a matter of taste). \n- The construction is very natural yet (perhaps surprisingly) new to the best of my knowledge. \n- While I did not proofread the appendices in full details, proofs seems well-written and I did not identify a major flaw at first glance. \n- [minor but still] The figures are of high-quality. \n\n# Weaknesses : \n- (1) To some extend, one may argue that the contributions of the work are somewhat limited, in sense that the construction follows the exact step of the usual SW distance and, up to few technical modification brought to the Radon transform, there is no specific \"breakthrough\" in this paper. \n- (2) The setting is restricted to the spherical case. While this yield elegant formulas and possible implementation overall, it is also somewhat disappointing in sense that the proposed method may be generalized quite well (typically intersecting [insert assumptions here] hypersurfaces with $V_{d,2}$). From a theoretical standpoint, that may lead to a more instructive construction. \n- (3) A reference and discussion to the recent preprint _Intrinsic sliced wasserstein distances for comparing collections of probability distributions on manifolds and graphs_, Rustamov et al., may be relevant. In this work as well, the authors build a SW-like distance on manifolds. They follow a fairly different approach though (relying on the eigenfunctions of the Laplace-Beltrami operator on the manifold). It is not clear to me how the two methods compare. While the approach proposed in the current paper seems (to me) more natural and easier to implement, the one of Rustamov et al., has the benefits of being more general and quite original. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HugkMemKb1", "reviewer": "Reviewer_dqQ6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "A novel variant of slice-based Wasserstein distance is proposed in the paper, namely the spherical sliced Wasserstein (SSW) distance. The SSW is particularly suited to evaluating the discrepancy between probability measures defined on spherical manifolds. By projecting measures defined on hyperspheres onto great circles with the proposed spherical Radon transform, the SSW is defined as the expectation of Wasserstein distances between obtained measures on great circles w.r.t to the uniform distribution over the Stiefel manifold. The proposed SSW has lower computational costs compared to standard Wasserstein distances on $R^d$, since Wasserstein distances on circles admit closed-form solutions. Experiment results show that the SSW indeed can better capture geometric information from data than the original SW when the data of interest lies on spherical manifolds.", "summary_of_the_review": "This paper introduced an interesting variant of SW distances which accommodates well with measures defined on spherical manifolds. The method is novel, and experiments show its effectiveness on several tasks.", "strength_and_weaknesses": "Strength:\n\nThe SSW is the first step towards defining slice-based Wasserstein distances for measures defined on manifolds. Closed-form solutions to projections onto great circles are derived in the paper and subsequently the spherical Radon transform is also defined analogously to the standard Radon transform.\n\nWeakness:\n\nThere are a few things that can be improved in the paper. \n(1) The projection efficiency is not mentioned in the paper, but this is something important to slice-based Wasserstein distances. Can we quantify how many projections are needed to gain a good approximation of the distance? \n(2) What is the computational cost of SSW compared to the standard SW? Runtime comparison shown in Figure 2 does not include standard SW. From the experiments, the standard SW also produced competitive results.\n(3) It is not explained clearly in the paper why the SSW has better performance than the standard SW. The overall argument is that the data manifold is taken into consideration in SSW, but detailed discussions and analysis to support this claim with visualisations, experiments, and theoretical analysis will make it a stronger paper.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6sfc-3aDwKx", "reviewer": "Reviewer_sqi7", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces Spherical Sliced-Wasserstein (SSW) as a \"pseudometric\" to measure the dissimilarity between distributions defined on a hypersphere. At its core, the paper utilizes the prior work proposed by Rabin et al. 2011a that studies transport-based distances on circles and proposes efficient numerical schemes for calculating these distances. The authors then propose to slice distributions defined on a hypersphere by projecting them onto the great circles leading to spherical slices. Lastly, following the vanilla-sliced Wasserstein framework, the Spherical Sliced-Wasserstein is defined as the expected Wasserstein distance between the spherical slices of two distributions. According to Rabin et al. 2011a, the 1-Wasserstein distance between two distributions defined on a circle has a closed-form solution and can be obtained efficiently.  In addition, the authors show that the 2-Wasserstein distance between a distribution defined on a circle and the uniform distribution also has a closed-form and computationally efficient solution (Proposition 1). \n\nThe paper provides extensive experiments (both in the main paper and in the appendix) showing that the proposed SSW \"pseudometric\" is effective in various applications, from density estimation to gradient flow and learning auto-encoders. The appendix also provides additional results on, for instance, self-supervised learning with spherical latent representations. The results indicate the robustness and effectiveness the proposed dissimilarity measure. ", "summary_of_the_review": "Overall, I think this is a fantastic paper. Here are my primary considerations in evaluating the work: \n\n* It is well-motivated, clear, and a pleasant read\n* It is algorithmically simple while being theoretically sound\n* It showcases the application of the proposed approach to a wide range of problems. \n\nWhile the experiments might be far from being large-scale, they remain relevant and challenging. \n\n", "strength_and_weaknesses": "### Strengths\n\n* The paper addresses the exciting problem of comparing distributions defined on a hypersphere, which appears in many applications.\n* The paper introduces a simple and computationally efficient pseudometric for comparing spherical distributions that is theoretically sound. \n* The paper is very well written, and it is easy to follow. \n* The authors show the application of the proposed SSW on a wide variety of applications. What I particularly appreciate is that while some of the experiments are not large-scale, they remain challenging to solve by existing methods. \n\n### Weaknesses\n\nI don't see any significant weaknesses in this paper. Below are not necessarily weaknesses but points that could improve the paper. \n\n* It would have been great if the authors could show that the proposed Radon transform is invertible, and hence SSW is indeed a metric as opposed to a pseudometric. \n* I am surprised that SW distance works so well compared to SSW for spherical distributions. I was expecting a more significant performance gap. \n\n ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster