Keep your distance: learning dispersed embeddings on $\mathbb{S}_{m}$
Authors: Evgeniia Tokarchuk
Hua Chang Bakker
Vlad Niculae
Venue: Accepted by TMLR ()

Abstract: Learning well-separated features in high-dimensional spaces, such as text or image embeddings,
is crucial for many machine learning applications. Achieving such separation can be effectively
accomplished through the dispersion of embeddings, where unrelated vectors are pushed
apart as much as possible. By constraining features to be on a hypersphere, we can connect
dispersion to well-studied problems in mathematics and physics, where optimal solutions are
known for limited low-dimensional cases. However, in representation learning we typically deal
with a large number of features in high-dimensional space, and moreover, dispersion is usually
traded off with some other task-oriented training objective, making existing theoretical and
numerical solutions inapplicable. Therefore, it is common to rely on gradient-based methods
to encourage dispersion, usually by minimizing some function of the pairwise distances.
In this work, we first give an overview of existing methods from disconnected literature,
making new connections and highlighting similarities. Next, we introduce some new angles.
We propose to reinterpret pairwise dispersion using a maximum mean discrepancy (MMD)
motivation. We then propose an online variant of the celebrated Lloyd’s algorithm, of
K-Means fame, as an effective alternative regularizer for dispersion on generic domains.
Finally, we revise and empirically assess sliced regularizers that directly exploit properties
of the hypersphere, proposing a new, simple but effective one. Our experiments show the
importance of dispersion in image classification and natural language processing tasks, and
how algorithms exhibit different trade-offs in different regimes

Published in Transactions on Machine Learning Research (08/2025)
Keep your distance: learning dispersed embeddings on Sm
Evgeniia Tokarchuk
cogeniia@lokare
Lenguage Technology Lab University of Amsterdam
Hua Chang Bakker University of Amsterdam
hua. hua.. chang, bwwkerrstudenn..ccoommmmm uvo.nd
Vlad Niculae
v.. w.nculae@@uvo..l
Language Technology Lab University of Amsterdam
Reviewed on OpenReview: https: openreview. net/ forum? id= 5JTQE6HeTa
Abstract
Learning well-separated features in high-dimensional spaces, such as text or image embeddings, crucial for many machine learning applications. Achieving such separation can effectively accomplished through the dispersion of embeddings, where unrelated vectors are pushed apart as much as possible. By constraining features to be on hypersphere, we can connect dispersion well-studied problems in mathematics and physics, where optimal solutions known for limited low-dinensional cases. However, in represent learning lypically deal with a large number of features in high-dimensional space, and moreover, dispersion is usually traded off with some other task-oriented training objective, making existing theoretical and numerical solutions inapplicable. Therefore, it is common to rely on gradient- basedd methods to encourage dispersion, usually by minimizing some function of the pairwise distances. In this work, we first give an overview of existing methods from disconnected literature, making new connections and highlighting similarities. Next, we int roduce some new angles. We propose to reinterpret pairwise dispersion using maximun mean discrepancy (MMD) motivation. We then propose an online variant of the cclebrated Lloyd's algorithm, of K-Means fame, as an effective alternative regularizer for dispersion on generic domains. Finally, we revise and empirically assess sliced regularizers that directly exploit properties of the hypersphere, proposing a new, simple but effective one. Our experiments show the importance of dispersion image classification and natural language processing tasks, and how algorithms exhibit different trade-offs in different regimes.
1 Introduction
Dispersion, sometimes referred to as spreading or uniformity, the property that collection of points in a domain cover the domain well, with no two points being too close or too far. When the domain is hypersphere, useful space for representing embeddings of text (Meng al,, 2019) and images (Liu et al., 2017; Mettes et al., 2019), optimal dispersion is challenging. Embedding clumping, i. e., occurrence of semantically distant embeddings that are close to each other in terms of distance metric, is known problem, and it has been shown before that it negatively impacts the performance of the downstream lasks, such as image classification (Wang & Isola, 2020; Liu et al., 2021; Trosten et al., 2023), image generation (Liu et al., 2021), text classification (Wang Isola, 2020) and text generation (Tokarchuk & Niculac, 2024). Beyond data embeddings, neural network layer weights have been shown also benefit from dispersion (Liu al., 2018; 2021; Wang et al., 2021).

---

Published in Transactions on Machine Learning Research (08/2025)
In general, the problem of dispersing points the surface of m-dimensional sphere, such that the angular distance between any two points is maximal, is an open mathematical problem known as the Tammes problem (Tammes, 1930). The optimal solutions for this problem are only known for small values of m and n (Fejes Tóth, 1943; Danzer, 1986; van der Waerden & Schütte, 1951; Robinson, 1961; Musin & Tarasov, 2012; 2015). The Tammes problem can also be formulated as problem of finding an optimal spherical code (Conway et al., 1999; Cohn, 2024). However, machine learning applications we typically deal with a large number of dimensions and many points. Moreover, perfect dispersion may not be optimal and may hurt the task-specific learning objective, in learning applications prefer being able to trade off between a task loss and dispersion regularizer. For these reasons, we cannot directly use classical results, and instead focus on hyperspherical dispersion as a gradient- based optimization of a regularization function.
Contributions. In this work, we study the regularization approach to dispersion in machine learning applications, proposing new algorithms as well as casting new light on known ones. We survey a number of dispersion objectives used in the literature and reveal relationships between them (62.4). We then show a new and fundamental connection between kernel-based dispersion and maximum mean discrepancy (MMD, Gretton et al., 2012) with a uniform measure ($3.1). Further, we adapt Lloyd's algorithm (Lloyd, 1982) as a stochastic dispersion regularizer ($3.2), and suggest an effective regularizer based on slicing dispersion along great circles ($3.3). We evaluate (54) old and new methods on synthetic small and large scale problems, as well real-world large-scale applications in computer vision and natural language processing, revealing different trade-offs and throughout confirming the importance of representation dispersion for task performance. Additionally, we highlight that using Riemannian optimization (Bonnabel, 2013; Becigneul & Ganea, 2019) on the hypersphere, rather than projecting Euclidean gradient updates, benefits dispersion and overall task performance. reusable library for spherical dispersion is available as open-source software: https://githie cowmmttomoaaataitecceeteee http:/lllwwallddo-toorrod
2 Background: Dispersion on the Hypersphere
2.1 Notation
We denote by Sm the m 1-dimensional hypersphere embedded in R" f.e., S.. e |-||| = =1}. For u, v E Rmm we denote their Fuclidean inner product by (u, u) == Em upt; The hypersphere is an embedded Riemannian submanifold of R". The tangent space of the hypersphere at point æ is T.Smm == {u Rm (z. v) = 0}, and the Riemannian inner product on it is inherited from RT i. c., for u, E T,Sm, (u, v)2 = (u, v). The geodesic distance or angular distance on Sm, is denoted by x2)x = arccos (xx x")), in contrast with the chordal distance, which is the Euclidean distance ambient space dx, = || - ='']. As special case, for circles (m = 2) it is more convenient to work in an isomorphic angular parametrization,
i. e., S2 is isomorphic to {0] | -T 0 < ] I with d') = 10 - 0"), and the embedding is A + (cos 0, sin s). We reserve the use of Greek letters 0, & for 1-d angles. We denote by II the set of permutations of (1....,n).
We denote as X = (Ii,...,, Tn) an (ordered) collection, configuration, or design of n points on the same sphere, é.e., each 2, E Sm. Sans- capitals, e.g., Y, denote random variables.
2.2 Riemannian Optimization
f())
Since the configurations we are considering consists of points on a sphere, the natural framework for optimization is Riemannian optimization, taking into account the geometry of the sphere Sn We briefly describe and illustrate the method for S. as an embedded submanifold of TR Given a point x E Sm and a function F Sm -> R, the Riemannian gradient is the projection of the standard (Euclidean) gradient onto the tangent plane at r: (Figure 1)
Rxp, (gred ((a))
Figure I: Euclidean vs. Rieman- nian optimization.
grad_ F(x: proji,e.. (V=F(x)) (T xr F(x)
2

---

Published in Transactions on Machine Learning Research (08/2025)
Vectors tangent at can be interpreted directions of travel for along the manifold. This intuition gives the Riemannian gradient method:
(1+1) (et (A grad F(x(1))),
where ( is a sequence of step sizes, and the retraction retr,, maps tangent vectors back onto the surface. The canonical retraction is the exponential map retrz (exp) (v) (v) = cos([ee])æ + sin((]e])) jep it moves from along the sphere surface with constant velocity e for one unit of time. For optimization, approximate retractions such as the projection-base based one also work and can be implemented slightly faster: retr! (v) + w)/l|x + u/l Riemannian gradient methods are due Luenberger (1972); Lichnewsky (1979); Gabay (1982). Stochastic versions where the gradient is replaced by estimation are studied by Bonnabel (2013), and accelerated versions like Riemannian Adam are due to Becigneul & Ganea (2019). While some works optimize spherical objectives ignoring the geometry and applying Euclidean gradient descent in R interspersed with projection back onto Sm: we note this is ill-defined: while formally it resembles the projected gradient method, S, is non-convex subset I Projection is not defined at the origin IR", and exhibits quick jumps around the origin. Despite also being convergent for certain simple objectives (Vu et al., 2019) and being often used, we experimentally show in Appendix E.3 that projected gradient is less well-behaved than Riemannian gradient for our purposes, and therefore stick to the natural choice of Riemannian optimization.
2.3 Measuring Dispersion
Intuitively, we say configuration of n points on Sm is dispersed if the points are as well spread out, with each point being neither too close nor too far from its neighbors, and therefore covering the surface well. In coding thoory, maximally-dispersed configurations form optimal spherical codes, and can be used to discrctize continuous spherical data. The problem of finding such optimal configurations is also known, equivalently, the sphere packing problem (Conway et al., 1999) and the Tammes problem (Tammes, 1930), and its general form is hard, with solutions found only for special values of n and m. Approximations can be found with continuous optimization methods, which also allow us to tackle multi-task objectives, where we seek to balance dispersion with some other task of interest. Tn order to develop dispersion regularizers that can be efficiently optimized, we will first review the traditionally used measures to quantify disporsion in the theoretical contexts listed.
The Tammes problom (Tammes, 1930) is the problem of finding a configuration that maximizes the minimum distance between all pairs of points. A natural measure of dispersion would then be:
Minimum distance.
duai (X) = min d(x;,
(1)
ISifiSh
where d is a distance. In the sequel we will always define dmin in terms of the geodesic distance dş. As a metric, however, dmin may be less useful for comparing dispersion of approximate solutions, as it cannot distinguish between a configuration with n - well-dispersed points and one e-close from another, from a configuration where all points are e-close.
Spherical variance. Seeing the points X as vectors in RR" we may consider their average p (Ei æ;) /n. This average is a vector lying inside the unit ball, and its length, sometimes called mean resultant length, gives variance estimator studied in directional statistics (Jammalamadaka & Sengupta, 2001; Mardia, 1975):
n
svar(X)
|| where /
Tz.
(2)
If the points are highly concentrated, a will be close the surface of the sphere, and so svar(X) would be close to zero. When points are dispersed, their directions cancel out, / is close to the origin, and so svar(X) is close to 1. Since y must be in the unit ball, svar(X) must be between and 1.
3

---

Published in Transactions on Machine Learning Research (08/2025)
Unlike dmii: configurations maximizing not necessarily well dispersed: the spherical variance of X = (r, -t, -x) is 1, but the minimum distance is 0 for any Nevertheless, its computational efficient. and highly global nature make it a useful metric, in ways that can complement minimum distance.
2.4 Optimizing for Dispersion
In this section review some of the methods for numerical optimization of dispersion in previous work. Such methods almost exclusively pairwise in nature, leading to at least quadratic complexity. Along the way, we make some new connections between the methods.
Closest-point objectives. It may be tempting to optimize dmin(() directly, as this is definitional measure of dispersion. However, the gradient of (X) is zero for almost all points, with the exception of the closest pair," 1 leading to slow convergence, as pointed out by Mettes et al. (2019) and Liu et al. (2018). An alternative to consider the closest point to every point in the configuration. The resulting max-min (MM) objective appears in several works:
(MM(X)
1 n
min d( (z) 372
(3)
Mettes et al. (2019) use cosine distance (equivalent to ) while Wang et al. (MMA, 2021) use ds. Plugging in a log-distance instead a distance gives the dispersion method proposed by Sablayrolles et al. (2019):
n log min d(x;, j
LKoLeo (X)
(4)
TI
which is to a constant the Kozachenko-Leoneekke estimator of differential entropy (Leonenko, 1987). Assuming the distance is the same, the Kozachenko- Leonenko estimator and the max-min distance are related by the following bound which follows from the Jensen incquality and the fact that -e 1
dmin(X) < LxM(()
< - exp (o s (x)
55)
We also note that technically the Kewachenko- on a lower-dimensional submanifold such as Sm: the differential entropy w.r.f. the Lebesgue measure on Rm is infinite. This issuc addressed in the estimator of Nilsson Kleijn (2007). Nevertheless, as Eq. (5) suggests, seen simply as an optimization objective, LooLeo has the intended dispersion effect.
estimator is not applicable when the data is supported
Energy and kernel methods. While the Tammes problem is defined in terms of the minimum distance, related problem is minimize an energy of the form:
Σ LMHE, n(n ISijjen
1
k(z,,
(6)
MHE stands for minimurm hyperspherical energy (Liu el al., 2018).² 2 When & is the Riesz 1-kernel with Euclidean distance, e.e k(I,z') = (=(,,x')=11 LMHE, (X) corresponds to minimizing the electrostatic energy between n electrons, and its minimization known as the Thomson problem (Thomson, 1904). This energy objective is minimized in a nunber of works on dispersion (Gautam & Vaintrob, 2013; Liu et al., 2018; 2021). Similarly, Wang & Isola (2020) propose related objective using the RBF (also known as Gaussian) kernel k(04,2y)) exp (7(1,,))) in the overall expression
n ΣΣ
Lwi(X) = loog n²
(7)
In the case of ties, function is differentiable. Picking any clasest pair ignoring tles gives subgradient. 2ame definitions omit. the scaling term n(n-1). 1).
4

---

Published in Transactions on Machine Learning Research (08/2025)
Denoting t(X) Eg k( 2t); we thus have exp(Lwi(X)) = (1 1/n2) (LMHE, + x(X)- For many kernels (including RBF, but not Riesz with 2 0), ((X) is a finite constant independent on X and thus, as exp is monotonic, the two objectives are optimization-eqqivaleee While in general such energy minimization objectives lead to different optimal solutions, some relationships to the Tammes problem exist. For example, using the Riesz kernel with 8 + 20, or the RBF kernel with 7 - 00, are equivalent to the Tammes problem in the limit (Appendix A.1).
We shall revisit kernel properties in $3.1.
Evaluating kernel energy requires the entire (or at least the upper triangle of the) sample Gram matrix K with (K), k( 2y), so these methods have at least quadratic complexity, just like the closest-point methods above. Liu et al. (2021) also show that the determinant of K is a dispersion mcasure; nevertheless calculating determinants require diagonalization, which has cubic complexity in n.
Von-pairwise objectives. Spherical variance (Eq. (2)) requires only linear time to evaluate. However, it is unfortunately not suitable for gradient-based optimization, as Va avar(()= -x/2u]llll (it is in the direction of z), and thus grad, svar(X) = O for any point x part of any configuration X. Intuitively, svar wants to shrink all points toward the origin, which can result no optimization progress on the sphere. As a generalization of spherical variance, it is possible to define more generic manifold notions of variance using Fréchet expectations, e. min, Σ; dr), (Pennec, 2006), or variants using kernels instead of distances (maximum polarization, Liu et al., 2021). While avoiding explicit pairwise comparisons, such measures require iterative bilevel optimization, which poses additional numerical and computational challenges; moreover, for certain symmetrical optimally-dispersa situations any point on S., solves the inner optimization and thus higher-order gradients can be ill- behaved.
Optimal transport and spherical sliced Wasserstein. Seeing dispersion as a semi-discrete distance between an empirical distribution (supported on a set of points X) and a uniform distribution u, is tempting to turn to optimal transport (Villani et al., 2008; Peyré et al., 2019) and optimize (2 u) min ~x [d(X, Y)²], where Up, u) is the set of all joint measures with marginals matching p and In most casos, including the case of the sphere, this Wasserstein distance is intractable, but (Bonet et al., 2023) show closed-form special case on S with the geodesic distance. On the circle, collection of points can be identified by their angular coordinates 0, S 02 S ... < 6n with -T < 0g S and the Wasserstein distance against the uniform measure has the closed-form expression
2
n
1
W (p. u)
8
n 1+ 20)0,
T1
n
n2
12
Bonet et al. (2023) suggest the spherical sliced Wasserstein (SSW) distance, an efficient approximation over Sm by taking the expectation over the projections onto random great circles, similar to the more general idea of sticed Wasserstein in Euclidean spaces (Rabin et al., 2012), and they suggest that the distance to uniform may be used to optimize for dispersion. A great circle can be identified by a pair of orthogonal directions, and the projection is given by the following result.
Lemma 1 (Projection onto great circle.) Letp, p, q E 3m with (p, g) = 0. Two such vectors determine a unique great circle Spy C Sm defined by Spg = (cos(()p + sin(O)g - S 0 < ) ~ Sj. The nearest point on Spq to a given E Sm is given by projs,, (x) = arctan2 ( ((x, g). (x.p)).
As SSW was not experimentally compared with other dispersion strategies, we provide empirical results in our work. Moreover, in $3.1 we show that enseg regularizers can be seen as optimizing different distance between distributions: the MMD distance rather than the Wasserstein distance, and in $3.3 we use the slicing idea to derive from first principles related but somewhat simpler regularizer with good performance.
3
New Insights Into Optimizing for Dispersion
In this section, we give a new interpretation of the uniformity regularizer discussed in $2, in terms of (squared) MMD, and then define two novel dispersion objectives with more appealing computational properties.
5

---

Published in Transactions on Machine Learning Research (08/2025)
3.1 Energy-based dispersion and MMD
A promising way of thinking about dispersion comes from the statistical perspective of sample-based tests of uniformity. Indeed, spherical variance appears in the Rayleigh test for uniformity on the hypersphere Sm (Mardia & Jupp, 1999, 10.4.1), with test statistic ma]|1 svar(X)]I². Liu et al. (2018) show further connections between Ajue's test (Mardia & Jupp, 1999, 10.4.1) and MHE with a specific choice of kernel.
An alternative statistical test for uniformity can be derived from the marimum mean discrepancy (MMD), which is a nonparametric test measuring the distance between two probability distributions (Gretton et al., 2012). The next result shows that on Sm; for kernels defined in terms of angles, the squared MMD from the uniform distribution simplifies to familiar form.
Lemma 2 (MMD2 with the uniform distribution on Sm.) Let p be a distribution on Sm, and de- note by u = Uniform(Sm) the uniform distribution on S.n- Let ho be a positive definite kernel on Sm such that k(x, v) - [(x,, (I) for some funclion f: [-], + R, and F be the unil ball in the reproducing kernel Hilbert space (RKHS) corresponding to k.
Then, for ii.d rondom variables X, X' with distribution p, we have
MMD2 PP, u| = Exxx~p |K(X, X1]
( m 1 ( 2
S d( (3)mmm8)/ 3)/2 is a constant w.r. t. p.
where c B
2'
While the constant is irrelevant for learning,
note that for the RBF du kernel, it takes the value of the Langevin distribution normalizing constant relative to the uniform measure, i.e., (tm/2)(/22 1m/2-1111 (7) (Mardia & Jupp, 1999, 9.3.6). The MMD is generally faster to approximate than the Wasserstein distance as there is no minimization over couplings. The assumption of expressing kernels in terms of the cosine (a, y) applies broadly and is required for rotational symmetry, and is analogous to expressing kernels terms of distances, as the geodesic and Euclidean distances can both be given in terms of the cosine. The proof of Lemma 2 is provided in Appendix A. As direct consequence, we may interpret LMHE,A - c as an unbiased estimator of squared MMD against the uniform distribution on S..:
Proposition 1 Let p be any distribution on Sm and k a positive definile kernel with the property from Lemma 2. Then, LMHEEA - e in Eq. (6) is an unbiased eslimator of MMT2 belween p and the uniform distribution the sphere, with respect to the wnit ball in the corresponding reproducible kernel Hilbert space.
Proof. Combine Lemma with Lemma 6 of Gretton et al. (2012) and identify with Eq. (6).
Since the terms involving are handled in closed-form (or constant), this estimator also has more favorable variance properties compared to the standard MMD, which would be estimated by drawing samples from u. We provide central limit theorem and expressions for the variance in Appendix A.3.
For our application, we take to be an empirical dis- tribution supported at finite number of points X. Since the uniform distribution is not in this family, MMD cannot reach zero, but we still have meaning- ful notion of distance between X and the continuous uniform distribution. This result provides valid in- terpretation of kernel-based dispersion objectives like MHE and WI without requiring taking a limit toward infinitely many points, as needed in the motivation of Wang & Isola (2020).
2.0
RBF de
Laplace da
1.5
<.. Riesz da
Igrad, yil
RBF dy
Laplace de
Riesz d.
0.5
0.0
m2
angle and (radians)
Choice of kernel. The most commonly-usod kernels for dispersion are 1-Riesz &Riesz,==1 (z, y) = d(z, y) (due to its connection to the Thomson problem), and
Figure 2: Norm of the gradient of the kernel between points, as the angle between points varies.
6

---

Published in Transactions on Machine Learning Research (08/2025)
RBF ARBF,7 = exp(-xd(zz, On the sphere (with de), the RBF kernel is unfortunately not positive definite, while the Laplace kernel k (x, x') exp(->d(ar, x")) is (Feragen et al., 2015). Inspired by Wang et al. (2021, Fig. 2), which shows the gradient norm of MMA is constantly 1 regardless of the angle to the nearest neighbor, we compare in Figure 2 the gradient norms for various kernels as a function of the angle to the other point. We find the geodesic Laplace kernel to be best-behaved from this perspective, as it is bounded and monotonically decreasing. All kernels that use dg, and also the RBF kernel with dg, first increase for small angles before decreasing. We give a detailed table of these kernels in Appendix A.1, alongside known results for positive definiteness. While this figure only depends on the angle and thus is the same in any dimension, the curse of dimensionality says that as dimension increases, angles converge to n/2.. We explore the sensitivity (variance) of kernels in high dimensions in Appendix D.1.
3.2 Lloyd's Algorithm
An alternative formulation of dispersion can be reached by considering dispersion as quantization of uniform measure. Quantization is the problem of approxinating given measure by an empirical measure supported at few centers. Lloyd's algorithm (Lloyd, 1982), henceforth Lloyd, is a celebrated algorithm, introduced initially for quantization of uniform intervals in R for signal processing, but subsequently broadly applied, e.g. in machine learning and graphics. It is an iterative algorithm that alternates between two stages. First, for cach of the current centers, we find its Voronoi cell: the set of points in the domain closer to it than to the other centers. Afterward, all centers are replaced with the center of their corresponding Voronoi cell.
When the target measure is another empirical measure, quantization recovers k-means clustering; for continuous measures, it can be scen as an infinite generalization thereof. Both the discrete and continuous cases generalize readily to Riemannian manifolds with an adequate choice of distance (Le Brigant & Puechmorel, 2019). While Lloyd's algorithm and k-means are originally batch algorithms, stochastic gradient versions have been developed (Bottou & Bengio, 1995; Sculley, 2010), including, independently, in the Riemannian case (Le Brigant & Pucchmorel, 2019). In general, given a domain D, which could be a manifold or a compact subset of one (for quantization), or a discrete dataset (for clustering), the n optimal centroids are a minimizer of
Lioye (X) = Ey-Unif(D)
min ! dY jEin 2
(9)
A stochastic gradient of the Lloyd regularizer can be obtained by drawing m uniform samples on D. Intuitively, each cluster center is pulled toward the barycenter of the uniform samples assigned to it; an approximation to the true Voronoi barycenter.
For dispersion on the sphere, we take ID = S,n and d = dg- While traditionally Lloyd's algorithm corresponds regularizer to move X closer to optimal Voronoi
to minimizing Lloyd alone, we propose using Liooy centers of the sphere, while also minimizing some main task-specific objective. The complexity of this regularizer is controlled by the number of samples: For efficiency, m should be much less than n. Notice that umassigned cluster centers are not updated in an iteration, but the stochastic gradient of assigned centers implicitly depends on all other centers competitively through the cluster assignment. We additionally discuss convergence guarantees for Lloyd optimizer in Appendix C.
3.3 Sliced Dispersion
The previously discussed algorithms are generally applicable to other manifolds. We now show how using properties of the sphere we may obtain an alternative algorithm. We propose a construction similar to the idea used in spherical sliced Wasserstein, discussed in $2.4. While in three or more dimensions it is hard to find the location of n evenly distributed points, on the S₂ circle this can be done efficiently. The following set. of angles is one optimal configuration:
n + 1
2rk n n
i- (#), ... Qn) where Φ
3Te target measure need be uniform. Le Brigant & Puechmorel (2019) give general conditions for existence.
7

---

Published in Transactions on Machine Learning Research (08/2025)
Any other optimal configuration must be rotation of this one, e.e., + + for E (-TT, [-T,,,,, n), followed by a permutation of these angles. Given a permutation a E 1n denote 2,- = (do(1):..... (((()):..,,(((n)). We can then write the set of all possible ordered optimally-dispersed configurations as
pprajs (x)
Riemannian update
F
D,S2 (T T (- 1mn.
(10)
Given an ordered configuration of angles θ 0.) S2, we define its (angular) distance to the maximally- dispersed set as:
ar
df ((0,
min dD
2 (0: 0.)
Figure 3: Sliced dispersion update along a (11) great circle Spq.
Projecting onto DnS2 means finding the closest dispersed configuration to a given one. The following lemma shows how to efficiently compute this projection, in the time it takes to sort n angles.
Lemma 3 Optimal circular dispersion. The projection arg min E 1/2 (0, - 0.)² is given by 0- -* + d, 1(), where a is the permutation such that oa(1) - (2) S ... S (o(n) and r* = Σ, O:/n.
In arbitrary dimensions, a similar construction not possible, since the optimal configurations do not have tractable characterizations.
A well-dispersed configuration over S. should remain fairly well-dispersed along any slice average. If we denote projsp, pq (X) = (projs,, (z1),... .rrjjs, (In)), we may capture this intention by the following sliced dispersion regularizer:
Lstioed (X) EpQ (proisana (nSpa)]
(12)
where de is defined in Eq. (11). The joint distribution of P and Q is distribution great circles. A sensible choice is the uniform distribution over great circles, which can be sampled by orthogonalizing 2 x matrix with standard normal entries. An alternative is to sample from the discrete uniform distribution over axis-aligned great circles, e., p e,, q = with i.j drawn without replacement from {L..... m.; the latter choice allows slightly faster sampling and projection.
Note that unlike algorithms such as principal geodesic analysis (Fletcher el. al., 2004), which keep X fixed and optimize for p, p to maximize variance, our intuition is the opposite: update X to increase dispersion along all great circles. The following proposition efficiently computes stochastic gradients of Csticed.
by Proposition 2 Denote 0elpg proje,,, and 8) pq the corresponding dispersion marimizer computed using Lemma 3. The Faclidean and Riemanmian gradients of Lssicedd are equal and given by:
grad Lstioed (X) = Epo (ONloo - 10:00000pp P0) (x,, P)Q (I,, Q
(13)
(xi, Q) + (xi, p.2
All proofs in this section are provided in Appendix B.
Since the objective is defined as expectation, applying stochastic Riemannian gradient is straightforward and we have an unbiased estimator. However, we remark that the standard convergence argument for stochastic Riemannian gradient does not hold for Sliced or for SSW, as the gradient not everywhere bounded. This shared issue is due to the great circle projection (Lemma 1), whose gradient (the fraction in Eq. (13)) tends to infinity whenever some I; tends toward the space orthogonal to the great circle. (The projection of the nort pole to the equator is not uniquely defined). In high dimensions and/or low numerical precision this may trigger numerical issues, but we find that handling them heuristically (e.g., g., via clipping) works well.
8

---

Published in Transactions on Machine Learning Research (08/2025)
minimum angle (degrees
50
30
10
uniform init.
MHE Laplace (dg)
MHE RBF (dg)
MHE Riesz (dg)
KoLeo (da)
MM (d.)
Lloyd
Sliced
SSw
Figure 4: Minimum angles (degrees) for each of the 24 points. The horizontal line indicates the angle for known optimal solution; in an ideal solution, all angles would be on the horizontal line.
4 Applications
We demonstrate the application of dispersion objectives and provide a comparative analysis on both synthetic and real-world tasks.
4.1 Tammes problem
We evaluate the dispersion methods introduced in $2 and $3 by verifying that they can approximate the known solution to the Tammes problem for n = 24 in three dimensions (Robinson, 1961), by considering the minimum angle between points of the optimal configuration. Uniformly sampled points are dispersed using the regularizers described in $2 and $3. Starting from uniform initialization, apply 10k iterations of Riemannian Adam (Becigneul & Ganea, 2019) with learning rate 0.005. Laplace and RBF kernels 7y 1, Riesz kernels use = 1. Lloyd uses 300 samples from the uniforı measure, number of projections for SSW is set to 50, while the sliced method uses a single random great circle per iteration.
The pairwise methods with min-max objective (MM and KoLco) perform best as shown in Figure 4, unsurprisingly, as they are relsedt to the Tammes objective; they are followed by Lloyd and SSW/Sliced. Among the MHE methods, the Laplace kernel with geodesic distance works best here. For comparison, in Appendix E.3 perform this experiment with Euclidean projected Adam instead of Riemannian Adam, finding decrease in performance.
4.2 Synthetic Embeddings
In practice, mostly interested in dispersion of large amount of points in dimension d > 3. We evaluate the behavior of the regularizers discussed in $2 on m = 64, under two initial conditions: uniform samples (thus already somewhat spread out), and clumped points (sampled from a power spherical distribution with scale x 100) (De Cao & Aziz, 2020).
method
steps//sec (+ s.e.)
20k points in dimension
MHE (RBF, de)
79.40 +: 0.06
KoLeo (dz)
90.52 0.07
MM (ds)
91.29 + 0.06
Lloyd
90.98 + 0.07
Sliced
87.72 0.14
We optimize using Riemannian Adam for 5k iterations with learning rate 0.001. Laplace and RBF kernels use 7 Riesz kernels use = 1, and the Sliced regularizer uses axis-aligned great circles for elficiency. We set the batch size and number of samples to result in the same order of magnitude of compu- tations: for all pairwise objectives (MHE, MM, KoLco) we use minibatches of 512 points. For Lloyd, we also take minibatches
SSW (uproj=13)
53.78 +: 0.25
SSW (uproj=1)
81.16 1. 0.12
Table Timing (mean gradient steps per second) and standard error of the mean for the synthetic experiments.
of size 512, and draw 512 random uniform points to estimate the expectation. For Sliced draw 13 random
9

---

Published in Transactions on Machine Learning Research (08/2025)
Clumped (PS K 100) init
Uniform init
Laplace (dg)
0.90
MHE Laplace (dg)
MHE RBF (dg)
1.00
0.80
MHE Riesz (dg)
MHE Riesz (dy)
Spherical variance
KoLeo (dg)
1.00
(d.)
Lloyd
1.00
Sliced
0.60
SSWI
0.99
0.50
0.40
0.99
1000
3000
1000
2000
3000
4000
5000
0.93
Minimum geodesic distance
0.80
0.91
0.90
0.70
0.89
0.88
0.65
0.87
0.60
1000
2000
3000
4000
5000
1000
2000
3000
4000
5000
Number of gradient steps
Number of gradient steps
Convergence and performance of different dispersion objectives synthetic data with n 20 000, m = 64. Mean and its standard error over three random initializations.
Figure 5:
axis-aligned circles (as 13n 512?). For SSW we provide results with and 13 random circles, since the model with 13 random circles shows instability in the clumped case and diverges. We report spherical variance and minimum distances in Figure 5, and additionally measure timings in Table 1.
In contrast to the small-scale case of the Tammes problem, we find the choice of kernel and distance in MHE to not be empirically important in this large-scale regime. We find somewhat different trends the clumped versus the uniform initialization scenarios. The clumped case roughly corresponds to a optimization alongside strong downstream objective that pushes against dispersion. In this case, we find that MHE and Sliced perform best, with the latter making faster progress initially. In the uniform scenario, when the points are already well-separated, MHE and Sliced remain strong in terms of spherical variance, but the MM-style regularizers are the only ones that manage to make progress in terms of dmin- In terms of speed (Table 1), even after balancing for amount of computation, we find MHE to be slower than the others.
Overall, these results suggest that Sliced is a contender for scenarios where initial dispersion is poor and fast progress wanted, while MaxMin objectives scem preferable after some dispersion has already been achieved. We include additional results for varying dimensionality in D.2. Of course, dmia and svar do not necessarily correlate with improvement downstream task metrics; the next sections are dedicated to such applications.
10

---

Published in Transactions on Machine Learning Research (08/2025)
4.3 Image Classification with Prototypes
Mettes et al. (2019) showed that learning protolypes with dispersion encouraged by minimizing the maximum cosine similarity on hypersphere improves classification results on ImageNet-200 (Le & Yang, 2015). In our naming system, this objective is LuM with dz- We first show in Table 2 that applying Riemannian
prototypes
50
100
200
Acc.
dmin
Acc.
dmii
Acc.
dmin
MM (de, projected)
41.13+0.31
1.22
43.39+0.46
1.36
43.43 43.4340.28
1.44
MM (d))
42.38
1.46
43. 43.27+0.31
1.52
42.88+0.22
1.56
KoLeo (dx)
41.72+00.9
1.37
42.93
1.44
42.5540.18
1.49
MM (dg)
42.11 10.28
1.39
43.23
1.46
42.73 42.731
1.51
MHE (Riesz, ds)
40.9
1.41
42.53+0.22
1.56
33.70+0.55
1.58
MHE (RBF, dz)
43.37 0036
1.22
42.64
1.57
33.3640.83
1.58
Lloyd
41.58
1.20
42.43+0.03
1.30
43.07
1.35
Sliced
40.91 10.11
1.10
42.68
1.20
42.62
1.33
SSW
40.44
1.08
42.34
1.18
43.01
1.29
Table 2: ImageNet-220 classification accuracy, Prototypes are trained with different separation conditions. In bold we emphasise the best accuracy in column. MM (dh, projected) is the objective of Mettes et al. (2019) optimized with Euclidean projected gradient; all other rows use Riemannian optimization.
gradient descent on the sphere, rather than projected gradient as done by Mettes et al. (2019), leads to the better class separation, and as result. better classification accuracy. Second, we compare the classification accuracy given the prototypes trained with different dispersion objectives discussed in $2 and $3. We use Riemannian gradient descent for all except the projection baseline; we use 7 = 1 for RBF and = 1 for Riesz kernels, 200 uniform samples in Lloyd, and single sampled great circle for Sliced. Also, Table 2 shows that, for m 50, MHE with the RBF kernel performs the best among all dispersion objectíves considered, even though the minimum distance is smaller compared to other pairwiseddistance based objectives. It proves that even though we can measure the dispersion using minimum distance, we cannot rely on this metric alone as a predictive factor of the downstream task accuracy.
Interestingly, when dimensionality equal to the number of points, MHE results degrade noticeably, as shown in Table 2. For both kernels considered, the minimum distance and median distance is approximately 1.57 radian or 90.3", which is close to an orthogonal solution. Orthogonality is suggested by Mettes al. (2019) to be insufficient despite being optimally dispersed.
4.4 Neural Machine Translation
Embeddings learned with the vanilla transformer model (Vaswani al., 2017) are known for their inefficiency in utilizing space effectively, leading to the collapse of token representations (Gao et al., 2019; Wang et al., 2020). This issue is particularly pronounced for rare tokens (Gong et al., 2018; Tokarchuk & Niculae, 2024; Zhang et al., 2022). Gong et al. (2018) proposed to alleviate the problem of rare tokens by learning frequency- agnostic embeddings, while Zhang el al. (2022) proposed to use contrastive learning. In our approach, we tackle this challenge by focusing on the concept of dispersion. Specifically, we train a neural machine translation (NMT) system and optimize the transformer weights W along with a dispersion regularizer on the decoder embeddings X (tied between the input and the output decoder layers):
C(W,)) LMT (M, + ALdisp(X),
(14)
where LMT is a standard conditional log-likelhood loss of the target sentence given the source sentence accumulated over a dataset (e.g., Bahdanau et al., 2015). We perform stochastic training: for every LMT minibatch gradient calculation, we calculate an (independent) stochastic gradient of the regularizer. In particular, for MHE, we estimate the dispersion gradient over a minibatch of 1000 tokens drawn from the whole vocabulary. For Lloyd use 100 uniform samples from the sphere, while for Sliced sample
11

---

Published in Transactions on Machine Learning Research (08/2025)
ro-en
en-de
model
BLEU
COMET
dmia
BLEU
COMET
dmin
Euclidean baseline
31.5
0.790
0.003
0.19
33.1
0.819
0.000
0.000
spherical baseline
32.2
0.793
0.001
0.57
33.7
0.826
0.001
0.408
+MHE (RBF, da)
32.3
0.795
0.001
0.56
33.9
0.825
0.001
0.410
+ +Looyd
32.4 "
0.791
0.001
0.60
33.4
0.822
0.001
0.414
+Sliced
32.3
0.795
0.435
0.99
33.5
0.821
0.222
0.999
Table 3: newstest2016 ro-en and en-de results discrete NMT. Embeddings 128 dim. All results marked with are statistically significant (p < 0.05) over baseline.
embeddings
t
dmin
BLEUt
pretrained R (normalized)
0.191
0.014
28.3
+offline MHE (RBF, da)
0.599
0.372
29.7
+offline Lloyd
0.585
0.004
27.7
+offline Sliced
0.979
0.106
29.6
pretrained 8m
0.573
0.001
29.9*
+MHE (RBF, da)
0.561
0.001
30.0"
+Lloyd
0.596
0.001
30.1
Sliced
0.999
0.435
30.0
Table 4: Dispersion impact on CoNMT results. We report BLEU scores on the newstest2016 for ro-en. Beam size is equal to 5. All results marked with are statistically significant (p 0.05) over baseline.
great circle, with no additional subsampling. We use y = 1 for RBF. The remaining NMT training settings are standard and provided in Appendix E.1. We report results on two WMT translation tasks: WMT 2016 Romanian--Fnglish (ro-en) with 612K training samples and WMT 2019 English++Gerin
(en-de) with 9.1M training samples (including back-translated data). We measure translation accuracy on the best checkpoint according validation BLEU score using SacreBLEU (Papinení et al., 2002; Post, 2018) and COMET (Rei et al., 2020).
Table 3 shows the BLEU and COMET results on newstest2016 for ro-en and 2016 en-de along with the dispersion metrics. Surprisingly, simply training the transformer with Riemannian gradient updates for X (and standard Euclidean gradients for other parameters) alone already improves dispersion and translation performance over the baseline. Additional dispersion regularization further improves dispersion and BLEU, but not always COMET.
To shed more light into the dispersion effect of Riemannian optimization, we plot the relationship between token frequency and gradient norm (Figure 6a) minimum pairwise distance (Figure 6b) in the late stage of training. While in the Euclidean model the embeddings of rare words can collapse close to their neighbor, the Riemannian model prevents this. The gradient norm rare words an order of magnitude higher with Riemannian gradients than with Euclidean gradients. We hypothesize that this increased gradient norm contributes better dispersion of rare tokens, thereby mitigating representation collapse. The training dynamics of gradient norms and minimum distances along the carlier training stages, shown in Appendix E.2, support this intuition.
4.5
Continuous- Output Neural Machine Translation
Contimuous-output NMT (CoNMT, Kumar & Tsvetkov, 2019) reformulates machine translation sequential continuous regression problem of predicting the embedding of the next word, instead of the more usual discrete classification formulation. Tokarchuk & Niculae (2024) recently showed that dispersion plays an important role and greatly impacts performance. We follow closely their setup and apply the dispersion
hntpps//wwv2... sww.at.tttt....
12

---

Published in Transactions on Machine Learning Research (08/2025)
10-3
Spherical Euclidean
1.0
Spherical Euclidean
Gradlent Norm
10
0.6
10-4
Min.
0.4
0.2
10
0.0
5000
10000
15000
20000
25000
5000
10000
15000 Frequency Rank
20000
25000
Frequency Rank
(a) Norm of the gradient of the embedding.
(b) Minimum de distance to the nearest embedding.
Figure 6: Gradient norms (a) and minimum distances (b) w.r. t. NMT embeddings, after 40k steps, for the standard (Euclidcan) transformer and the spherical version where decoder embeddings are optimized on Sm- Frequency rank is the index of the token in a sorted vocabulary: higher means less frequent.
regularizers in order to achieve dispersion. Pre-trained embeddings come from the well-trained discrete models from $4.4 and are frozen while the continuous model is trained from scratch. We present results for WMT 2016 ro-en with 612k training samples. Table 4 shows the BLEU score results on newstest2016 for CoNMT models with different larget embeddings X, alongside dispersion measures defined in $2.3
As baselines, we include (normalized) target embeddings from the best Euclidean discrete model (labeled pretrained RT"). For offline dispersion we optimize starting from the pretrained embeddings for 500 epochs, using batch size 2000 in MHE, RBF 7 = 1, 100 uniform samples in Lloyd, and single sampled great cirele for Sliced. Second, as have already trained discrete NMT models with spherical dispersion in $4.4, we use the resulting embeddings for CoNMT (rows labeled pretrained S.. ). All CoNMT models were trained for 50k steps. Spreading out the projected embeddings generally results in BLEU score improvements. For all dispersion regularizers, we can see that svar is increasing. However, duan decreases for Lloyd, which may explain the lower BLEU score. Consistent with the synthetic results ($4.2), the Sliced regularizer is best at quickly increasing svar.
4.6 Discussion
Pairwise objectives are good choice when the full matrix of pairwise distances can be calculated efficiently, e., when the number of datapoints and/or dimensionality are relatively small. In practical MI. applications, dimensionality and number of points are typically large, as in $4.4 and $4.5, and given the limitations of computational resources we can either calculate pairwise distance for random batch of datapoints use alternatives like Lloyd and Sliced, since they are more computationally efficient as shown in Table 5.
5 Additional Related Work
method
complexity
A prominent related but distinct construction is that of determinantal point processes (Kulesza & Taskar, 2012), often used to ensure diversity or dispersion in extractive models. Given a set of n points, in the scope of our paper we study optimizing the locations of the points; in contrast, finite DPPs are intended for selecting subsets of k < such points in way that en- courages dispersion of the selected points. Technically, DPPs induce a probability distribution over all subsets. Like the pairwise regularizers we study in $2, DPPs form an n x n kernel matrix of pairwise similaritics. In order to draw samples, or evaluate the probability of given subset, an eigendecomposition (or equivalent)
pairwise
(n"m)
Lloyd
(mm)
Sliced / SSW
O(nm - sort(n))
(axis-aligned)
O(n sort(n))
Table 5: Computational complexity of the disper- sion regularizers for n samples in S.. For Lloyd, Sliced, and SSW the complexities are w.r. I. single Monte Carlo sample and complexity scales linearly in case of multiple samples. sort(n) is the complexity of sorting a list of length n, í. e.e., O(n) with radix sort. and On log n) with comparison sorts.
13

---

Published in Transactions on Machine Learning Research (08/2025)
of the kernel matrix is necessary, leading to O(n2) time complexity for such operations. Maximization, i. e., finding optimally-disperseed subset under DPP, is NP-hard to solve exactly, but approxinations may be worth exploring. Contimuous DPPs can be defined on compact sets (Lavancier et al., 2014), including on Sm (Møller et al., 2018). Relatedly, numerical integration it is essential to find good quadratures or discretizations of continuous measures. Methods include Sobol sequences (Sobol, 1967), defined on unit. hybercubes, and kernel herding, for arbitrary measures (Bach et al., 2012). Particle systems such as Stein variational gradient descent Liu & Wang (2016); Liu & Zhu (2018) can be used to sample from uniform measures. Sisouk et al. (2025) discuss various strategies for sampling on Sm while ensuring good coverage. They show that orthonormal sampling (Rowland et al., 2019) specifically is the best in high-dimensional scenario. However, sampling is not directly applicable in a regularization application where a task-specific objective also present. Nevertheless, further connections in this direction seem worth developing.
Dispersion is also closely connected to contrastive learning (Chen et al., 2020a; He et al., 2020; Hjelm et al., 2019; Chen al., 2020b), where model outputs corresponding different classes are pushed away from each other. Wang & Isola (2020) in particularly showed that widely used contrastive learning objective can be interpreted in terms of "alignment" (similar features for similar samples) and "uniformity" (feature distribution is close to uniform distribution). In our work we focus on paramcter dispersion, which can more easily be quantified.
Marbut et al. (2023) explore other ways to quantify dispersion in Euclidean space, and studied their consequences for word embeddings. Exploring spherical extensions thereof direction for future work.
6 Conclusion
In this work, we evaluate several dispersion objectives on the hypersphere, including one that is equivalent the widely used maximum mean discrepancy (MMD) method, as well two novel approaches: Lloyd and Sliced. We compare these objectives against various pairwise distance-based methods previously explored in the literature. Our experimental results show that these methods can approximate the Tammes problem solution, and also allow improvement on few-shot image classification with prototypes, machine translation and the CoNMT tasks, which uses cosine distance both for training and decoding.
Acknowledgments
This work is supported by the Dutch Research Council (NWO) via Venii.212222 The authors also thank SURF www.ssurf.l)) for the support in using the National Supercomputer Snellius. Finally, we thank all the members of the UvA Language Technology Lab for their valuable foedback on our work, with special thanks to Sergey Troshin for his input on the manuscript.
References
Kavosh Asadi and Michael L. Littman. An alternative softmax operator for reinforcement learning. In Doina Precup and Yee Whye Tch (eds.), Proceedings of the 34th International Conference Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 243-252. PMLR, 06-11 Aug 2017. URL https://proceedinn mlr. www.mmlw....///ie.. wwwppp/ws////eeee 20770/aaai111 htmll.l..
Francis Bach, Simon Lacoste-Julien, and Guillaume Obozinski. On the Equivalence between Herding and Conditional Gradient Algorithms. In ICML 2012 International Conference on Machine Learning, pp. -, Edimburgh, United Kingdom, June 2012. URL https://hal www.nccehal--000111111 1tt
Damitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations, January 2015.
Gary Becigneul and Octevian-Eugen Ganea. Riemannian adaptive optimization methods. In International Conference Learning Representations, 2019. URL. http:p////gggaeeeeeeeeeee//aaaaa
14

---

Published in Transactions on Machine Learning Research (08/2025)
Dmitriy Bilyk and Peter Grabner. Positive definite singular kernels on two-point homogeneous spaces, 2024. URL
22104.
Clément Bonet, Paul Berg, Nicolas Courty, François Septier, Lucas Drumetz, and Minh Tan Pham. Spherical Blicod-Wasserstinn. In The Eleventh International Conference on Learning Representations, 2023. URL
https://openreeiew..u wttp://www....../ooooorrii///iid
Silvere Bonnabel. Stochastic gradient descent on Riemannian manifolds. IEEE Transactions on Automatic Control, 58(9):2217 2229, 2013.
Léon Bottou and Yoshua Bengio. Convergence properties of the KMeans algorithm. In Proc. of NeurIPS, 1995. URL http: http://eoon..ttttto org. wttpp/paprs///ttttt--aaaaa-ooooa
Wacha Bounliphone, Eugene Belilovsky, Matthew B. Blaschko, Ioannis Antonoglou, and Arthur Gretton. A test of relative similarity for model selection in generative models. CoRR, abs/1511.04555 2015. URL
batps://aphl hetppissemmaiccccolleeee sttaaaaiiiccool....ooooooooosaao
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Internalional conference machine learning, pp. 1597-1607. PMLR, 2020a.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33: 22243 22255, 2020b.
Henry Cohn. Table of spherical codes. https://dspace smptppacc..ii...........eeaeette mit wwwpe/eaaeeee/11.1///////// 1/1117221.1111553311 2024. Accessed: 2024-05-28.
John Horton Conway, Neil James Alexander Sloane, and Elichi Bannai. Sphere-packings, lattices, and groups. Grundlchren der mathematischen Wissenschaften 290. Springer, New York, 3rd ed edition, 1999. ISBN 0387985859.
Ludwig Danzer. Finite point-sets on 52 with minimum distance as large as possible. Discrete Mathematics, 60:3 66, 1986. ISSN 0012-365X. doi: //www www.sceirecccrrc....m wtt./c/ceeee//rrriiiee..ooe
URL https:
Nicola De Cao and Wilker Aziz. The power spherical distrbution. Proceedings of the 37th International Conference on Machine Learning, INNP+, 2020.
László Fejes Tóth. Über eine Abschätzung des kürzesten Abstandes zweicr Punkte eines auf einer Kugelfläche ialllemnlkker-VVeeenninnn Vereinigung, 53:66 68, 1943. URL
liegenden Punklsystemss Jahresberichl der Dealschen Mathemaliker- http://dml dml mathdoc fectthoo..rritttee wtt.:ffftttttteeeeeeeeeeeeeeee /ttpa//aaa.///////////////////1/
Aasa Feragen, Francois Lauze, and Soren Hauberg. Geodesic exponential kernels: When curvature and linearity conflict. In Proceedings of the IEEE conference on computer vision and pattern recogrition, pp. 3032 3042, 2015.
P Thomas Fletcher, Conglin Lu, Stephen M Pizer, and Sarang Joshi. Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE trunsactions on medical imaging, 23(8):995-100 1005, 2004.
Daniel Gabay. Minimizing a differentiable function over a differential manifold. Journal of Oplimizalion Theory and Applications, 37:177-219, 1982.
Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tieyan Liu. Representation degeneration problem in training natural language generation models. In International Conference on Learning Representations, 2019. URL /tttps://pperreviiee http:///ppeeeeeeeeeeeeeoro//n/ee
Simanta Gantam and Dmitry Vaintrob. novel approach to the spherical codes problem. In MIT, Cambridge, MA, USA, Tech. Rep, 2013. URI. https::/ semanticscholla . wowp:/oooprs/./ 126477839.
15

---

Published in Transactions on Machine Learning Research (08/2025)
Chengyue Gong, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tic- Yan Liu. Frage: Frequency-agnostic word representation. Advances in neural information processing systems, 31, 2018.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723 72237)3733 2012. URL http://jmlr. org/ hatpars/v13/geettoo2222 html.
Godfrey Harold Hardy, John Edensor Littlewood, and György Pólya. Inequalities. Cambridge University Press, 1952.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual reprosentation learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9729-9733 2020.
R Devon Hjelm, Alex Fodorov, Samuel Lavoie-Marchildeee Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In International Conference Learning Representations, 2019. URL https:// http:///ppprrrreeeeeffomrmm net/forum? id-Bx1r3jOcKX.
Wassily Hoeffding. Probability inequalitics for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13 30, 1963. ISSN 01621459, 1537274X. URL http://www.jstott .rgg// stable/2282962.
S Rao Jammalamadaka and Ambar Sengupta. Topics in circular slatistics. Series on multivariate analysis; vol. 5. World Scientific, Singapore, 2001. ISBN 9810237782.
Taku Kudo and John Richardson. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing In Eduardo Blanco and Wei Lu (eds.), Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pp. 66 71, Brussels, Belgium, November 2018. Association for Computational Linguisties. doi: 10. 10.18833vv/DDL 2012. URL http:sss//aclanttllgooorr///////
Alex Kulesza and Ben Taskar. Determinantal point processes for machine learning. Foundations and Trends@ in Machine Learning, ((2-3):123 286, December 2012. ISSN 1935-8237, 1935-8245. doi: 10.1561 10.1561 /2200000044 URL
Publisher: Now Publishers, Inc.
Sachin Kumar and Yulia. Isvetkov. von Mises-Fistt Fisher loss for training sequence to sequence models with continuous outputs. In International Conference on Learning Representations, 2019. URL https:// openreview.net/forre www.://orrumiidrrloooooo..
Frédéric Lavancier, Jesper Møller, and Ege Rubak. Determinantal point process models and statistical inference. Journal of the Royal Statistical Society Series B: Statistical Methodology, 77(4):853-80 877, 12 2014. 12096. URL https://doi. org/10 http:/10.1111//11/ 1111/rssb. 12096.
ISSN 1369-7412. doi: oet.attttttettaooroooooorrrraaar
Ya Le and Xuan S. Yang. Tiny ImageNet visual recognition challenge. Technical report., 2015. URL https://apt: sea.tticcsaaatt.cclll...rrr.rrr.
Alice Le Brigant and Stéphane Puechmorel. Quantization and clustering on riemannian manifolds with an application to air traffic analysis. Journal of Multivariate Analysis, 173:685 703, 2019. ISSN 0047-259X. doi: www.///ciene//rttaeeeeeeeeeeeeee article/
https:///do..or/11..00000 j0t.///0..00.....a..0000011 pJ1//50047290X18033311
hmwma.019.00000000 URL
Nikolai N Leonenko. Sample estimate of the entropy of a random vector. Problemy Peredochi Informatsti, 23
(2):9-16, 1987.
Alain Lichnewsky. Minimisation des fonctionnelles définies sur une variété par la methode du gradiënt confugué. PhD thesis, These de Doctorat d'Etat. Paris: Université de Paris-Sud, 1979.
Chang Liu and Jun Zhu. Riemannian stein variational gradient descent for bayesian inference, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
16

---

Published in Transactions on Machine Learning Research (08/2025)
Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural informalion processing systems, 29, 2016.
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Te Song. Sphereface: Deep hypersphere embedding for face recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Weiyang Liu, Rongmei Lin, Z. Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song. Learning lowards minimum hyperspherical energy. In Neural Information Processing Systems, 2018. URL https: //api. semanticscholar ptg/CorppuIDD 43921092.
Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Schölkopf, and Adrian Weller. Learning with hyperspherical uniformity. In Arindam Banerjec and Kenji Fukumizu (eds.), Proceedings of The 24th International Conference on Artificial Intelligence and Slatistics, volume 130 of Proceedings of Machine Learning Research, pp. 1180 1188. PMLR, 13 15 Apr 2021. URL https: https://proceeding/ mlr prres// v130/1iu21d, html.
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726-742, 2020. doi: 10.1162/tacl a 00343. URL
hatps////c/aa 1tt.ao/gg.rr///0...a1111........ tac1-1.47
Stuart P Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2): 129,137, 1982. doi: 10.1109/TIT. 19822106688992115548888
David G. Luenberger. The gradient projection method along geodesics. Management Science, 18(11):620 631, 1972. ISSN 00251909, 15265501. URL http://www.jsttet arg/stablee262215..
Anna Marbut, Katy McKinney-Bock, and Travis Wheeler. Roliable measures of spread in high dimensional latent spaces. In International Conference on Machine Leurning, pp. 23871 23885. PMLR, 2023.
Kanti V. Mardia and P. E. Jupp. Directional statistics. John Wiley & Sons, Inc., 1 1999. doi: 10.1002/ 9780470316979. URL https: http:///...rr//1.../////////0//0
Kantilal Varichand Mardia. Statistics of directional data. Journal of the Royal Statistical Society Series B: Statistical Methodology, 373333349-371, 1975.
Yu Meng, Jiaxin Huang, Guangyuan Wang, Chao Zhang, Honglci Zhuang, Lance Kaplan, and Jiawei Han. Spherical lext embedding. In Advances in neural informalion processing systems, 2019.
Pascal Mettes, Elise der Pol, and Cees G M Snoek. Hyperspherical prototype networks. In Advances in Neural Information Processing Systems, 2019.
Jesper Møller, Morten Niclsen, Emilio Porcu, and Ege Rubak. Determinantal point process models on the sphere. Bernoulli, 24(2):1171 1201, 2018. ISSN 13507265. URL http://www.jsst ott.ooobrg/////ee/////// 26491978.
Oleg R. Musin and Alexey S. Tarasov. The strong thirteen spheres problem. Discrete and Computational Geometry, 48(1):128 141, 2012. doi: 10.1007/800554-01--9999 URL https://abi.orr//0.. s00454-014 011--011-9000 0119992-2..
Oleg R. Musin and Alexey S. Tarasov. The Tammes problem for N = 14. Eaperimental Mathematics, 24(4): 460-468, 2015. doi: 1022842.
URL attps://doi: wwt.://///...//.//////r/.. 2015.
Mattias Nilsson and W. Bastiaan Kleijn. On the estimation of differential entropy from data located on embedded manifolds. IEEE Transactions on Information Theory, 53(7):2330-255 2341, 2007. doi: 10.1109/ TTT TTT.2007.899533.
17

---

Published in Transactions on Machine Learning Research (08/2025)
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit. for sequence modeling. In Proceedings of NAACI-HLT 2019: Demonstrations, 2019.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin (eds.), Proceedings of the 40th Annual Meeling of the Associalion for Compulational Linguistics, pp. 311 318, Philadelphia, 1073135.
Pennsylvania, USA, July 2002. Association for Computational Linguisties. doi: URL attps://aclanthollee org/ 1040.
Xavier Pennec. Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measure- ments. Math Imaging Vis, 25:127-154, 2006. URL https://link www.kksspsinnscoeooo com/arttccee/1..1007 s10851-006- 6228-4.
Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and TrendsB in Machine Learning, 11(5-6):355 607, 2019.
Matt Post. A call for clarity in reporting BLEU scores. In Ondřcj Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Lucia Specia, Marco Turchi, and Karin Verspoor (eds.), Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 186-191, Brussels, Belgium, October 2018. Association for Computational Linguistics. doi: 10. 10.18653/v (1653//v//oo ///1186631. URL
.agg:o..g//io. org//18-63111
Julien Rabin, Gabriel Peyré, Julie Delon, and Marc Bernot. Wasserstein barycenter and its application to texture mixing- In Alfred M. Bruckstein, Bart M. ter Haar Romeny, Alexander M. Bronstein, and Michael M. Bronstein (eds.), Scale Space and Variational Methods in Computer Vision, pp. 435-446, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg. ISBN 978-3-642-247855
Ricardo Rei, Craig Stewart, Ana Farinha, and Alon Lavie. COMET: neural framework for MT evaluation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), ) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2685 2702, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.eer 12066//////20000mmpmmmmmm..... URL https://aclaanthology org/2020. emnlp- maap..mmain......m 213.
Raphael M Robinson. Arrangement of 24 points on a sphere. Mathemalische Annalen, 144:17-48, 1961. URL http://eudml/ httpall.o/////////..7././
Mark Rowland, Jiri Hron, Yunhao Tang, Krzysztof Choromanski, Tamas Sarlos, and Adrian Weller. Orthogonal estimation of wasserstein distances. In Kamalika Chaudhuri and Masashi Sugiyama (eds.), Proceedings of the Twenly-Secom International Conference on Arlificial Intelligence and Stalistics, volume 89 of Proceedings of Machine Learning Research, pp. 186-195. PMLR, 16 18 Apr 2019. URL https://proceeddngg mlr.. press/v89// https//oolaad1eee htmm.
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, and Hervé Jégou. Spreading vectors for similarity search. In International Conference on Learning Representations, 2019. URL https://openreview.nnt net/
forum?id-SkhuG2R65
Joseph Salmon and François-David Collin. Soft-max and Soft-argmax, 2024. URL https::/ eu/blog/soffrn soft////.
D Sculley. Web-scale K-mcans clustering. In Proc. of WWW, 2010.
Robert Serfling. Approzimation theorems of mathematical statistics. John Wiley & Sons, 2009.
Keanu Sisouk, Julie Delon, and Julien Tierny. A user's guide to sampling strategics for sliced optimal transport. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL /ttps://openrevier. het//Eorrumm.ddEECllgTTTTEEE Survey Certification.
18

---

Published in Transactions on Machine Learning Research (08/2025)
Ila M Sobol. Distribution of points in a cube and approximate evaluation of integrals. USSR Computational mathemalics and mathemalical physics, 7:86-112, 1967.
Danica J. Sutherland. Unbiased estimators for the variance of mmd estimators. ArXiu, www.ssemaaiicoolla........ Seemaaaiiicccol.....o////oooaop :10orppu 174799385.
2019.
URL
Pieter Merkus Lambertus Tammes. On the origin of number and arrangement of the places of exit on the surface of pollen- grains. PhD thesis, University of Groningen, 1930. Relation: De Bussy.
Rights:
JJ Thomson. On the structure of the atom: an investigation of the stability and periods of oscillation of number of corpuscles arranged equal intervals around the circumference of circle; with application of the results to the theory of atomic structure. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 7(39):237 265, 1904. doi: (rgbi..og//i 10. 1/t/:/10.100/1141111 1111000/117764449946811111
URL https:
Evgeniia Tokarchuk and Vlad Niculae. The unreasonable effectiveness of random target embeddings for contimuous-output neural machine translation. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapler of the Associalion for Compulational Linguistics: Human Language Technologies (Volume 2: Short Popers), pp. June 2024. Association for Computational Linguistics. doi: https: http:s///cclathhlloo.orrggg
Mexico City, Mexico, URL
mwacc-shorraaaaaaea 56.
Daniel Trosten, Rwiddhi Chakraborty, Sigurd Løkse, Kristoffer Wickstrom, Robert Jenssen, and Michael Kampffmeyer. Hubs and hyperspheres: Reducing hubness and improving transductive few-shot learning with hyperspherical embeddings. In Proceedings of the TEEE/CVE conference on compuler vision and pottern recognition, pp. 7527-7536, 06 2023. doi: 10.1109 /ttp//C/222229222...00020
Bartel Leendert van der Waerden and Kurt Schütte. Auf welcher kugel haben 5, 6, 7, 8 oder punkte mit mindestabstand eins platz? ? Mathematische Annalen, 123:96-12 124, 1951. URL http: //edd//e.mmmm///// org/oogg 160237.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Aduances in Neural Information Processing Systems, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Cédric Villani et al. Optimal transport: old and new, volume 338. Springer Berlin, Heidelberg, 2008
Trung Vu, Raviv Raich, and Xiao Fu. On convergence of projected gradient descent for minimizing a large-scale quadratic over the unit sphere. In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1-6 6. IEEE, 2019.
Lingxiao Wang, Jing Huang, Kevin Huang, Ziniu Hu, Guanglao Wang, and Quanquan Gu. Improving neural language generation with spectrum control. In International Conference on Learning Representations, 2020. URL
net/ffrumm www.orummmmdyyyyyyttt.....
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Hal Daumé IIT and Aarti Singh (ods.), Proceedings of the 97th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 9929 99999999..911 PMLR, 13 18 Jul 2020. URL https:://poooooo httpps://proceedings mmr. www.mmlrppp.e..... ppress/vi19/ 11199//ang0k.... html.
Zhennan Wang, Canqun Xiang, Wenbin Zou, and Chen Xu. Mma regularization: Decorrelating weights of neural networks by maximizing the minimal angles, 2021.
Wikipedia contributors. Atan2 - Wikipedia, the free encyclopedia. tttps://en.wwikipedia.e org/ wtg/w///gg w/index. (ttp:tteeeetttt2kooodddd222//-0
2024. [Online; accessed 25-Septemuber-2024]
Tong Zhang, Wei Ye, Baosong Yang. Long Zhang, Xingzhang Ren, Dayiheng Liu, Jinan Sun, Shikun Zhang, Haibo Zhang, and Wen Zhao. Frequency-aware contrastive learning for neural machine translation. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):117 Jun. 2022. doi: 10. 10609//aai.v3611..o
URL https://ojs. aaai weteaaiiorg/inndeee http:///wa.aagp////////aaaaaaaaa
19

---

Published in Transactions on Machine Learning Research (08/2025)
Appendix
A Kernels and MMD Dispersion
A.1 Spherical Kernels and Relationships
In this section we give a convenient table of the expressions of the kernels used in this paper (Table 6), and additionally show the limit-case relationship to the Tammes problem.
name
d hyperpar.
expression k(x,y)
(conditionally) positive definite yes
RBF
E 00)
exp (7(x, 3))
RBF
S
? (0, 00)
exp (arccos (x, v))
no (Feragen et al., 2015)
Laplace
R
7 (0, 0o)
exp
722 2- (s, v)
yes
Laplace
S
E (0, oo)
exp (-7 exp(-7arccus arccos (I, 1/
yes (Feragen et al., 2015)
Riesz
S
8 E (0, oo)
(arccos (z. y))
cond., for < m - (Bilyk & Grabner, 2024, Cor. 2)
log arccos (z. y)
cond., (Bilyk & Grabner, 2024, Cor. 2)
Riesz
E (0, oc)
(2 (1r, u) =/2
cond., for < m 1 (Bilyk & Grabner, 2024, Cor. 1) cond., (Bilyk & Grabner, 2024, Cor. 1)
1/2 log 2 (z, y/)
Table 6: Expressions the kernels considered in this work and in related work on hyperspherical dispersion. In all cases, the domain is a sphere S The Fuelidean-distance RBF kernel is up constant equal to the one in ambient space. Riesz kernels for negative are sometimes defined K (2, y) -kiissz. (2,
Limit cases. First, let's consider the RBF and Laplace kernels. For simplicity let R denote the set of pairs of distinct indices from to n, and 2, be the cosine similarity between the rth pair of points in X. Then, the geodesic objective of the Tammes problem can be written (X) arccos (maxreR 2,) and since arccos is nondecrcasing, the optima of the Tammes problem thus also maximize maXren 2r. The "soft maximum" operator log-sum-exp satisfies a well-known limit property (Salmon & Collin, 2024; Asadi & Litlman, 2017). Denoting x, = max,e 2, and assuming no ties,
lim 7 log Σ exp((22) = lim 7 log Σ exp((((z 2, tz)) 7+0
7-00 rER
rell
lim
log
exp(12.)
Σ FER
exply (2r-
(15)
70000
<0
Since log-sum-exp is continuous, presence ties, the same limit holds as the ties can be broken by some going zero. Introducing or not normalization of [ before the sum does not change this result, - log R - log 7 -1 log [t] and the latter term goes to zero in the limit. Moreover, any strictly monotonic transformation of the cosine similarities also preserves the optima, which means all versions of the RBF and Laplace kernels shown in Table 6 are equivalent to Tammos the limit of y - cc. The limit case of the Riesz kernel as s - oo can be proven similarly and is related to the definition of the || Iloo norm. Let's redefine now 2, 1/d, be the inverse of a distance (so still a similarity):
1/s
Zr
lim 300
2
24
lim
1+
(16)
80000
Z+
FER
rEml
The MHE regularizer with Riesz kernel as + oo unbounded, but its sth root converges an objective equivalent to the Tammes problem.
20

---

Published in Transactions on Machine Learning Research (08/2025)
A.2 MMD and spherical uniform densities: Proof of Lemma
The squared MMD of two probability distributions p and q is equal (Gretton al., 2012, Lemma 6)
MMD"(p = Exxxxp((XXXx) 2Bxxp, (K(, Y) Byy,,
We show that the last two expectations constant, when p is distribution the hypersphere Sm and g is (n Let. z, E Sm and let Q be rotation madrix such that Q= ='=... Note that Y ~ Uni((Sm) if and only if QTY ~ Unif(Sm), and (Qz, 2) = QT,.).)...))))))) It then follows that
Ey-tnif(s,.) [k2 Y)] = Uu.-
)k [k(2', Y)],
since k(r,y) = -((r,))) Hence, there exists a E IR such that for all z E S. we have
Uyyuuit(s..,Ikke = =c
Consoquently, Ex-ppY-UUnf(s., Y)] = and By,y-Umir(ss.)) |KY, Y'] =c
To work out an expression for c we use a standard strategy used, e.g. by Mardia Jupp (1999, 9.3.1). We reparametrize the sphere as (t, u) tz + -1- Fuu for E [-1,1] and on the unit sphere in the space orthogonal to (f. ||u|| that ()) = [()), allowing to eventually reduce to a 1d integral. The domain of u is isomorphie to 8m-1, which is easiest to see if we take (O.....0.1) 0. which gives [vI - tu, t] for u E Sm-1- Putting everything together gives
1 and (u,z) 0). This reparamctrization has the essential property
( du [ dtf(t((vi -2) m-3 =B 8m--
1 m
1)
atf(t)(vI- tt) mm3
(17)
Surf(Sm)
2' 2
where B the Bela function and we used the fact that Surl(S..)/ Surl( B(1/2, (m 1)/2). particular, for f(t) exp(1t) this recovers the Langevin distribution normalizing integral as derived by Mardia & Jupp (1999, 9.3.6).
A.3 Variance for a generic positive-definite kernel
Since the MMD estimator Lemma 2 treats the terms involving u as a constant and thus does not need to estimation, limits and bounds for variance can be better derived. In particular, because U-statistic (Hoeffding, 1963) in terms of & itself, Theorem 10 from (Gretton et al., 2012) can be modified as follows:
(+1ln/21 K2
Pr|MMD', MMD* > > exp
e.e the denominator inside the exp is smaller by factor of compared to the formulation of (Gretton et al., 2012), and thus bounded in j0, K). In terms of variance, we use the expressions of Serfling (2009) as in (Bounliphone et al., 2015) and (Sutherland, 2019) (as well as the CLT in (Gretton et al., 2012) Corollary 16) to obtain the asymptotic expression
4(n - 2) n(n -
Var[MMD, u)
- + Onn
or the similar second-order expression. These expressions are much simpler than the full MMD case since only the p terms are ncoded, and the terms involving u are constants. Note that in case of dispersion, which is our primary study focus, p is an empirical measure and thus (with full- optimization) our regularizer is exact, but these variance results could be used to analyze the variance of the minibatch estimator.
21

---

Published in Transactions on Machine Learning Research (08/2025)
B Sliced Dispersion: Proofs
B.1 Optimal Circle Dispersion
We aim to prove the assertion that the projection
1
(0: 0.)²
arg min deD 1=1
2
is given by 0 7* ()-)(1), where is the permutation such that da(1) S 2o(2) S Br(n), and r'
0: -
By definition, per Eq. (10), Ô = + do and thus we may write the problem equivalently
-o(() 2o(1) (0,0000 -)
arg min rEl-
Finding the permutation. In terms of a the objective takes the form - E, 0,00(1 + const, so we must find the permutation that maximizes Σ, (:0 = E, (,) By the rearrangement incquality (Hardy el al., 1952, Thms. 368 369), since Φ is in ascending order, this sum maximized when 0g-1()) is ascending order; the optimal must be the inverse of the permutation that sorts O.
Finding T. Ignore the constraints momentarily, and set the gradient of the objective zero:
0r - 0, 6o()) - 7)² = Σ + do(s) - 0,) = implying nt = Σ 0: Σ 0i Σ 0,,
the last equality by choice of the zero-centered reference configuration $ Since all 0, E [-T,, T), so is their average, and thus the constraints are satisfied, concluding the proof.
B.2 Projection onto a great circle
The projection we seek to compute is
projspa (x) = arg min (G(c sin(O)g,z). -550<m
Since the geodesic distance satisfies ds(:,)) :) arccos (., and arccos is strictly decreasing on (-1,1), we have
projsp (z) arg max (cos(0)p + sin(O)g, z).
As side note, this shows that it doesn't matter whether geodesic Euclidean distance to define this projection. Setting the gradient to zero yields
(). = (p,
equivalently tan(0)= (g. x (p, æ). The unique solution [-ग, π)) is given by the two-argument arctangent function (arctan2), also known as the argument of complex number (p,x) + i(g, (Wikipedia contributors, 2024).
B.3 Gradient of sliced distance
We first compute the Euclidean gradient of the desired expression:
xy Lsticod (X) = Vz. Ep.g dg (X), D DnSpp)
(18)
22

---

Published in Transactions on Machine Learning Research (08/2025)
First, by writing
(nSpp) min Σ 2 (0: - -0)
2
see this may be interpreted Euclidean projection and
00; Dr = (0, (0,-02 -;).
But 0, = -proispp (xi) and we can write
00: a
projsp.o (I:)
Ox:
dr:
00, tan -1
(g. x)) (p, x))
dr;
(p. 0)
(4, xpPp
x) + (p, x))
Putting the two together via the chain rule yields
V xy Csiiced (0elpp 8.1pp)
x))) (g. xiiP (q, x,)² + (p, xx,²
(19)
Notice that the second term vector in R that is orthogonal x; because:
(x 24)q (g, xx)p) (p, 2.) 2g) (q, m. (p, 2.) O.
Therefore,
grad Lsticed (X) = Vx, Ssliced
C Convergence of the Lloyd Regularizer
The convergence of Lloyd with Riemannian SGD can be shown with direct application of Bonnabel (2013, Theorem 1).
Let Vi( = (s Sm d(ss x,) S d(s,, x,) for all j * i} denote the spherical Voronoi cell around cluster center
24. The gradient w. w.r.t. all parameters is:
grad Croyd (xX,Q(X
(20)
or, blockwise r.t. centroid at time,
grad Lioood (X,,)
(21)
where Q.(X.v) can be defined as:
Qi(X,y) = Log,,, (y)1vx(()-
The length the Riemannian Log for the sphere is the geodesic distance, which is bounded:
IQ w)1l d(F.,))lvx(1) ~
and therefore for the total gradient also have:
23

---

Published in Transactions on Machine Learning Research (08/2025)
(Q(x,,)) S aVn.
As the sphere is compact and has injectivity radius n, and Q bounded shown, in the conditions of Theorem of Bonnabel (2013), and therefore Riemannian SGD with step sizes satisfying Σiso gi <
oc, Ev>o% " = converges.
D Effect of Dimensionality
D.1 Kernel sensitivity
Points on 8 concentrate near its equator, I.C., when m grows, the angles between some 2o and a uniform point on the sphere concentrate around 90 degrees. All kernels we consider appear to exhibit similar linear convergence log-log space (see Figure 7), but some are more sensitive in this critical area. Spherical kernels are generally more sensitive than Euclidean ones, with the exception of RBF, which justifies using Euclidean RBF in high-dimensional cases, at least when the data starts close to uniform distribution, despite more altractive properties of other kernels.
Laplace de
101
Laplace da
RBF d;
da
Riesz da
10
Riesz dg
Std [k(7a. 211
1
10-2
10-3
10a
102
10a
dimension m
Figure 7: Concentration of angles for different kernels.
D.2 Additional results
To study the impact of dimensionality further provide additional empirical comparison of the minimum geodesic distance and spherical variance for m E (64, 256, 512) for 10k and 20 in Figures 8 and 9. The maximum dmin increases while dimensionality grows. The red line shows the maximum achievable separation (90 degrocs). In torms of spherical variance, all methods converge to 1.0 except for Lloyd.
E
Additional experimental details and results
E.1 Neural Machine Translation: Experimental Setup
For subword tokenization we used the same SentencePiece (Kudo & Richardson, 2018) model, specifically the used in the MBart multilingual model (Liu et al., 2020). This choice allows for unified preprocessing for all
24

---

Published in Transactions on Machine Learning Research (08/202))
Clumped (PS 100) init n=10000
1.6
Minmum goodelic desance
1.4
1.2
1.0
MM
MIEE (d (d.)
Lioyd
0.8
RBF
Sliced
KIRE Riesz Idg
MIK
- uniform samples
0.6
KoLao (dg)
64
256
512
Dimensionality
Clumped (PS x- 100) init n=10000
1.0
Sahorica varaacce
0.9
0.8
KiNE placece
MM
Laplace (dl.)
Lloyd
KINE REF
Sliced
SSW
KIKE Riesz Idl)
samples
KoLoc
0.5
64
256 Dimansionality
Figure 8: Maximum minimum angle and spherical variance for synthetic data with different dimensionalities and number of points 10000.
languages we cover. We used fairseq (Ott et al., 2019) framework for training our models. Bascline discrete models (Fuclidean baseline) are trained with cross-entropy loss, label smoothing equal to 0.1 and effective batch size 65.5K tokens. All models are trained with learning rate 5 10-4 and 10k warm-up steps for 50k steps in total. Spherical baseline and models with dispersion regularizer are trained by defining decoder's embeddings layer as a manifold parameter. We tune learning rate for Riemannian Adam (Becigneul & Ganea, 2019) in the 10-4,5- .5 10-3) and report results with the learning rate 10-3 33, We used SacreBLEU (Post,
range [5-10- 15-10-5_5 5,5 2018) with the following signature nrefs: case: mixed leff no tok: 13a smooth : wersiooeeeoooio COMET (Rei et al., 2020) with ibbabel-cmme library version 2.2.2 and Unbabel comelllmottmtmmmmt--mmooo- model.
1 and
E.2 Neural Machine Translation: Gradient Norms
We show in Figure 10 how gradient norms and minimum distances target language embeddings vary throughout the training process. Note that at the step=0, the norms and minimum distances are the same.
E.3 Riemannian vs Euclidean Optimization
We compare the results of optimal angle approximation using projected gradient Adam RM against Riemannian Adam over Sm. All other parameters are the same as described in $4.1. Results are shown in Figure 11. However, for all other methods except KoLeo we can see that Riemannian optimization performs better overall.
E.4 Sample Convergence of the Sliced Regularizer
For better insight. into the number of great-circle samples required to estimate Lsticed in high dimensions, perform numerical experiment. Fixing X to be uniformly-sampae configuration of 10k points
"nttps://githubb.co/bbbbaa com/Ubbabbb////CIM
25

---

Published in Transactions on Machine Learning Research (08/2025)
Clumped (PS x x=100) init n000
1.4
1.0
Leplece Md
Llyd
MHE RBF (dg)
MilE Rleez tdc)
unffirm war ples
0.8
35
64
256
532
Dimencinellty
Clumped (PS x- 1001 init n=20000
0.
0.74
Laptace
NHE RBF (o))
0.6
MHE Riosz 10.)
MIEE: Rimaa tdi)
uniform somples
64
256 Dimensionaity
512
Figure 9: Maximum minimum angle and spherical variance for synthetic data with different dimensionalities and number of points 20000.
in dimension m 128, we draw up 10k uniformly-sampled great circles and report the Monte Carlo estimated average objective in Figure 12. The value converges relatively quickly and 1000 samples seem than enough. In experiments, we find it sufficient to draw single sample per update, because we perform sulficiently many updates.
E.5 Spherical Sliced Wasserstein: Additional Results
We do comparison between Sliced and SSW on larger scale using synthetic data. We tune the number of great circles sampled in one iteration and the learning rate. We report the best results for different dimensionalities and the number of points in Figure 13. We can see that in lower dimensions and with a smaller number of points, SSW can achieve a better minimum geodesic distance than Sliced. But as number of points grows, they provide identical results.
26

---

Published in Transactions on Machine Learning Research (08/2025)
Spherical
10-3
Euclidean
1.0
10-4
Gradient
0.8
10-1
0.7
Spherical
10
Euclidean
0.5
5000
15000
20000
5000
20000
25000
Frequency Rank
Frequency Rank
(a) Gradient norms and minimum distance for step=1000
10-3
Spherical Euclidean an
1.0
Norm
0.8
10
10
0.4
0.2
Spherical Euclidean
10
5000
10000
15000
20000
25000
5000
10000
15000
20000
25000
Frequency Rank
Frequency Rank
(b) Gradient norms and minimum distance for step=8000
10-3
Spherical Euclidean
Spherical
Euclidean
0.8
Gradient Norm
10-4
Miniuum
0.6
10-1
0.4
0.2
10-4
5000
10000
15000
20000
25000
5000
10000
15000
20000
25000
Frequency
Frequency
(c) Gradient norms and minimunn distance for step=11p000 12000
10:
Spherical
Spherical Euclidean
Euclide an
Gradient Norm
10-4
Minimum
10-1
0.4
0.2
10-6
0.0
5000
10000
15000
20000
25000
20000
Frequency Rank
Frequency Rank
(d) Gradient norms and minimum distance for step=20000
10
Spherical Euclidean
1.0
Spherical Euclidean
0.8
Gradient Norm
10-4
Min. Dist.
10
0.4
0.2
10-4
5000
10000
15000
20000
25000
5000
10000
15000
20000
25000
Frequency Rank
Frequency Rank
27
(e) Gradient norms and minmium distance for slep=32000
Figure 10: Training dynamic of gradient norms and minimum distances of the larget language embeddings.

---

Published in Transactions on Machine Learning Research (08/2025)
60
40
10
2
3
4
5
60
minimum angle (degrees)
30
10
uniform Init. MHE Laplace (dy) MHE RBF (dg) MHE Rlesz (d,)
KoLeo (dw)
MM (d.)
Lloyd
Sliced
Figure 11: Minimum angles (degrees) for each of the N=24 points with respect to optimization methods.
Top: projected gradient in Euclidean space. Bottom: Riemannian gradient the sphere.
3.50
3.25
3.00
2.75
(iiced(X)
2.50
2.25
2.00
2000
4000
6000
8000
10000
number samples used in estimation
Figure 12: Convergence of the sliced regularizer.
28

---

Published in Transactions on Machine Learning Research (08/2025)
Sliced vs Spherical Sliced Wasserstein
Sliced (n-1000)
1.4
SSW (n-1000)
Sliced (n-00000)
SSW (-- 10000)
geadesic distance
1.3
(0=0))
SSW 20000)
1.2
1.1
1.0
X
64
512
1024
Dimension
Minimum geodesic distance for different dimensions and number of points optimized by Sliced and SSW. Points generated with PS distribution with 1000
Figure 13:
29


Reviews:
[{"review_id": "pwKeURFREJ", "reviewer": "Reviewer_18EA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strength\n* This paper evaluated many regularizers in several settings, such as machine translation tasks and image classification tasks.\n\n## Weakness\n* The manuscript presents different methods across sections, especially in Sec. 3, and there is no discussion to compare them in Sec. 3. As a result, it becomes challenging to follow the overall argument.\n* It was not clear how this paper relates to the previous study. This paper explains several regularizers in Sec. 3.1-3.3, but it is unclear whether these regularizers are existing or proposed methods. As far as I understood, the authors explained how we can use the existing loss, e.g., MMD, Eq. (8), as the regularizer for learning embeddings on the Hypersphere. \n* At the beginning of Sec. 3, the authors claimed that \"then define two novel dispersion objectives with more appealing computational properties\", but I could not understand what the benefit is in terms of computational properties.\n* In this paper, Tables 3 and 4 experimentally validate the effectiveness of many regularizations, but the improvement is very small, with only a gain of 1 or less in the BLUE score.\n* For the experimental results shown in Sec. 4.4 and 4.5, the error bars were not shown. Since the improvement is small, it is important to evaluate these regularizers with several seed values and show whether these regularizers can consistently improve the scores.\n* For the experimental results shown in Sec. 4.4 and 4.5, the stepsize was not tuned by grid search.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studied the loss functions for learning embeddings on the Hypersphere. These loss functions work as regularisers in many tasks, e.g., machine translation, and help us to obtain embeddings that are evenly distributed on the hypersphere. The main contribution of this paper is that it experimentally evaluated the effectiveness of many regularizers on several tasks, such as machine translation and image classification.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Could you clarify the relationship between this paper and existing papers, and clarify which part the new ideas in this paper are?\n* Can the authors add error bars in Tables 2-4?\n* Although each regularizer has been compared experimentally, can the authors discuss the difference in the more intuitive advantages of each regularizer?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KMhQx16nfb", "reviewer": "Reviewer_Dr9k", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n- Theoretical unification and clarity: Shows that popular hyperspherical‐dispersion objectives (max–min, Kozachenko–Leonenko entropy, energy/MHE, “uniformity” loss) are all pair-wise potentials differing only by the kernel $k$. Provides a rigorous link between Minimum Hyperspherical Energy and MMD. This statistical interpretation unifies dispersion with kernel two-sample testing and offers principled hyper-parameter choices.\n- Novel, computationally efficient objectives:  Online Lloyd regulariser introduces a mini-batch, Riemannian variant of spherical $k$-means that is linear in the mini-batch size. Sliced dispersion reduces computation from $O(n^2)$ to $O(n)$ per batch via random great-circle projections and has an analytic Riemannian gradient.\n- Manifold-aware optimisation: Convincingly demonstrates that Riemannian Adam steps on $\\mathbb{S}^m$  are smoother and avoid the oscillations seen with Euclidean gradients and projection.\n\n## Weaknesses\n\n- No convergence guarantees for the new regularisers: The online Lloyd objective is presented as a stochastic Riemannian variant of spherical $k$-means, yet the paper offers neither a proof of almost-sure convergence to a local optimum nor a bound on the expected quantisation error after $T$ updates. For sliced dispersion, the manuscript states that each step is a “stochastic Riemannian gradient”, but omits any analysis of step-size conditions under which the iterates remain stable or converge.\n- Unbiasedness without variance control: The MHE–MMD theorem establishes that $\\mathcal{L}_{\\text{MHE},k}(X)$ is an unbiased estimator, but the authors do not derive finite-sample variance bounds or asymptotic normality, crucial for understanding regularization strength and hyper-parameter tuning.\n- Dimension-dependent behavior is unanalyzed:  Theoretical claims are stated for any $m$, yet the manuscript lacks discussion of the *concentration of measure $\\mathbb{S}^m$ when $m\\to\\infty$. In high dimensions, pairwise dot products concentrate around zero, which may decay the repulsive force of kernels such as Laplace or RBF; no bound quantifies this effect.\n- Lack of optimality against Tammes code: While experiments show that max–min and Lloyd approach the optimal Tammes energy for $n\\le 24$ on $\\mathbb{S}^2$, the theory provides no guarantee that the proposed objectives achieve a constant-factor approximation to the Tammes optimum in general $m,n$.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- The paper unifies a diverse family of hyperspherical‐dispersion objectives, including max–min distance, Kozachenko-Leonenko entropy, Minimum Hyperspherical Energy (MHE) and uniformity losses, by showing that they are all pair-wise distance functionals that differ only in the choice of potential or kernel, thereby clarifying their computational cost and optimization geometry.\n\n- It proves that MHE with any positive-definite kernel is an unbiased estimator of the squared Maximum-Mean-Discrepancy (MMD) from the uniform measure on the sphere, giving a principled statistical interpretation to popular “energy” regularizers.\n\n- Building on this insight, the authors introduce two new, computationally efficient dispersion regularisers: i) Online Lloyd regularizer, which treats dispersion as on-the-fly quantisation of $U(\\mathbb{S}^m)$ and applies a stochastic Riemannian variant of Lloyd’s $k$-means algorithm within mini-batches; ii)  Sliced dispersion, which projects embeddings onto random great circles, computes the closed-form optimal 1-D code, and penalizes the orthogonal distance, yielding an $\\mathcal{O}(n)$ per-batch method with an explicit stochastic Riemannian gradient.\n\n- The work proposes manifold-aware optimization by demonstrating that Riemannian Adam updates on $\\mathbb{S}^m$ provide smoother training dynamics and superior dispersion compared with Euclidean gradients followed by projection.\n\n- Finally, the paper provides practical guidance on when to deploy each regularizer, noting that max–min excels in fine-tuning well-spread codes, while sliced dispersion is preferable for very large or initially clustered vocabularies, and online Lloyd offers a favorable speed-quality trade-off as a drop-in mini-batch loss.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Give formal convergence guarantees for both new objectives:  For online Lloyd, state assumptions on step–size, mini-batch sampling and curvature of $\\mathbb{S}^m$; prove almost-sure convergence to a stationary point *or* provide an upper bound on the expected quantization error after $T$ updates. For sliced dispersion: show that the stochastic Riemannian gradient is unbiased and that, with a diminishing-step schedule, the expected squared gradient norm vanishes.\n- Augment the MHE–MMD analysis with finite-sample variance control: Derive variance for a generic positive-definite kernel and give an asymptotic CLT. Use this to justify the regularization weight and the number of Monte-Carlo samples for the uniform-term estimate.\n- Analyze high-dimensional behaviour:  Discuss how concentration of measure on $\\mathbb{S}^m (m\\to\\infty)$  affects the repulsive force of commonly used kernels (Laplace, RBF). Provide bounds (or empirical plots) showing how the minimum achievable pairwise angle $\\theta_{\\min}$ scales with $m,n$ under each objective.\n- Establish an approximation guarantee w.r.t. the Tammes optimum or explain its impossibility:  Either prove that each objective attains a constant-factor approximation in general $m,n$ or supply a hardness argument clarifying why only empirical proximity can be expected.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CdJKEBLkjj", "reviewer": "Reviewer_c9uQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper provides an interesting overview of methods which can be used to enforced dispersion on high dimensional spheres. It also provides nice connections between different objectives, such as energy and kernel methods, with the Tammes problem which is equivalent in the limit, and the MMD on the sphere. It also introduces a new sliced algorithm which seems to be very competitive with the other methods. Moreover, the comparison are done on several Machine Learning tasks. So overall, I believe this is a nice contribution to the community.\n\n\n**Strengths:**\n- Description of several criteria to measure dispersion, and several methods to enforce it\n- Connection between different objectives\n- New sliced algorithm demonstrating good results\n- Comparison of the methods on several settings, interesting to the ML community.\n\n**Weaknesses:**\n- I believe that some methods which could also solve this kind of problem are lacking.\n- This work lack some discussion about the complexity of the methods (or could be clearer), and if the different objectives come with optimization guarantees or not.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work studies the problem of learning dispersed embeddings on the sphere. The authors first provide an overview of different ways to measure the dispersion on the sphere, and of objectives that can be minimized to learn dispersed features. They make a connection between existing methods based on the minimization of an energy with kernels, and the minimization of the MMD on the sphere. They also make connections between these objectives and the Tammes problem. Additionally, relying on the observation that the optimal dispersion of $m$ points is know on a circle, they propose an objective based on the slicing process to enforce the dispersion in higher dimensional spaces. Finally, they compare the introduced methods on different applications, ranging from synthetic embeddings to image classification with prototypes, and neural machine translation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In Section 2.3, and the description of criteria to measure dispersions, I believe that any distance between probability distributions on the sphere could be added (for instance in particular optimal transport distances such as the Wasserstein distance or the Spherical Sliced-Wasserstein distance [1]).\n\nThe spherical Sliced-Wasserstein distance introduced in [1] can be efficiently computed between any discrete distribution and the uniform measure on the sphere. Thus, it can be optimized to generate dispersed points (see e.g. https://pythonot.github.io/auto_examples/backends/plot_ssw_unif_torch.html#sphx-glr-auto-examples-backends-plot-ssw-unif-torch-py). I am wondering how it would compare with methods used in this paper?\n\nThe recent paper [2] compared several methods to sample uniform points on the sphere (e.g. quasi Monte Carlo methods...) to then approximate the integral with respect to the uniform measure on the sphere. I believe that any method of [2] could also be compared with the methods of this paper, and conversely, the methods of this paper could be used to estimate integral. I think a discussion (or even comparisons) about the methods from this paper would be a nice addition.\n\nIt could also be a nice addition to add a table somewhere summarizing some of the results, and with computational complexities.\n\nIn Table 5, there is a description of kernels on the sphere. But some of them are known to not be positive definite. It could be nice to add a column with this information. And a related question is, is it an issue?\n\nIn the experiments, why are you always sampling only 1 great circle in the sliced method? I guess it would not be much more costly to take several directions, and it should give a better approximation. Also, it would be nice to have a comparison between the proposed sampling method along axis, and the uniform one.\n\nIn Section 4.1, Figure 4 is not refered to.\n\nIn the related works Section, it is written that \"DPPs are intended for selecting subsets of $k \\le n$\". It is true for finite DPPs, but there are also continuous DPPs, see e.g. [3].\n\nTypos:\n- Page 2: \"the projection the standard\"\n- Equation (5): $\\mathcal{L}\\_{\\mathrm{Max-Min}}$ should be $\\mathcal{L}\\_{\\mathrm{MM}}$?\n\n\n[1] Bonet, C., Berg, P., Courty, N., Septier, F., Drumetz, L., & Pham, M. T. (2022). Spherical sliced-wasserstein. arXiv preprint arXiv:2206.08780.\n\n[2] Sisouk, K., Delon, J., & Tierny, J. (2025). A User Guide to Sampling Strategies for Sliced Optimal Transport. arXiv preprint arXiv:2502.02275.\n\n[3] Fanuel, M., & Bardenet, R. (2021). Nonparametric estimation of continuous DPPs with kernel methods. Advances in Neural Information Processing Systems, 34, 24124-24136.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)