Identifying latent distances with Finslerian geometry
Authors: Alison Pouplin
David Eklund
Carl Henrik Ek
Søren Hauberg
Venue: Accepted by TMLR ()

Abstract: Riemannian geometry provides us with powerful tools to explore the latent space of generative models while preserving the underlying structure of the data. The latent space can be equipped it with a Riemannian metric, pulled back from the data manifold. With this metric, we can systematically navigate the space relying on geodesics defined as the shortest curves between two points.

Generative models are often stochastic, causing the data space, the Riemannian metric, and the geodesics, to be stochastic as well. Stochastic objects are at best impractical, and at worst impossible, to manipulate. A common solution is to approximate the stochastic pullback metric by its expectation. But the geodesics derived from this expected Riemannian metric do not correspond to the expected length-minimising curves.

In this work, we propose another metric whose geodesics explicitly minimise the expected length of the pullback metric. We show this metric defines a Finsler metric, and we compare it with the expected Riemannian metric. In high dimensions, we prove that both metrics converge to each other at a rate of $\mathcal{O}\left(\frac{1}{D}\right)$. This convergence implies that the established expected Riemannian metric is an accurate approximation of the theoretically more grounded Finsler metric. This provides justification for using the expected Riemannian metric for practical implementations.

Published in Transactions on Machine Learning Research (10/2023)
Identifying latent distances with Finslerian geometry
Alison Pouplin
alpu@dlu.. dk
Technical University of Denmark
David Eklund
dwvid.eklund@ri..s
Research Institutes of Sweden
Carl Henrik Ek University of Cambridge
che29Gcam.ocr ac. uk
Søren Hauberg Technical University of Denmark
sohau @ dtv. dk
Reviewed on OpenReview: https: // openreview. net/ forum? id= d=Q2CCOTTAA
Abstract
Riemannian geometry provides us with powerful tools to explore the latent space gener- ative models while preserving the underlying structure of the data. The latent space can be equipped it with Riemannian metrie, pulled back from the data manifold. With this metric, we systematically navigate the space relying geodesics defined the shortest curves between two points.
Generalive models are oflen stochastic, causing the data space, the Riemannian metric, and the geodesics, be stochastic as well. Stochastic objects are at best impractical, and worst impossible, to manipulate. A common solution is to approximate the stochastic pullback metric by its expectation. But the geodesics derived from this expected Riemannian metric do not correspond to the expected length-min minimising curves.
In this work, we propose another metric whose geodesics explicitly minimise the expected length of the pullback metric. We show this metric defines a Finsler metric, and we compare it with the expected Riemannian metric. In high dimensions, prove that both metries converge to each other at a rate of O ()) This convergence implies that the established ex- pected Riemannian metric is an accurate approximation of the theoretically more grounded Finsler metric. This provides justification for using the expected Riemannian metric for practical implementations.
1 Introduction
Generative models provide a convenient way to learn low-dimensional latent variables 2 corresponding to data observations x through smooth function this learnt manifold, one can generate new data or compare observations by interpolating or computing distances. However, doing so by using the Euclidean distance in the latent space misleading (Hauberg, 2018a), because the latent variables manifold (Fefferman et al., 2016), we want to equip our latent space with metric that preserves distance measures on it. Figure 1 (left panel) illustrates the need for defining geometric-aware distances on manifolds.
C X' c R.D, such that x = f(=). Through
not statistically identifiable. If our observations lying near
Distances on a manifold can be precisely defined using a norm, which is a mathematical function that exhibits several desirable properties such as non-ncgativity, homogeneity, and the triangle incquality. In particular, norm can be induced by an inner product (i.c., a quadratic function) that associates each pair of points the manifold with scalar value.

---

Published in Transactions on Machine Learning Research (10/2023)
To derive the standard Riemannian interpretation of the latent space, first compute the infinitesimal Euclidean norm according to the data space. Using the Taylor expansion, we have: [[(=+ f(=)ll A=
[I]()) + J(=)a== f(=)lī² A J(=) Az) As a first approximation, the norm defined in the latent space locally preserves the Euclidean norm defined in the data space. The curvature of our data manifold is condensed in the Riemannian metric tensor G2 JJ (z))((zz, which serves Riemannian metric: 9s (u, + u Gsv. In mathematical jargon, we say that the Riemannian manifold (2, 9) is obtained by pulling back the Euclidean metric through the map f.
proxy to define the
Riemannian geometry enables the exploration of the latent space in precise geometric terms, and quantities of interest such as the length, the energy or the volume can be directly derived from the pullback metric. These goometric quantities are, by construction, known to be invariant to reparamctrizations of the latent space 2, and are thus statistically identifiable (Hauberg, 2018a). While this geometric framework exclusively handles deterministic objects, generative models are often stochastic. The learnt map f, that mathemat- ically describes those models, is stochastic too. For example, in the celebrated Gaussian Process Latent Variable Model (GPLVM) (Lawrence, 2003). this function is a Gaussian process. The pullback metric is then stochastic, and standard differential geometric constructions no longer apply. Navigating the latent manifolds through geodesics (length-ming minimising curves) becomes practically infeasible.
Previous research tried circumvent this problem by approximating the stochastic pullback metric with the expected value of the Riemannian metric tensor, but the derived length is unintuitive quantities that do not correspond to the expected length:
(()) letai= /
1100)ll dt * [(((r) |a] = ( dt.
Instead of taking the expectation of the metric tensor, which serves surrogate to define norm, we propose to take the expectation of the norm directly.
In this paper, we compare our expected norm with the norm induced by the the commonly used expected metric tensor. The main findings are:
The expected norm defines Finsler metric. Finsler geometry is gencralisation of Riemannian geometry.
2. For Gaussian processes, the stochastic norm obtained through the pullback metric follows non- central Nakagami distribution, so our Finsler metric has closed-form expression.
3. In high dimensions, for Gaussian processes, our Finsler metric and the previously studied Riemannian metric converge to each other at a rate of O ()), with D the dimension of the data space.
We conclude that the Riemannian metric serves as a good approximation of the Finsler metric, which is theoretically more grounded. further justifics the use of the Riemannian metric practice when exploring stochastic manifolds.
1.1 Outline of the paper
The paper explores the geometry of latent spaces learned by generative models, which encode latent low-dimensional manifold that represents observed high-dimensional data. The latent manifold denoted Z R29 and the data manifold is denoted X' c RD.
Assuning the manifold hypothesis holds, need to define an infinitesimal norm in the latent manifold to compute distances that respect the underlying geometry of the data. Such norm can be constructed by pulling back the Euclidean distance through the smooth function that maps the latent manifold to the data manifold. This map, f: Z + X, mathematically describes the decoder of a trained generative model. Those models being often stochastic, we consider f being a stochastic process. It means that the pullback metric tensor, G J, and its induced norm, ll+llc u + VuT Gu, are also stochastic. In section 2, we mathematically define the notion of stochastic pullback metric and stochastic manifolds.
2

---

Published in Transactions on Machine Learning Research (10/2023)
Figure In this illustration, we show the 2-dimensional representation (0, () of the sphere parametrised in R with f (0,4p) + cos(O) sin(g), sin(0) cos(p), sin(p). In practice, we don't have access to well-parametrised manifolds, but instead those manifolds are being shaped by data points in RD. The data points are depicted as tiny dots for illustrative purposes. On the left figure, the map f is deterministic, and on the right figure, we imagine that f is stochastic. When we pullback, through S, circle from the sphere to the plane, Those ellipses, called indicatrices, are the fingerprints of the metric and they showcase the deformation of the plane. Left figure: blue, the Euclidean distance, is represented by the straight line in the plane. It is not identifiable and it does not represent a geodesic on the sphere. In orange, the bength-minimssing curve obtained through the pullback metric of the mapping leads to a great circle on the sphere. Following the great circle is the fastest way to go from one point another. Effectively, the pullback metric leads to a geodesic, contrary to the Euclidean metric. Notice that, in the plane, the orange geodesic also follows the direction of the indicatrices, since is sength-minimise Right figure: We imagine that generative model maps latent space to data space to the data space using stochastic process f = J2........ This leads to a stochastic pullback metric, stochastic indicatrices, and stochastic paths in the latent space.
circle deformed to an ellipse.
To circumvent all the challenges posed by this stochastic component, a deterministic approximation of the norm is nceded. It can be defined by taking the expectation of the metric tensor. This norm, that we will note ||-Il и + | has been studied before by Tosi et al. (2014), and explained in section 2.2. this paper, we propose instead to directly take the expectation of the stochastic norm. This expected norm, noted I|:p:: E [[]w[la]+ is introduced in section 2.4. The norm ||-| is defined by Riemannian metric, and we show that the norm I|- defines a Finsler metric. We cxplain the general difference between Finsler and Riemannian geometry in section 3.
The aim of this paper is to compare those two norms. We first draw absolute bounds in section 3.2, and then relative bounds in 3.3. We also investigate the relative difference of the norms when the dimension of the data space increases, in section 3.4. Finally, we perform some experiments in Section 4, that illustrate the Riemannian and Finsler norm in the same latent space.
1.2 Related works
Riemannian geometry for machine learning
When navigating a learnt latent space, the geodesics obtained through the pullback metric not only follow geometrically coherent paths, they also remain invariant under different representations. While two runs of the same model will produce distinct latent. manifolds, the geodesics connecting the same chosen points should have the same length. We say that the pullback metric effectively solved the identifiability problem (Hauberg, 2018a). This has led to the growing adoption of Riemannian geometry in machine learning applications. In robotics, Yoekk-Mohammadi et al. (2021) used the pullback metric from variational autoencoder to safely navigate the space of motion patterns, and Scannell et al. (2021) use geodesics under the expected metric in a GPLVM to control quadrotor robot. In proteins modelling, Detlessen el al. (2022) showed that the derived geodesics on the space beta-lactamase follow their phylogenic tree structure, In game content generation, González-Duque et al. (2022) generated game levels more coherent and reliable by interpolating data along the geodesics. Jørgensen & Hauberg (2021) used the expected Riemannian metric from obscrvations of pairwised distances through a GPIVM to build probabilistic dimensionality reduction model.
3

---

Published in Transactions on Machine Learning Research (10/2023)
Finsler geometry in machine learning
Our work crucially relies on Pinslerian geometry, which has been well-studied mathematically, but has only seen very limited use in machine learning and statistics. We point to two notable exceptions, which are quite distinct from our work. Lopez al. (2021) use symmetric spaces to represent graphs and endow these with Finsler metric to capture dissimilarity structure in the observational data. Ratliff et al. (2021) discuss the role of differential geometry in motion planning for robotics. Along the way, they touch upon Finslerian geometry, but mostly as neat tool to allow for generalizations. To the best of our knowledge, no prior work has investigated the links between stochastic and Finslerian geometry.
Strategies to deal with stochastic Riemannian geometry
Tosi et al. (2014) and Arvanitidis et al. (2018) introduced approximation of the pullback metric by taking the expectation of the metric tensor. In those two cases, the map f is respectively a trained Gaussian process, or the decoder a VAE. In this paper, the derivations only hold if f is a smooth stochastic process (Definition 2.2), which not the case of the VAEs, and hence, our results are not directly applicable to those models.
In addition to the work of Tosi et al. (2014), solution to circumvent the randomness of the metric tensor is to consider that the data follows specific probability distribution. Instead of looking at the shortest path on the data manifold, Arvanitidis et al. (2021) borrow tools from information geometry and consider the straightest paths on the manifold whose elements are probability distributions.
2 Expectation on random manifolds
The metric pulled back by a stochastic mapping is, de facto, stochastic and endows a random manifold. Unfortunately, we are not yet equipped to derive geometric objects on random manifold. Instead, we dodge this problem by seeking a deterministic approximation of this stochastic metric. As mentioned above, common solution is to approximate such metric by its expectation. In section 2.2, we study the expected Riemannian metric.
The solution suggested by this paper is to approximate the expectation of the lengths instead of the random metric itself. In section 2.4, we show that this new metric is not Riemannian but Finslerian (Proposition 2.2), and has a closed-form expression when the map f is a Gaussian process (Proposition 2.3).
2.1 Random Riemannian geometry
The pullback metric is defined Riemannian metric if and only if the mapping is immersion, which is differentiable function whose derivatives are injective everywhere the manifold (Lee, 2013, Proposition 13.9). A manifold equipped with a Riemannian metric is called a Riemannian manifold.
The pullback of the Euclidean metric through the immersion f 2 - X' is Rieman- nian metric. It is defined as the inner product 9s TTZ) - E4 : (u, v) + u Gv, at specific point z in the manifold 2. u and u are vectors lying in the tangent plane T.Z (ie: the set of all tangent vectors) of the manifold. = JT with the Jacobian of f.
Definition 2.1.
Since Riemannian metric is an inner product, it induces norm: Il.lIg- We then define the curve length and curve energy on a manifold: La(?) = fo (1))l dt and Ea(r) = So (1 dt, with a curve defined on 2, and y its derivative. A locally fngth--inimising curve between two connecting points is a geodesic. To obtain a geodesic, we can minimise the curve length, but in practice minimising the
the decoder of a VAE, while decudes to Gaussian, cannot be considered reason because the independence of the probability of the data: p(a[z) = II p(æu)a.). Let assume the opposite: the decoder is Caussian process. The covariance of the Gaussian process would be diagonal matrix because of the independence of the probability of the data. The covariance would correspond dirac distribution: cov(z;, &i.- However, stochastic process differentiahle only the covariance is differentiable, which the of the dirac distribution.
differentiable stochustic prucess. One
4

---

Published in Transactions on Machine Learning Research (10/2023)
curve energy is more efficient. On the manifold, we also define volume measure in order to integrate probability functions: for U C 2, Joo Iu
with Vn(z) vGz the volume measure.
In addition, considering the case where the immersion is stochastic process. The outputs of our trained model, E X, which represent our data, are random variables.
Definition 2.2. A stochastic process is collection of random variables (X(I,w), E T} indexed by an index set T defined on a sample space Ω which represents the set of all possible outcomes. An outcome in Ω is denoted by w, and realisation of the stochastic process is the sequence of ((,w) that depends on the outcome w.
In this framework, our index set is our latent manifold T = 2, and our sample space Q defined as the set of the model evaluations. For every point z E 2, every time we execute our model, the output x = f() is a random variable following specific distribution. When the data x follow a Gaussian distribution, the stochastic process is called Gaussian process. A GP-LVM (Lawrence, 2003) is model that learns how to map the data from latent space to data space through Gaussian process.
When is a stochastic immersion, the metric tensor becomes a random matrix. In this paper, we call a manifold equipped with the stochastic pullback metric a random manifold, noted (Z,g). As a consequence of the stochastic aspect the metric, all the functionals are stochastic themselves, and they are no longer trivial to manipulate.
Definition 2.3. A random Riemannian metric tensor a matrix-valued random field (ie: a collection of matrix-valued random variables (G(z,w), E Z)), whose realisation for a specific evaluation E Ω is Riemannian metric tensor. A random Riemannian metric is metric induced by random Riemannian metric tensor: Oz (T,2,T.)) T. R+ (u,v) u Gv. For the rest of the paper, the associated stochastic norm is noted:
ll TuZ R + V (u,u) Vu Gu
If this stochastic norm induced by f defined as a Ganssian process, then ||lle follows a non-central Nakagami distribution. This is explained in the proof of Proposition 2.3.
2.2 Norm induced by the expected metric tensor
One way approxinate random metric tensor is take its expectation with respect to the collection of random metrics induced by the stochastic process. This has been introduced before by Tosi et al. (2014) GP-LVMs.
Definition 2.4. Let G be stochastic Riemannian metric tensor the manifold 2. We refer to E|G] the expected metric tensor. It induces a Riemannian metric and norm on Z. We will note the norm induced by the expected metric tensor as:
|| T,2 R. u ||ullzje V TEGGG
Like any Riemannian metric, we can define the following functionals: LR(7) = So ()())B((G)))) di,
En(r) = 5 (()) T[() ((G)(( dt E(En(r),, and Vı(z) /det E[G].
2.3 Expected paths on random manifolds
Approximating the stochastic metric by its expectation seems natural but also ad-hoc solution. If we want. to explore manifold, might prefer to representative quantity, such as the lengths between

---

Published in Transactions on Machine Learning Research (10/2023)
data points. The expectation of the lengths can give us idea about how, average, two points are connected random manifold. The expected curve length, and its corresponding curve energy on the random manifold (Z. are defined as: Lp(r) = fo /i(t) Gi(t) = E[La(>)), and Ef(r) =
So Vi((t) G+
(V()) dt.
One observation made by Ekhund & Hauberg (2019) is that the length (Lx) derived from the expected Riemannian metric is not equal to the expected curve length (Ir), and their respective energy curves differ by a variance term:
En(?) Er(7) / 5(1) (()TB(GH)()) E [ViL) dt
/ E [110012]] [[lx00]l]]* de L Var [[+(0] de
This term can be regarded as a regularisation term for the Riemannian energy curve: the curve energy En might be penalised when the curve goes through regions with high-variance. In practice, for a Gaussian process with a stationary kernel, this variance term is upper bounded by the posterior variance that is relatively low next the training points and is high outside of the support of the data. Later, we will also see that the functionals agree in high dimensions, leading to the same geodesics (Section 3).
Eklund & Hauberg (2019) also noted that these quantities are bounded by the number of dimensions:
Proposition 2.1. (Eklund & Hauberg, 2019) Let f: R RRD be stochastic process such that the sequence: f2........ ... fb} has uniformly bounded moments. There is then constant C such that:
LR LF <
C
0 0
LR
8D
2.4 Expected norm and Finsler geometry
Our work builds on Eklund & Hauberg (2019)'s rescarch. We are interested in approximating the stochastic norm instead of the metric lensor, and by doing so, the derived curve length and curve energy are the same ones studied by Eklund & Hauberg (2019). We go further as we not only compare curve lengths, but the deterministic norms obtained with the stochastic metric.
Definition 2.5. Let G be stochastic Riemannian metric tensor the manifold 2. It induces stochastic norm, I|-lla on Z. We will note the expected norm
11-11 TeZ R4 : + [[llella] E [
While it cannot be induced by an inner-product, it is sufficiently convex to be defined as Finsler metric.
Definition 2.6. Let TZ + R,, be continuous non-negative function defined on the tangent bundle TZ of differentiable manifold Z.
We say that F is Finsler metric if, for each point of Z and T,2, have (1) Positive homogeneity: AA E R4, F(Ae) >F()). (2) Smoothness: F is O function on the slit tangent p² bundle T2 (0). (3) Strong convexity criterion: the Hessian matrix (v) 2 de (v) positive definite for non-zero v.
A differentiable manifold equipped with a Finsler metrie called a Finsler manifold. Finsler geometry can be seen as an extension of Riemannian geometry, since the requirements for defining a metric are less restrictive.

---

Published in Transactions on Machine Learning Research (10/2023)
Proposition 2.2. Let G be stochastic Riemannian metric lensor. Then, the function F2 T - lk | defines Finsler metrie, but it is not induced by Riemannian metric.
If F was induced by Riemannian metrie, then this metric would be defined as: fa 129 1 R+ v2) + Bl Guz]a. Since Riemannian metric is an inner product, it should be symmetric, positive, definite and bilinear. Here, we see that fo is not bilinear, fz is not Riemannian metric. However, can prove that F, I - R : E[ve Go] is positive, homogeneous, smooth and strongly convex, and so Fz is a Finsler metric (Shen & Shen, 2016, Definition 2.1). For the full proof, see Section B.1.
Proof.
So far, we have assumed that f is an immersion and a stochastic process. If we consider f to be a Gaussian Process in particular, the Finsler norm can be rewritten in a closed form expression.
Proposition 2.3. Let f be a Gaussian process and its Jacobian, with J ([E[J), 2)). The Finsler norm can be written as:
r + i
1 D w
F3 : T,Z+RRA |pl|== = = -2 Ev
r(
2 2 2
with FF as the confluent hypergeometric function of the first kind and w = (TT [ EJJb)).
Proof. We suppose that f is Gaussian process, and so is its Jacobian. G follows a non-central Wishart distribution: G=J (2 E, E. E[J] E[J]). v Gu is scalar and also follows a non-central Wishart distribution: GG ~ Wi(D, o,w), with σ = v Ev and w = (u Ev T E[J] [[J]] (Kent & Muir- head, 1984, Definition 10.3.1). The square-root
non-central Wishart distribution follows a non-central Nakagami distribution (Hauberg, 2018b). Then, by construction, the stochastic norm l|l| follows non- central Nakagami distribution. The expectation of this distribution is known, and it has closed-form expression.
The confiuent hypergeometric function of the first kind, also known as the Kummer function, is special function that is defined as the solution of specific second-order linear differential equation. The term w appears from the non-central Wishart distribution. When w is non-zero, the distribution of the Jacobian shifts away from the origin, and w represents the magnitude and the direction of this shift, balanced by the correlation between the variables. In Section 3.4, to prove our results in high-dimensions, will assume that our manifold Z is bounded, and is
3
Comparison of Riemannian and Finsler metrics
3.1 Theoretical comparison
In geometry, need to define norm to compute functionals, and the Riemannian metric is conveniently obtained by constructing an inner product. Because of its bilinearity, it greatly simplifies subsequent com- putations, but it is also restrictive. Relaxing this assumption and defining a metric as more general? norm has been studied by Finsler (1918), who gave his name to this discipline.
Finsler geometry is similar to Riemannian geometry without the bilinear assumption, most of the functionals (curve length and curve energy) are defined similarly to those obtained in Riemannian gcometry. However, the volume measure is different, and there are at least two definitions of volume measure used in Finsler geometry: the Ausemann-Fausdorff volume and the Holmes Thomson volume measure (Wu, 2011). In this
2The actually needs be strongly convex, but it is not necessary symmetrie. This have non reversible Finsler metric: (e) F2( v). Intuitively, this means that the path used to counect points would be different depending on the starting point. This asymmetric property becomes valuable when studying the genmetry of anisotropic media (Markvorsen, 2016), example. In our case, our Finsler metric reversible.
that, for vector e,
7

---

Published in Transactions on Machine Learning Research (10/2023)
paper, decided to focus the and easier to derive. If the Finsler metric is a Riemannian metric, the definition of volume naturally coincides with the Riemannian volume measure.
definition (Definition 3.1), which is more intuitive
Definition 3.1. For a given point 2 on the manifold, we define the Finsler indicatrix as the set
vectors the tangent space such that the Finsler metric equal to: {u E F2()) = 1J). We call (1) the Euclidean unit ball, and vol(-) the standard Euclidean volume. In local coordinates (et...., on Finsler manifold M, the Busemann-Hausdorf volume form is defined dVp
V=(z)et A... ee, with:
vol(B"(1)) V=(2) = (2 <c)
In the definition above, introduce the notion of indicalrie. An indicatrix is way to represent the distortion induced by the metric unit circle. If our metric is Euclidean, will only have a linear transformation between the latent and the observational spaces, and the indicatrix would still be circle. Because the Riemannian metric is quadratic, it will always generate an ellipse in the latent space. The Finsler indicatrix, however, would have a convex, even asymmetrical, shape. This difference can be observed in the indicatrix-field represented Figure 2: The Finsler indicatrices in purple can have almost rectangular shape, while the Riemannian indicatrices, in orange, are ellipses.
A
B
Figure 2: Indicatrice field over the latent space of the pinwheel data (in grey) representing the Riemannian (in orange) and Finslerian (in purple) metrics (See Section C). (A) The indicatrices computed grid in the latent space. (B) The indicatrices are computed along geodesic: the Riemannian and Finslerian metrics coincide.
There are also a few observations to note in Figure 2. First, in the area of low predictive variance (where data points lie in the latent space), the Finsler and Riemannian indicatrices are alike. This follows from the preceding comment that the metrics diverge by a variance term. If our mapping f was deterministic, both metrics would agree. Second, for every point, the Riemannian indicatrices are always contained by the Finslerian ones, illustrating Proposition 3.1 absolute bounds in the following section.
3.2 Absolute bounds on the Finsler metric
The Finsler norm is upper bounded with the Riemannian norm obtained from the expected metric tensor. It is also lower bounded:

---

Published in Transactions on Machine Learning Research (10/2023)
2
+( r(2)
Proposition 3.1. We define
The Finsler norm: ||-|| bounded by two norms,
I1l,,s and Il-lRR induced by the two respective Riemannian metric tensors: the covariance tensor aL,, and the expected metric tensor [[
V(z,0) x2xT:2: ||laax S | s ||l|
Proof.
The full proof is detailed in Section B.2.1, and it can be summarised the following way. The upper
bound ||e|| & ||vll also rewritten as: E[[v' Gvl & V is obtained by applying Jensen's inequality, knowing that the square root x > v is a concave function. The lower bound || I ||l]ee rewritten Vel aL & E[veTGu), is obtained using the closed form expression of the Finsler function.
The result is illustrated in Figure (lower right). Four metric tensors (G1, G3, Ga), each following a non-central Wishart distribution with a specific mean and covariance matrix, have been computed. For each of them, we have drawn the indicatrices (fv T,Z | ||a|| 1}) induced by the norms: I|-Il II-Il and Il-laax- As expected, can notice that the o'-indicatrix contains the Finsler indicatrix, itself containing R-indicatrix.
By bounding the Finsler metric, we are able to bound their respective functionals:
Corollary 3.1. The length, the energy and the Busemann- Haundorrf volume of the Finsler metric are bounded respectively by the Riemannian length, energy and volume the covariance tensor aΣ (noted Laz, Eas, Vas) and the expected metric E[G] (noted Ln, En, VR):
22 E 2, Las(z) & LF(2) & Ln(z) Eax(z) S EF(x) S ER(z) Vaz(z) S V=(z) s VR(z)
The full proof is detailed in Section B.2.1. From Proposition 3.1, we need to integrate each term of the inequality to obtain the length and the energy. The volume is less trivial, since use the Busemann- Hausdorff definition for measuring Vr. We recast the problem in hyperspherical coordinates, and show that the Finsler indicatrix is still bounded.
Proof.
3.3 Relative bounds on the Finsler metric
Proposition 3.2. Let be stochastic immersion. f induces the stochastic norm llll.- defined in Section 2. The relative difference between the Finsler norm ||-I|p and the Riemmanian norı ||-l| is:
|
|
Var
|
0
<
<
|||||
2'
2E
Proof.
This proposition direct application of the Sharpened Jensen's incquality (Liao Berg, 2019).
The previous proposition is valid for any stochastic immersion. We that the metrics become equal when the ratio of the variance over the expectation shrinks to zero. This happens in two cases: when the variance converges to zero, which is similar to having deterministic immersion, and when the number of dimensions increases. The latter case is investigated below for a Gaussian process 3.
*Interestingly, the term /Vr]//Var(p), when follows a central Nakagami distribution, called a shape parameter. It has been introduced by Nakagami himself study the intensity of fading in radio wave propagation (Nakagami, 1960). When / with & (1 D), then D/2. This is particular result obtained from Proposition 3.3, when E[J]

---

Published in Transactions on Machine Learning Research (10/2023)
Proposition 3.3. Let f be a Gaussian process. We note w = (v En) T B]J] jacobian of f, and 2 the covariance matrix of J. The relative ratio between the Finsler norm I|-1l and the Riemmanian norm ||-1l is:
B(J]o), B[J)), with J the
|ell
| R
D+ w
(D+
Gu follows one-dimension non-central Wishart distribution: v vT (D,,w, with 0 Ev and w (o Eu) (v"TV " [[]]]/[[]oo We use the theorem of the moments to obtain both the expectation and the variance, which leads us to the result.
Proof.
As we have seen that the metrics are bounded, it is easy to show that the functionals derived from those metrics are also bounded:
Corollary 3.2. When f is Gaussian Process, the relative ratio between the length, the energy and the volume of the Finsler norm (noted Lp, Ep, VF) and the Riemannian norm (noted Ln, ER. Vn) is:
LR(2)= - LF(Z)
W
max VET.2 2
LR(Z)
D +
(D (D+w)
w)
0 ER(2) - Ep(z)
2
1 2w
2w
< max TET.2
+
+
+
ER(2)
D +
(D + w)2
(D + w)
(D
w
0 VR(z) VF(22)
w
Vn(=)
<
1- max vE7.2
D+
+
(D+
Proof. We directly use Proposition 3.3. To obtain the inequalities with the lengths and the energies, we first multiply all the terms by the Riemannian metric, and we integrate every term. To obtain the inequality with the volume, similarly to Corollary 3.1, we place ourselves in hyperspherical coordinates and bound the radius of the Finsler indicatrix. The full proof is in Section B.2.2.
A
D
-45
-25
400
Figure 3: Difference volume for data embedded in the latent space. (A) Riemannian volume measure, (B) Finslerian (Busemann- Hausdorrf)) volume measure, (C) Variance of the Gaussian process, (D) Ratio between the Riemannian and Finslerian volume: (VR(=) V=<=)/VVn(z). All heatmaps are computed in logarithm scale.
In Figure 3, can compare the volume measures obtained from the Riemannian and Finsler metrics, and in particular, their ratio in the top right image. When the metrics are computed next to the data points in area where the variance is very low, we can see that the ratio of the volume measure is al the order of magnitude 10-4, Further away from the data points, the variance increases and so does the difference between the Riemannian and Finsler volume measures.
3.4 Results in high dimensions
Proposition 3.3 and Corrollary 3.2 indicate that the metrics become similar when the dimension (D) of the observational space increases. If we assume that the latent space is a bounded manifold, the metrics converge to each other at rate of O (b), as do their functionals.
10

---

Published in Transactions on Machine Learning Research (10/2023)
We assumee that the latent manifold is bounded. Then, can deduce that (1) the term w, which represents the non-centrality of the data, does not grow faster than the number of dimensions (See lemma B.2.2, in Section B.2.3) and (2) we that the metrics are finite.
Corollary 3.3. Tet. f be a Gaussian Process. high dimensions, we have:
LR(Z) LF(2) (R(z)
= - ER(Z) Ep(z) (o(z)))p(() =0 ( and VR(2)-Vo(( VF(2) O ( En(z)
Vπ(=)
When D converges toward infinity: LR LF ER Ep and VR Vp-
t00
400
Proof. This result. follows from Corollary 3.2, assuming the latent manifold bounded. The full proof can be found in Setion B.2.3.
Corollary 3.4. Let f be Gaussian Process. In high dimensions, the relative ratio between the Finsler norm I and the Riemmanian norm || is:
| 1 =0 (0(1)
||l|
And, when D converges toward infinity: Ve T,2, ||ll too |1l1
Proof. Similarly, from Proposition 3.3, in a bounded manifold, both metrics converge to each other high dimensions.
Male0--100 with respect to the number of dimensions
0.16
average difference volume
fx) =
0.12
dim:
dim: 5
dim:
dim:
0.08
0.06
0.04
0.02
0.00
Lower bound metric lensor indicalrix Finslerian indicatrix
20
40
60
80
100
Riemannian indicatrix
dimensions
Figure Left: Ratio volumes (VR - V5)/VR decreasing with respect to the number of dimensions. The results obtained from using collection of matrices {G.} following non-central Wishart distribution. Upper right: The Finsler and Riemannian indicatrices converge towards each other when increasing the number of dimensions. Lower right: Mustration of the absolute bounds in Proposition 3.1 with the aE-indicatrices, Riemannian indicatrices and Finsler indicatrices.
4 Experiments
We want to illustrate cases where these metrics differ in practice. For this, we use two synthetic datasets, consisting of pinwheel and concentrie circles mapped to sphere, and four real-world datasets: font
11

---

Published in Transactions on Machine Learning Research (10/2023)
dataset (Campbell & Kautz, 2014), dataset representing single-cells (Guo et al., 2010), MNIST (LeCun, 1998) and fashionMNIST (Xiao et al., 2017). We trained GPIVM with and without using stochastic active sets (Noreno-Muñoz et al., 2022) to learn a latent manifold. From the learnt model, we can access the Riemannian and Finsler metrics, and minimise their respective curve energies to obtain the corresponding geodesics. All the code has been built using Stochman (Detlefsen et al., 2021), a python library to efficiently compute geodesics on manifolds.
4.1 Experiments with synthetic data showing high variance
The synthetic data correspond to simple patterns pinwheel and concentric circles that have been projected onto sphere. In those examples, we plotting curves that does not follow the data points and so, go through regions of high variance. Notably, the background of the latent manifold represents the variance of the posterior distribution in logarithmic scale. For both cases, we have the 3-dimensional data space and the corresponding learnt latent space. The curves are mapped from the latent space to the data space using the forward pass of the GPLVM.
A.2
B.1.
-8
B.2.
-5
-20
25
-2
-18
-35
observed data
Riemann
Finsler
Euclid
Figure The Riemannian (purple), Finslerian (orange) and Euclidean (dotted gray) geodesics obtained by pulling back the metric through the Gaussian processes of trained GPIVM. The models, trained on the 3-dimensional syuthetic data (Figures A.2, B.2) learnt their latent representations (Figures A.1, B.1)
We can see that the Riemanniin geodesics tend to avoid area high variance in both cases: they are attracted the data, at the detriment of following longer paths in R33 The Finslerian geodesics, on the opposite, not perturbed by the variance term, and will explore regions without any data, following shorter paths in R3 The geodesics have been plotted by computing a discretized manifold, obtained from 10x10 grid with Detlefsen el al. (2021). This approach proved to be more elfective than minimizing the energy along a spline, which was prone to getting trapped in local minima. The GPLVM has been coded in Pyro (Bingham et al., 2019). All the implementation details can be found the Appendix C.
4.2 Experiments with font dataset and qPCR dataset
The font dataset (Campbell & Kautz, 2014) represents the contour letters in various font, and the qPCR dataset (Guo al., 2010) represents single-cells stages. We trained a GPLVM model, also using Pyro (Bingham et al., 2019), to learn the latent space. From the optimised Gaussian process, we access the Riemannian and Finsler metric, and minimise their respective curve energies to obtain geodesics.
Similar to the previous experiment, the background colour represents the variance of the posterior distribution in logarithunic scale. We can notice that the Riemannian and the Finsler geodesics agrees with each other. This experiment also agrees with our finding that in high dimensions (the dimensions of the single-cells data 48, the font data 256), both metrics converge to each other. This concludes that, in practice, the Riemannian metric is a good approximation of the Finsler metric, and Finsler geometry doesn't need to be used in high dimensions.
12

---

Published in Transactions on Machine Learning Research (10/2023)
A.
B.1.
B.2.
1.0
0.6
1.90
1.85
92
1.80
Riemannian geodesic
Finslerian geodesic Observed data
Figure 6: The Riemannian (plain line) and Finslerian (dotted line) geodesics obtained by pulling back the metric through the GPLVM. The models trained on single-cells data (which cannot be represented) and font data (Figure B.2) learnt their respective 2-dimensional latent representation (Figures A, B.1).
4.3 Experiments with MNIST and FashionMNIST
GPLVMs are often hard train, but they can be scaled effectively using variational inference and inducing points. Using stochastic active sets has been shown to give reliable uncortainty estimates, and the model also learn more meaningful latent representations, shown in the original paper by Moreno-Muñoz el al. (2022). For those experiments, we used those models on two well-known benchmarks: MNIST and FashionMNIST.
In both experiments, because the difference of the variance the posterior learnt by the GPLVM across the latent manifold low, as we can notice in the background of the plots on the right, and because the data is high dimensional, we cannot any difference between the Finslerian and the Riemannian geodesics. Again, in practice, the Riemannian metric good approximation of the Finslerian metric.
5 Discussion
Generative models often used to reduce data dimension in order to better understand the mechanisms behind the data generating process. We consider the general setting where the mapping from latent vari- ables to observations is driven by a smooth stochastic process, and the sample mappings span Riemannian manifolds. The Ricmannian geometry machinery has already been used in the past to explore the latent space.
In this paper, we have shown how curves and volumes can be identified by defining the length of a latent its expected length measured in the observation space. This is natural extension of classical differential geometric constructions to the stochastic realm. Surprisingly, give rise to a Riemannian metric over the latent space, even if sample mappings do. Rather, the latent representation naturally becomes equipped with a Finsler metric, implying that stochastic manifolds, such as those spanned by Latent Variable Models (LVMs), are inherently more complex than their deterministic counterparts.
have shown that this does not
The Finslerian view of the latent representation gives us suitable general solution to explore a random manifold, but it does not immediately translate into practical computational tool. As Riemannian mani- folds better understood computationally than Finsler manifolds, we have raised the question: How good approximation of the Finsler metric can be achieved by Riemannian metric? The answer turns out to be: quite good. We have shown that as data dimension increases, the Finsler metric becomes increasingly Riemannian. Since LVMs are most commonly applied to high-dimensional data (as this where dimension- ality reduction carries valuc), we have justification for approximating the Finsler metric with Riemannian metric such that computational tools become more easily available. In practice we find that geodesics under the Finsler and the Riemannian metric near identical, except in regions of high uncertainty.
13

---

Published in Transactions on Machine Learning Research (10/2023)
2.0
250
Finslerian geodesic Riemannian geodesic
2.54
046
0.52
034
Figure The Riemannian (purple) and the Finslerian (orange) geodesics plotted. Upper figure: learnt latent space with the images (left) and data points (right) of Fashion MNIST. Lower figure: learnt latent space with the images (left) and data points (right) MNIST.
Acknowledgments
The authors would like to thank Professor Steen Markovsen for the initial discussions on Finsler geometry, and Dr. Cilic Feldager for her invaluable help in taming GP-LVMs. This work was funded in part by the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Science (NNF200C0002606) Tt also received funding from the European Research Council (ERC) under the European Union Horizon 2020 research, innovation programme (757360). SH was supported in part by research grants (15334, 42062) from VILLUM FONDEN.
Code availability
The code used for the experiments this study available on GitHub https: http::///gthhb.... com/a-pouplin latent dwwtaacess_finnneecc ffnessfinnshh finsler.
14

---

Published in Transactions on Machine Learning Research (10/2023)
Notations
2, X
Smooth differentiable latent (2) and data. (At) manifold, A stochastic immersion f 2 C IR - X C RP, Jacobian of the stochastic function f.
f
J
G
Stochastic metric tensor defined as the pullback metric through f: G GJJJJ, Tangent space of the manifold Z at a point 2,
T.2
Σ
IF f is Gaussian process, then J ~ I N(u, E),
lHla
Stochastic induced norm: ||v|| = Vu T Gv,
Riemannian induced norm: ||e|| R Gu = Vg(u, v), Finsler norm: ||ج|| == ElvuTGi Gu] F(v),
Lns Ens VR LF, LF,FF, VF
Length, energy and volume obatined from the Riemannian induced norm I1:l Length, energy and Busemann Hausdorff volume obtained from the Finsler norm ||-1I
References
Sumon Ahmed, Magnus Rattray, and Alexis Boukouvalas. Grandprix: scaling up the bayesian gplvm for single-cell data. Bioinformatics, 35(1):47 54, 2019.
Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent Space Oddity: on the Curvature of Deep Generative Models. In International Conference on Learning Representations (ICLR), 2018.
Georgios Arvanitidis, Miguel González- Dunzue, Alison Pouplin, Dimitris Kalatzis, and Søren Hauberg. Pulling back information geometry, 2021. URL https:://arxxw..or//bbb///
05367.
Hadi Beik- Haik--Mohammdii, Søren Hauberg, Georgios Arvanitidis, Gerhard Neumann, and Leonel Rozo. Learning riemannian manifolds for geodesic motion skills. arXiv preprint arXiv:2106.044 044401, 2021.
Eli Bingham, Jonathan P Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D Goodman. Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research, 20(1):973-977) 2019.
Neill D. F. Campbell and Jan Kautz. Learning a manifold fonts. ACM Trans. Graph., 33(4), jul 2014. ISSN 0730-0301. doi: 10.1145/2601000 261100002299222222 URL attps://dolt http:////o.oo/////////// 1145/2601097 2601212.
Nicki S. Detlefsen, Alison Pouplin, Cilie W. Feldager, Cong Geng, Dimitris Kalatzis, Helene Hauschultz, Miguel González Duque, Frederik Warburg, Marco Miani, and Søren Hauberg. Stochman. GitHub. Note: 2021.
htlps:///illuub.com// Maw.chiineeeerriiaaaaaeeeeeeieie
Nicki Skafte Detlefsen, Søren Hauberg, and Wouter Boonsma. Learning meaningful representations of protein sequences. Nature communications, 13(1):1914, 2022.
David Eklund and Søren Hauberg. Expected path length on random manifolds. arXiv preprint arXiv:1908.074 07377, 2019.
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4):983-1049, 2016.
Paul Finsler. Ueber kurven und Flächen in allgemeinen Räumen. Philos. Fak., Georg-August- Univ., 1918.
Miguel Conzález-Duquu Rasmus Berg Palm, Soren Hauberg, and Sebastian Risi. Mario plays on manifold: Generating functional content in latent space through differential geometry. In 2022 IEEE Conference Games (CoG), pp. 385-30 392.. IEEE, 2022.
Guoji Guo, Mikael Huss, Guo Qing Tong, Chaoyang Wang, Li Li Sun, Neil D. Clarke, and Paul Robson. Resolution of cell fate decisions revealed by singlo-cell gene expression analysis from zygote blastocyst. Developmental Cell, 18(4):675-685, 2010. ISSN 1534-5807. doi:
20..000.. 012. URL http://pwwwgggoooooeccccoooaaaaa
15

---

Published in Transactions on Machine Learning Research (10/2023)
Søren Hauberg. Only bayes should learn manifold (on the estimation of differential geometric structure from data). orXiv preprint arXiv:1806.0490) 2018a.
Søren Hanberg. The non-central Nakagami distribution. Technical report, Technical University of Denmark, 2018b. URL http://www2. compute http://///ssaaaaappppaaaaaaaaaaa 2a0iraaaaam.20008//aaaaaaaa... pdf.
Martin Jørgensen and Soren Hauberg. Isometric gaussian process latent variable model for dissimilarity data. In International Conference on Machine Learning, pp. 5127-5136. PMLR, 2021.
John T. Kent and R. J. Muirhead. Aspects of Multivariate Statistical Theory. The Statistician, 1984, ISSN 00390526. doi: 10.2307 10.2307/2987858 /2987858.
ar preprint Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1422 1908000, 2014.
Neil Lawrence. Gaussian process latent variable models for visualisation of high dimensional data. Advances in neural information processing systems, 16, 2003.
Yann LeCun. The mnist database of handwritten digits. http://yarm.. lecun, com/erdly /tt////ddm/m/////////mm/m//// 1998.
John M Lec. Smooth manifolds. In Introduction to smooth manifolds, pp. 1-31. Springer, 2013.
J. G. Liao and Arthur Berg. Sharpening jensen's inequality. The American Slatistician, 73(3):278 2212281,,118 2019.
doi: 000000000310052000000
Federico Lopez, Beatrice Pozzetti, Steve Trettel, Michael Strube, and Anna Wienhard. Symmetric spaces for graph emboddings: A finsler-ricmannian approach. In Marina Meila and Tong Zhang (ods.). Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learn- ing Research, pp. 7090-7101. PMLR, 18 24 Jul 2021. URL https: https: http:///ooeeeenn...// .mlringgnnmmm............ mlr, wttp:nnn..........rsreeeee press/v139/ lopez21a. html.
Steen Markvorsen. A finsler geodesic spray paradigm for wildfire spread modelling. Nonlinear Analysis: Real World Applications, 28:208-221 2016.
Pablo Moreno-Muñoz, Cilie Feldager, and Søren Hauberg. Revisiting active sets for gaussian process de- coders. Advances in Neural Information Processing Systems, 15:6603-6614, 2022.
Minoru Nakagami. The m-distributiona general formula of intensity distribution of rapid fading, In Statistical methods in radio wave propagation, pp. 3-36. Elsevier, 1960.
Pyro. Gaussian process latent variable model, 2022. URL https://pyrrs http:tt//ppppo..a/eaaaaall////// ww/paaapple///ppl.aaa....
Carl Edward Rasmusson and Christopher K. I. Williams. Gaussian Processes for Machine Learning (Adaptive Compulation and Machine Learning). The MIT Press, 2005. ISBN 026218253X.
Nathan D. Ratliff, Karl Van Wyk, Mandy Xic, Anqi Li, and Muhammad Asif Rana. Generalized nonlincar and finsler geometry for robotics. In 2021 IEEE International Conference on Robolies and Aulomation (ICRA), pp. 10206-10212, 2021. doi: N0.1109/TCRA4556.. 200.0..92221995143
Aidan Scannell, Carl Henrik Fk, and Arthur Richards. Trajectory optimisation in learned multimodal dynamical systems via latent-ode collocation. 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 12745 12751. IEEE, 2021.
Yi-Bing Shen and Zhongmin Shen. Introduction to Modern Finsler Geometry. Co-published with HEP, 2016. doi: 10. URL
Alessandra Tosi, Søren Hauberg, Alfredo Vellido, and Neil D Lawrence. Metrics for probabilistic geometries. arXiv preprinl Xr 1. 7432, 2014.
J. G. Wendel. Note on the gamma function. The American Mathematical Monthly, 55(():5633-56 564, 1948. ISSN 00029890, 19300972. URL http://wwwz jatorr...r... wttg:stable/230446000
16

---

Published in Transactions on Machine Learning Research (10/2023)
Bingye Wu. Vohume form and its applications in finsler geometry. Publicationes Mathematicae, 78, 04 2011. doi: 10.5486/PMD.201 1PMMD20011.444 1.999.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist novel image dataset for benchmarking machine learning algorithms. Xiv preprint Xrxi 1 077407777, 2017.
17

---

Published in Transactions on Machine Learning Research (10/2023)
A A primer on Geometry
The main purpose of the paper is to define and compare two legitimate metrics to compute the average length between random points. Before going further, it's important to formally define the two metrics (Riemannian and Finsler metrics, respectively) which we do sections A.2 and A.3. They are both constructed on topological manifolds, the definition of which is recalled in section A.1. We finally introduce the notion of random manifold in section A.4, which is the last notion needed to frame problem of interest: which metric should we use to compute the average distance random manifold?
A.1 Topological and differentiable manifolds
This section aims to define core concepts in differential geomelry that will be used later to define Riemannian and Finsler manifolds. Recall that two topological spaces are called homeomorphie if there is a continuous bijection between them with continuous inverse.
Definition A.1. A d-dimensional topological manifold M is second-countable Hausdorff topo- logical space such that every point has open neighbourhood homcomorphic to open subset of R4.
Let M be a topological manifold. This mcans that for any E M there an open neighbourhood Uz of and a homeomorphism QU, Un + Ie onto open subset of Rd, Suppose that x, E M such that U₂ n Uu 0, let U = Un. V Uy and consider the so-called coordinate change map
Φv (Ujjo ou +R.
We call M together with an open cover (Uz)zEM as above a differentiable or smooth manifold the coordinate maps are infinitely differentiable.
Beyond these technical definitions, one imagine differentiable manifold as well-behaved smooth surface that possesscs locally all the topological properties of Euclidean space. All the manifolds in this paper are assumed to be differentiable and connected manifolds.
Definition A.2. We also define, for differentiable manifold M, the tangent space T,M the set of all the tangent vectors at M, and the tangent bundle TM the disjoint union of all the tangent spaces: TM = U M.
ZEM
So far, we have only defined topological and differential properties of manifolds. In order to compute geometric quantitics, we need to equip those with a metric that helps us derive useful quantitics such as lengths, energies and volumes. A metric is a scalar valued function that is defined for each point on the topological manifold and takes as inputs one or two vectors (depending on the type metric) from the tangent space at the specific point. Such a function can either defined as a scalar product between two vectors, this is the case of Riemannian metric or, in the case of Finsler metric, defined similarly the norm of vector. We will formally define these metrics and highlight their differences in the following sections.
A.2 Riemannian manifolds
Definition A.3. Let M be a manifold. Riemannian metric is map assigning at each point x M a scalar product G(:,)) T.M x T.M - IR, with a positive definite bilinear map, which is smooth with respect x. A smooth manifold equipped with Riemannian metric is called Riemannian manifold. We usually express the metric symmetric positive definite matrix G, where have for two vectors u, E T:M: G(u,e) v) = u Gv. We further define the induced norm: u E M, ||ull = v).
The Riemannian metric here can either refer the scalar product G itself, or the associated metric tensor G.
18

---

Published in Transactions on Machine Learning Research (10/2023)
T&M
Iv (2, v)
df(w)
df(u)
M
Figure f is an immersion that maps a low dimensional manifold to a high dimensional manifold M. On M, a tangent plane T&M is draw at The indicatrice of the Euclidean metric is plotted in blue. When this metric is pulled-back through f. the low dimensional space now equipped with the pullback metric which is a Riemannian metric by definition. The vectors df (u) and (v) are called the push-forwards of the vectors and through f.
Definition A.4. We consider curve (t) and its derivative +t) Riemannian manifold M equipped with the metric g. Then, define the length of the curve:
La(>)= = Intilll dt - /f (V(()())))))
where gi = --(1)) Locally length-minimising curves between two connecting points are called Geodesics.
Definition A.5. The curve energy is defined as:
Ea(?) = / [1(00) di f (000--((00000 (+()))(()))
There are two interesting properties to note about the length of a curve and the curve energy. First, the length is parametrisation invariant: for any bijective smooth function on the domain of y we have that La(ron) = LG(y). We also say the Riemannian metric gives us intrinsic coordinates to compute the length. Secondly, for given curve 7, we have: Lo(>)² < 27G(7). Because of the invariance of the curve, when we aim minimise it, solver convex and will lead to unique solution. Thus, to obtain geodesic, instead of solving the corresponding ODE equations, or directly minimising lengths, it is casier in practice to minimisc the curve energy, as a minimal energy gives a minimal length.
find infinite number of solutions. On the other hand, the curve energy is
The Riemannian metric also provides us with an infinitesimal volume element that relates our metric G to an orthonormal basis, the same way the Jacobian determinant accommodate for a change of coordinates the change of variables theorem.
Definition A.6. In local coordinates (e'..... ,e"), the volume form of the Riemannian manifold M, equipped with the metric tensor G, is defined as: dVG Vo(x)e A... Aed, with:
Va(x) = dee((G).
Remark. The symbol A represents the wedge product and it is used to manipulate differential k-forms. Here, the basis vectors (e',.-.,ee) ,ea) form a d d-dimensional parallelepiped (e' A A... A e²) with unit volume.
19

---

Published in Transactions on Machine Learning Research (10/2023)
(x(
f(1)
Figure Oace the low dimensional manifold equipped with a metric that captures the inherent structure of the high dimensional manifold, we can compule geodesic 7, by minimising the energy functional between two points. The geodesic f(r) will be the shortest path between two points on manifold M. The volume measure Vg can also be used to integrate functions over regions of the manifold, be linked to the density the data: the data points are uniformly distributed over the nigh-dimensional manifold, in the low-dimensional manifold, low volume would correspond to high density of data. It is a useful way give more information about the distribution the data.
would do in the Fuclidean space. Tt can also
A.3 Finsler manifolds
Finsler geometry is often described more general way, lifting the quadratic constraint. In particular, the norm of Riemmanian metric is a Finsler metric, but the converse is not true.
extension of Riemannian geometry, since the metric is defined in
Definition A.7. Let F' TM + R be a continuous non-negative function defined on the tangent bundle TM of differentiable manifold M. We say that is a Finsler metric if, for each point æ of M and on T,M, we have:
1. Positive homogeneity: E R,, F(Ao) = XF()).
2. Smoothness: F is C function the slit tangent bundle TM {0).
1 F2 3. Strong convexity criterion: the Hessian matrix (v) zero v.
2 de (v) is positive definite for non-
A differentiable manifold M equipped with a Finsler metric is called a Finsler manifold.
Here, is worth noting that, for a given point in the manifold, the Finsler metric is defined with only one vector in the tangent space, while the Riemannian metric is defined with two vectors. Moreover, from the previous definition, can deduce that the metric is:
Positive definite: for all M and T,M, F(U) 2 and F(u) if and only if 0.
2. Subadditive: F(u + w)) < F(v) + F(w) for all x e M and E T.M.
We say that F is a Minkowski norm on each tangent space T.M. Furthermore, if F' satisfies the reversibility property: F(v) F(-v), it defines norm T.M in the usual sense.
Similarly to Riemannian geometry, lengths, energies and volumes can be defined directly from the Finsler metric:
Definition A.8. We consider a curve and its derivative á on a Finsler manifold M equipped with the metric F. We define the length of the curve as follows:
Lx(1) / F(i(t))dt.
20

---

Published in Transactions on Machine Learning Research (10/2023)
Fu)
T&M
L
M
M
Figure 10: f an immersion that maps a low dimensional manifold to a high dimensional manifold M. On M, a tangent plane T&M is draw the all the vectors u E T&M such that Fg (u) = 1, is not necessarily an ellipse. It can be asymmetric if the metric is asymmetric itself. It always convex.
Compared the Riemannian manifold, the Finsler indicatrix, which represents
Definition A.9. The curve energy is defined as: Ep(7) d(
Not only are the definitions strikingly similar, they also share the same properties. The curve length is also invariant under reparametrisation, and upper bounded by the curve energy. Computing geodesics on a manifold reduced to a variational optimisation problem. These propositions are proved in detail in Lemmas B.1.4 and B.1.5, in the appendix.
In Riemannian geometry, the volume measure defined by the metric unique. In Finsler geometry, different definitions of the volume exist, and they all coincide with the Riemannian volume element when the metric is Riemannian. The most common choices of volume forms are the Buseman--Haudort measure and the Holmes-Thommson measure. According Wu (2011), depending on the Finsler metric and the topological manifold, some choices seem more legitimate than hers.. In this paper, we decided to only focus on the Busemann- Hausdorff Bausdon volume, as its definition is the most commonly used and leads to easier derivations. We will later show that in high dimensions, our Finsler metric converges to a Riemannian metric, and thus, the results obtained for the Busemann-Hausdorn volume measure also valid for the Holmes-Thomee volume measure.
Definition A.10. For given point the manifold, we define the Finsler indicatrix as the set of vectors in the langent space such that the Finsler metrie is equal to one: {u E T,M)F(u) 1H). We denote the Euclidean unit ball in Rd by Bd (1) and for measurable subsets S [ IRd we use vol(S) to denote the standard Euleidean volume S. In local coordinates (c!,.... edd) on a Finsler manifold M, the Busemann- Srnemmnn-Hausdorff Hausdorff volume form is defined as dVF = VF(æ)e) A .... ed, with:
(L)(B((1)
Vz(x) = vol((u E T.M F()) < 1])
We can interpret the volume the ratio between the euclidean ball, and convex ball whose radius is defined a unit Finsler metric. If the Finsler metric is replaced by Riemannian metric, the volume of the indicatrix will be an ellipsoid whose semi-axis are equal to the inverse of the squareroot of the metric's eigenvalues. The Finsler volume then reduces to the definition of the Riemannian volume.
A.4 Random manifolds
So far, we have only considered deterministic data points lying manifold. If we consider our data to be random variables, we will need to define the associated random metric and manifold,
21

---

Published in Transactions on Machine Learning Research (10/2023)
(1) = J3,
I
M
Figure 11: Usually the immersion f would deterministic. In the case of most generative models, where f is described by GP-LVM, the decoder of VAE, the immersion is stochastic. The pullback metric is stochastic de facto.
As said previously, if we have function f : 12 + R that parametrises manifold, then we can construct Riemannian metric G = J Jy. with Jy the Jacobian of the function In the previous cases, assumed J to be a deterministie function, and so is the metric. We construct a stochastic Riemannian metric in the same way, with f being a stochastic process. A stochastic process f R2 + RD is a random map in the sense that samples of the process are maps from Rº to RD (the so-called sample paths of the process).
Definition A.11. A stochastic process / R + RDD is smooth the sample paths of f are smooth. We call a smooth process a stochastic immersion the Jacobian matrix of its sample paths has full rank everywhere. We can then define the stochastic Riemannian metric = JJ Jy.
The terms stochastic and random are used interchangeably. The definition of the stochastic immersion is fairly important, it means that its Jacobian is full rank. Since the Jacobian is full rank, the random metric G is positive definite, necessary condition to define a Riemannian metric. Another definition of a stochastic Riemannian metric would be the following:
Definition A.12. stochastic Riemannian metric on RRY matrix-valued random field on R whose sample paths are Riemannian metrics. A stochastic manifold is differentiable manifold equipped with a stochastic Riemannian metric.
Any matrice drawn from this stochastic metric would be proper Riemannian metric. When using the random Riemannian metric on two vectors u, E T,M, C(u,v) = T Gv is a random variable, but both are deterministic vectors. From this definition, it follows that the length, the energy and the also volume are random variables.
Object
Riemann
Finsler
metric
g T: x T.M M + R
F: P:TM- + +R+
length structure
La(?) = S (g((((0),()))
LF(T) = f F({(t) dt
energy structure
Ea(r) gi(((t), (()) dt
Ep(7) 1 2()(t)) dt
volume element
Va(x) detetGH
V^(z) vol (1))/vol({r ())ovv))) e F.M(F(z, Busemann-HHausdorll volume measure
< <1)
Table Comparison of Riemannian and Finsler metries.
22

---

Published in Transactions on Machine Learning Research (10/2023)
B Proofs
One of the main challenges of this paper is to find coherent notations while respecting the tradition of two geometric fields. In Riemannian geometry, we principally use a metric, noted : TpM x TnM + R++ that is defined as an inner product and thus can induce a norm, but is not a norm. In Finsler geometry, we call intercheangeably Finsler function, Finsler metric or Finsler norm, the norm traditionally noted F: T. R., with Fp(u) == F(p. u) defined at a point p E M for a vector E TpM. We will assume that all metric are always defined for specific point (or p) on our manifold 2 (or M), and we will just drop this index. The following notations will be used:
Stochastic pullback metric tensor
G= J
Stochastic pullback metric Expected Riemannian metric
g: (u( + -u Gu
9: 0;(u,) - GGu
Stochastic pullback induced norm Expected Riemannian induced norm
Illla: : + Vu
/uTE = u(
Finsler metric
I1 : u- El u Gu) = F(u)
B.1 Finslerian geometry of the expected length
In this section, we will always let f TR9 - DDD be a stochastic immersion, Jy its Jacobian, and G = JJ metric tensor. We will first prove that the function F : TM - R : - E Vo Gu is a Finsler metric. Then, for the specific case where J, follows a non-central normal distribution, the Finsler metric F de- fined as the expected length follows a non-central Nakagami distribution and can be expressed in closed form.
To prove that the function F is indeed Finsler metric, we will need to verify the criteria above, among them the strong convexity criterion is less trivial to prove than the others. It will be detailed in Lemma B.1.3. Strong convexity means that the Hessian matrix (Hess(F(u))) non-negative v. This matrix, when F is a Finsler function, is also called the fundamental form and plays an important role in Finsler geometry. To prove the strong convexity criterion, we will need the full expression of the fundamental form, detailed in Lemma B.1.1.
d2p² T (v) is strictly positive definite for
Lemma B.1.1. The Hessian matrie I Hess (v)))) of the function F(w) = E Vo Gv is given by
2
Hesst F(6)3) Hess(F(()))= = (o) E (0 (T)) - - (v Go) G - + E [w (ITTGO) Gv) ( To_.
(
Proof. Let G be a random positive definite symmetric matrix and define R Vv Go where v is considered a column vector. We would like to know the different derivatives of g with respect to v. We name by default Jy and Hg: its Jacobian and Hessian matrix. Using the chain rule, we have: Jn = Gu G and Hg = (v T - (u Go) (GouT
For the rest of the proof, we need to show that derivatives and expectation values commute.
Using the Fubini theorem, we can show that tha derivatives and the expectation values commute.
For F: + :RUHH E Go),
Hess F) = E(H,) = E [T - Gvv G)
F [[J,]] = (1 ( Gu)-
We now consider the function : RR9 + R e + E V Goj² = F(u)². Using the chain rule and changing the order of expectation and derivatives, we have its Hessian
H 2F Hess FF] + 2VP VF
[HH 23[J,] [[Ja)-
23

---

Published in Transactions on Machine Learning Research (10/2023)
Finally, replacing Jg and Hg previously obtained in this expression, we conclude:
B [TT +G - Go) T o] + E Go) [ [t (b)+
2 Hn (2, E
Go) = vo
Remark. Before going further, il's imporrann to note that G G= JJ definite: it is symmetric by definition and has full rank. The later statement is justified by the assumption that the stochastic process f R + RD is an immersion, then Jy is full rank.
is a random malrix that is posilive
Lemma B.1.2. The function F(u)= = =B [VoTGG is:
1. posilive homogeneous: AA E RR,, F()v)= XF(v)
2. smooth: F(v) is a C function on the slit tangent bundle TM \ {0}
Proof. 1) Let X R, then we have: F(Ae) E X2 Go col = |>| Go
Gul
2) The multivariate function: [[) + Rt positive definite. The function R, - RT - v is also C Finally, E2 + R. x + E[r] by definition differentiable. By composition, F(v) is C function the slit tangent bundle TM (0).
v Ge is C and strictly positive, since G G= GJJ.J. Jz is
Lemma B.1.3. The function F(U) E
Vo
Gt satisfies the strong converity criterion.
Proof. Proving that F satisfies the strong convexity criterion is equivalent to show that the Hessian matrix H (ess(F()))) is strictly positive definite. Thus, need to prove that Vw E R²MO), w Hw 0. According to Lemma B.1.1, because the expectation is a positive function, it's straightforward to see that Vw E (0), w Hw 2 O. The tricky part of this proof is to show that w Hw > 0. This can be obtained if one of the terms (F Hess(F) or VF VF) is strictly positive.
First, lot's decompose H as the sum of matricos: H FHess (F) VF (Lemma B.1.1), with:
F. Hess(Hs))) F) = E (w Gu)
E (v Go)- -
VF VF = E Gv) Gv) t vo..
We will study two cases: when w span(v), and when & span(v). We will always assume that u * 0, and so by definition: F(v)= 0.
Let w E span(v). We will show that wt T F VFW > 0. We have av,a E R. Because F is 1- homogencous and using Euler theorem, we have: VF(F)u= F(v). Then VFF a²F(r)²>0.
= F²F2, and
Let w & span(u). being a scalar function, we have: w FHess[F]w F Hess[F]w. We would like to show that: w Fess > O. The strategy is the following: if prove that the kernel of Hess[F) is equal to the span(r), then clude that: w: Hess (Fjw > 0. Let's prove span(u) E ker(Hess( F)). We know that Hess(F)o = 0. since F' is I-homogeneous, so we have span(u) E ker(Heess(F)) (F)). To obtain the equality, we just need to prove that the dimension of the kernel is equal to 1. Let 2 E sp) which (Gu) 2 = 0. We have din(span(v' M))
span(v) is equivalent to say that & ker(Hess(F)) and can con-
1, and thus: dim(span(r') (T)
q q 1. Furthermore, 2 [ess(F]=
= ZTE (u Mo) 3 x 0, so we can deduce that dim (im (Hess F1)) q-11 1. Using the Rank Rank-Nullity theorem, we conclude that dim dim(ker(Hess/ F))) q dim (immiimHess(AA = 1, which concludes the proof. In conclusion, Vu E (0), w Hess( F(N)2)w > 0. The function F satisfies the strong convexity criterion.
24

---

Published in Transactions on Machine Learning Research (10/2023)
Proposition 2.2. Let G be stochastic Riemannian metric lensor. Then, the function F2 T.2 - lk | defines Finsler metric, but it is not induced by Riemannian metric.
Proof. Let's define F as Riemannian metric: F IR x R + R (V1, U2) + E Voj G If F were Riemannian metric, then would be bilinear, which clearly not the case. Thus, F not a Riemannian metric. According Lemma B.1.2 and Lemma B.1.2, F is Finsler metric.
Proposition 2.3. Let f be Gaussian process and its Jacobian, with J (BBJ), E). The Finsler norm can be written as:
1 + r(
2' 2 )
1 D w
Fg : T.Z + R4 | = V2vo Ev
PP
with Fi as the confluent hypergeometric function of the first kind and w (v T -v)--(v1 [()oo.
The objective of the proof is to show that, if the Jacobian J follows non-central normal distribution, then, Vo E R", the expectation E[vJJ, Jyul will follow a non-central Nakagami distribution. This is a particular case of the derivation of moments of non-central Wishart distributions, previously shown and studied by Kent & Muirhead (1984); Hauberg (2018b).
Proof.
By hypothesis, J follows a non-central normal distribution: J, NN & Σ). Then, = JJ follows a non-central Wishart distribution: G Wa(D, E, ",E,,,,EE"EJJ]EEJJJ According to (Kent & Muirhead, 1984, Theorem 10.3.5.), T Gu will also follow a non-central Wishart distribution: v Gv W2(D, (T,,)) with: (v -v [[J]][[J [J))).
To compute E[ Guj, shall look at the derivation of moments. (Kent & Muirhead, 1984, Theorem 10.3.7.)
states that: if X Wg(D, with s D, then B((det(X)))
(det (det{E)))2vt) T. + +k) (-kk, 2. 12). We directly apply the theorem to our case, knowing that Gv TGu
F)
is scalar term, so det 0 Gu) = v. Gv, q = 1, and k= 2
+ +2 I(2)
D 7
D 1
|0|| = EI Go v2v
Ee
1F(
2' 2 2 w
Lemma B.1.4. The length of a curve using a Finsler metric is incoriant by regaramelrisatioo.
Proof. The proof is similar to the one obtained on Riemannian manifold (Lee (2013), Proposition 13.25), where we make use of the homogeneity property of the Finsler metric.
Let (M, F) be Finsler manifold and [[a bj + M a piccewise smooth curve segment. We call 7 reparametrisation of 7, such that 7 == y o with o: (c. d + [a, b] diffeomorphism. We want to show that Lr(7) = Ly (7).
d LF(7) F((()) di -f [ f (To di (0) (())F(( -10-00 )
d dl
' F(+(t)) Lp(r)
Lemma B.1.5. If curve globally minimizes its energy on a Finsler manifold, then it also globally minimizes its length and the Finsler function F. of the velocity vector along the curve is constant.
25

---

Published in Transactions on Machine Learning Research (10/2023)
Proof. The curve energy and the curve length are defined as: Ef(?) So d) and LF(7) f d( with y: [0, + RS. Let's define f and two real-valued functions such that: J:R-RR 1:tH F(3(I)) and g: IR +R +R: 1R: 1. Applying Cauchy-Schwarlz inequality, we directly obtain:
P(HOOD))) S $f. [(G())P J." 1'dt, which means:
The equality is obtained exactly when the functions f and g are proportional, hence, when the Finsler function is constant.
B.2 Comparison of Riemannian and Finsler metrics
We have defined both a Riemannian (g (v1, 02) + 1. B[]0)) and Finsler (F : (2, 7 B[ve Gol) metric, in the hope to compute the average length between two points on a random manifold created by the random field f: G JJ.JJ The main idea of this section in what extend they differ in terms of length, energy and volume. From now on, f R" + RR will always be defined as a stochastic non-central gaussian process. Its Jacobian Jy also follows a non-central gaussian distribution, G Jr non-central Wishart distribution, and F (x,u) E[Vu Go] non-central Nakagami distribution (Proposition 2.3). The Finsler metric can be written in closed form.
better compare those two metrics and
In section B.2.1, we will see that the Finsler metric is upper and lower bounded by two Riemannian tensors (Proposition 3.1), and we can deduce an upper and lower bound for the length, the energy and the volume (Corollary 3.1). Then, in section B.2.2, will show that the relative difference between the Finsler norm and the Riemannian induced norm is always positive and upper bounded term that is inversely proportional to the number of dimensions D (Proposition 3.3). Similarly,
will deduce the same for the length, the energy and the volume (Corollary 3.2). From this last results, we can directly conclude in section B.2.3 that both metrics are equal in high dimensions (Corollary 3.4). A possible interpretation is that in high dimensions the data distribution obtained on those manifolds becomes more and more concentrated around the mean, reducing the variance term to zero. The manifold becoming deterministic, both metrics become equal.
Remark. Most of the following proofs will be a bit technical, as they rely on the closed form expression of the non-central Nakagami distribulion. Once proving the main propositions, oblaining the corollaries is straightforward. While we do not have closed form expression of the indicatria, we will show that it's a monotoneous function which con upper and lower bounded.
B.2.1 Bounds on the Finsler metric
2
r(p +1) [(
The Finsler norm: ||:1|p is bounded by two norms, Ih and I|- induced by the two respective Riemannian metric tensors: the covariance tensor a. and the expected metric tensor [.
Proposition 3.1. We define
V(2,2) E 2x T.Z: ||l|as < ||l|| S ||
Proof. Let's first recall that the Finsler function can be writlen as:
[ +
1 D
|| = F(e) /2V T
Fi
2' 2' w)
T(2)
(b)a ; with (a)x and (b)x being the Pochhammer symbols. Note that, despite their confusing notation, they are defined as rising factorials. By definition, we have: (ale P(a=k) 1(6). We the Kummer transformation to obtain:
The confluent. hypergeometric function is defined as: (a,
b, (a)
(b)x T(6+k [(a)*
26

---

Published in Transactions on Machine Learning Research (10/2023)
(a, -2) = a, a.b, 2). Replacing - b 2 and 200, finally get:
r + 2 k) 2
= V Ev-e
r + k)
k!
1) Let's show that: Yo E T:M: VU aLv S F(V), with = [(++
T(4)
The Pochhammer symbole is defined as (z)x x(x + 1) ... (x + k 1) (z), S (x+ 2)). Thus, [(]+)) I r((+3+8) +(
[(++k) T(m)
For x E R7, we have:
The Gamma function being strictly positive on R,, we obtain:
IT4)
r( + 1
1(2 + 1 kk)
r(
T( +k)
I(2 + 2)
2 k)
2
V2VU Ex
< V2V
Ev
T(
M!
T( k) k!
k=0
km0
r( 7)
k)
2 k!
22
Vu Vu S V2VV
[(2)
T( D 2
+ k)
VuTaCN S F(D).
2) Let's show that: Ve T:M: F(o) S E.G.
Wendel (1948) proved:
r(+ty) T(x)
I r" for > 0 and E 1. With x
+ k,
D 2
2, we obtained
-( D + k, which leads to: F(r) s 22 Ev. Lk=0 V 2
D +
+(
2
Furthermore,
Ee e
and the function
V + is concave. Then by Jensen's in-
equality:
E
0k= V
D 2
+
& / - 2t Knowing that Ex E = ze., we have:
eT 2k=0 2 T
0 +2.
And with z 2, obtain: F(v) Ve E(D+ +5b.
From (Kent & Muirhcad, 1984, 442), the expectation of non-central Wishart distribution (G E,,)), is: E[G] = DE + ΣΩ. This finally leads to:
F(v) S V 1 TEG.o.
Remark. As a side note, the second part of the inequality F(v) S Vo E|Glu can be obtoined using directly Proposition 3.2.
Corollary 3.1. The length, the energy and the Busemann-Haasddonn volume of the Finsler metric are bounded respectively by the Riemannian length, energy and volume of the covariance tensor aΣ (noted LON, Eas, V.s) and the expected metric [EG] (noted LR, ER, VR):
Yz Z, Las(z) < LF(z) < LR(Z)
Fax(2> & Er(2) & Eπ(z)
Vax(2) < Vr(2)= s Vz(=)
27

---

Published in Transactions on Machine Learning Research (10/2023)
Proof. From Proposition 3.1, we have v(z, u) E M vh(o) F(o) vg(u), with h + +
and g : uo " Riemannian metries. We also define the parametric curve: VI E R, 7() = and {tt) U.
1) Let's show that (x(x)= I (p(x) S In(x). Because of the monotonicity of the Lebesgue integrals, we directly have: I vh(i(t))dt S d( I dt(
2) Let's show that Ex(x) I Er(x) S En(x). Since all the functions are positive, we can raise them to the power two, and again, with the monotonicity of the Lebesgue integrals, we have: J h((()))d & ((0)) SJ Vg((()))
3) Let's show that (x(x)= I Vp(x) & Vn(x)) We write the vectors v E T.M in hyperspherical coordi- nates: U re, with r ||a|| the radial distance and the angular coordinates. With re, have: Me) Sr F(e) S g(e) = Vi(e) F(e)-t v Vg(e)
We want to identify an incquality between the indicatrices, noted vol( vol(Ty), vol(Ir), formed by the functions h, 9 and F. Let's define: To h(y)(e) rh g(eg rre(e) 1. For every angular coordinate e, we obtain: Th 2 rF 2 Tg. Intuitively, this means that the finsler indicatrix will always be bounded by the indicatrices formed by and g. The Busemann-Hause (s( = (1))vo()))))((()) with (1)) the volume of the unit. ball and vol(T,) the volume of the indicatrix formed by f. The previous incquality and the definition of the Bassmmnnn-aasddore volume implies that: vol(Ix) 2 vol(Ir) 2 vol(Ig) (h)) S (F()) < (g()). The functions g and h being Riemannian, we have: an(h)= = dde((aE) and (g(g) = (E) which concludes the proof.
volume of a function f is defined as:
B.2.2 Relative bounds between the Finsler and the Riemannian metric
Proposition 3.2. Let be a stochastic immersion. f induces the stochastic norm |llle. defined in Section 2. The relative difference between the Finsler norm ||-I|p and the Riemmanian norm ||-I is:
||0|| R
|wlll
Var
|
0 <
<
|10|
2B
||el
Proof. We will directly use sharpen version of Jensen's inequality obtained by Liao & Berg (2019): Let X be one-dinensional random variable with mean μ and P(X E b)) = 1, where -00 S S b < +00. Let @ twice derivable function on (a, We further define: (x), d(w)-A(p) d'(n)
(=-1)* x- Then:
inf zE(a,))
< ((E()) < sup xE(a,))
In our case, we will chose Φ 2 - v/2 with z onc-dimensional random variable defined as = = TT.. a= 0, = too and μ = =[z]. h(2,p) = (Vz - (p) μ4) (2(2 - -1 Because its first derivative o' is convex, the function x > h(x p) is monotonically increasing. Thus:
Vp 2n
inf (M(x, lim =E(0,+0o)
and
sup (h(2,p)) lim 0. =0(0++o0))
20++00
It. finally gives:
T
Var
<
E
V
E
S O.
2μ²
28

---

Published in Transactions on Machine Learning Research (10/2023)
Replacing || == F(o) B[y²] and |=
/ V vp concludes the proof.
]
Lemma B.2.1. Let o, 2t) following a one-dimensional non-central Wishart distribution. Then:
Var[z] 2
Ω
= D+Ω + (D (D+90 22)2
Proof.
(Kent & Muirhead, 1984, Theorem 10.3.7.) states that if Wi(D,a,w) then E[z*]
2+ r( +k)
1(4) [Pi(-kk 2 =2). In particular, for k and k = 2, we have (5i(-1,,,0) - and F (,, =1 - + NO+Iy- We also have T( 2 and r( I(
2 1): which leads to: E|a] (D + +2) and E|2²| = 0² (2w + 2(D + w) + (D + 2) Finally, we conclude:
T(
Var 3
2
2w
(D + w)2 + D +
3 2
Proposition 3.3. Let f be Gaussian process. We note w (uT - E[J] with J the jacobian of f, and Σ the covariance matrix of J. The relative ratio between the Finsler norm I1-Il and the Riemmanian norm ||-1 is:
|0l| R
|
0 <
|00|
D w
w)
Proof.
The result is directly obtained using Proposition 3.2 and Lemma B.2.1.
Corollary 3.2. When f is Gaussian Process, the relative ratio between the length, the energy and the volume of the Finsler norm (noted Lp, Ep, VF) and the Riemannian norm (noted LR, ER, VR) is:
0 x Ln(z)= - LF(2)
W
< max VET.2
LR(Z)
D +
(D + w)2
0 < En(2)- - Er(z)
2
1 2w
2w
< max TET.2
+
+
FxR(z)
D +
(D + w)
(D + ww3
(D ()+
0 05
VR(2) V=(z)
<
1 max vET.2
+ (D w)
w
Vn(2)
D+w
Proof. Let's call M max,e7, (Dtw)) + Diw).
From Proposition 3.3, we have:
1 +
0 < || R - -lull S M ||ll with M
max TET.M
D
w)+
1) By the monocity of the Lesbesgue integral, we can directly integrate the previous norms along a curve 7: which immediately leads to: & Ln(x)= Lr(x) z MLn(z).
2) Since all the functions are positive: 0 ||l| < | < M|lle + ||ll leads to: |
|| | + 2M [ ||lll + || and replacing ||
s ||l|| in the right hand
term: ||0/| < |||l < (M² 2M] 2 + ||u| and finally: 0g |||| || < (M² 112+77 2| Again,
by continuity of the Lebesgue integral, we directly obtain: 0 5 En(x) Ex(x) s (M² + +2M))R(())
29

---

Published in Transactions on Machine Learning Research (10/2023)
3) In order to compare the volume between the Finsler and the Riemannian metric, we need to compare the volume of their indicatrices, noted: vol(T)) and vol(Tp) respectively. We write the vectors E M in hypershperical coordinates, with v re, = I|el| the radial distance and e the angular coordinates. The volume of the indicatrices obtained in dimension q (dimension of the latent space) can be written as: dVV = drd with defining the different angles. We will note rF and rg the radial distances of the Finsler and Riemann metrics such that: ||0||p- = rrllell = 1 and | ryyle ry]|e|læ obtained for a specific angle e.
vol((p)-vol(tg) ( 19-1dr dr -r ( dΦ
o 44
1
- rs
dΦ
<
I J
dd
d$.
1
-
Fg rJ
and by definition: vol(Ip) = (2(33//d Furthermore, for a specific angle e, we have: rg/rF Vg(c)/F(e)) 2 1 I-M, from Proposition 3.3. We have:
os vol(I=) - vol((g) s.1 vol(IF)
[g [T]
S (1-(1-M),
and by the definition of the Busemann Hausdorff volume: proof.
V=(n)-Vo(s) Vr(x)
vol(Ig)-vol(t)) conclude the vol(Fr)
B.2.3 Implications in High Dimensions
In this scction, we want to show that the difference between the Finsler norm and the Riemannian inducod norm, as well their respective functionals, tend to zero at rate of ( We need to be sure that w O(D). This can be obtained if assume that every element of the expectation of Jacobian is upper bounded (3m E ji, B[Jy] S m). This happens Vi,j
doesn't grow faster than D, in other terms:
in at least two cases: (1) [[f] is somehow Lipschitz continuous; or (2) if f is a Gaussian Process and its covariance is upper bounded. The latter case happens when the process is defined over a bounded domain.
Lemma B.2.2. Our Finsler metric v - E[VuTGu) is defined with v Gu Wi(D TT,, W), and w E[J]
(v [v [o)
If the Finsler manifold is bounded, them: w S DM, with M R..
Proof. By definition, Σ does not depend on D. We assume the manifold is bounded, which means that every element of the expected Jacobian is upper boundod: B(Jli I m, with m E R2. We call v EE R1.
We have:
50-1 Σ m mlll DM,
4
with M M-o o E Ri, and M does not depend on D.
Corollary 3.3. Let ff be Gaussian Process. In high dimensions, we have:
LR(z) LF(2) ( ER(z) - Er(x) = O ( and VR(=) - VF(2) (e(=)-Vr(2) Ln(2)
En(())
Vx(²)
When D converges toward infinity: La Lr, En
Ep and Vn Vp.
t00 too
to0
Proof. From Corollary 3.2, directly obtained the results in high dimensions.
30

---

Published in Transactions on Machine Learning Research (10/2023)
We assume that our latent space is bounded, then by B.2.2, we have: 0 w MD, with M E R+. For the length, we have:
Lo(x) - LF(x
W
2 max
(O(I)
wE
D
w
(D +
+ M
D
For the energy functional, we have:
Eg(x) Ex(x)
2 +
1+ 2w
+ 2w
w²
< max TET, M
(D+w) +
EC(x)
D+w w
(D+w)²
(D w)
+ 2M
1 2M M²
D
D
lim sup D Ex(x) - -Ep(x)
M + 2M +1
< limsup 2(1 + M) +
+ 2(1 + M)
D-420
EG(x)
D-+0c
D
For the volume, we have:
Vo(œ) - VV(x)
1
(D+wp))
S 1- max "ET. M
Va(x)
D+W
+ M
<
Using Taylor series expansion, when x ~ O, we have: 1 - (1-z)² = qu. + olx?). Let's call < 1, and rewrite the Taylor series:
Va(x)- - Vi((t)
1 M
1+M + +eq D
Vo(x)
D
lipsup D D-000 q
Va(x) Vi(x))
< (1+ M)(1 +8)
Va(x)
The difference between the functionals can converge to zero if they are similar in high dimensions, or if they all diverge to infinity. This latter case does not happen as we assume the latent manifold being bounded, and so the metrics are then finite, which concludes the proof.
Corollary 3.4. Let f be a Gaussian Process. high dimensions, the relative ratio between the Finsler norm I| and the Riemmanian norm |1-11 is:
| R |
O (
|
And, when D converges toward infinity: Vo T,Z, || ~ ||
toc
Proof. Similar to the 3.3, assuming that our latent space is bounded, from B.2.2, we have 0 S w S MD. From 3.3, we deduce:
| full
|
1
W
0 <
<
+
||0||
D +
(D + w)
< + M
D
31

---

Published in Transactions on Machine Learning Research (10/2023)
In bounded manifold, the metric are finite. We deduce that they converge to each other in high dimensions.
C
Experiments
C.1
Datasets
C.1.1
Font data
The datasct represents 46 different font for cach letter (upper and lower case) whose contour is paramctrisod by a spline (or two splines, depending on the letter used) obtained from at least 500 points Campbell Kautz (2014).
In case, choose to learn the manifold of the letter f. The dataset is composed of 46 different fonts, each letter being drawn by 1024 points. We reduce this number from 1024 to 256 by sampling one point every 4. The dimension of the observational space is then 256.
C.1.2 qPCR
The qPCR data, gathered from Guo et al. (2010), was used to illustrate the training of a GPLVM in Pyro Pyro (2022) and available at the Open Data Science repository Ahmed al. (2019). It consists of 437 single-cell qPCR data for which the expression of 48 genes has been mcasured during 10 different cell stages. We then have 437 data points, 48 observations, and 10 classes. Before training the GP-LVM, the data is grouped by the capture time, as illustrated in the Pyro documentation.
C.1.3 Pinwheel on a sphere
A pinwheel in 2-dimension is created and then projected onto sphere using stereographic projection method. The final dataset is composed of 1000 points with their coordinates in 3-dimensions.
C.2 GP-LVM training
We learn our two-dimensional latent space by training GP-LVM Lawrence (2003) with Pyro Bingham et al. (2019). The Gaussian Process used is a Sparse GP, defined with kernel (RBF, or Matern) composod of one-dimenssonal lengthscale and variance. The parameters learnt with the Adam optimiser Kingma & Ba (2014). The number of steps and the initialisation of the latent space vary with the dataset.
datasets
pinwheel
font data
qPCR
Number of data points
500
46
437
Number of observations
256
48
initialisation
PCA
PCA
custom
kernel
RBF
Matern52
Matern52
steps
17000
5000
5000
learning rate
1c-3
1c-4
1c-4
hscale
0.24
0.88
0.15
variance
0.95
0.30
0.75
noise
1e-4
1c-3
1c-3
Table 2: Description of the datasels trained with a GP-LVM.
C.3 Computing indicatrices
An indicatrix of function g at a point z defined such that: E (g))) < 1. In other terms, the indicatrix the representation of a unit ball in our latent space. If we use an euclidean metric, our indicatrix in our 2-dimensional latent space would be unit ball, need to solve: T.M, ||l|| <
32

---

Published in Transactions on Machine Learning Research (10/2023)
For Riemannian metric, our indicatrix is necessarily ellipse, whose semi axis are the square-roots of the eigenvalues of the metric tensor G:: v E TrM, v Gu < 1. For our Finsler metrie, we don't have an analytical solution, and so it's difficult to predict the shape of the convex polygon.
In this paper, the indicatrices are drawn the following way: for a single point in our latent space, we compute the value of v Gv and F(x, w) for v varying over the space. We then extract the contour when v Gu and F(x, v) are equal to Computing the area of the indicatrices will be used in the section C.4 compute the volume measures.
C.4 Computing the volume forms
For the figures used in this paper, by default, the background of the latent space represents the volume measure of the expected Riemannian metric (Vg vE[G]) on logarithm scale. In figure 3, the volume mcasure of the Finsler metric is also computed.
C.4.1 Finsler metric
To compute the volume measure of our Finsler metric, we choose the Busemann-Hausdonn definition, which is the ratio of a unit ball over the volume of its indicatrix: While our Finsler function has an analytical form, its expression doesn't allow to directly solve the equation: M, F(x, v) < 1. Instead we approximate its indicatrix describe in C.3, using a contour plot and extracting the paths vertices. We can then compute the area of the obtained polygon, and divide with the volume of unit ball: v1)( = π.
(1)) E T,M F(xr, u) < 1)).
The volume measure can then be computed for each point over a grid (32 x 32, in figure 3), and we interpolate all the other points. Note that this method can only be used when our latent space of dimension 2.
C.4.2 Expected Riemannian metric
There is two ways to compute the volume measure of the expected Riemannian metric. One way is to directly use the metric tensor: VG VEGG Another one is to remember that any Riemannian metric is a Finsler metric, and thus, the Busernann- Haunnn- definition also applied for our metric: Va = vol (1)) MM B[G]u < 1)). Solving v E[G]u < 1 for v E T.M equivalent to solving the area of an ellipse.
For the first method, we can either sample multiple times the metric, which is computationally expensive, or use the fact that our metrie (ensor is a non-central Wishart matrix: G JTJ WD, E, EEEE] [[J], with Σ the covariance of the Jacobian J and D the dimension of the observational space. In this case, its expectation is: E[G] = [[]] + DE. We can access the derivatives of the function f (detailed in section D.2), and compute both quantities [[J] and Σ needed to estimate the expected metric and its determinant.
For the second method, we can compute the area of the ellipse in the same way we compute the Finsler volume mcasure.
C.5 Experiments when increasing the number of dimensions
In Figure 4, we computed the volume ratio and draw indicatrices while varying the number of dimensions, to illustrate that both our Finsler metric and the expected Riemannian metric seem to converge when D increases.
The main issue with this experiment is to vary only one factor, the number of dimensions D, while keeping the other factors unchanged. This is difficult for two reasons: 1) Even with a very low dimensional observational, both metrics are already very similar to each other. It would be difficult to illustrate convergence while increasing the number of dimensions. 2) the function f needs to be learnt again each time we increase the number of dimensions of the observational space, and the parameters of the Gaussian Process will change too.
Instead, we try illustrate our results by computing empirically the stochastic metric tensor JJJ, using its Jacobian J N[(E(]], E), D y matrix. The number of dimensions is modified by simply
33

---

Published in Transactions on Machine Learning Research (10/2023)
truncatenating the Jacobian J. In Figure 4, the volume ratio is computed for 12 Jacobians obtained with different random parameters B and E. The Finsler and Riemannian indicatrices (lower right) are drawn for only one Jacobian selected randomly.
D Computations
D.1 Computing geodesics with Stochman and minimising the curve energy functionals
An essential task to compute shortest paths, or geodesics, between data points in the latent space. Those shortest paths can be obtained in two ways: either by solving corresponding system of ODEs, by minimising the curve energy in the latent space. The former being computationally expensive, we favour the second approach which consists in optimising a parameterised spline on the manifold. This method is already implemented in Stochman Detlefsen et al. (2021), where we can easily optimise splines by redefining the curve energy function of a manifold class.
We need two curvge energy functional: one for the expected Riemannian metric and one for the Finsler metric.
D.1.1 Curve energy for the Riemannian metric
We know that the stochastic metric tensor G, defined on a point t follows non-central Whishart distribution. Thus, we can compute its expectation E[G]] knowing the Jacobian covariance and expectation: E[J[ and E. The next Section D.2 explains how to compute those quantilies.
Assuming the spline is discretized into N points, we compute the curve energy with:
Ea(r((t) ( ((t) (t)) d(G.)(()) MT (E[JI [B]A+ + DE.) n.
D.1.2 Curve energy for the Finsler metric
In order to compute of the curve energy {()), we must first derive the expectation E[J[] and covariance Σ of the Jacobian of f, which should follow a normal distribution: Jj Öf, ~ V E). We assume the points of, are independent samples with the same variance drawn from a normal distribution. We can then compute the Finsler metric which follows non-central Nakagami distribution (See Proposition 2.2):
N 8(((1) = S F(, & 24d E
r + T(
2
1 D Wi
F
2 2'
2
with Fi the confluent hypergeometric function of the first kind and w; = (4, Ex)) 26T 94:1)) and Ω = Σ, "[[[]T[[J.]
This function has been implemented in Pytorch using the known gradients for the hypergoometric function: 8 (a, = (a 1,b 1,x). 1,
D.2 Accessing the posterior derivatives
We assune that the probabilitic mapping f from the latent variables X the observational variables Y follows normal distribution. We would like to obtain the posterior kernel E, and expectation μ4, such that p(a,flY,X) ~ N NN (u.,E.).
We make the hypothesis the observed variables are modelled with a gaussian noise e whose variance is the same in every dimension. In particular, for the th latent (æ) and observed (y) variable in the jth dimension: Yo.j = (2n.)) + Thus, the output variables have the same variance, and the posterior kernel Σ, then isotropic with respect to the output dimensions: Σ, = Ip.
There two ways of obtaining the posterior variance and expectation:
34

---

Published in Transactions on Machine Learning Research (10/2023)
We use the gaussian processes to predict the derivative (0af) of the mapping function and we multiply the obtained posterior kernel by the curve derivative (dxc), following the chain rule: fl (Section: D.2.1)
dt
We discretize the derivative of the mapping function as the difference of this function evaluated at two close points. We use a linear operation to obtain the posterior variance and expectation: 0. O.f(c(()) ((c(((41)) - f(c(4)). (Section: D.2.2)
D.2.1 Closed- form expressions
We assume that f a Gaussian process. Hence, because the differentiation is a linear operation, the derivative of a Gaussian Process also Gaussian Process Rasmussen & Williams (2005).
The data Y E R follows normal distribution, so can infer the partial derivative of one data point
(J dys with i = 1... D and j = =l...d d. We have: 8my"
Y (J)
D
N
([
Py
].
1
K(x,x) 0K x
0K ((,,,,,)) 1).
0K
Moy
1=1
and J can be predicted:
(Y) II N (u.. i=1
with:
He = [K(C..)) K(x,x)-+ (y - Myy) + pau
E. = K (x., - (K(Ta, K ((x,x)= 0K x.).
Finally, 0.f is obtained:
D ,) II N (
D.2.2 Discretization
One can notice that: aeS(c(()) [c(li+1)) Se((t)). We know that ((((1+1)) and [(e(()) both follows a normal distribution.
f(((ts) f(c(ta4)) -
D II N ([
].
oi oi
1)
M4 Mi+l
1=1
y = AX affine transformation of a multivariate Gaussian X ~ NNn, o²), then Y is also a multivariate Gaussian with: Y ~ NAA,, 2² In our case, we choose AT = [-1, We have:
- ~ NN,,, II).
with:
P. = Pi+1 pi
0 o +
2 041,1411 20
35

---

Published in Transactions on Machine Learning Research (10/2023)
1.00
0.75
0.50
0.25
0.00
-0.25
0.50
-075
-1.00
-0.75
-0.50
-0.25
0.00
0.25
0.50
0.75
1.00
predictions
computing posterior
discretization
theoretical derivatives
Figure 12: Illustration of the derivatives obtained with trained GP on simple parametrised function: both methods give the correct derivatives if enough points are sampled.
36


Reviews:
[{"review_id": "lMocQQr6iY", "reviewer": "Reviewer_X3sD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. The paper is novel in investigating the metric whose geodesics explicitly minimize the expected length of the pullback metric via the Finsler geometry framework. It does indeed prove that this metric converges to the expected Riemannian metric for high dimensions, providing a practical justification for using the Riemannian metric for high dimensional data.\n\n2. Ideas in the paper are clearly presented, Figures 2-4 provide great visual aids for the presented material.\n\n## Weaknesses\n\n1. The impact of the paper is relatively marginal given the fact that the largest result is to provide a justification for a previous practice in a niche subfield. The test tasks seem very simple/toy, but are ultimately sufficient to demonstrate what the paper wanted. I would have liked to see more investigation into the low-dimensional cases where the Riemannian and Finsler metrics are very different, particularly for low dimensions. Are there any (common) cases for which the traditional Riemannian approximation completely fails/misrepresents geometry, but the Finsler metric does well (in essence, a deeper investigation of what was done in Figure 5).\n\n2. In terms of writing, the paper could be more formal in presenting certain topics. For example, in Section 3.1 writing the parenthetical in \"...define a metric (a norm)\" is a bit confusing. A metric and a norm are very different, yet the parenthetical seems to imply they are interchangeable. Additionally, language such as \"We have to place ourselves in hyperspherical coordinates...\", present in the proof for Corollary 3.1, is rather informal and should be altered. For example, \"We recast the problem in hyperpsherical coordinates...\". \n\n## Verdict\n\nThe paper presents a novel investigation of the Finsler metric whose geodesics explicitly minimize the expected length of the pullback metric in the context of existing papers such as Tosi et al. (2014), and shows that this metric converges to the expected Riemannian metric for high dimensions, thereby providing a theoretical justification for using this approximation for high dimensions. It also empirically investigates the differences between the Finsler and expected Riemannian metric, showing them to be quite similar for high dimensions. Although the impact is a bit limited due to the narrow scope and the writing could use some work, I believe the claims made in the submission are well supported and the findings of the paper would be interesting to some portion of TMLR's audience. As such, both criteria for acceptance to TMLR are satisfied; ergo, I recommend an accept rating for the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The work studies tools for investigating the latent space geometry of generative models. It first notes that traditionally, one would like to investigate said geometry by pulling back a Riemannian metric (say, the Euclidean metric) from the data manifold. Since generative models are often stochastic, previous work approximates the stochastic pullback metric by its expectation. However, this is not fully principled since the geodesics derived from the expected Riemannian metric do not correspond to the expected length-minimizing curves. To resolve this, the work proposes a Finsler metric whose geodesics explicitly minimize the expected length of the pullback metric. The paper introduces bounds to compare the expected Riemannian metric with this Finsler metric and further prove that the metrics converge to each other at a rate of $\\mathcal{O}(1/D)$. Several experiments are given in Section 4; both synthetic test tasks that demonstrate these metrics can behave quite differently for low dimensions, as well as test tasks that demonstrate these metrics behave almost identically for high dimensions. The ramifications of the paper are that it provides a justification for using the expected Riemannian metric (used in papers such as Tosi et al. (2014) and Arvanitidis et al. (2018)) for practical implementations when the data dimension is quite high (e.g. more than $100$).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Content-wise, I think the work is complete as is. In terms of writing, I gave some parts of the paper that I think should be rewritten/clarified above, in point 2 of the Weaknesses section. I would like to suggest some minor writing corrections that will improve the writing via the following non-exhaustive list:\n\n- Abstract, second paragraph: \"Generative models are often stochastic causing the data space, the Riemannian metric, and the geodesics to be stochastic as well\" -> \"Generative models are often stochastic, causing the data space, the Riemannian metric, and the geodesics, to be stochastic as well\"\n\n- Introduction (Section 1), fifth paragraph: \"The pullback metric is then stochastic, and standard differential geodesic constructions no longer applies.\" -> \"The pullback metric is then stochastic, and standard differential geodesic constructions no longer apply.\"\n\n- Introduction (Section 1), sixth paragraph: \"...but the derived length are unintuitive quantities that do not correspond to the expected length\" -> \"...but the derived length is an unintuitive quantity that does not correspond to the expected length\"\n\n- Section 1.1, fourth paragraph: \"...when the dimension of the data space increase\" -> \"...when the dimension of the data space increases\"\n\n- Section 1.1, fourth paragraph: \"...we perform some experiments in Section 4, that illustrates\" -> \"...we perform some experiments in Section 4, that illustrate\"\n\n- Footnote on page 4: \"the decoder...\" -> \"The decoder...\"\n\n- Section 2.1, first paragraph: \"...metric is defined as an Riemannian metric\" -> \"...metric is defined as a Riemannian metric\"\n\n- Section 3, third paragraph: \"...euclidean\" -> \"...Euclidean\"\n\n- Section 4, first paragraph: \"...consisting of of pinwheel and concentric circles\" -> \"...consisting of pinwheel and concentric circles\"\n\n- Section 4.2, second paragraph: \"...Finler geometry\" -> \"...Finsler geometry\"\n\n- Section 4.2, first paragraph: \"Using stochastic active sets has been shown to give reliable undertainty estimates, and the model also learn more meaning latent representations\" -> \"Using stochastic active sets has been shown to give reliable uncertainty estimates, and the model also learns more meaningful latent representations\"\n\n- Discussion (Section 5), third paragraph: \"In practice we find that geodesics under the Finsler the Riemannian metric are near identical\" -> \"In practice we find that geodesics under the Finsler and the Riemannian metrics are nearly identical\"", "broader_impact_concerns": "The paper is of a fairly theoretical nature; as such, I have no ethical concerns and do not believe the paper requires a broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UEdaqxl4V1", "reviewer": "Reviewer_yi4b", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1) The proposed method is interesting and has nice connections to Finslerian geometry\n2) Paper does a thorough comparison of the two types of geometry. In particular, the paper does a very good job of connecting the data density, the Finselerian norm and the Riemannian norm. The paper bounds the Finserlerian quantities in terms of the Riemannian ones. The strength of the bound depends on the data density. \n3) The theoretical results are supported by experimental validation. The figures are well illustrated and help the reader understand the relevant concepts. \n\n**Weaknesses**\n\nSee questions. \n\nSee questions\n\n**Questions**\n\n0) What is the Jacobian of a stochastic map? Why do we expect in general realizations of the stochastic map to be differentiable? I think there are assumptions missing in definition 2.3.\n\n1) Bottom of page 5, it says $L_F(\\gamma) = \\int\\mathbb{E}\\sqrt{\\dot{\\gamma}^T G \\dot{\\gamma}}dt =  \\mathbb{E}[L_R(\\gamma)]$. But that second equality does not seem correct. We need to replace $G$ by $\\mathbb{E}[G]$ for that to hold. This equality not being true *seems to be the main point of the paper*. \n\n2) What is a Gaussian process on a manifold? Is that for any finite subset of points on the manifold, the marginal distribution is multivariate Gaussian in the standard sense?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the pull back metric associated with a stochastic map. Pull back metrics are defined by the Jacobian of the map. However, if the map is stochastic, then the Jacobian of the map is stochastic. The paper states computing the expected metric tensor results in Riemannian metric. However, if we delay computing the expectation to after computing the quadratic form, then this is no longer Riemannian, but is now instead Finslerian. The paper then derives various properties of this geometry in relation to the Riemannian one obtained by taking the expectation earlier. \n\nThe paper supports the theoretical results with numerical simulations and even provides an example where the Finserlerian geometry is more useful. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In the introduction's first paragraph. It would be nice to distinguish between the maps from $\\mathcal{X}$ to $\\mathcal{Z}$. Generative models learn $\\mathcal{Z} \\to \\mathcal{X}$. However, the introduction wants to learn the inverse. ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "dvd5q9FPSJ", "reviewer": "Reviewer_uaUo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n-----------\n* The method is motivated through a natural geometric framework.\n* The theory provides a justification for previous methods.\n* The experiments showcase that the correspondence between the Finslerian and Riemannian metrics does emerge for real-world datasets (MNIST and fashion MNIST).\n\nWeaknesses\n--------------\n* The general setting of the paper is rather narrow (proving that a **specific** approximation to a **specific** Riemannian metric on a **specific** type of generative model is justified). I am concerned that this nicheness, when coupled with the rather uncommon background required to understand the paper, can leave this paper out-of-scope of the TMLR audience.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper analyzes differential-geometric based metrics used in some analyses of generative models like VAEs or GPs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "N/A", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Mentioned above. Claims are justified. All the reviewers believe the paper is now in a better shape than the earlier versions. Please take care of the changes asked in the camera-ready version.

Note: this was a major revision of an earlier submission id=bm2XSzY6o7.
Decision: Accept (poster)