Title: Estimating Riemannian Metric with Noise-Contaminated Intrinsic Distance
Venue: NeurIPS 2023
Accepted: yes
Keywords: metric learning

manifold learning

local metric

dissimilarity

geometry
Abstract: We extend metric learning by studying the Riemannian manifold structure of the underlying data space induced by similarity measures between data points. The key quantity of interest here is the Riemannian metric, which characterizes the Riemannian geometry and defines straight lines and derivatives on the manifold. Being able to estimate the Riemannian metric allows us to gain insights into the underlying manifold and compute geometric features such as the geodesic curves. We model the observed similarity measures as noisy responses generated from a function of the intrinsic geodesic distance between data points. A new local regression approach is proposed to learn the Riemannian metric tensor and its derivatives based on a Taylor expansion for the squared geodesic distances, accommodating different types of data such as continuous, binary, or comparative responses. We develop theoretical foundation for our method by deriving the rates of convergence for the asymptotic bias and variance of the estimated metric tensor. The proposed method is shown to be versatile in simulation studies and real data applications involving taxi trip time in New York City and MNIST digits.

Estimating Riemannian Metric with Noise-Contaminated Intrinsic Distance

Abstract

We extend metric learning by studying the Riemannian manifold structure of the underlying data space induced by similarity measures between data points. The key quantity of interest here is the Riemannian metric, which characterizes the Riemannian geometry and defines straight lines and derivatives on the manifold. Being able to estimate the Riemannian metric allows us to gain insights into the underlying manifold and compute geometric features such as the geodesic curves. We model the observed similarity measures as noisy responses generated from a function of the intrinsic geodesic distance between data points. A new local regression approach is proposed to learn the Riemannian metric tensor and its derivatives based on a Taylor expansion for the squared geodesic distances, accommodating different types of data such as continuous, binary, or comparative responses. We develop theoretical foundation for our method by deriving the rates of convergence for the asymptotic bias and variance of the estimated metric tensor. The proposed method is shown to be versatile in simulation studies and real data applications involving taxi trip time in New York City and MNIST digits.

1 Introduction

With increasing data complexity, it has received increasing attention for capturing the geometric structure of the data space characterized by a Riemannian manifold, which consists of two vital parts: the coordinate system and the Riemannian metric. Manifold learning attempts to find low-dimensional representations as coordinates for the high-dimensional data [23, 27, 7]. Yet existing methods generally assume the local geometry is given, e.g., by the Euclidean distance between ambient data points, leading to non-isometric methods that distort the data geometry in the embedded space. On the other hand, many methods have been proposed to consider the Riemannian metric, including parametric estimation [17, 16, 15]; multi-metric learning [12]; pushforward via the Laplacian operator [21]; pull-back via Jacobian of generative models [1, 2]; Bayesian approach based on arc length of Gaussian process [14]; and dynamics of evolving probability when time-indexed samples are available [24]. It is also possible to recover the manifold in an abstract setting with only similarity measures among data points [9].

We propose to estimate the Riemannian metric utilizing similarity measures between data points when data coordinates are available. Suppose that data are generated from an unknown Riemannian manifold, while the Euclidean distance between the coordinates may not reflect the underlying geometry. We model the observed similarity measures between data points as noise-contaminated intrinsic distances, which are then used to characterize the intrinsic geometry on the Riemannian manifold. The targeted Riemannian metric is estimated in a data-driven fashion, which enables path finding via geodesics and calculus on the manifold in which the data reside.

This problem is closely related to metric learning [3, 26] where a Mahalanobis distance is commonly used to obtain the best distance for classification [11] and clustering [29] tasks. While a global metric is often the focus of earlier works, multiple local metrics [10, 28, 22, 5] are found to be useful because they better capture the data space geometry. Despite the resemblance, our target is to learn the Riemannian metric instead of the distance metric , which fundamentally differentiates our proposal from metric learning. We emphasize the data space geometry rather than obtaining a metric optimal for subsequent supervised learning tasks. The additional smoothness of the Riemannian metric tensor field allows analysis of finer structures, while the coefficient matrix for the Mahalanobis distance is locally constant for distance metric learning.

To formulate the problem, let ( M , G ) be a Riemannian manifold with Riemannian metric G and induced geodesic distance dist ( · , · ) which measures the true intrinsic difference between points. Knowing the coordinates of data points x 0 , x 1 ∈ M , we identify each point via its coordinates (a tuple of real numbers). The noisy measures y of the intrinsic distance between data points are referred to as similarity measures (equivalently dissimilarity). The response is modeled flexibly, and we consider the following common scenarios: (i) noisy distance, where y = dist ( x 0 , x 1 ) 2 + glyph[epsilon1] for error glyph[epsilon1] , (ii) similarity/dissimilarity, where y = 0 if the two points x 0 , x 1 are considered similar and y = 1 otherwise, and (iii) relative comparison, where a triplet of points ( x 0 , x 1 , x 2 ) are given and y = 1 if x 0 is more similar to x 1 than to x 2 and y = 0 otherwise. The binary similarity measurement is common in computer vision [e.g. 6], while the relative comparison could be useful for perceptional tasks and recommendation system [e.g. 25, 4]. We aim to estimate the Riemannian metric G and its derivatives using the coordinates and similarity measures among the data points.

The major contribution of this paper is threefold. First, we formulate a framework for probabilistic modeling of similarity measures among data on manifold via intrinsic distances. Such a framework also justifies the distance metric learning from a geometrical perspective and unveils its approximation error for the first time as we know. Second, a theoretical foundation is developed for the proposed method including asymptotic consistency. Last and most importantly, the proposed method provides a geometric interpretation for the structure of the data space induced by the similarity measures, as demonstrated in the numerical examples that include a taxi travel and an MNIST digit application.

2 Background in Riemannian Geometry

For brevity, metric now refers to Riemannian metric while distance metric is always spelled out. Throughout the paper, M denotes a d -dimensional manifold endowed with a coordinate chart ( U, ϕ ) , where ϕ : U → R d maps a point p ∈ U ⊂ M on the manifold to its coordinate ϕ ( p ) = ( ϕ 1 ( p ) , . . . , ϕ d ( p ) ) ∈ R d . Without loss of generality, we identify a point by its coordinate as ( p 1 , . . . , p d ) , suppressing ϕ for the coordinate chart. Upper-script Roman letters denote the components of a coordinate, e.g., p i is the i -th entry in the coordinate of the point p , and γ i is the i -th component function of a curve γ : R ⊃ [ a, b ] →M when expressed on chart U . The tangent space T p M is a vector space consisting of velocities of the form v = γ ′ (0) where γ is any curve satisfying γ (0) = p . The coordinate chart induces a basis on the tangent space T p M , as ∂ i | p = ∂/∂x i | p for i = 1 , . . . , d , so that a tangent vector v ∈ T p M is represented as v = ∑ d i =1 v i ∂ i for some v i ∈ R , suppressing the subscript p in the basis. We adopt the Einstein summation convention unless otherwise specified, namely v i ∂ i denotes ∑ d i =1 v i ∂ i , where common pairs of upper- and lower-indices denotes a summation from 1 to d [see e.g., 18, pp.18-19].

The Riemannian metric G on a d -dimensional manifold M is a smooth tensor field acting on the tangent vectors. At any p ∈ M , G ( p ) : T p M× T p M→ R is a symmetric bi-linear tensor/function satisfying G ( p )( v, v ) ≥ 0 for any v ∈ T p M and G ( p )( v, v ) = 0 if and only if v = 0 . On a chart ϕ , the metric is represented as a d -byd positive definite matrix that quantifies the distance traveled along infinitesimal changes in the coordinates. With an abuse of notation, the chart representation of G is given by the matrix-valued function p ↦→ G ( p ) = [ G ij ( p )] d i,j =1 ∈ R d × d for p ∈ M , so the distance traveled by γ at time t for a duration of dt is [ G ij ( γ ( t )) ˙ γ i ( t ) ˙ γ j ( t )] 1 / 2 . The intrinsic distance induced by G , or the geodesic distance , is computed as dist ( p, q ) = inf α ∫ 1 0 ¨ ∑ 1 ≤ i,j ≤ d G ij ( α ( t )) ˙ α i ( t ) ˙ α j ( t ) dt for two points p, q on the manifold M , where infimum is taken over any curve α : [0 , 1] →M connecting p to q .

A geodesic curve (or simply geodesic ) is a smooth curve γ : R ⊃ [ a, b ] →M satisfying the geodesic equations , represented on a coordinate chart as

where over-dots represent derivative w.r.t. t ; Γ k ij = 1 2 G kl ( ∂ i G jl + ∂ j G il -∂ l G ij ) are the Christoffel symbols at p ; and G kl is the ( k, l ) -element of G -1 . Solving (2.1) with initial conditions 1 produces geodesic that traces out the generalization of a straight line on the manifold, preserving travel direction with no acceleration, and is also locally the shortest path.

Considering the shortest path γ connecting p to q and applying Taylor's expansion at t = 0 , we obtain dist ( p, q ) 2 ≈ ∑ 1 ≤ i,j ≤ d G ij ( p )( q i -p i )( q j -p j ) , showing the connection between the geodesic distance the Mahalanobis distance. Our estimation method is inspired by this approximation, and we will discuss the higher-order terms shortly which unveil finer structure of the manifold.

3 Local Regression for Similarity Measurements

3.1 Probabilistic Modeling for Similarity Measurements

Suppose that we observe N independent triplets ( Y u , X u 0 , X u 1 ) , u = 1 , . . . , N . Here, the X uj are locations on the manifold identified with their coordinates ( X 1 uj , . . . , X d uj ) ∈ R d , j = 1 , 2 , and Y u are noisy similarity measures of the proximity of ( X u 0 , X u 1 ) in terms of the intrinsic geodesic distance dist ( · , · ) on M . To account for different structures of the similarity measures, we model the response in a fashion analogous to generalized linear models. For X u 0 , X u 1 lying in a small neighborhood U p ⊂ M of a target location p ∈ M , the similarity measure Y u is modeled as

where g is a given link function that relates the conditional expectation to the squared distance. Example 3.1 . We describe below three common scenarios modeled by the general framework (3.1).

Continuous response being the squared geodesic distance contaminated with noise:

where ε 1 , . . . , ε u are i.i.d. mean zero random variables, and σ : M→ R + is a smooth positive function determining the magnitude of noise near the target point p . This model will be applied to model trip time as noisy measure of cost to travel between locations.

Binary (dis)similarity response:

for some smooth function h : M→ R , where logit( µ ) = log ( µ/ (1 -µ )) , µ ∈ (0 , 1) is the logit function. This models the case when there are latent labels for X uj and Y u is a measure of whether their labels are in common or not. The function h ( p ) in (3.3) describes the homogeneity of the latent labels for points in a small neighborhood of p . The latent labels could have intrinsic variation even if measures are made for the same data points x = X u 0 = X u 1 , and the strength of which is captured by h ( p ) .

Binary relative comparison response, where we extend our model for triplets of points ( X u 0 , X u 1 , X u 2 ) , where Y u stands for whether X u 0 is more similar to X u 1 than to X u 2 :

so that the relative comparison Y u reflects the comparison of squared distances.

3.2 Local Approximation of Squared Distances

It is the squared distance that determines the responses in our model (3.1), which is justified by the following local approximation. Proposition 3.1 provides a Taylor expansion for the squared geodesic distance between two geodesics with same starting point but different initial velocities (see Figure S5.1 for visualization). It is the key tool to estimate model (3.1) through local regression. Furthermore, (3.5) characterize the error of approximating geodesic distance with Mahalanobis distance. For a point p on the Riemannian manifold M , let exp p : T p M → M denote the exponential map defined by exp p ( tv ) = γ ( t ) where γ is a geodesic starting from p at time 0 with initial velocity γ ′ (0) = v ∈ T p M . For notational simplicity, we suppress the dependency on p in geometric quantities (e.g., the metric G is understood to be evaluated at p ). For i = 1 , . . . , d , denote δ i = δ i ( t ) = γ i ( t ) -γ i (0) as the difference in coordinate after a travel of time t along γ .

Proposition 3.1 (spread of geodesics, coordinated) . Let p ∈ M and v, w ∈ T p M be two tangent vectors at p . On a local coordinate chart, the squared geodesic distance between two geodesics γ 0 ( t ) = exp p ( tv ) and γ 1 ( t ) = exp p ( tw ) satisfies, as t → 0 ,

where for i, j, k, l, m = 1 , . . . , d ,

δ i 0 = γ i 0 ( t ) -p i , δ i 1 = γ i 1 ( t ) -p i , and δ i 0 -1 = δ i 0 -δ i 1 , i.e., δ i 0 , δ i 1 are differences in i -th coordinates of γ 0 ( t ) and γ 1 ( t ) to the origin p , respectively, and δ i 0 -1 = δ i 0 -δ i 1 is the coordinate difference between γ 0 ( t ) and γ 1 ( t ) ;

G ij and Γ j kl are the elements of the metric and Christoffel symbols at p , respectively.

Proof.

To the RHS of (3.5), the first term is the quadratic term in distance metric learning. The second term is the result of coordinate representation of geodesics. It vanishes under the normal coordinate where the Christoffel symbols are zero. 2 It inspires the use of local regression to estimate the metric tensor and the Christoffel symbols. For X u 0 , X u 1 in a small neighborhood of p , write the linear predictor as

a function of the intercept β (0) and coefficients β (1) ij , β (2) ijk , where δ i u 0 = X i u 0 -p i , δ i u 1 = X i u 1 -p i , and δ i u, 0 -1 = δ i u 0 -δ i u 1 , for i, j, k, l = 1 , . . . , d , and u = 1 , . . . , N . The intercept term β (0) is included for capturing intrinsic variation (e.g., h ( p ) in (3.3)) and can otherwise be dropped. The link function connects the linear predictor to the conditional mean via µ u := g -1 ( η u ) ≈ E ( Y u | X u 0 , X u 1 ) as indicated by (3.1) and (3.5), where µ u is seen as a function of the coefficients β (0) , β (1) ij , and β (2) ijk . Therefore, upon the specification of a loss function Q : R × R → { 0 } ∪ R + and non-negative weights w 1 , . . . , w N , the minimizers

subject to

are used to estimate the metric tensor and Christoffel symbols, obtaining

where ˆ G kl is the matrix inverse of ˆ G satisfying ˆ G kl ˆ G kj = 1 { j = l } . The symmetry constraints (3.8) are the result of the symmetries in the metric tensor and Christoffel symbols, and are enforced by optimizing over only the lower triangular indices 1 ≤ i < j ≤ d without constraints. Asymptotic

results provide the positive-definiteness of the metric estimate, as will be shown in Proposition 4.1. To weigh the pairs of endpoints according to their proximity to the target location p , we apply kernel weights specified by

for some h > 0 and non-negative kernel function K . The bandwidth h controls the bias-variance trade-off of the estimated Riemannian metric tensor and its derivatives.

Altering the link function g and the loss function Q in (3.7) enables flexible local regression estimation for models in Example 3.1.

Example 3.2 . Consider the following loss functions for estimating the metric tensors and the Christoffel symbols when data are drawn from model (3.2)-(3.4), respectively.

Continuous noisy response: use squared loss Q ( y, µ ) = ( y -µ ) 2 with g being the identity link function so µ u = η u .

Binary (dis)similarity response: use log-likelihood of Bernoulli random variable

and g the logit link, so µ u = logit -1 ( η u ) . The model becomes a local logistic regression.

Binary relative comparison response: apply the same loss function (3.11) and logit link as in the previous scenario, but here we formulate the linear predictor based on dist ( X u 0 , X u 2 ) 2 -dist ( X u 0 , X u 1 ) 2 ≈ η u 1 -η u 2 and

Locally, the difference in squared distances is approximated by

for δ i u 2 = X i u 2 -p i and δ i u, 0 -2 = δ i u 2 -δ i u 0 , i = 1 , . . . , d . Here η u 1 and η u 2 are constructed in analogy to (3.6) using ( X u 0 , X u 1 ) and ( X u 0 , X u 2 ) pair respectively.

Examples in Section 5 will further illustrate the proposed method in those scenarios. Besides the models listed, other choices for the link g and loss function Q can also be considered under this local regression framework [8], accommodating a wide variety of data. To efficiently estimate the metric on the entire manifold M , we apply a procedure based on discretization and post-smoothing, as detailed in Section S3 of the Supplement. In short, kernel smoothing (weighted average) of estimated component functions ˆ G ij over some grid points provides a smooth tensor field, which ease the burden of re-estimation for every points on the manifold.

4 Bias and Variance of the Estimated Metric Tensor

This subsection provides asymptotic justification for model (3.2) with E ( Y u | X u 0 , X u 1 ) = dist ( X u 0 , X u 1 ) 2 under the squared loss Q ( µ, y ) = ( µ -y ) 2 and the identity link g ( µ ) = µ . The estimator we analyzed here fits a local quadratic regression without intercept and approximates the squared distance by a simplified form of (3.6):

for u = 1 , . . . , N . Given a suitable order of the indices i, j for vectorization, we rewrite the formulation into a matrix form. Denote the local design matrix and regression coefficients as

so that the linear predictor η u = D T u β . Further, write D = ( D 1 , . . . , D N ) T , Y = ( Y 1 , . . . , Y N ) T , and W = diag ( w 1 , . . . , w N ) , with weights w u specified in (3.10). The objective function in (3.7) becomes ( Y -D β ) T W ( Y -D β ) , and the minimizer is ˆ β = ( D T WD ) -1 D T W Y , for which we will analyze the bias and variance.

To characterize the asymptotic bias and variance of the estimator, we assume the following conditions are satisfied in a neighborhood of the target p . These conditions are standard and analogous to those assumed in a local regression setting [8].

(A1) The joint density of endpoints ( X u 0 , X u 1 ) is positive and continuously differentiable.

(A2) The functions G ij , Γ k ij are C 2 -smooth for i, j, k = 1 , . . . , d .

(A3) The kernel K in weights (3.10) is symmetric, continuous, and has a bounded support.

(A4) sup u var ( Y u | X u 0 , X u 1 ) < ∞ .

Proposition 4.1. Under (A1)-(A4), bias GLYPH<128> ˆ β | X GLYPH<138> = O p ( h 2 ) , var GLYPH<128> ˆ β | X GLYPH<138> = O p ( N -1 h -4 -2 d ) , as h → 0 and Nh 2+2 d →∞ , where X = { ( X u 0 , X u 1 ) } N u =1 is the collection of observed endpoints.

Proof.

The local approximation (4.1) is similar to a local polynomial estimation of the second derivative of a 2 d -variate squared geodesic distance function, explaining the order of h in the rates for bias and variance.

The positive definite constraints (as in distance metric learning) are dropped based on the asymptotic properties, thus the estimated metric could fail to be positive definite at some locations. This is alleviated by sufficient data and post-smoothing that averaging neighboring estimation.

5 Simulation

We illustrate the proposed method using simulated data with different types of responses as described in Example 3.1. We study whether the proposed method well estimates Riemannian geometric quantities, including the metric tensor, geodesics, and Christoffel symbols. Additional details including assessment on stability and positive-definiteness are included in Section S4 of the Supplement. Subsection S4.2 also provides an ellipsoid example for manifold of non-constant curvature.

5.1 Unit Sphere

The usual arc-length/great circle distance on the d -dimensional unit sphere is induced by the round metric, which is expressed under the stereographic projection coordinate ( x 1 , . . . , x d ) as ˚ G ij = 4 GLYPH<128> 1 + ∑ d k =1 x k x k GLYPH<138> -2 1 { i = j } , for i, j = 1 , . . . , d . Under the additive model (3.2) in Example 3.1, we considered either noiseless or noisy responses by setting σ ( p ) = 0 or σ ( p ) > 0 respectively.

Experiments were preformed with d = 2 and the finding are summarized in Figure 5.1. For continuous responses, the left panel of Figure 5.1a visualizes the true and estimated metric tensors via cost ellipses (S2.1) and the right panel shows the corresponding geodesics by solving the geodesic equations (2.1) with true and estimated Christoffel symbols. The metrics and the geodesics were well estimated under the continuous response model with or without additive errors, where the estimates overlap with the truth. Figure 5.1c evaluates the relative estimation errors ∥ ∥ ∥ ˆ G -G ∥ ∥ ∥ F / ‖ G ‖ F and ∥ ∥ ∥ ˆ Γ -Γ ∥ ∥ ∥ F / ‖ Γ ‖ F w.r.t. the Frobenius norm (S2.2) for data from the continuous model (3.2).

For binary responses under model (3.3), Figure 5.1b visualizes the data where the background color illustrates h . Figure 5.1d and left panel of Figure 5.1a suggest that the intercept and the metric were reasonably estimated, while the geodesics are slightly away from the truth (Figure 5.1a, right). This indicates that the binary model has higher complexity and less information is provided by the binary response (see also Figure S4.1b in the Supplement).

metric tensor

geodesics

-2

-1

0

1

2

0.50

0.75

1.00

1.25

1.50

-0.6

-0.3

0.0

0.3

0.6

-2

-1

0

1

2

x1

x2

method

truth

estimated with no error

estimated with additive error

estimated with binary response

-2

0

2

-2

0

2

x1

x2

10

20 30

intercept

binary response

1

0

(b) Binary responses, with segments showing comparisons, background colored to h .

metric

-2

-1

0

1

2

-2

-1

0

1

2

x1

x2

0.025

0.050

0.075

0.100

0.125

relative

error

Christoffel Symbol

-2

-1

0

1

2

-2

-1

0

1

2

x1

x2

0.25

0.50

0.75

1.00

relative

error

estimated intercept

-2

-1

0

1

2

-2

-1

0

1

2

x1

x2

0.25

0.50

0.75

relative

error

(c) Relative errors in term of Frobenius norm of the estimated tensors for the continuous response model (3.2) with additive error.

(d) Errors for estimating h with binary responses (3.3).

5.2 Relative Comparison on the Double Spirals

A set of 7 × 10 4 points on R 2 were generated around two spirals, corresponding to two latent classes A and B (e.g., green points in Figure 5.2a are from latent class A ). We compare neighboring points ( X u 0 , X u 1 , X u 2 ) to generate relative comparison response Y u as follows. For u = 1 , . . . , N , Y u = 1 if X u 0 , X u 1 belong to the same latent class and X u 0 , X u 2 belong to different classes; otherwise Y = 0 . Figure 5.2b shows a portion of the N = 6 , 965 , 312 comparison generated, where the hollow circles in the middle of each wedge correspond to X u 0 .

Here, contrast of the two latent classes induces the intrinsic distance, so the distance is larger across the supports of the two classes and smaller within a single support. Therefore, the resulting metric tensor should reflect less cost while moving along the tangential direction of the spirals compared to perpendicular directions. Estimates were drawn under model (3.4) by minimizing the objective (3.7) with the link function (3.12) and the local approximation (3.13).

The estimated metric shown in Figure 5.2c is consistent with the interpretation of the intrinsic distance and metric induced by the class membership discussed above. Meanwhile, the estimated geodesic curve unveils the hidden circular structure of the data support as shown in Figure 5.2d.

-2

0

2

-2

0

2

class

A

B

(a) The points and their la-

tent classes

-2

0

2

-2

0

2

response

0

1

latent class

A

B

(b)

Relative

comparison.

Circles show

X

u

0

, branch

to

X

, X

.

-2

-1

0

1

2

-2

-1

0

1

2

class

A

B

(c) Estimated metric

-2

-1

0

1

2

-2

0

2

(d) Geodesics starting from

(1

,

0)

with initial velocity

pointing to 9 o'clock under

estimated metric.

u

1

u

2

6 New York City Taxi Trip Duration

We study the geometry induced by taxi travel time in New York City (NYC) during weekday morning rush hours. New York City Taxi and Limousine Commission (TLC) Trip Record Data was accessed on May 1st, 2022 3 to obtained business day morning taxi trip records including GPS coordinates for pickup/dropoff locations as ( X u 0 , X u 1 ) and trip duration as Y u . Estimation to the travel time metric was drawn under model (3.2) with Q ( y, µ ) = ( y -µ ) 2 and g ( µ ) = µ .

Figure 6.1a shows the estimated metric for taxi travel time. The background color shows the Frobenius norm of the metric tensor, where larger values mean that longer travel time is required to pass through that location. Trips through midtown Manhattan and the financial district were estimated to be the most costly during rush hours, which is coherent to the fact that these are the city's primary business districts. Moreover, the cost ellipses represent the cost in time to travel a unit distance along different directions. This suggests that in Manhattan, it takes longer to drive along the east-west direction (narrower streets) compared to the north-south direction (wider avenues).

Geodesic curves in Figure 6.1b show where a 15-minutes taxi ride leads to starting from the Empire State Building. Each geodesic curve corresponds to one of 12 starting directions (1-12 o'clock). Note that we apply a continuous Riemannian manifold approximation to the city, so the geodesic curves provide approximations to the shortest paths between locations and need not conform to the road network. Travel appears to be faster in lower Manhattan than in midtown Manhattan. The spread of the geodesics differs along different directions, indicating the existence of non-constant curvature on the manifold and advocating for estimating the Riemannian metric tensor field instead of applying a single global distance metric.

7 High-dimensional Data: An Example with MNIST

The curse of dimensionality is challenging for nonparametric methods applied on data sources like images and audios. However, it is often found that apparent high-dimensional data actually lie close to some low-dimensional manifold, which is utilized by manifold learning literature to produce reasonable low-dimensional coordinate representations. The proposed method can be applied to the resulting low-dimensional coordinates to recover the high-dimensional geometry even implicitly defined, demonstrated in the following MNIST example.

We embed images in MNIST to a 2-dimensional space via tSNE [13]. Similarity between the objects was computed by the sum of the Wasserstein distance between images 4 and the indicator of whether the underlying digits are different (1) or not (0). The goal is to infer the geometry of the embedded

3km

2km

40.85°N

time

40.80°N

norm

(minutes)

15

0.5

0.4

10

0.3

40.75°N

0.1

0.2

5

40.70°N

74.00°W

73.95°W

73.90°W

(a) Estimated metric tensors for trip duration: cost ellipses and Frobenius norm (background color).

(b) Geodesics correspond to 15-minute taxi rides from the Empire State Building heading to 1-12 o'clock.

data induced by this similarity measures. Estimation was drawn under model (3.2) with squared loss Q ( y, µ ) = ( y -µ ) 2 and the identity link. An intercept term ( β (0) in (3.6)) is used to capture intrinsic variation partially due to the non-injective dimensional reduction.

The geodesics estimated from our method tend to minimize the number of switches between labels. For example, the geodesic A in Figure 7.1 remains '4' (1st row of panel (b)) throughout, while the straight line on the tSNE chart translates to a path of images switching between '4' and '9' (2nd row of panel (b)); similar phenomenon occurs for geodesics B and C (corresponding to 7th and 12th rows in (b)). Also, our estimated geodesics produce reasonable transition and reside in the space of digits, while unrestricted optimal transport (3rd, 8th, and 13th rows of panel (b)) could produce unrecognizable intermediate images. Our estimated geodesic is faithful to the geometric interpretation that a geodesic is locally the shortest path. Moreover, our proposal provide sensible image transition, especially compared to the shortest paths along neighborhood graph in the embedded space (dotted blue lines in (a) and 4, 9, 14 rows in (b)) or kNN graphs in the original Euclidean space R 28 × 28 (5, 10, 15 rows in (b)). This could be useful for complex data interpolation that preserves geometry.

8 Discussion

We present a novel framework for inferring the data geometry based on pairwise similarity measures. Our framework targets data lying on a low-dimensional manifold where observations are expected to be dense near the locations where we wish to estimate the metric. However, these assumptions are general enough for our method to be applied to manifold data with high ambient dimension in combination with manifold embedding tools. Context-specific interpretation of the geometrical notions, e.g., for Riemannian metric and geodesics, has been demonstrated in the taxi travel and MNIST digit examples. Our method could inspire additional compelling applications to other topics such as cognition and perception research, where psychometric similarity measures are commonly made. Moreover, further development including (but not limited to) handling sparse data and ensuring positivity in the Riemannian metric estimation is needed, as discussed in Section S1 of the Supplement.

A

B

C

-2

-1

0

1

2

-2

-1

0

1

2

tSNE.1

tSNE.2

4

q

4

多

3

1

999

5

6

2

2

2

力

4

4

4

4

4

4

4

4

4

4

4

>

44

44

4

4

4

4

4

4

4

Q

力

7

4

4

4

Q

4

4

力

4

4

4

4

>

4

y

人

入

3

3

X

3

3

S

3

5

D

8

5

8

8

8

8

S

S

S

S

S

S

5

5

5

S

8

88886

8

SS5558888

Acknowledgments and Disclosure of Funding

The authors would like to thank Dr. Zhengyuan Zhu and the reviewers for the helpful feedback and discussion. The research was partially supported by NSF grant DMS-2329879.

References

Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: On the curvature of deep generative models. In International Conference on Learning Representations , February 2018.

Georgios Arvanitidis, Søren Hauberg, and Bernhard Schölkopf. Geometrically enriched latent spaces, August 2020.

Aurélien Bellet, Amaury Habrard, and Marc Sebban. Metric learning. Synthesis Lectures on Artificial Intelligence and Machine Learning , 9(1):1-151, January 2015.

Adam Berenzweig, Beth Logan, Daniel P.W. Ellis, and Brian Whitman. A large-scale evaluation of acoustic and subjective music-similarity measures. Computer Music Journal , 28(2):63-76, June 2004.

Shuo Chen, Lei Luo, Jian Yang, Chen Gong, Jun Li, and Heng Huang. Curvilinear distance metric learning. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) , volume 1, pages 539-546 vol. 1, June 2005.

Ronald R. Coifman and Stéphane Lafon. Diffusion maps. Applied and Computational Harmonic Analysis , 21(1):5-30, July 2006.

Jianqing Fan and I. Gijbels. Local Polynomial Modelling and Its Applications . Number 66 in Monographs on Statistics and Applied Probability. Chapman & Hall, London ; New York, 1st ed edition, 1996.

Charles Fefferman, Sergei Ivanov, Matti Lassas, and Hariharan Narayanan. Reconstruction of a Riemannian manifold from noisy intrinsic distances. SIAM Journal on Mathematics of Data Science , 2(3):770-808, September 2020.

Andrea Frome, Yoram Singer, Fei Sha, and Jitendra Malik. Learning globally-consistent local distance functions for shape-based image retrieval and classification. In 2007 IEEE 11th International Conference on Computer Vision , pages 1-8, October 2007.

T. Hastie and R. Tibshirani. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence , 18(6):607-616, June 1996.

Søren Hauberg, Oren Freifeld, and Michael J. Black. A geometric take on metric learning. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25 , pages 2024-2032. Curran Associates, Inc., 2012.

Geoffrey E Hinton and Sam Roweis. Stochastic Neighbor Embedding. In Advances in Neural Information Processing Systems , volume 15. MIT Press, 2002.

Martin Jørgensen and Soren Hauberg. Isometric Gaussian process latent variable model for dissimilarity data. In Proceedings of the 38th International Conference on Machine Learning , pages 5127-5136. PMLR, July 2021.

Tam Le and Marco Cuturi. Unsupervised Riemannian metric learning for histograms using Aitchison transformations. In Proceedings of the 32nd International Conference on Machine Learning , pages 2002-2011. PMLR, June 2015.

G. Lebanon. Metric learning for text documents. IEEE Transactions on Pattern Analysis and Machine Intelligence , 28(4):497-508, April 2006.

Guy Lebanon. Learning Riemannian metrics. In Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence , UAI'03, pages 362-369, San Francisco, CA, USA, 2002. Morgan Kaufmann Publishers Inc.

John M. Lee. Introduction to Smooth Manifolds . Number 218 in Graduate Texts in Mathematics. Springer, New York ; London, 2nd ed edition, 2013.

John M. Lee. Introduction to Riemannian Manifolds . Springer Berlin Heidelberg, New York, NY, 2018.

Wolfgang Meyer. Toponogov's theorem and applications. Lecture Notes, Trieste , 1989.

Dominique Perraul-Joncas and Marina Meila. Non-linear dimensionality reduction: Riemannian metric estimation and the problem of geometric discovery, May 2013.

Deva Ramanan and Simon Baker. Local distance functions: A taxonomy, new algorithms, and an evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 33(4):794-806, April 2011.

Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science , 290(5500):2323-2326, December 2000.

Christopher Scarvelis and Justin Solomon. Riemannian Metric Learning via Optimal Transport. In The Eleventh International Conference on Learning Representations , February 2023.

Matthew Schultz and Thorsten Joachims. Learning a distance metric from relative comparisons. In Advances in Neural Information Processing Systems , volume 16. MIT Press, 2003.

Juan Luis Suárez, Salvador García, and Francisco Herrera. A tutorial on distance metric learning: Mathematical foundations, algorithms, experimental analysis, prospects and challenges. Neurocomputing , 425:300-322, February 2021.

Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science , 290(5500):2319-2323, December 2000.

Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research , 10(2), 2009.

Eric Xing, Michael Jordan, Stuart J Russell, and Andrew Ng. Distance metric learning with application to clustering with side-information. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems , volume 15. MIT Press, 2002.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel mechanism for learning a Riemannian metric from distance observations. This is important for applications where relative observations are available (e.g. \"objects x1 and x2 are different, while x2 and x3 are similar\"), such as perception studies. The approach is based on local regression from which a Riemannian metric is deduced from a Taylor expansion of the geodesic distance. Limited experimental results demonstrate feasibility.\n\nSTRENGTHS:\n* The paper approaches an important question that is highly understudied in the machine-learning community.\n* The proposed Riemannian metric estimator is new and sufficiently simple to be practically useful.\n* It is neat that the estimator also comes with a direct estimate of the Cristoffel symbols as these can otherwise be tedious to compute.\n* The examples given throughout the paper are instructive and provide a nice assistance to the reader.\n* The illustrative examples are instructive.\n\nWEAKNESSES:\n* The approach seems to be difficult to scale to higher dimensional data. I don't think this is a significant problem as the current application areas where distance observations are available often revolve around low-dimensional observations, e.g. in perception studies in psychology.\n* The computational procedure seems to rely on a discretization of the input space, which will only work in low-dimensional cases.\n* The kernel smoothing used to produce the Riemannian metric (eq. 3.10) is rather ad hoc. It would have been nice if this was more closely tied to the local regression. That being said, the approach basically follows established procedures, see e.g. [10].\n\n### Related work\n* I am not fond of the phrasing of lines 37-39, which suggests that the proposed work is the first time that Riemannian metrics are learned. The cited work of Lebanon [14] and Hauberg et al. [10] follows the same philosophy as the present paper, so I think a rephrasing would be appropriate.\n* The authors are not the first to learn metrics from observations of distances. I think it would be good to at least acknowledge the vast literature on multi-dimensional scaling, which does not learn a Riemannian metric but learns from distances. Perhaps closer to the present work is the paper \"Isometric Gaussian Process Latent Variable Model for Dissimilarity Data\" (Jørgensen et al., ICML 2021) which learns a low-dimensional manifold and its Riemannian metric from noisy distance observations. The taken approach is quite dissimilar from the present paper, but the ambition is similar.\n\n### Minor things\n* Typo in line 219: \"without or without\" --- I suppose this should be \"with or without\".\n\nQUESTIONS:\n* What does the $\\circ$ operator denote in Eq. 2.1? As I read the equation, all quantities are scalars, so I do not think that this denotes a Hadamard product, but this is nonetheless my best guess.\n* I do not quite understand Eq. 3.2: Are there no further restrictions in $\\epsilon_n$? If so does that not mean that $P(Y < 0) > 0$, which is rather odd for distance observations which must be strictly non-negative.\n* There has been quite extensive work on pull-back metrics in autoencoder-like models (see e.g. \"Latent Space Oddity: on the Curvature of Deep Generative Models\", Arvanitidis et al., ICLR 2018). These can be viewed as smooth interpolations of local linear regressions since the metric is formed by Jacobian matrices. Do you see links between this line of work on the present paper?\n\nLIMITATIONS:\nThe main paper only makes a brief mention of the limitations, while a more extensive discussion is given in the supplements. I appreciate the supplementary discussion but would encourage the authors to move this to the main paper to increase openness regarding limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims to extend metric and manifold learning by learning Riemannian metrics from functions of the observed data that are not related to an embedding space metric as is the usual case. Assuming there is an underlying Riemannian metric structure and that the observed dissimilarity is a known function of this structure, the authors show how a Riemannian metric can be learned from dense data. The method is tested on trip time data in New York and MNIST represented on a 2d manifold.\n\nSTRENGTHS:\n- methods for estimating a Riemannian metric from dissimilarity measures are derived\n- in principle, it can be a good idea relying on other distances than embedding space distances to learn geometric structure\n\nWEAKNESSES:\n- learning Riemannian metrics has a long history in the literature, although not in this exact form\n- I am unsure of the usefulness of the method:\n-- are there any guarantees that a Riemannian metric that is compatible with the observed structure exists? I believe if the objective is actually a (sufficiently smooth) distance there are existence results, but it would be nice with arguments for the more general cases in sec 3.1\n-- it is assumed that a chart is known. One can always find a mapping to a lower-dimensional subspace such as is done in the MNIST example, but whether the data is actually 2d is unknown, and the uncertainty in the chart estimation is not taken into account\n-- in practice, the result would likely be a Riemannian metric approximation in a chart that is estimated with high uncertainty and with strong assumptions on the dimensionality. The data would likely not be dense in higher dimensions, and so the estimated structure would very likely be a poor approximation. This can then be used for e.g. geodesic interpolation, but whether this is actually useful is not clear to me\n-- what happens if the model is misspecified? The link functions are assumed to be known a priori, right?\n- the developed estimation techniques have merit, but the fact that a Riemannian metric can be recovered from the metric (distances) is not a new result\n\nQUESTIONS:\nConvincing counterarguments to the weaknesses listed above.\n\n- do you require the charts to be normal? If so, that restricts the set of metrics you can possibly learn, right? I.e., it could be that a true underlying Riemannian structure was not orthonormal with respect to the chosen chart\n\nLIMITATIONS:\nthe limitations of the method could be described more precisely", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops a theory for estimating the Riemannian metric tensor for a given set of observations and some addition information. This additional information includes a (noisy) measure of similarity between the given points in a pairwise fashion. Examples of this information includes the geodesic distance, or a binary response about the similarity of types, or a binary response about relative comparison. \n\nThe formulation relies on a formula for the intrinsic distance between corresponding points on two geodesics shot from the same base p.  This formula approximates the said distance using the Euclidean chords, Riemannian metric tensor, and the Christoffel symbols. The first term computes the Riemannian metric between the shooting vectors and the second term accounts for the curvature. The paper seizes on this linear approximation and sets up a regression problem for estimating these matrices from the given data. In this way, it estimates the metric tensors and the Christoffel symbol at point p. The paper further develops the estimation theory (bias and variance) for the metric tensor estimation. \n\nThis theory is demonstrated using several experiments, some based on simulated data and some on real data sets. The simulated data helps validate the method for a known geometry (sphere, double spiral). The real data experiments involve learning metrics for the taxi travel times and MNIST images embedded in R^2 using tSNE.\n\nSTRENGTHS:\n-- The problem of learning Riemannian metric from the data is quite important and challenging. \n\n-- The paper formulates this problem in a novel and interesting way and provides analytical insights, rather than the current systems that apply deep neural networks to such problems. \n\n-- The paper goes on to develop estimation theory for the metric tensor. There is some interesting statistical contributions here.\n\nWEAKNESSES:\n-- The simulation experiments seem to involve manifolds with constant curvature (unless the curvature changes along the spiral). Perhaps the authors can try their method for ellipsoids or some variable curvature manifolds. \n\n-- How does the method work if the sampled points on the manifold are sparse? I feel that the linear regression model derived here requires the manifold points to be close otherwise the errors will start piling up.\n\nQUESTIONS:\n-- I believe that the equation connecting geodesic distance to the Mahalanobis distance is the same kind that is used to derive Cramer-Rao bound on estimation error using the Fisher-Rao Riemannian metric. It would be interesting to make that connection if there is one. \n\n\n-- What are the practical situations where the pairwise geodesic distances between points on a manifold are given, and one does not know the Riemannian metric? I understand the taxi example but for the MNIST example one has to use an embedding which is somewhat of an arbitrary choice.\n\nLIMITATIONS:\nThe authors have not explicitly address the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method to estimate the Riemannian metric of data space when coordinate representations of each data point and some noisy similarity measurements among data points are provided. The similarity measurements of different types, such as noise-contaminated distances, similarity/dissimilarity labels, and comparative similarity labels, are probabilistically modeled as functions of intrinsic distances between data points. The squared geodesic distances are approximated to be linear to the Riemannian metric and Christoffel's symbol via Taylor's expansion, leading the metric estimation problem to be like a maximum likelihood estimation using a generalized linear model with each entry of the Riemannian metric and the Christoffel's symbol as its parameters. Asymptotic convergence rates of the bias and variance of the estimator are derived for the case where noisy distances are given as the similarity measure. Experiments are performed using some simulated data, New York taxi trip duration data, and MNIST data sets to demonstrate the proposed estimator's benefits in capturing the underlying geometry of the data space.\n\n**Post rebuttal** I have increased my score from 4 to 5. Lingering concerns involve providing more practical applications and comparisons to other methods.\n\nSTRENGTHS:\n* The paper is in general well-written.\n* The paper provides some interesting ideas to characterize the Riemannian geometry of data space by utilizing noisy similarity measurements among data points, such as continuous noisy distances, binary similarity labels, and binary relative comparison labels, in a unified framework.\n* The proposed method to estimate the Riemannian metric is quite simple, seems original, and shows reasonable experimental results.\n* This paper provides complete proof of a proposition for the local approximation of the geodesic distance and that for the asymptotic convergence analysis of the estimator.\n\nWEAKNESSES:\n* The considered problem setting seems to be quite restrictive. It would be rare that a problem provides both low-dimensional coordinate representations and meaningful intrinsic distances among data points. Obtaining each of them has been an important research topic for decades.\n* The experiments need to provide practical applications of the proposed method. For example, obtaining geodesics in the experiments is usually done to demonstrate the validity of the estimated Riemannian metrics but not for further use. Suggesting more practical uses for real data based on the estimated Riemannian metric, computed geodesics, or other subsequent geometric quantities such as lengths or volumes, would significantly benefit the community.\n* This paper lacks any comparison with other Riemannian metric estimation methods. A direct comparison would be possible to the method in Perrault-Joncas et al., which considers a mapping $f$ from the data submanifold embedded in the ambient space (endowed with an ambient space metric) to a coordinate chart and estimates the push-forward metric in the coordinate chart via the mapping $f$.\n* Experiments in high-dimensional settings are lacking. How the method behaves according to the dimension would be valuable information. A worry here is that local regions to estimate the Riemannian metrics would increase exponentially with respect to the dimension.\n\nQUESTIONS:\n* When extracting the Riemannian metric from binary similarity labels or binary relative comparison labels, there may not be any ground truth Riemannian metrics. Is it then appropriate to call the proposed framework an 'estimation'?\n* How can the proposed probabilistic models be justified? Distance scale may significantly affect the results according to the current choice of models in (3.3) and (3.4). Also, different options of probabilistic models would result in other Riemannian metrics.\n* Is there any guarantee that the local squared distance in (3.5) is always positive when the estimation is performed based on (3.7)? This may also be related to the point that the estimated metric is not guaranteed positive-definite. I wonder if solving for the case (3.11) and (3.12) can make the estimated squared distances, i.e., $\\eta_u$s, negative.\n* There are no explanations for obtaining estimates for Christoffel's symbols at arbitrary points, i.e., $\\hat{\\Gamma}\\circ \\gamma(t)$, which are required for obtaining geodesics.\n* Regarding the post-smoothing of the obtained Riemannian metrics explained in the supplement, the weighted averaging of Riemannian metrics seems strange. For example, we cannot simply add tangent vectors on different tangent spaces but should apply parallel transport for them to be on the same tangent space. \n\n**[Minor comments]**\n* It must be explicitly stated in the main text the location of the supplement containing the proof of each proposition.\n* Since the proposed approximations and estimation methods are valid only locally, it would be better to explain how the methods can be applied globally in the main text, not in the supplement.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations of the paper. For other possible limitations, please refer to the weaknesses and questions above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}