Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper develops a theory for estimating the Riemannian metric tensor for a given set of observations and some addition information. This additional information includes a (noisy) measure of similarity between the given points in a pairwise fashion. Examples of this information includes the geodesic distance, or a binary response about the similarity of types, or a binary response about relative comparison. 

The formulation relies on a formula for the intrinsic distance between corresponding points on two geodesics shot from the same base p.  This formula approximates the said distance using the Euclidean chords, Riemannian metric tensor, and the Christoffel symbols. The first term computes the Riemannian metric between the shooting vectors and the second term accounts for the curvature of the metric. The paper seizes on this linear approximation and sets up a regression problem for estimating these matrices from the given data. In this way, it estimates the metric tensors and the Christoffel symbol at point p. 

The paper illustrates this solution on two simulated data (a unit sphere and a points on double spirals) and one real data set (time take by taxis between points in New York City. 
strength_and_weaknesses: 
Strengths: 

Seems like a good idea to learn the local geometry of a manifold. In case that additional information (pairwise geodesic distances or labels or relative labels) is available and the points are relatively dense, then it seems reasonable to be able to learn the metric from the data. I don't know if any one has taken this approach in the past, but it seems quite novel to me. 

Weakness: 
•	How are the base points p (the basepoint for Eqn. 3.5) chosen in practice. The underlying approximation is valid only for a small t, so the base p should be close to the given data points. Does the method work if the given points are sparse?

•	What are the practical situations where the pairwise geodesic distances between points on a manifold are given, and one does not know the Riemannian metric? I understand the taxi example but I can’t think of too many other situations. 

•	It seems strange that the nonlinearity of the manifold is being learnt through a linear system of equations. 

•	The experiments are limited in scope. There are two simulation experiments – one on a unit sphere and one on spiral curves. The third example uses time of travel by taxis. A unit sphere is a special case where it is easy to learn the geometry through only the first order information (points and tangent vectors). It would be been nice to see an example of learning geometry of a more complicated manifold. 

clarity,_quality,_novelty_and_reproducibility: 
The paper is well written and defines all the quantities that are used in the development. It would be useful to provide some intuitive explanation behind the two terms on the right in Eqn. 3.5. 
summary_of_the_review: 
This is an interesting paper which solves a very specific problem. Estimating Riemannian metrics in general is much more ambitious and challenging but it assumes the knowledge of (noisy) pairwise geodesic distances which limits the scope and  makes the problem easier.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper is concerned with learning a Riemannian metric such that geodesic distances matched (direct or indirect) measured distances. A general framework is presented that captures the common situations of (1) observations are distances, (2) observations are binary indicating if two points are similar or not, or (3) observations are triplets where one pair is more similar than others. It is then proposed to estimate the Riemannian metric through a local Taylor expansion. This is demonstrated to work well on fairly simple low-dimensional problems.
strength_and_weaknesses: ## Strengths
* The derived Taylor expansion is $\mathcal{O}(t^4)$ indicating that it is quite accurate as an estimate of a local metric.
* The method seems both quite general and novel.

## Weaknesses
* I was unable to determine if the estimated Riemannian metric is always positive definite. It would be nice with a clarification on that.
* As far as I can tell, Eq. 3.10 relies on Euclidean distances through the kernel functions. The main motivation of the work is that we should not trust the Euclidean distance, so if my reading is correct, then this seems a bit problematic.
* I was unable to determine how geodesics were computed in practice.
* The paper contains some simple toy experiments to verify that the method works and an application to travel times in Manhattan. While these experiments are nice, I miss a compelling application of the method. I believe that this line of work can be quite useful, but it would be good to demonstrate this to a greater degree.
* I miss a more careful investigation in the synthetic experiments. For example, in the sphere example, it should be trivial to vary the sphere dimensionality and ask, "does the method keep working?" Given that only very low-dimensional problems are considered, I worry that the method does not scale. It would be nice with some evidence to counter such a belief.
* The discussion section of the paper feels very rushed. Perhaps this is the usual time pressure of conference deadlines.
clarity,_quality,_novelty_and_reproducibility: I found the paper to be quite clear to read, and the idea seems both sensible and novel. I would be unable to reproduce the results as it is not clear how to compute geodesics.
summary_of_the_review: The paper presents an interesting idea, but I find that the experiments are lacking.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: To measure the difference between two samples $A$ and $B$,  this paper assumes the samples $A$ and $B$ are endpoints of two different geodesics starting from the same point $P$ in a small neighborhood and approximates their squared geodesic distances using Taylor extension which leads to a local linear regression model. Geometric characteristics like Riemannian metric and the Christoffel symbols serve as parameters of the proposed linear regression model. For the special case using squared loss, the authors established asymptotic bounds on the bias and variance of the estimator. Some intuitive results are shown through numerical experiments on unit spheres, spirals, and taxi trip duration. 
strength_and_weaknesses: [Strength]

1). This paper proposes an interesting linear regression model which stems from a rigorous expression of Taylor approximation to the squared geodesic distances on a Riemannian manifold. 

2). Bounds on the bias and variance of the estimator are quantified in an asymptotic sense. 

[Weakness]

1). Assumptions behind the local linear model may be too strict for real applications. 

The proposed local regression model assumes the samples lie in small neighborhood of another point. However, in real applications using binary similarity measurements in computer vision, perception tasks, and recommendation systems, the compared samples may be far from each other and the assumption of localness may be violated. 

2). Insufficient empirical verifications for real applications.

Through experiments on unit spheres, spirals, and taxi duration data, the authors provide nice visualizations of the estimate geometric parameters. However, neither comparison results with other methods nor experimental results on modern CV&ML datasets are given. 

clarity,_quality,_novelty_and_reproducibility:  [Quality and novelty of the proposed model, Score: 65/100]  

The proposed local linear regression model is new and has a solid differential geometry foundation given in Proposition 3.1.

(1). Although Proposition 3.1 seems a combination of lemmas similar to the ones in (de Carmo, 1992) and (Lang, 1999), the expression of Taylor expansion still has its novelty.  

(2). It is interesting to find that although (4.1) does not involve the Christoffel symbols, they do affect the performance of estimation. Could the authors give some intuitive explanations?  

[Quality regarding empirical use in modern CV&ML applications, Score: 20/100]
 
In my mind, the underlying assumption of the proposed model is a bit strict and may seriously limit its real applications. The lack of experiments on modern CV&ML datasets may also reflect this disadvantage. 

[Clarity and Presentation, Score: 50/100] 
This paper is in general clearly written. The somewhat complicated geometric notions are relatively well introduced. However, this paper has typos and not introduced symbols examples as follows:

(1) In Eq. (3.7), the iterator in the summation symbol should be $u$.

(2) In Proposition C.1, the Riemannian curvature tensor $R(v,w)w$ should be defined/explained in advance.

(3) In Line 3, P17, it should be $ H(s)=dist(\gamma_0(s),\gamma_1(s))^2=\|\|T\|\|^2\big|_{s,t}$.

(4) In Line 5, P18, $Rm(\cdot,\cdot,\cdot,\cdot)$ should be introduced. 

[Reproductivity, Score: 50/100] 

Although no code is uploaded as supplemental materials, the details are relatively sufficient to reproduce the results. 
summary_of_the_review: I recommend "reject, not good enough" due to the scores regarding Clarity, Quality, Novelty And Reproducibility.
[Quality and novelty of the proposed model, Score: 65/100]  [Quality regarding empirical use in modern CV&ML applications, Score: 20/100]
[Clarity and Presentation, Score: 50/100] [Reproductivity, Score: 50/100]
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a method to recover the parameters of a Riemannian metric using noisy observations of distances between points on the manifold (described through coordinates). While interesting and (re)introducing the ICLR audience to a problem area that was heavily studied about 10 years ago, the paper suffers from a few fatal flaws that do not make it a suitable addition to the program as it is. All experiments are toy, and the authors have no strategy to make this work generalize to datatypes that are of interest. As a result the paper is rejected, but this round of reviews has provided the authors with a lot of feedback.
justification_for_why_not_higher_score: Experimental section is too weak. Method does not scale.
justification_for_why_not_lower_score: na
Decision: Reject