SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition 4.1. Under (A1)-(A4), bias  $\left(\hat{\beta}|\mathbf{X}\right) = O_p(h^2)$ , var  $\left(\hat{\beta}|\mathbf{X}\right) = O_p\left(N^{-1}h^{-4 - 2d}\right)$ , as  $h \to 0$  and  $Nh^{2 + 2d} \to \infty$ , where  $\mathbf{X} = \{(X_{u0}, X_{u1})\}_{u=1}^{N}$  is the collection of observed endpoints.
Proof. See Section S6 in the Supplement.
The local approximation (4.1) is similar to a local polynomial estimation of the second derivative of a  $2d$ -variate squared geodesic distance function, explaining the order of  $h$  in the rates for bias and variance.
The positive definite constraints (as in distance metric learning) are dropped based on the asymptotic properties, thus the estimated metric could fail to be positive definite at some locations. This is alleviated by sufficient data and post-smoothing that averaging neighboring estimation.
5 Simulation
We illustrate the proposed method using simulated data with different types of responses as described in Example 3.1. We study whether the proposed method well estimates Riemannian geometric quantities, including the metric tensor, geodesics, and Christoffel symbols. Additional details including assessment on stability and positive-definiteness are included in Section S4 of the Supplement. Subsection S4.2 also provides an ellipsoid example for manifold of non-constant curvature.
5.1 Unit Sphere
The usual arc-length/great circle distance on the  $d$ -dimensional unit sphere is induced by the round metric, which is expressed under the stereographic projection coordinate  $\left(x^{1},\ldots ,x^{d}\right)$  as  $\mathring{G}_{ij} = 4\left(1 + \sum_{k = 1}^{d}x^{k}x^{k}\right)^{-2}1_{\{i = j\}}$ , for  $i,j = 1,\dots,d$ . Under the additive model (3.2) in Example 3.1, we considered either noiseless or noisy responses by setting  $\sigma (p) = 0$  or  $\sigma (p) > 0$  respectively.
Experiments were preformed with  $d = 2$  and the finding are summarized in Figure 5.1. For continuous responses, the left panel of Figure 5.1a visualizes the true and estimated metric tensors via cost ellipses (S2.1) and the right panel shows the corresponding geodesics by solving the geodesic equations (2.1) with true and estimated Christoffel symbols. The metrics and the geodesics were well estimated under the continuous response model with or without additive errors, where the estimates overlap with the truth. Figure 5.1c evaluates the relative estimation errors  $\left\| \hat{G} - G \right\|_F / \| G \|_F$  and  $\left\| \hat{\Gamma} - \Gamma \right\|_F / \| \Gamma \|_F$  w.r.t. the Frobenius norm (S2.2) for data from the continuous model (3.2).
For binary responses under model (3.3), Figure 5.1b visualizes the data where the background color illustrates  $\hbar$ . Figure 5.1d and left panel of Figure 5.1a suggest that the intercept and the metric were reasonably estimated, while the geodesics are slightly away from the truth (Figure 5.1a, right). This indicates that the binary model has higher complexity and less information is provided by the binary response (see also Figure S4.1b in the Supplement).
Figure 5.1: Simulation results for 2-dimensional sphere under stereographic projection coordinate.
5.2 Relative Comparison on the Double Spirals
A set of  $7 \times 10^{4}$  points on  $\mathbb{R}^2$  were generated around two spirals, corresponding to two latent classes  $\mathcal{A}$  and  $\mathcal{B}$  (e.g., green points in Figure 5.2a are from latent class  $\mathcal{A}$ ). We compare neighboring points  $(X_{u0}, X_{u1}, X_{u2})$  to generate relative comparison response  $Y_{u}$  as follows. For  $u = 1, \dots, N$ ,  $Y_{u} = 1$  if  $X_{u0}, X_{u1}$  belong to the same latent class and  $X_{u0}, X_{u2}$  belong to different classes; otherwise  $Y = 0$ . Figure 5.2b shows a portion of the  $N = 6,965,312$  comparison generated, where the hollow circles in the middle of each wedge correspond to  $X_{u0}$ .
Here, contrast of the two latent classes induces the intrinsic distance, so the distance is larger across the supports of the two classes and smaller within a single support. Therefore, the resulting metric tensor should reflect less cost while moving along the tangential direction of the spirals compared to perpendicular directions. Estimates were drawn under model (3.4) by minimizing the objective (3.7) with the link function (3.12) and the local approximation (3.13).
The estimated metric shown in Figure 5.2c is consistent with the interpretation of the intrinsic distance and metric induced by the class membership discussed above. Meanwhile, the estimated geodesic curve unveils the hidden circular structure of the data support as shown in Figure 5.2d.
class A B
1
(a) The points and their latent classes
response
latent class
0
A
B
(b) Relative comparison. Circles show  $X_{u0}$ , branch to  $X_{u1}, X_{u2}$ .
class A B
(d) Geodesics starting from  $(1,0)$  with initial velocity pointing to 9 o'clock under estimated metric.
(c) Estimated metric
Figure 5.2: Relative comparison on double spirals. Gray curves (solid and dashed) in the background represent the approximate support of the two latent classes.
6 New York City Taxi Trip Duration
We study the geometry induced by taxi travel time in New York City (NYC) during weekday morning rush hours. New York City Taxi and Limousine Commission (TLC) Trip Record Data was accessed on May 1st,  $2022^{3}$  to obtained business day morning taxi trip records including GPS coordinates for pickup/dropoff locations as  $(X_{u0}, X_{u1})$  and trip duration as  $Y_{u}$ . Estimation to the travel time metric was drawn under model (3.2) with  $Q(y, \mu) = (y - \mu)^{2}$  and  $g(\mu) = \mu$ .
Figure 6.1a shows the estimated metric for taxi travel time. The background color shows the Frobenius norm of the metric tensor, where larger values mean that longer travel time is required to pass through that location. Trips through midtown Manhattan and the financial district were estimated to be the most costly during rush hours, which is coherent to the fact that these are the city's primary business districts. Moreover, the cost ellipses represent the cost in time to travel a unit distance along different directions. This suggests that in Manhattan, it takes longer to drive along the east-west direction (narrower streets) compared to the north-south direction (wider avenues).
Geodesic curves in Figure 6.1b show where a 15-minutes taxi ride leads to starting from the Empire State Building. Each geodesic curve corresponds to one of 12 starting directions (1-12 o'clock). Note that we apply a continuous Riemannian manifold approximation to the city, so the geodesic curves provide approximations to the shortest paths between locations and need not conform to the road network. Travel appears to be faster in lower Manhattan than in midtown Manhattan. The spread of the geodesics differs along different directions, indicating the existence of non-constant curvature on the manifold and advocating for estimating the Riemannian metric tensor field instead of applying a single global distance metric.
7 High-dimensional Data: An Example with MNIST
The curse of dimensionality is challenging for nonparametric methods applied on data sources like images and audiios. However, it is often found that apparent high-dimensional data actually lie close to some low-dimensional manifold, which is utilized by manifold learning literature to produce reasonable low-dimensional coordinate representations. The proposed method can be applied to the resulting low-dimensional coordinates to recover the high-dimensional geometry even implicitly defined, demonstrated in the following MNIST example.
We embed images in MNIST to a 2-dimensional space via tSNE [13]. Similarity between the objects was computed by the sum of the Wasserstein distance between images and the indicator of whether the underlying digits are different (1) or not (0). The goal is to infer the geometry of the embedded
<sup>3</sup> https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page. Data format changed after our download.
After rescaling, see Subsection S4.5 of the Supplement.
(a) Estimated metric tensors for trip duration: cost ellipses and Frobenius norm (background color).
Figure 6.1: New York taxi travel time during morning rush hours.
(b) Geodesics correspond to 15-minute taxi rides from the Empire State Building heading to 1-12 o'clock.
data induced by this similarity measures. Estimation was drawn under model (3.2) with squared loss  $Q(y, \mu) = (y - \mu)^{2}$  and the identity link. An intercept term  $(\beta^{(0)}$  in (3.6)) is used to capture intrinsic variation partially due to the non-injective dimensional reduction.
The geodesics estimated from our method tend to minimize the number of switches between labels. For example, the geodesic A in Figure 7.1 remains "4" (1st row of panel (b)) throughout, while the straight line on the tSNE chart translates to a path of images switching between "4" and "9" (2nd row of panel (b)); similar phenomenon occurs for geodesics B and C (corresponding to 7th and 12th rows in (b)). Also, our estimated geodesics produce reasonable transition and reside in the space of digits, while unrestricted optimal transport (3rd, 8th, and 13th rows of panel (b)) could produce unrecognizable intermediate images. Our estimated geodesic is faithful to the geometric interpretation that a geodesic is locally the shortest path. Moreover, our proposal provides sensible image transition, especially compared to the shortest paths along neighborhood graph in the embedded space (dotted blue lines in (a) and 4, 9, 14 rows in (b)) or kNN graphs in the original Euclidean space  $\mathbb{R}^{28\times 28}$  (5, 10, 15 rows in (b)). This could be useful for complex data interpolation that preserves geometry.
We present a novel framework for inferring the data geometry based on pairwise similarity measures. Our framework targets data lying on a low-dimensional manifold where observations are expected to be dense near the locations where we wish to estimate the metric. However, these assumptions are general enough for our method to be applied to manifold data with high ambient dimension in combination with manifold embedding tools. Context-specific interpretation of the geometrical notions, e.g., for Riemannian metric and geodesics, has been demonstrated in the taxi travel and MNIST digit examples. Our method could inspire additional compelling applications to other topics such as cognition and perception research, where psychometric similarity measures are commonly made. Moreover, further development including (but not limited to) handling sparse data and ensuring positivity in the Riemannian metric estimation is needed, as discussed in Section S1 of the Supplement.
(a) The estimated geodesic curves (solid red), shortest path along graph (dotted blue), and straight lines on the chart (dashed black).
Figure 7.1: Geometry induced by a sum of Wasserstein distance and same-digit-or-not indicator.
(b) Image transitions corresponding to A, B, and C in (a). Every 5 rows correspond to a set of paths sharing the same pair of starting and ending images, where the 1st - 5th row correspond to the estimated geodesics, the straight lines on the chart, the optimal transport (path not shown in (a)), the paths along neighborhood graph on the chart, and along kNN graph in the Euclidean space (path not shown in (a)), respectively.
The authors would like to thank Dr. Zhengyuan Zhu and the reviewers for the helpful feedback and discussion. The research was partially supported by NSF grant DMS-2329879.
[1] Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: On the curvature of deep generative models. In International Conference on Learning Representations, February 2018.
[2] Georgios Arvanitidis, Søren Hauberg, and Bernhard Schölkopf. Geometrically enriched latent spaces, August 2020.
[3] Aurélien Bellet, Amaury Habrard, and Marc Sebban. Metric learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 9(1):1-151, January 2015.
[4] Adam Berenzweig, Beth Logan, Daniel P.W. Ellis, and Brian Whitman. A large-scale evaluation of acoustic and subjective music-similarity measures. Computer Music Journal, 28(2):63-76, June 2004.
[5] Shuo Chen, Lei Luo, Jian Yang, Chen Gong, Jun Li, and Heng Huang. Curvilinear distance metric learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[6] S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), volume 1, pages 539-546 vol. 1, June 2005.
[7] Ronald R. Coifman and Stéphane Lafon. Diffusion maps. Applied and Computational Harmonic Analysis, 21(1):5-30, July 2006.
[8] Jianqing Fan and I. Gijbels. Local Polynomial Modelling and Its Applications. Number 66 in Monographs on Statistics and Applied Probability. Chapman & Hall, London; New York, 1st ed edition, 1996.
[9] Charles Fefferman, Sergei Ivanov, Matti Lassas, and Hariharan Narayanan. Reconstruction of a Riemannian manifold from noisy intrinsic distances. SIAM Journal on Mathematics of Data Science, 2(3):770-808, September 2020.
[10] Andrea Frome, Yoram Singer, Fei Sha, and Jitendra Malik. Learning globally-consistent local distance functions for shape-based image retrieval and classification. In 2007 IEEE 11th International Conference on Computer Vision, pages 1-8, October 2007.
[11] T. Hastie and R. Tibshirani. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6):607-616, June 1996.
[12] Søren Hauberg, Oren Freifeld, and Michael J. Black. A geometric take on metric learning. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 2024–2032. Curran Associates, Inc., 2012.
[13] Geoffrey E Hinton and Sam Roweis. Stochastic Neighbor Embedding. In Advances in Neural Information Processing Systems, volume 15. MIT Press, 2002.
[14] Martin Jørgensen and Soren Hauberg. Isometric Gaussian process latent variable model for dissimilarity data. In Proceedings of the 38th International Conference on Machine Learning, pages 5127-5136. PMLR, July 2021.
[15] Tam Le and Marco Cuturi. Unsupervised Riemannian metric learning for histograms using Aitchison transformations. In Proceedings of the 32nd International Conference on Machine Learning, pages 2002–2011. PMLR, June 2015.
[16] G. Lebanon. Metric learning for text documents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4):497-508, April 2006.
[17] Guy Lebanon. Learning Riemannian metrics. In Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence, UAI'03, pages 362-369, San Francisco, CA, USA, 2002. Morgan Kaufmann Publishers Inc.
[18] John M. Lee. Introduction to Smooth Manifolds. Number 218 in Graduate Texts in Mathematics. Springer, New York; London, 2nd ed edition, 2013.
[19] John M. Lee. Introduction to Riemannian Manifolds. Springer Berlin Heidelberg, New York, NY, 2018.
[20] Wolfgang Meyer. Toponogov's theorem and applications. Lecture Notes, Trieste, 1989.
[21] Dominique Perraul-Joncas and Marina Meila. Non-linear dimensionality reduction: Riemannian metric estimation and the problem of geometric discovery, May 2013.
[22] Deva Ramanan and Simon Baker. Local distance functions: A taxonomy, new algorithms, and an evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4):794-806, April 2011.
[23] Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323-2326, December 2000.
[24] Christopher Scarvelis and Justin Solomon. Riemannian Metric Learning via Optimal Transport. In The Eleventh International Conference on Learning Representations, February 2023.
[25] Matthew Schultz and Thorsten Joachims. Learning a distance metric from relative comparisons. In Advances in Neural Information Processing Systems, volume 16. MIT Press, 2003.
[26] Juan Luis Suárez, Salvador García, and Francisco Herrera. A tutorial on distance metric learning: Mathematical foundations, algorithms, experimental analysis, prospects and challenges. Neurocomputing, 425:300-322, February 2021.
[27] Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319-2323, December 2000.
[28] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research, 10(2), 2009.
[29] Eric Xing, Michael Jordan, Stuart J Russell, and Andrew Ng. Distance metric learning with application to clustering with side-information. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15. MIT Press, 2002.