NeurIPS 2025

Connecting Neural Models Latent Geometries with
Relative Geodesic Representations
Hanlin Yu1
University of Helsinki
Berfin Inal
University of Amsterdam
Georgios Arvanitidis
DTU
Søren Hauberg
DTU
Francesco Locatello
IST Austria
Marco Fumero1
IST Austria
Abstract
Neural models learn representations of high-dimensional data on low-dimensional
manifolds. Multiple factors, including stochasticities in the training process, model
architectures, and additional inductive biases, may induce different representations,
even when learning the same task on the same data. However, it has recently been
shown that when a latent structure is shared between distinct latent spaces, relative
distances between representations can be preserved, up to distortions. Building
on this idea, we demonstrate that exploiting the differential-geometric structure of
latent spaces of neural models, it is possible to capture precisely the transformations
between representational spaces trained on similar data distributions. Specifically,
we assume that distinct neural models parametrize approximately the same un-
derlying manifold, and introduce a representation based on the pullback metric
that captures the intrinsic structure of the latent space, while scaling efficiently
to large models. We validate experimentally our method on model stitching and
retrieval tasks, covering autoencoders and vision foundation discriminative models,
across diverse architectures, datasets, pretraining schemes and modalities. Code is
available at https://github.com/marc0git/RelativeGeodesics.
1
Introduction
Figure 1:
Neural models trained on similar
data learn parametrizations of the same mani-
fold. NNs learn parametrizations (D1, D2) of the
same underlying manifold Y up to isometries T.
Pulling back the metric from Y makes relative
geodesic representations invariant to transforma-
tions T between latent spaces Z1 and Z2.
Neural models learn meaningful representations
of high-dimensional data generalizing to many
task...