# Discovering Data Manifold Geometry via Non-Contracting Flows

David Vigouroux, Lucas Drumetz, Ronan Fablet, François Rousseau

► To cite this version:

David Vigouroux, Lucas Drumetz, Ronan Fablet, François Rousseau. Discovering Data Manifold Geometry via Non-Contracting Flows. 2026. hal-05486024

HAL Id: hal-05486024

https://hal.science/hal-05486024v1

Preprint submitted on 30 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
David Vigouroux<sup>1,2</sup> Lucas Drumetz<sup>3,4,5</sup> Ronan Fablet<sup>3,4,5</sup> François Rousseau<sup>2,5</sup>

Abstract

We introduce an unsupervised approach for constructing a global reference system by learning, in the ambient space, vector fields that span the tangent spaces of an unknown data manifold. In contrast to isometric objectives, which implicitly assume manifold flatness, our method learns tangent vector fields whose flows transport all samples to a common, learnable reference point. The resulting arc-lengths along these flows define interpretable intrinsic coordinates tied to a shared global frame. To prevent degenerate collapse, we enforce a non-shrinking constraint and derive a scalable, integration-free objective inspired by flow matching. Within our theoretical framework, we prove that minimizing the proposed objective recovers a global coordinate chart when one exists. Empirically, we obtain correct tangent alignment and coherent global coordinate structure on synthetic manifolds. We also demonstrate the scalability of our method on CIFAR-10, where the learned coordinates achieve competitive downstream classification performance.

# 1. Introduction

Under the manifold hypothesis, high-dimensional data are assumed to concentrate near a low-dimensional manifold. When this manifold is known *a priori* or provided with a predefined parametrization, the underlying geometry is fixed and learning reduces to approximating functions or embeddings defined on it. In contrast, when the manifold itself is unknown and only accessible through sampled data, as considered here, learning an appropriate representation necessarily entails recovering its geometric structure, making representation learning for unknown manifolds a

<figure><img src="image_2.png" /><figcaption>Figure 1. (a) A local isometry maps a neighborhood of a surface to its tangent plane while preserving intrinsic distances. On curved surfaces such as a half sphere, this is impossible: Due to curvature, a geodesic neighborhood that is circular on the surface appears as an ellipse in the tangent plane. This illustrates why exact isometric learning cannot be achieved on non-flattenable surfaces. (b) A manifold is parallelizable if it admits a global frame of linearly independent tangent vector fields. The half sphere is shown with smoothly varying tangent directions, illustrating the notion of parallelizability.</figcaption></figure>

central problem (Meilä & Zhang, 2023).

A large class of unsupervised manifold-learning methods aim to infer geometric structure directly from data by preserving local distances or approximate metric relations. A subset of these approaches seeks an *isometric* representation, in which the latent space preserves intrinsic distances or the underlying Riemannian metric of the data manifold. Methods such as Isomap (Tenenbaum et al., 2000), Locally Linear Embedding (Roweis & Saul, 2000), diffusion maps (Coifman & Lafon, 2006), and spectral embeddings (Belkin & Niyogi, 2003), as well as more recent metric-consistent autoencoder and latent-variable models, fall into this category. However, isometric learning relies on a strong and often implicit assumption: that the data manifold is globally flat, i.e., admits a distance-preserving embedding, with respect to its usual metric, into a Euclidean space of zero curvature. This assumption is rarely justified in practice, and there is no general theoretical or empirical evidence that real-world data manifolds satisfy it.

In this work, we focus on a more general class of manifolds that admit a global coordinate representation and seek to discover the geometry of this unknown manifold from data. Such global coordinate representations are particularly valuable for downstream tasks—including classification, regression, clustering, and generative modeling—where geometric consistency directly impacts performance and interpretability. Even under this setting, constructing global, smooth, and geometrically consistent co-

<footnote n="1">¹IRT Saint Exupery, ANITI, Toulouse, France ²IMT Atlantique, LaTIM UMR 1101 INSERM, Brest, France ³IMT Atlantique, Lab-STICC, UMR CNRS 6285, Plouzané, France ⁴INRIA, ODYSSEY team-project, Brest, France ⁵SequoIA. Correspondence to: David Vigouroux <david.vigouroux@irt-saintexupery.com>.</footnote>
ordinates from data is ill-posed and remains a challenge
(Bronstein et al., 2017).

Our approach is motivated by the concept of parallelizability of the underlying data manifold (see Fig. 1 for a comparison with isometric representations). A parallelizable *m*-dimensional manifold admits a set of *m* linearly independent vector fields defined everywhere on the manifold that span the tangent space at each point. Under this assumption, rather than relying on pairwise distance constraints, our method directly learns *m* globally-defined vector fields, each representing a one-dimensional direction in the tangent space. Together, these vector fields recover the full tangent space at every point, thereby defining a global frame. While such a representation naturally suggests an underlying system of ordinary differential equations, our approach avoids the computational burden of an explicit integrations. We exploit the commutativity of the learned vector fields within a flow-matching-style objective to derive a scalable and numerically-stable learning procedure. Importantly, parallelizability is used here as a conceptual tool rather than as a strict topological assumption. While the method is naturally formulated in terms of global vector fields, it also applies to non-parallelizable manifolds as demonstrated in our experiments.

Our contributions can be summarized as follows:

*   **Novel geometry discovery method for data manifolds.** We propose a new unsupervised method that learns tangent vector fields directly from data, without prior knowledge of the underlying manifold, and we define a flow-based global coordinate chart.

*   **Theoretical guarantees** We prove that the proposed optimization problem under non-contracting flow constraints has a well-defined optimum and that any minimizer recovers the targeted flow-based global coordinate chart.

*   **Scalable training via commuting flows and flow matching.** We derive a scalable, integration-free training objective inspired by flow matching that exploits the commutativity of the learned vector fields, enabling efficient training for high-dimensional data.

## 2. Related work

**Learning unconstrained maps.** Many widely used representation-learning approaches aim to learn a map from data lying on an unknown low-dimensional manifold to a latent space, without assuming any prior knowledge of the manifold's structure. This category includes reconstruction-based models such as autoencoders and variational autoencoders (Kingma & Welling, 2014; Rezende et al., 2014), adversarially learned representations

such as BiGAN and ALI (Donahue et al., 2017; Dumoulin et al., 2017), and contrastive or self-supervised encoders that rely on invariance to data augmentations rather than geometric fidelity (Chen et al., 2020; He et al., 2020; Zbon玄 tar et al., 2021). These latent representations are generally defined only up to arbitrary nonlinear transformations, so distances and directions lack intrinsic meaning. Imposing geometric constraints is crucial when interpretability, global structure, or coordinate consistency are desired. Disentanglement is often formulated as learning latent variables whose dimensions correspond to approximately independent generative factors, a perspective most commonly explored in variational autoencoders and their regularized variants (Higgins et al., 2017; Kim & Mnih, 2018). From a geometric perspective, this corresponds to learning coordinates aligned with intrinsic directions of variation on the data manifold. However, disentanglement is fundamentally ill-posed without additional inductive biases or structural assumptions (Locatello et al., 2019).

**Isometric learning and metric-consistent representations.** Various works aim to recover the geometry of data manifolds by learning latent representations that preserve intrinsic distances, typically in the sense of approximating the underlying Riemannian metric whose induced path lengths define geodesic distances. This includes classical manifold learning methods targeting approximate isometries, as well as recent deep generative approaches based on normalizing flows and flow matching, where the Euclidean metric in latent space is pulled back to the data manifold to obtain a metric-consistent representation (Diepeveen & Needell, 2025; Shao et al., 2018; de Kruiff et al., 2025). Such isometric objectives effectively enforce a flat latent geometry and recover coordinates only up to isometries. In contrast, our work adopts a *frame-learning* perspective: rather than learning a metric-equivalent embedding, we learn tangent directions whose commuting flows organize the manifold into a global coordinate system. This construction relies on parallelizability as a structuring principle for learning directions, but does not impose flatness or restrict the manifold's intrinsic geometry. Beyond pure distance preservation, some methods directly regularize the learned Riemannian metric using curvature-driven objectives, including Ricci-type flows, but their reliance on higher-order differential operators makes them challenging to scale in practice (Lazarev, 2025).

**Relation to Bundle Networks.** The notion of fiber bundles has also recently appeared in machine learning, most notably in Bundle Networks (Courts & Kvinge, 2022). They introduce deep generative architectures inspired by the differential-topology concept of a fiber bundle. Bundle Networks model many-to-one mappings by decomposing the input space locally into the product of a base space and
a fiber space, enabling conditional generation of multiple plausible inputs for a given output value. This local trivialization perspective is useful for exploring the fiber structure of a task and for generative modeling of conditional distributions. While both Bundle Networks and our work draw on geometric notions from manifold theory, they operate on fundamentally different objectives. Bundle Networks leverage fiber bundle structure to model local base, whereas our approach explicitly recovers intrinsic tangent directions and organizes them into a global coordinate chart, while scaling in more complex high-dimensional datasets.

**Implicit manifold representations.** Some approaches represent data manifolds as level sets of scalar functions or energy landscapes in the ambient space. This includes implicit neural representations, score-based models, and energy-based formulations, where the manifold is characterized as the set of points minimizing an energy or satisfying a constraint (LeCun et al., 2006; Song et al., 2020; Béthune et al., 2025; Diepeveen et al., 2025). Such approaches are effective for modeling complex data distributions and for projection or sampling near the manifold, but they do not provide intrinsic coordinates or explicit tangent structure. In contrast to our frame-learning approach, implicit representations describe where the manifold lies in ambient space rather than how to traverse it along intrinsic directions, and therefore do not directly support global coordinate construction or directional manipulation.

**Lyapunov and contraction-based vector field learning.** Another related line of work focuses on learning dynamical systems under explicit stability constraints, typically via Lyapunov functions (Khalil, 2002; Rodriguez et al., 2022) or contraction metrics. These methods enforce dissipativity by requiring a Lyapunov function to decrease along trajectories, or equivalently that the Lie derivative of a metric is negative semidefinite, ensuring convergence and robustness (Chang et al., 2019). In contrast, our work adopts the opposite geometric objective: rather than enforcing contraction through a negative semidefinite Lie derivative, we impose a positive semidefinite condition on the Lie derivative of the Euclidean metric. This avoids bringing neighboring flow lines arbitrarily close, thereby preventing degeneration into a lower-dimensional structure.

# 3. Method

## 3.1. Principle

**Main idea.** Our method is designed to recover a global coordinate representation of an unknown data manifold by learning vector fields in the ambient Euclidean space whose restrictions parameterize the manifold's tangent spaces. These vector fields induce flows that transport data points

along intrinsic directions of the manifold toward a common global reference point, thereby yielding an interpretable global coordinate system embedded in the ambient space.

Formally, we consider data sampled from an unknown $m$-dimensional manifold smoothly embedded into $M \hookrightarrow \mathbb{R}^n$. Our goal is to learn a set of $m$ vector fields that capture the intrinsic directions of $M$ and whose induced flows provide global coordinates. Each vector field is defined in the ambient space $\mathbb{R}^n$ as the solution of a minimization problem to behave consistently with the geometry of the manifold. Our method is inspired by parallelizability, though it does not strictly rely on global parallelizability; see Appendix B.

**Flow-based view of global coordinates.** The core idea is to learn a flow-based mapping that moves each data point $x \in M$ along the manifold to a reference point $C \in \mathbb{R}^n$, which serves as the origin of a global coordinate frame. This map is constructed as a composition of $m$ flows, $\phi_m \circ \cdots \circ \phi_1$, where each flow $\phi_i$ captures one intrinsic degree of freedom of the manifold.

**Definition 3.1 (Flow).** Each flow $\phi_i$ is defined by a pair $(F_i, T_i)$, where $F_i : \mathbb{R}^n \to \mathbb{R}^n$ is a vector field and $T_i : \mathbb{R}^n \to \mathbb{R}_+$ is a time horizon. Specifically,

$$ \phi_i(x) = \Phi_{F_i}^{T_i}(x) = z_i(T_i(x)), \quad \left\{ \begin{array}{l} \frac{d}{dt} z_i(t) = F_i(z_i(t)), \\ z_i(0) = x. \end{array} \right. \qquad (1) $$

The aim of our method is to construct a sequence of flows acting along tangent directions of the manifold, where each successive flow removes one intrinsic dimension by projecting the manifold along that direction: the first flow resolves one direction, the second another, and so on, until all points are transported to the same reference location $C$. As a result, the arc-lengths $(\ell_1(x), \dots, \ell_m(x))$ traveled along each flow – see Appendix A – serve as the coordinates of $x$ in the learned global frame.

**Frame learning loss.** To learn the vector fields and time horizons, we introduce the *Frame learning loss*

$$ \min_{\{F_i\}, \{T_i\}, C} \mathbb{E}_{x \sim X} \left[ \| C - (\phi_m \circ \cdots \circ \phi_1)(x) \|_2^2 \right], $$

where $X$ denotes the data distribution supported on $M$.

The purpose of this loss is not merely to collapse the data to a point. Rather, the sequential structure of the flows is constructed so that each vector field aligns with a distinct intrinsic direction of the manifold. When each $F_i$ is tangent to $M$, i.e., $F_i(x) \in T_x M$ for all $x \in M$, its integral curves follow one coordinate direction of the manifold, and the composition of these flows yields a geometrically-consistent global parameterization of $M$.
<figure><img src="image_3.png" /><figcaption>**Figure 2. Main Principle of Our Method.** Given data to an unknown manifold, our goal is to construct a global reference frame defined by a trainable center $C$. (a) Learning a single unconstrained vector field is insufficient: such a field can always be rescaled to collapse the entire manifold onto $C$. (b) Imposing a non-shrinking constraint—by requiring the Lie derivative of the metric to be positive semi-definite (see Appendix A)—prevents this collapse and enforces geometric consistency. (c) By sequentially combining $m$ tangent vector fields (matching the intrinsic dimension of the manifold), we can transport every point to a unique location associated with the global frame’s center while ensuring trajectories remain on the manifold.</figcaption></figure>

**Avoiding degenerate solutions.** Without additional structure, the optimization problem admits degenerate solutions in which the vector fields shrink all trajectories so that every point collapses directly to the reference point $C$, ignoring the geometry of the manifold (Fig. 2a). The resulting arc-lengths do not encode meaningful coordinates.

To prevent this behavior, we impose a non-shrinking condition on each vector field. Intuitively, nearby trajectories should not locally contract distances along the flow. Formally, this is enforced by requiring the Lie derivative of a conformally Euclidean metric $g_i$ along to each vector field $F_i$ to be positive semidefinite (see Appendix A). This condition enforces local non-contraction and rules out degenerate collapsing solutions (Fig. 2b). This notion is formally introduced in Definition 3.3.

**Geometric outcome.** Under this constraint, each learned vector field becomes tangent to $M$ and defines, at every point $x \in M$, a one-dimensional intrinsic direction. Composing $m$ such flows recovers a full global reference frame: successive flows resolve the intrinsic degrees of freedom one by one, and the final composition maps the entire manifold to a single reference point while yielding a complete and interpretable global coordinate system (Fig. 2c). From a differential-geometric perspective, each vector field induces a smooth line subbundle of $TM$, and the resulting global frame realizes a decomposition of the tangent bundle into one-dimensional subbundles (Lee, 2013, Ch. 10).

**Training and scalability.** To make this framework computationally practical, we exploit the commutativity of the learned vector fields. Informally, commutativity (see Appendix A – Figure 3) means that flowing along one vector field and then another yields the same result regardless of the order in which the flows are applied. We leverage this property to replace a minimization objective based on the numerical integration of the flows with a flow-matching-inspired objective that directly aligns the vector fields with the desired transport directions. As

a result, training becomes integration-free, scalable, and numerically-stable, even as the intrinsic dimension of the manifold grows.

## 3.2. Theoretical Results

We now formalize the framework introduced in the previous section and develop the mathematical setting required to state our main theoretical results. We recall that the objective of our method is to recover a global coordinate system on an unknown manifold by composing flows generated by suitably constrained vector fields.

The global parametrization considered in this work only applies to manifolds satisfying a mild regularity condition, formalized through the notion of *admissible manifolds*, namely compact embedded submanifolds of $\mathbb{R}^n$ that admit a global chart onto an Euclidean domain. This assumption ensures the absence of topological obstructions and provides a natural target for the coordinate functions produced by our method.

**Definition 3.2** (Admissible Manifolds with a Global Chart). Let $\mathcal{M}_m$ denote the class of all compact, connected, smoothly embedded $m$-dimensional submanifolds with boundary of $\mathbb{R}^n$ that admit a global chart in the sense of manifolds with boundary:

$$ \psi: M \xrightarrow{\approx} U \subset \mathbb{R}^m, $$

where $M \in \mathcal{M}_m$ and $U$ is a compact set with nonempty interior.

The flows used to parameterize the manifold are generated by vector fields subject to a non-shrinking condition. It requires the Lie derivative of the metric along the vector field to be non-negative (see Appendix A for a formal definition of the Lie derivative). This ensures that the induced dynamics follow the tangent structure of the manifold and avoid collapse, so that arc-lengths along integral curves can be interpreted as intrinsic coordinates. In addition, each vector field is equipped with a time
horizon function that forces all trajectories to terminate at a common reference point, thereby enabling the construction of a coherent global frame.

**Definition 3.3** (Admissible maps induced by flows). Let $\mathcal{F}$ denote the space of smooth flows $\phi: \mathbb{R}^n \to \mathbb{R}^n$ obtained from a triple $(F, T, \sigma)$, where

* $T: \mathbb{R}^n \to \mathbb{R}^+$ is a smooth bounded time horizon function.

* $\sigma: \mathbb{R}^n \to \mathbb{R}$ be a bounded smooth scalar field defining a conformally Euclidean Riemannian metric

$$g(x) = e^{\sigma(x)} I_n.$$

* $F: \mathbb{R}^n \to \mathbb{R}^n$ is a nowhere-vanishing smooth bounded vector field such that the Lie derivative of the conformally Euclidean metric $g$ along $F$ satisfies

$$\mathcal{L}_F g \ge 0 \quad \text{on } \mathbb{R}^n, \quad |\langle \nabla \sigma(x), F(x) \rangle| \le K < \infty \\ \langle \nabla T(x), F(x) \rangle = -1 \quad \forall x \in M \in \mathcal{M}_m$$

For each such triple, following Def 1, the flow is defined as:

$$\phi(x) = \Phi_F^T(x),$$

and all such maps $\phi$ are collected in $\mathcal{F}$.

The flows are then composed sequentially. Each vector field spans a one-dimensional subbundle of the tangent bundle of the manifold, and the length traveled along its integral curves defines a scalar quantity $\ell_i(x)$. These quantities are interpreted as intrinsic coordinates associated with the successive flow directions, and their joint behavior under composition is central to our theoretical analysis.

**Definition 3.4** (Lengths of field line segments). Given a sequence of flows $\{\phi_1, \dots, \phi_k\}$ generated by integrating $F_i$ during the time $T_i(x)$, and a point $x \in \mathbb{R}^n$, the quantity $\ell_i(x)$ denotes the length of the field line segment traced by $F_i$ during time $T_i$, starting from

$$(\phi_{i-1} \circ \dots \circ \phi_1)(x).$$

A formal definition of $l_i$ is provided in Appendix A.

Under the admissibility conditions described above, the optimization of the frame learning loss admits a well-defined geometric interpretation. For a known intrinsic dimension $m$, the minimizer yields vector fields that are tangent to $M$, whose flows cover the manifold, and whose associated intrinsic lengths $(\ell_1, \dots, \ell_m)$ define a global chart. This behavior is formalized in the following theorem.

<figure><img src="image_4.png" /><figcaption>**Figure 3.** Effect of commuting vector fields on flow simplification When the vector fields commute, the sequential application of flows generated by $(F_1, T_1)$ and $(F_2, T_2)$ is equivalent to integrating a single combined vector field. Our formulation exploits this property to replace a sequence of $n$ ODEs with a single ODE producing the same transport.</figcaption></figure>

**Theorem 3.1** (Loss Minimization when the Dimension is Known). Let $M \in \mathcal{M}_m$. Let $\{\phi_i\} \in \mathcal{F}$. Let $X$ be a random variable drawn from a probability distribution $\rho$ whose support coincides with the manifold $M$. Define

$$\mathcal{L}_m = \min_{\{T_i, F_i, \sigma_i\}_{i=1}^m, C} \mathbb{E}_{x \sim X} \left[ \| C - (\phi_m \circ \dots \circ \phi_1)(x) \|_2^2 \right]$$

Then, the minimal value $\mathcal{L}_m$ is equal to 0 and the vector fields generating the flows are tangent to $M$ and the flows generated by these vector fields cover $M$. Moreover, the functions $(\ell_1, \dots, \ell_m)$ provide a system of global coordinates on $M$.

When the intrinsic dimension of the data manifold is unknown, one must determine an appropriate value of $m$ for the model. A natural strategy is to choose the smallest $m$ for which the loss can be driven to zero. The following result guarantees the validity of this approach: if $m$ is chosen too small, then the model lacks sufficient degrees of freedom to represent the manifold, and the loss remains strictly positive. Thus, vanishing loss occurs if and only if $m$ matches (or exceeds) the true intrinsic dimension.

**Theorem 3.2** (Main Characterization via Loss Minimization). Let $M \in \mathcal{M}_m$, and suppose we have $k < m$. Let $\{\phi_i\} \in \mathcal{F}$. Let $X \sim \rho$ be a random variable sampled from the distribution supported on $M$. There exists $c > 0$ such that $\mathcal{L}_k \ge c$ where:

$$\mathcal{L}_k = \min_{\{T_i, F_i, \sigma_i\}_{i=1}^k, C} \mathbb{E}_{x \sim X} \left[ \| C - (\phi_k \circ \dots \circ \phi_1)(x) \|_2^2 \right]$$

All proofs of the theorems are provided in the appendix C.

### 3.3. Training Formulation

**Commutativity-induced flow** Evaluating the composite flow $\phi_m \circ \dots \circ \phi_1$ requires integrating $m$ distinct Neural ODEs in sequence, which becomes increasingly expensive as $m$ grows. By enforcing that the field line bundles commute (see, the proof of Frobenius integrability theorem -
Lee, 2013, Theorem 19.12 Frobenius), the composition of flows can be equivalently evaluated as a single flow. As illustrated in Figure 3, flowing first along $F_1$ for time $T_1$ and then along $F_2$ for time $T_2$ reaches the same point $C$ as integrating the single combined field $T_1F_1 + T_2F_2$. This geometric property of commuting line bundles is analogous to moving along orthogonal directions on a manifold. It ensures that ordered compositions of flows can be replaced by a single autonomous flow. Consequently, the entire sequence of $m$ Neural ODEs can result into the integration of one vector field without changing the resulting transport. Theorem 3.3 formalizes this property and reformulates the frame learning loss in Theorem 3.1 according to a single Neural ODE.

**Theorem 3.3 (Equivalent Loss under Commuting Flows).** Let $T: \mathbb{R}^n \to \mathbb{R}_+^m$ be defined componentwise by $T(x) = (T_1(x), ..., T_m(x))$, let $F: \mathbb{R}^n \to \mathbb{R}^{n \times m}$ be defined as $F(x) = (F_1(x), ..., F_m(x))$ and let $\sigma: \mathbb{R}^n \to \mathbb{R}^m$ be defined as $\sigma(x) = (\sigma_1(x), ..., \sigma_m(x))$. Assume that the vector fields $F_1, ..., F_m$ commute and satisfy $JT(x)F(x) + I_m = 0_m$. Then the loss in Theorem 3.1 can be rewritten as:

$$ \mathcal{L}_m = \min_{T,F,\sigma,C} \mathbb{E}_{x \sim X} \lim_{t \to \infty} \left[ \| C - \phi^{\top} \sum_{i=1}^{m} T_i F_i(x) \|_2^2 \right], $$

where $\phi$ denotes the flow defined in Definition 1. The proof of the theorem is provided in Appendix C.

**Integration-free training via path-wise convergence.** Reducing the computation to a single autonomous Neural ODE significantly alleviates the burden compared to sequentially integrating $m$ flows. However, the resulting ODE must be evolved to infinite time in order to reach its equilibrium, which is not achievable in practice and leads to numerical and optimization difficulties.

To circumvent this issue, and inspired by flow-matching schemes (Kapuńniak et al., 2024), we introduce an interpolating curve $u_s: [0, 1] \times \mathbb{R}^n \to \mathbb{R}^n$ whose endpoints are fixed to be the input $x$ and output $C$, and we match the instantaneous velocity of this curve with the velocity generated by the autonomous vector field. Crucially, the first argument of $u_s$ indexes the trajectory location rather than the physical time: while the ODE converges to its endpoint only asymptotically in time, learning is performed on the geometry of paths in the state space and is independent of this temporal convergence.

In other words, rather than requiring the numerical trajectory to land exactly at $C$, we only require its speed at each time $t$ to align with the speed of a known reference path. This removes the dependency on high-precision ODE integration and produces a much more stable optimization problem.

Let us introduce a neural network $s: [0, 1] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}^n$ parameterizing the interpolating curve $u_s$, defined by

$$ u_s(t, x, C) = (1-t)x + tC + t(1-t)s(t, x). \quad (2) $$

which satisfies the boundary conditions $u_s(0, x, C) = x$ and $u_s(1, x, C) = C$.

Instead of supervising only the endpoint, we match the instantaneous velocity of the flow to the time derivative of the reference path $u_s(t, x, C)$ and we rewrite the loss of Theorem 3.3 as

$$ \begin{gathered} \min_{T,F,s,C} \mathbb{E}_{x \sim X} \int_0^1 L_c(F, T, s, x, t) dt, \\ L_c(F, T, s, x, t) = \left\| F(u_s(t, x, C)) T(u_s(t, x, C)) - \partial_t u_s(t, x, C) \right\|_2^2. \end{gathered} $$

This integral form is conceptually useful but not required for computation, since the objective is driven to zero. For efficiency, the evaluation time $t$ is instead sampled uniformly, yielding the stochastic objective:

$$ \min_{T,F,s,C} \mathbb{E}_{x \sim X} \mathbb{E}_{t \sim \mathcal{U}[0,1]} L_c(F, T, s, x, t) \quad (3) $$

### Enforcing positive semidefiniteness of the Lie derivative

To enforce the positive semidefiniteness (PSD) of the Lie derivative of the metric, directly computing the full Jacobian matrix of each vector field is computationally prohibitive and does not scale to high-dimensional data. Instead, we rely on a trace-based surrogate that avoids explicit Jacobian construction. Concretely, we regularize:

$$ \begin{gathered} \mathcal{R}_{\lambda}(x, F_i, T_i, \sigma_i) = h(\Lambda(A(F_i, \sigma_i, x)), T_i(x)), \\ A(F_i, \sigma_i, x) = \operatorname{sgn}(T_i(x))(JF_i(x) + JF_i^{\top}(x) \\ + \langle \nabla \sigma_i, F_i(x) \rangle > I_n), \end{gathered} \quad (4) $$

where $\Lambda(\cdot)$ denotes the eigenvalues of the symmetric matrix $A(F_i, \sigma_i, x)$, capturing the aggregate local metric expansion induced by the vector field $F_i$ and $h$ is a function to penalize negative eigenvalues in order to enforce positive semidefiniteness. Each regularization term is rescaled by the corresponding time function $T(x)$. It prevents degenerate solutions in which the vector field magnitude is arbitrarily reduced while the time horizon is increased proportionally to satisfy the regularization. In our theoretical framework, we assume that the time functions are positive. This constraint can be relaxed by enforcing that the Lie derivative of the metric is positive semidefinite for forward time and negative semidefinite for backward time. See Appendix E for computational details and definition of the function $h$.
In our experiments, we randomly sample a single vector field per minibatch when evaluating the regularization term. This stochastic approximation significantly reduces the computational cost while remaining stable in practice.

Furthermore, we control the contractiveness of the vector field by regularizing the conformal factor $\sigma_i$ as follows:

$$ \mathcal{R}_{\text{metric}} = m(\langle \nabla \sigma_i, F_i \rangle T_i)^2. \quad (5) $$

**Time and commutativity regularization.** As in the regularization term enforcing positivity of the Lie derivative, we rescale the time and commutativity regularization terms by the time function $T_i$. This rescaling improves the stability of the model and leads to robust weighting choices throughout our experiments.

$$ \mathcal{R}_{\text{commute}}(x, F, T) = \sum_{i \neq j} \| [F_i(x), F_j(x)] \|_2^2 T_i(x) T_j(x), \quad (6) $$

where [., .] denotes the Lie bracket (see Appendix A).

$$ \begin{aligned} \mathcal{R}_{\text{time}}(x, F, T) &= \| S(x, F, T)(JT(x) F(x) + I_m) \|_2^2, \\ S(x, F, T)_{i,j} &= \delta_{ij} + (1 - \delta_{ij}) \|F_j(x)\|_2^{-1} \|\nabla T_i(x)\|_2^{-1}. \end{aligned} \quad (7) $$

**Overall objective.** The full training objective (see Algorithm 1) combines a flow-matching term with four geometric regularization terms controlling contraction and enforcing commutativity and temporal consistency. In all experi-

**Algorithm 1 Training loss evaluation and update**

**Require:** Batch $\{x^{(b)}\}_{b=1}^B$, vector fields $F = \{F_i\}_{i=1}^m$, time maps $T = \{T_i\}_{i=1}^m$, conformal factors $\sigma = \sigma_i$, field $s$, reference point $C \in \mathbb{R}^n$

1: **for** $b=1$ **to** $B$ **do**

2: &nbsp;&nbsp;&nbsp;&nbsp;Sample $i \sim \mathcal{U}\{1, \dots, m\}$ and $t \sim \mathcal{U}[0, 1]$

3: &nbsp;&nbsp;&nbsp;&nbsp;Compute the trajectory point $z \leftarrow u_s(t, x^{(b)}, C)$ (defined in 2)

4: &nbsp;&nbsp;&nbsp;&nbsp;Compute losses:

$$ \mathcal{L} \leftarrow L_c(F, T, s, x^{(b)}) \quad (\text{defined in 3}) $$

$$ +\alpha \mathcal{R}_{\lambda}(z, F_i, T_i, \sigma_i) \quad (\text{defined in 4}) $$

$$ +\beta \mathcal{R}_{\text{commute}}(z, F, T) \quad (\text{defined in 6}) $$

$$ +\zeta \mathcal{R}_{\text{time}}(z, F, T) \quad (\text{defined in 7}) $$

$$ +\eta \mathcal{R}_{\text{metric}}(z, F_i, \sigma_i) \quad (\text{defined in 5}) $$

5: **end for**

6: Update parameters of $(F, T, \sigma, s, C)$ by backpropagating $\frac{1}{B} \sum_{b=1}^B \mathcal{L}$

ments, the regularization coefficients $\alpha, \beta$, and $\zeta$ are set to

$1$ and $\sigma_i = 0$ and $\eta = 0$. We observe that the regularization factors have little influence (see Appendix F).

**Coordinate system.** To construct a global coordinate system, we use the learned time functions as a proxy. These functions represent the flow time required to reach a reference point, which coincides with the exact distance whenever the vector field has unit norm.

## 4. Experiments

The experiments are designed to illustrate the behavior of the proposed method, empirically validate its theoretical properties, and assess its practical scalability. Our method addresses a setting that is not directly covered by existing approaches, namely the unsupervised learning of global coordinate structures via ambient-space vector fields, under explicit geometric constraints. As a result, the experiments are designed to (i) verify the necessity and effect of the proposed regularization terms in controlled settings, (ii) illustrate the geometric behavior of the learned vector fields on both parallelizable and non-parallelizable manifolds, and (iii) demonstrate that the approach scales to high-dimensional image data. Architectural details, and training procedures are provided in Appendix E.

**Linear manifold sanity check.** We first consider a linear manifold defined as a uniformly sampled 3-dimensional plane embedded in a 4-dimensional ambient space. This controlled setting allows us to directly assess the necessity of learning the correct number of vector fields.

<figure><img src="image_5.png" /><figcaption>Figure 4. Linear manifold in $\mathbb{R}^4$: Frame learning loss versus number of learned vector fields $m$ for 3-dimensional plane dataset.</figcaption></figure>

To validate Theorem 3.1 and Theorem 3.2, we vary the number of learned vector fields $m \in \{1, 2, 3, 4\}$. We also perform an ablation study in which the Lie-derivative regularization is disabled. The resulting frame learning loss is reported in Figure 4. When the number of vector fields does not match the intrinsic dimension of the manifold, the loss cannot vanish. In contrast, without the regularization term, a single vector field suffices to collapse the manifold and achieve zero loss despite the dimensional mismatch,
yielding a degenerate and geometrically meaningless solution. This experiment highlights the need for geometric constraints and an appropriate number of vector fields to achieve near-zero loss up to numerical error.

**Curved manifolds in $R^3$.** We next study 2-dimensional manifolds embedded in 3-dimensional space (Table 1): sphere, torus, Swiss roll and Hyperbolic paraboloid surface. These examples allow us to evaluate both tangency and global consistency of the learned vector fields. Tangency is assessed by computing the cosine similarity between the learned vector fields and the analytical normal vectors of the manifolds. Across all examples (Table 1), the learned vector fields are nearly orthogonal to the normal directions, confirming that they are tangent to the manifold.

<table><caption>**Table 1. Evaluation.** Angular error (in degrees) between the estimated and ground-truth tangent spaces on synthetic datasets. An isometric auto-encoder (Iso AE) is used as a baseline.</caption><thead><tr><th>Data set</th><th>Ours</th><th>Iso AE</th><th>Parallelizable</th><th>Flat</th></tr></thead><tbody><tr><td>Paraboloid</td><td><b>0.55° ± 0.07</b></td><td>2.17°</td><td>✓</td><td>✓</td></tr><tr><td>Swiss Roll</td><td><u>3.11° ± 1.05</u></td><td><b>1.75°</b></td><td>✓</td><td>✓</td></tr><tr><td>Sphere</td><td><b>0.97° ± 0.11</b></td><td>29.90°</td><td>✗</td><td>✗</td></tr><tr><td>Torus</td><td><u>6.93° ± 0.7</u></td><td>24.07°</td><td>✓</td><td>✗</td></tr></tbody></table>

<figure><img src="image_6.png" /><figcaption>**Figure 5.** Learning tangent spaces on the sphere (a) The combined vector field *FT*. Consistent with the Hairy Ball Theorem, no smooth global tangent vector field exists on the sphere, resulting in singularities near the poles. (b) The two learned tangent vector fields defining local tangent directions.</figcaption></figure>

This experiment also illustrates the behavior of our method when a global coordinate system does not exist. In particular, the sphere is not parallelizable and does not admit a smooth global frame. As shown in Figure 5, singularities emerge near the poles, in accordance with the Hairy Ball Theorem, while the vector fields remain well-defined almost everywhere else on the manifold.

**CIFAR-10** To evaluate scalability, we apply our method to the CIFAR-10 dataset. Following (Brown et al., 2023), we consider under the hypothesis that the data lie on a manifold of intrinsic dimension 20. We freeze the learned time functions and train a lightweight classifier on top, without data augmentation. We compare this approach against an autoencoder baseline with the same architecture and training protocol. First, although the eigenvalue-based

constraints are computationally demanding, the method remains scalable, as evidenced by the reported training-time runtime and memory costs. Secondly, both methods achieve comparable classification accuracy 2.

<table><caption>**Table 2.** Evaluation on CIFAR-10. Comparison of classification accuracies between our method and an autoencoder baseline, trained without data augmentation. Reported times (in seconds) and memory consumption (in GiB) correspond to the backward pass for a single batch of size 100 measured on a RTX4090.</caption><thead><tr><th>DATA SET</th><th>Ours</th><th>ISOMETRIC AE</th><th>AE</th></tr></thead><tbody><tr><td>ACCURACY</td><td>42.0%</td><td>22.8%</td><td>45.0%</td></tr><tr><td>TRAINING TIME</td><td>2.03s</td><td>0.09s</td><td>0.008s</td></tr><tr><td>MEMORY</td><td>11.3 GiB</td><td>5.4 GiB</td><td>0.5 GiB</td></tr></tbody></table>

As expected, the frame training loss does not converge to 0 (~0.05 per dimension), reflecting that CIFAR-10 is better modeled as a collection of distinct manifolds rather than a single globally parameterizable one (Brown et al., 2023), yet the representation remains useful for downstream tasks.

# 5. Limitations and Future Work

While the present work focuses on a specific geometric setting, it naturally opens several directions for extension. First, the method assumes the existence of a global coordinate map, an assumption that does not hold for many real-world datasets. A natural extension is to learn multiple local reference systems, which can be viewed as a geometric analogue of extending VAEs to VQ-VAEs. Second, our approach is designed to learn the geometry of the manifold but does not explicitly model the data density along it. Incorporating density estimation would enable generative modeling and likelihood-based evaluation. Finally, designing neural architectures that satisfy Lie derivative constraints by construction would be a promising way to reduce training cost and improve scalability.

# 6. Conclusion

We introduced a new unsupervised learning framework for recovering global coordinate structures of data manifolds by learning ambient-space vector fields under explicit geometric constraints. Unlike classical manifold learning approaches that focus on local embeddings, our method directly models intrinsic geometry and global structure. The framework is grounded in differential geometry, supported by theoretical guarantees, and validated empirically on both controlled synthetic manifolds and high-dimensional image data. By demonstrating that such geometric principles can be enforced at scale, this work opens a promising direction for geometry-aware representation learning beyond local or purely embedding-based methods.
## Acknowledgements

This work was carried out within the DEEL project,<fnref n="1" /> which is part of IRT Saint Exupéry and the ANITI AI cluster. This work is also carried out in collaboration with the SequoIA AI Cluster, in particular via AI chair GENESIS. The authors acknowledge the financial support from DEEL's Industrial and Academic Members and the France 2030 program - Grant agreements n°ANR-10-AIRT-01, n°ANR-23-IACL-0002 and ANR-23-IACL-0009.

## Impact Statement

This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.

## References

Belkin, M. and Niyogi, P. Laplacian eigenmaps for dimensionality reduction and data representation. *Neural Computation*, 15(6):1373–1396, 2003. doi: 10.1162/089976603321780317.

Béthune, L., Vigouroux, D., Du, Y., VanRullen, R., Serre, T., and Boutin, V. Follow the energy, find the path: Riemannian metrics from energy-based models. In *The Thirty-ninth Annual Conference on Neural Information Processing Systems*, 2025. URL https://openreview.net/forum?id=BOiQ7Kd5Lx.

Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., and Vandergheynst, P. Geometric deep learning: Going beyond euclidean data. *IEEE Signal Processing Magazine*, 34(4):18–42, 2017. doi: 10.1109/MSP.2017.2693418.

Brown, B. C., Caterini, A. L., Ross, B. L., Cresswell, J. C., and Loaiza-Ganem, G. Verifying the union of manifolds hypothesis for image data. In *The Eleventh International Conference on Learning Representations*, 2023. URL https://openreview.net/forum?id=Rvee9CAX4fi.

Chang, Y.-C., Roohi, N., and Gao, S. Neural lyapunov control. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), *Advances in Neural Information Processing Systems*, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/2647c1dba23bc0e0f9cdf75339e120d2-Paper.pdf.

Chen, R. T. Q., Behrmann, J., Duvenaud, D., and Jacobsen, J.-H. *Residual flows for invertible generative modeling*. Curran Associates Inc., Red Hook, NY, USA, 2019.

Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual representations. In *Proceedings of the 37th International Conference on Machine Learning*, ICML'20. JMLR.org, 2020.

Coifman, R. R. and Lafon, S. Diffusion maps. *Applied and Computational Harmonic Analysis*, 21(1):5–30, 2006. ISSN 1063-5203. doi: https://doi.org/10.1016/j.acha.2006.04.006. URL https://www.sciencedirect.com/science/article/pii/S1063520306000546. Special Issue: Diffusion Maps and Wavelets.

Courts, N. and Kvinge, H. Bundle networks: Fiber bundles, local trivializations, and a generative approach to exploring many-to-one maps. In *International Conference on Learning Representations*, 2022. URL https://openreview.net/forum?id=aBXzcPPOuX.

de Kruiff, F., Bekkers, E. J., Öktem, O., Schönlieb, C.-B., and Dieveveen, W. Pullback flow matching on data manifolds, 2025. URL https://openreview.net/forum?id=mBXLtNKpeQ.

Dieveveen, W. and Needell, D. Manifold learning with normalizing flows: Towards regularity, expressivity and isoriemannian geometry, 2025. URL https://arxiv.org/abs/2505.08087.

Dieveveen, W., Batzolis, G., Shumaylov, Z., and Schönlieb, C.-B. Score-based pullback Riemannian geometry: Extracting the data manifold geometry using anisotropic flows. In Singh, A., Fazel, M., Hsu, D., Lacoste-Julien, S., Berkenkamp, F., Maharaj, T., Wagstaff, K., and Zhu, J. (eds.), *Proceedings of the 42nd International Conference on Machine Learning*, volume 267 of *Proceedings of Machine Learning Research*, pp. 13746–13773. PMLR, 13–19 Jul 2025. URL https://proceedings.mlr.press/v267/dieveveen25a.html.

Donahue, J., Krähenbühl, P., and Darrell, T. Adversarial feature learning. In *International Conference on Learning Representations*, 2017. URL https://openreview.net/forum?id=BJtNZAFgg.

Dumoulin, V., Belghazi, I., Poole, B., Lamb, A., Arjovsky, M., Mastropietro, O., and Courville, A. Adversarially learned inference. In *International Conference on Learning Representations*, 2017. URL https://openreview.net/forum?id=B1E1R4cgg.

<footnote n="1">¹https://www.deel.ai/</footnote>
Frans, K., Levine, S., and Abbeel, P. A stable whitening optimizer for efficient neural network training. In *The Thirty-ninth Annual Conference on Neural Information Processing Systems*, 2025. URL https://openreview.net/forum?id=0T8i3uXq30.

Guillemin, V. and Pollack, A. *Differential topology*. Prentice-Hall, Ine., Englewood Cliffs, New Jersey, 1974.

He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Momentum contrast for unsupervised visual representation learning. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2020.

Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. beta-VAE: Learning basic visual concepts with a constrained variational framework. In *International Conference on Learning Representations*, 2017. URL https://openreview.net/forum?id=Sy2fzU9gl.

Hutchinson, M. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. *Com munications in Statistics - Simulation and Computation*, 19(2):433–450, 1990.

Jia, Z., Zaharia, M., and Aiken, A. Beyond data and model parallelism for deep neural networks. In Talwalkar, A., Smith, V., and Zaharia, M. (eds.), *Proceedings of Machine Learning and Systems*, volume 1, pp. 1–13, 2019. URL https://proceedings.mlsys.org/paper_files/paper/2019/file/b422680f3db0986ddd7f8f126baaf0fa-Paper.pdf.

Kapuńniak, K., Potaptchik, P., Reu, T., Zhang, L., Tong, A., Bronstein, M., Bose, A. J., and Di Giovanni, F. Metric flow matching for smooth interpolations on the data manifold. In *Proceedings of the 38th International Conference on Neural Information Processing Systems*, NIPS '24, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 9798331314385.

Khalil, H. K. *Nonlinear systems*. Prentice-Hall, Upper Saddle River, NJ, 2002. URL https://cds.cern.ch/record/1173048.

Kim, H. and Mnih, A. Disentangling by factorising. In Dy, J. and Krause, A. (eds.), *Proceedings of the 35th International Conference on Machine Learning*, volume 80 of *Proceedings of Machine Learning Research*, pp. 2649–2658. PMLR, 10–15 Jul 2018. URL https://proceedings.mlr.press/v80/kim18b.html.

Kingma, D. P. and Welling, M. Auto-encoding variational bayes. In Bengio, Y. and LeCun, Y. (eds.), *2nd International Conference on Learning Representations*, ICLR

2014, Banff, AB, Canada, April 14-16, 2014, *Conference Track Proceedings*, 2014. URL http://arxiv.org/abs/1312.6114.

Krämer, N., Moreno-Muñoz, P., Roy, H., and Hauberg, S. Gradients of functions of large matrices. *Advances in Neural Information Processing Systems*, 37:4948–49518, 2024.

Lazarev, A. *Metric regularization in machine learning with curvature functionals*. Theses, Université de Toulouse, May 2025. URL https://theses.hal.science/tel-05223442.

LeCun, Y., Chopra, S., Hadsell, R., Ranzato, A., and Huang, F. J. A tutorial on energy-based learning. In *Predicting Structured Data*, 2006. URL http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf.

Lee, J. M. *Introduction to smooth manifolds*. Graduate texts in mathematics. Springer, New York, 2nd edition edition, 2013.

Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B., and Bachem, O. Challenging common assumptions in the unsupervised learning of disentangled representations. In Chaudhuri, K. and Salakhutdinov, R. (eds.), *Proceedings of the 36th International Conference on Machine Learning*, volume 97 of *Proceedings of Machine Learning Research*, pp. 4114–4124. PMLR, 09–15 Jun 2019. URL https://proceedings.mlr.press/v97/locatello19a.html.

Meilä, M. and Zhang, H. Manifold learning: what, how, and why, 2023. URL https://arxiv.org/abs/2311.03757.

Rezende, D. J., Mohamed, S., and Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. In Xing, E. P. and Jebara, T. (eds.), *Proceedings of the 31st International Conference on Machine Learning*, volume 32 of *Proceedings of Machine Learning Research*, pp. 1278–1286, Beijing, China, 22–24 Jun 2014. PMLR. URL https://proceedings.mlr.press/v32/rezende14.html.

Rodriguez, I. D. J., Ames, A., and Yue, Y. LyaNet: A Lyapunov framework for training neural ODEs. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), *Proceedings of the 39th International Conference on Machine Learning*, volume 162 of *Proceedings of Machine Learning Research*, pp. 18687–18703. PMLR, 17–23 Jul 2022. URL https://proceedings.mlr.press/v162/rodriguez22a.html.
Roweis, S. T. and Saul, L. K. Nonlinear dimensionality reduction by locally linear embedding. *Science*, 2000.

Shao, H., Kumar, A., and Fletcher, P. T. The riemannian geometry of deep generative models. *2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)*, pp. 428–4288, 2018. URL https://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w10/Shao_The_Riemannian_Geometry_CVPR_2018_paper.pdf.

Song, Y., Sohl-Dickstein, J. N., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. *ArXiv*, abs/2011.13456, 2020. URL https://openreview.net/pdf/ef0eadbe07115b0853e964f17aa09d811cd490f1.pdf?ref=news-tutorials-ai-research.

Tenenbaum, J. B., de Silva, V., and Langford, J. C. A global geometric framework for nonlinear dimensionality reduction. *Science*, 2000.

Teschl, G. *Ordinary Differential Equations and Dynamical Systems*, volume 140 of *Graduate Studies in Mathematics*. American Mathematical Society, 2012. ISBN 978-0-8218-8328-0. Graduate Studies in Mathematics 144, Amer. Math. Soc., Providence, 2012.

woo Lee, H., Choi, H., and Kim, H. Robust weight initialization for tanh neural networks with fixed point analysis. In *The Thirteenth International Conference on Learning Representations*, 2025. URL https://openreview.net/forum?id=Es4RPNDtmq.

Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S. Barlow twins: Self-supervised learning via redundancy reduction. In *Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learning*, volume 139 of *Proceedings of Machine Learning Research*, pp. 12310–12320. PMLR, 18–24 Jul 2021. URL https://proceedings.mlr.press/v139/zbontar21a.html.
# A. Appendix - Background

Before introducing the Lie derivative formally, we provide an intuitive interpretation of the quantity we seek to control. A vector field defines a continuous deformation of space by moving points along its flow. As points are transported, the distances between nearby points may increase, remain unchanged, or decrease. In many learning settings, unconstrained vector fields tend to collapse regions of space, shrinking distances and destroying geometric structure. To prevent such degeneracies, we need a way to measure how a vector field locally changes distances between infinitesimally close points. The Lie derivative of the metric provides exactly this information: it quantifies the instantaneous rate at which the squared distance between nearby points changes when they are pushed along the flow of the vector field. Requiring this quantity to be non-negative enforces a local non-contraction property, ensuring that the flow does not shrink distances to first order.

## A.1. Lie Bracket of Vector Fields

Let $M$ be a smooth manifold and let $X, Y \in \mathcal{X}(M)$ be smooth vector fields. Interpreting vector fields as first-order differential operators, their Lie bracket is given by the commutator

$$[X, Y] = XY - YX.$$

In local coordinates, writing the vector fields as

$$X = \sum_{i} X^{i} \partial_{x^{i}}, \quad Y = \sum_{i} Y^{i} \partial_{x^{i}},$$

the Lie bracket can be expressed in matrix form as

$$[X, Y] = (J_{Y}) X - (J_{X}) Y,$$

where $J_X$ and $J_Y$ denote the Jacobian matrices of the component functions of $X$ and $Y$, respectively.

## A.2. Pushforward and the Pushforward Identity

Let $\varphi : M \to N$ be a smooth map between manifolds. The pushforward of a tangent vector $v \in T_x M$ is the linear map

$$\varphi_* : T_x M \to T_{\varphi(x)} N, \quad \varphi_*(v)(f) = v(f \circ \varphi)$$

for all $f \in C^\infty(N)$.

If $\{\varphi_t\}$ is the flow of a vector field $X$, then the infinitesimal change of a vector field $Y$ under transport by this flow is given by the pushforward identity:

$$\left. \frac{d}{dt} \right|_{t=0} (\varphi_t)_* Y = [X, Y].$$

## A.3. Lie Derivative of the Metric (Euclidean Case)

In Euclidean space $\mathbb{R}^n$ equipped with the standard metric

$$g_x(u, v) \equiv g(v, u) = u \cdot v,$$

the metric is constant and coincides with the dot product. For a smooth vector field $X : \mathbb{R}^n \to \mathbb{R}^n$, the Lie derivative of the metric with respect to $X$ is the symmetric (0, 2)-tensor

$$(\mathcal{L}_X g)(Y, Z) = X(Y \cdot Z) - [X, Y] \cdot Z - Y \cdot [X, Z].$$

In coordinates, writing

$$X = (X^1, \dots, X^n), \quad Y = (Y^1, \dots, Y^n), \quad Z = (Z^1, \dots, Z^n),$$

and denoting by $\partial_i$ the partial derivative with respect to $x^i$, this expression reduces to

$$(\mathcal{L}_X g)_{ij} = \partial_i X_j + \partial_j X_i.$$
Thus, in $\mathbb{R}^n$ the Lie derivative of the metric is simply the symmetric part of the Jacobian matrix of $X$. A vector field satisfies $\mathcal{L}_X g = 0$ if and only if

$$ \partial_i X_j + \partial_j X_i = 0, $$

which characterizes infinitesimal rigid motions (translations and rotations).

For a conformal Euclidean metric of the form $g(x) = e^{\sigma(x)} I_n$, the Lie derivative of the metric along a vector field $X$ is given by

$$ (\mathcal{L}_X g)_{ij} = \langle \nabla \sigma, X \rangle \delta_{ij} + \partial_i X_j + \partial_j X_i. $$

Let $\Phi_X^t$ denote the flow generated by $X$. For any $x \in \mathbb{R}^n$ and any $v \in T_x \mathbb{R}^n \simeq \mathbb{R}^n$,

$$ \left. \frac{d}{dt} \right|_{t=0} \| d\Phi_X^t(x) v \|^2 = \left. \frac{d}{dt} \right|_{t=0} g(d\Phi_X^t(x) v, d\Phi_X^t(x) v) = (\mathcal{L}_X g)_x(v, v). $$

Hence, if $\mathcal{L}_X g \simeq 0$, then for all $x$ and all $v$,

$$ \left. \frac{d}{dt} \right|_{t=0} \| d\Phi_X^t(x) v \|^2 \geq 0, $$

i.e., the squared norm of any infinitesimal displacement does not decrease to first order under the flow, which corresponds to local non-contraction of distances.

### A.4. Commuting Vector Fields

Two vector fields $X, Y \in \mathcal{X}(M)$ are said to commute if their Lie bracket vanishes:

$$ [X, Y] = 0. $$

This condition is equivalent to any of the following:

* the flows of $X$ and $Y$ commute locally;

* there exist local coordinates in which both appear as coordinate vector fields;

* for all $f \in C^\infty(M)$, $X(Yf) = Y(Xf)$.

### A.5. Arc-length

For each flow $\phi_i$ defined by $(F_i, T_i)$, let $z_i : [0, T_i(x)] \to \mathbb{R}^n$ denote the corresponding trajectory,

$$ \phi_i(x) = z_i(T_i(x), x), \quad \begin{cases} \frac{d}{dt} z_i(t, x) = F_i(z_i(t, x)), \\ z_i(0, x) = \phi_{i-1} \circ \dots \circ \phi_1(x). \end{cases} $$

We define the arc-length with the metric $G_i$ traveled along the $i$-th flow by

$$ \ell_i(x) = \int_0^{T_i(x)} \sqrt{\dot{z}_i(t)^\top G_i(z_i(t)) \dot{z}_i(t)} dt = \int_0^{T_i(x)} \sqrt{F_i(z_i(t))^\top G_i(z_i(t)) F_i(z_i(t))} dt. $$

The arc-lengths traveled along each flow, $(\ell_1(x), \dots, \ell_m(x))$, serve as the coordinates.

## B. Discussion on parallelizability

Parallelizability is a concept in differential geometry that characterizes the global structure of a manifold's tangent bundle. A smooth $m$-dimensional manifold $M$ is said to be *paralleizable* if its tangent bundle $TM$ is trivial, meaning that there
exist $m$ smooth vector fields that are everywhere linearly independent and form a global frame. Equivalently, parallelizability ensures the existence of a smoothly varying basis of the tangent space at every point of the manifold (Lee, 2013, Cor. 10.20, p. 259 and Example 10.10c).

Importantly, our approach does *not* assume strict global parallelizability as a topological property of the underlying data manifold. Instead, it is motivated by the geometric intuition provided by parallelizable manifolds and relies on the existence of smooth vector fields that span the tangent space on a large subset of the manifold. This distinction is crucial, as many manifolds of practical interest are not globally parallelizable, yet still admit well-defined tangent directions almost everywhere.

Results in differential topology show that for any smooth $n$-dimensional manifold, one can construct collections of smooth vector fields that are linearly independent outside a closed subset of codimension at least two. In particular, this exceptional set has measure zero with respect to any smooth volume form. As a consequence, the tangent bundle admits a smooth framing on an open dense subset of the manifold. These results follow from parametric transversality arguments applied to sections of the tangent bundle—specifically, from the transversality theorem ensuring that generic vector fields are transverse to the zero section (Guillemin & Pollack, 1974, Transversality Theorem, p68). This viewpoint is consistent with the local triviality of vector bundles and the theory of smooth frames described in modern treatments of smooth manifolds (Lee, 2013, Corollary 10.20, p259).

From the perspective of our method, this means that a global framing need not exist everywhere for the construction to be meaningful. Learning vector fields that form a frame almost everywhere is sufficient to recover intrinsic directions and define a coherent coordinate system on the data manifold, up to a negligible singular set. This relaxed viewpoint explains why the method remains applicable even when strict global parallelizability fails.
## C. Appendix - Proofs

**Definition C.1** (Tangent and normal decomposition). For a smooth manifold $M \subset \mathbb{R}^n$ and a vector field $F \in \mathcal{X}(\mathbb{R}^n)$, we write $T_x M$ is the tangent space of $M$ at $x$ and $N_x M = (T_x M)^\perp$ its normal space. The projection operator onto the tangent space is denoted by $P_{T_x M}$ and onto the normal space by $P_{N_x M}$.

**Definition C.2.** Let $M$ be a subset of $\mathbb{R}^m$. We denote by $\mathcal{R}_M^e$ the set of all Euclidean $m$-dimensional hyperrectangles whose $2^m$ vertices lie in $M$.

**Definition C.3** (Projected side lengths). Let $R$ be an $m$-dimensional hyperrectangle with vertex set $\mathcal{V}(R)$ and edge sets $\mathcal{E}_1(R), \dots, \mathcal{E}_m(R)$ corresponding to the coordinate directions. Let

$$ \mathcal{V}_{\phi}(R) := \{\phi(v) : v \in \mathcal{V}(R)\} \subset \phi(M) $$

be the projected vertices under the flow $\phi$.

For each $i \in \{1, \dots, m\}$, we define the $i$-th projected side length by

$$ \ell_i^{\phi}(R) := \min_{\{v,w\} \in \mathcal{E}_i(R)} \|\phi(v) - \phi(w)\|. $$

We order these as

$$ \ell_{(1)}^{\phi}(R) \le \ell_{(2)}^{\phi}(R) \le \dots \le \ell_{(m)}^{\phi}(R), $$

and set

$$ \mathrm{lc}_1^{\phi}(R) := \ell_{(1)}^{\phi}(R), \quad \mathrm{lc}_2^{\phi}(R) := \ell_{(2)}^{\phi}(R). $$

**Lemma C.1** (Two-direction non-collapse bound). Let $M \in \mathcal{M}_m$ with $m \ge 2$, and $\phi$ a flow, induced by pair $(F, T)$. Then there exists a constant $c > 0$, depending only on the geometry of $M$ and the admissible pairs, such that

$$ \max_{r \in \mathcal{R}_M^e} \mathrm{lc}_2^{\phi}(r) \ge c > 0. $$

*Proof. Step 1 — Equal-time distances are nondecreasing.* (A) *Infinitesimal expansion in the conformal metric.* Fix $z_0 \in U$ and set $z(t) := \Phi_t^F(z_0)$. Let $\delta(t)$ be a variational vector along $z(t)$, i.e.

$$ \dot{\delta}(t) = J_F(z(t)) \delta(t). $$

Define the conformal energy

$$ E(t) := \frac{1}{2} \|\delta(t)\|_{g(z(t))}^2 = \frac{1}{2} e^{\sigma(z(t))} \|\delta(t)\|^2. $$

Differentiating and using $\dot{z}(t) = F(z(t))$ gives

$$ \begin{aligned} E'(t) &= \frac{1}{2} e^{\sigma(z(t))} (F(z(t)) \cdot \nabla \sigma(z(t))) \|\delta(t)\|^2 + e^{\sigma(z(t))} \langle \delta(t), J_F(z(t)) \delta(t) \rangle \\ &= \frac{1}{2} e^{\sigma(z(t))} \delta(t)^\top \left( (F(z(t)) \cdot \nabla \sigma(z(t))) I + (J_F(z(t)) + J_F(z(t))^\top) \right) \delta(t) \ge 0. \end{aligned} $$

Hence

$$ t \mapsto \|\delta(t)\|_{g(z(t))} \text{ is nondecreasing.} $$

(B) *Consequence for equal-time distances.* Let $\gamma: [0, 1] \to U$ be any $C^1$ curve from $x$ to $y$ and define $\Gamma_t(s) := \Phi_t^F(\gamma(s))$. Then $\partial_s \Gamma_t(s) = D\Phi_t^F(\gamma(s)) \gamma'(s)$, and by (A),

$$ \|\partial_s \Gamma_t(s)\|_{g(\Gamma_t(s))} \ge \|\gamma'(s)\|_{g(\gamma(s))} \quad \forall s \in [0, 1]. $$

Integrating over $s$ yields the length expansion

$$ \mathrm{Length}_g(\Gamma_t) \ge \mathrm{Length}_g(\gamma). $$

Taking the infimum over all curves $\gamma$ joining $x$ to $y$ gives

$$ d_g(\Phi_t^F(x), \Phi_t^F(y)) \ge d_g(x, y), $$
so $t \mapsto d_g(\Phi_t^F(x), \Phi_t^F(y))$ is nondecreasing.

**Step 2** — Construct a local $(m-1)$-dimensional rectangle inside a level set. By standard dimension theory, every level set of a continuous map $M \to \mathbb{R}$ has topological dimension $m-1$, and therefore cannot be a single point. Hence the level set

$$S_{\tau} := T^{-1}(\tau)$$

contains at least the two distinct points $x$ and $y$ with $T(x) = T(y) = \tau$. Thus

$$\operatorname{diam}(S_{\tau}) \geq \|x - y\| > 0, \quad c_T := \sup_{\tau \in T(M)} \operatorname{diam}(S_{\tau}) > 0.$$

It is a geometric fact that every Euclidean ball in $\mathbb{R}^{m-1}$ contains an inscribed axis-aligned hypercube $R$ whose nodes lie in the manifold $M$.

**Step 3** — Propagation through the admissible flow. For any $x, y \in R$, we have $T(x) = T(y) = \tau$. Therefore, by the nondecrease of equal-time distances,

$$\|\phi(x) - \phi(y)\| = \|\Phi_{\tau}^{F}(x) - \Phi_{\tau}^{F}(y)\| \geq \|x - y\|.$$

Hence the image $\phi(R)$ is an $(m-1)$-dimensional subset of $\phi(M)$ with the same or larger side lengths.

**Step 4** — Conclusion. The projected vertex set $V_{\phi}(R)$, augmented with an additional zero-length coordinate is inside $\phi(M)$. In particular, the second smallest projected side length satisfies

$$\operatorname{lc}_{2}^{\phi}(R) > 0.$$

Therefore, there exists a constant $c > 0$, depending only on $M$ and on the local geometry of $(F, T)$, such that

$$\max_{r \in \mathbb{R}_M^e} \operatorname{lc}_2^\phi(r) \geq c > 0.$$

This completes the proof.

<figure><img src="image_7.png" /></figure>

**Lemma C.2** (Tangency Limit from Hyperrectangle Collapse). Let $M \subset \mathbb{R}^n$ be a smooth $m$-dimensional embedded manifold $(m \geq 1)$, and let $\phi \in \mathcal{F}$ be a flow defined by $(F, T)$. Denote by $\mathcal{R}_M^e$ the family of $m$-dimensional euclidian hyperrectangles with vertices contained in $M$. If

$$\max_{r \in \mathbb{R}_M^e} \operatorname{lc}_1^\phi(r) = 0,$$

then the vector field $F$ becomes asymptotically tangent to $M$ in the sense that

$$\max_{x \in M} \|P_{N_x M} F(x)\| = 0.$$

*Proof.* We argue by contraposition. Assume

$$\max_{x \in M} \|P_{N_x M} F(x)\| > 0.$$

Then there exists $x_0 \in M$ and $\varepsilon > 0$ such that

$$\|P_{N_{x_0} M} F(x_0)\| \geq \varepsilon.$$

We choose the value $\tau_0 := T(x_0)$ and select a local chart around $x_0$ of the form

$$\Theta : (y, s) \in U \subset \mathbb{R}^{m-1} \times \mathbb{R} \mapsto M, \quad T(\Theta(y, s)) = \tau_0 + s,$$
so that the level set $T^{-1}(\tau_0)$ corresponds to $\{s=0\}$ and the slices

$$S_s := \{\Theta(y, s) : y \in \mathbb{R}^{m-1}\}$$

are the nearby $(m-1)$-dimensional level sets of $T|_M$, and the vector field $F$ crosses these slices transversally. In particular, near $x_0$ the “time” direction $t$ has a uniform normal component of size at least $\epsilon$, while the $y$-directions are tangent to $M$ and tangent to each $S_t$.

In these coordinates, we focus on the central level set

$$S_0 := \{\Theta(y, 0) : y \in \mathbb{R}^{m-1}\}.$$

By Lemma C.1 applied on $S_0$ and the associated flow $\phi$ defined by $(F, T)$, there exists an $(m-1)$-dimensional rectangle $R \subset S_0$ and a constant $c_T > 0$ such that all projected tangential side lengths of $R$ satisfy

$$\ell_i^\phi(R) \ge c_T, \quad i = 1, \dots, m-1.$$

Moreover, since $\|P_{N_{x_0}M}F(x_0)\| \ge \epsilon$, the $s$-direction at $x_0$ is strictly transverse to $S_0$. Hence we can choose two points $x_0^\pm \in M$ on the normal line through $x_0$ with $s$-coordinates $\pm\sigma$ such that their images under $\phi$ satisfy

$$\|\phi(x_0^+) - \phi(x_0^-)\| \ge c_N$$

for some $c_N > 0$ depending only on $\epsilon$ and the local geometry of $(F, T)$. Combining $R$ with this normal segment yields an $m$-dimensional combinatorial hyperrectangle in $\phi(M)$ whose $m$ side lengths are all bounded below by $\min\{c_T, c_N\}$.

Let $c := \min\{c_T, c_N\} > 0$. Then all $m$ side lengths of $\phi(r)$ are at least $c$, so the smallest side length satisfies

$$\mathrm{lc}_1^\phi(r) \ge c.$$

In particular,

$$\max_{r \in \mathcal{R}_M^e} \mathrm{lc}_1^\phi(r) \ge c > 0,$$

which contradicts the assumption

$$\max_{r \in \mathcal{R}_M^e} \mathrm{lc}_1^\phi(r) = 0.$$

Therefore, the contraposition yields

$$\max_{x \in M} \|P_{N_x M} F(x)\| = 0,$$

i.e., $F$ is everywhere tangent to $M$.

□

### Proof of Theorem 3.1

*Proof. Step A (Achievability: $\mathcal{L}_m = 0$).* Because $M \in \mathcal{M}_m$ admits a global chart $\psi: M \to U \subset \mathbb{R}^m$ is a diffeomorphism, its differential $d\psi_p: T_pM \to T_{\psi(p)}U$ is a linear isomorphism for every $p \in M$ (Lee, 2013, Prop. 3.6 - p. 55). Pulling back the standard coordinate vector fields on $U$ therefore yields a smooth global frame on $M$, and hence the tangent bundle $TM$ is smoothly trivial (Lee, 2013, Cor. 10.20 - p. 259). For each $i \in \{1, \dots, m\}$, this pullback of the standard coordinate vector fields write

$$E_i := \psi^* \left( \frac{\partial}{\partial u^i} \right) \in \mathcal{X}(M).$$

Equivalently, $E_i$ is the unique smooth vector field on $M$ that is $\psi$-related to $\partial/\partial u^i$ in the sense of pushforward and pullback under a diffeomorphism (Lee, 2013, Corollary 3.22, p. 68). Each $E_i$ spans a rank-one smooth subbundle $L_i = \mathrm{span}\{E_i\} \subset TM$, i.e., a line bundle. By the Fundamental Theorem on Flows (Lee, 2013, Theorem 9.12, pp. 212–213)
and the Frobenius Theorem Lee, 2013, Theorem 19.12, p497, a smooth nowhere-vanishing vector field defines a family of nonintersecting integral curves whose images form a smooth one-dimensional foliation of M.

Moreover, compactness of M implies that every smooth vector field on M is complete (Lee, 2013, Corollary 9.17, p. 216). Consequently, each $E_i$ generates a globally defined smooth flow $\Phi_{E_i}^t: M \to M$ for all $t \in \mathbb{R}$. Fix a reference location $u_* \in U$ and denote $C := \psi^{-1}(u_*) \in M$ (and its image under the embedding into $\mathbb{R}^n$). Since each $E_i$ is smooth and complete, it generates a globally defined smooth flow $\Phi_{E_i}: \mathbb{R} \times M \to M$, which depends smoothly on both time and initial conditions (Lee, 2013, Thm. 9.12, p212-p213). Using the smoothness of the flow map $(t, x) \mapsto \Phi_{E_i}^t(x)$ and standard transversality arguments (see Guillemin & Pollack, 1974, Transversality Theorem, p 68), one may choose, for each $i$, a smooth codimension-one embedded submanifold $\Sigma_i \subset M$ that is transverse to $E_i$ and intersects each leaf of the $E_i$-foliation exactly once. Such transversals can be realized as regular level sets of smooth functions. By the Regular Level Set Theorem (Lee, 2013, Cor. 5.14, p. 106), the preimage of a regular value of a smooth map $f: M \to \mathbb{R}$ is a smooth embedded submanifold of codimension equal to the dimension of the codomain, here equal to one. Choosing $f$ so that $df(E_i) \neq 0$ along the level set yields a codimension-one submanifold $\Sigma_i$ transverse to $E_i$, meaning that $E_i$ is nowhere tangent to $\Sigma_i$.

The associated hitting-time function $T_i: M \to \mathbb{R}$, defined by the unique $t$ such that $\Phi_{E_i}^t(x) \in \Sigma_i$, is smooth, and the map

$$ \phi_i(x) := \Phi_{E_i}^{T_i(x)}(x) $$

projects $M$ along the integral curves of $L_i$ onto $\Sigma_i$. Iterating this construction produces a nested sequence

$$ M = \Sigma_0 \supset \Sigma_1 \supset \cdots \supset \Sigma_m = \{C\}, \quad \dim \Sigma_k = m-k, $$

so that one intrinsic dimension is eliminated at each step. After $m$ iterations, the manifold collapses to a single point, and hence

$$ \mathbb{E} \|C - (\phi_m \circ \cdots \circ \phi_1)(X)\|_2^2 = 0, $$

showing that the infimum defining $\mathcal{L}_m$ is achievable.

Finally, since $M$ is a smoothly embedded submanifold of $\mathbb{R}^n$, any tangent vector field on $M$ (in particular, each $E_i$) admits a smooth extension to an ambient vector field on $\mathbb{R}^n$ by the standard extension lemma for vector fields (Lee, 2013, Lemma 8.6, p. 177). Consequently, the intrinsic flow constructions on $M$ are compatible with the ambient-flow parametrization used in the definition of $\mathcal{F}$.

With this geometric achievability in place, the following paragraph addresses the remaining admissibility condition $\mathcal{L}_F g \succeq 0$.

Let the manifold be equipped with a Riemannian metric $g_0$, for instance the Euclidean metric inherited from the ambient space. For any smooth function $\sigma$, the conformal change $g = e^{2\sigma}g_0$ satisfies the standard identity

$$ \mathcal{L}_F g = \mathcal{L}_F(e^{2\sigma}g_0) = e^{2\sigma}(2F[\sigma]g_0 + \mathcal{L}_F g_0). \quad (8) $$

see (Lee, 2013, Corollary 9.39 - p 230). At a fixed point $x \in M$, write $\mathcal{L}_F g_0(x)$ as a $g_0(x)$-self-adjoint operator with minimal eigenvalue $\mu_{\min}(x) \in \mathbb{R}$. If we can find a smooth $\sigma$ such that

$$ F[\sigma](x) = a(x) \quad \text{with} \quad 2a(x) + \mu_{\min}(x) > 0 \quad \forall x \in M, \quad (9) $$

then from (8) we obtain, in a $g_0$-orthonormal basis,

$$ \lambda_{\min}(\mathcal{L}_F g(x)) = e^{2\sigma(x)} \min_i (2a(x) + \mu_i(x)) \geq e^{2\sigma(x)} (2a(x) + \mu_{\min}(x)) > 0, $$

hence $\mathcal{L}_F g(x) > 0$ pointwise. Thus the problem reduces to proving the existence of a smooth solution $\sigma$ of the transport equation $F[\sigma] = a$ with $a$ as in (9).

The equation

$$ F[\sigma] = \langle \nabla \sigma, F \rangle = a $$

is a first-order linear partial differential equation on $M$. Since the vector field $F$ is smooth, nowhere vanishing, it defines a smooth one-dimensional foliation of $M$ by its integral curves (Lee, 2013, Theorem 9.12, pp. 212–213). By the method
of characteristics, such an equation admits a global smooth solution $\sigma$ obtained by integrating $a$ along the flow lines of $F$. Indeed, along each integral curve $\gamma(t)$ of $F$, the equation reduces to the ordinary differential equation

$$ \frac{d}{dt} \sigma(\gamma(t)) = a(\gamma(t)), $$

which always admits a unique smooth solution when $F \neq 0$ everywhere and $F$ is bounded (hence globally Lipschitz), as guaranteed by the Picard–Lindelöf (Cauchy–Lipschitz) theorem (Teschl, 2012) on existence and uniqueness of solutions to ordinary differential equations.

**Step B (Vanishing loss implies tangency of the learned vector fields along the manifold.)** We aim to rule out a potential degeneracy, showing that the training procedure cannot converge to a spurious minimizer: achieving arbitrarily small loss without learning tangent directions. Specifically, we aim to prove that there does not exist a constant $\eta > 0$ such that for every $\varepsilon > 0$, there exist parameters $\{(F_i, T_i)\}_{i=1}^m$ and a constant $C$ for which the loss is below $\varepsilon$ while some generator has a uniformly non-negligible normal component along the manifold. Formally,

$$ \exists \eta > 0 \text{ s.t. } \forall \varepsilon > 0, \exists \{(F_i, T_i)\}_{i=1}^m, C \text{ with } \mathcal{L}(\{F_i, T_i\}, C) \le \varepsilon \text{ and } \max_{1 \le i \le m} \sup_{x \in M} |\langle F_i(x), n_M(x) \rangle| \ge \eta, $$

where $n_M(x)$ denotes a unit normal field on $M$. Ruling this out is crucial: otherwise one could drive the objective to (numerically) zero using flows that are not close to tangent, so low loss would not certify that the learned directions recover the tangent bundle.

We argue by contradiction. Fix $\eta > 0$ and assume that for every $k \in \mathbb{N}$ there exist parameters $\{(F_i^{(k)}, T_i^{(k)})\}_{i=1}^m$ and $C^{(k)}$ such that

$$ \mathcal{L}(\{F_i^{(k)}, T_i^{(k)}\}, C^{(k)}) \le \frac{1}{k} \quad \text{and} \quad \max_{1 \le i \le m} \sup_{x \in M} \|P_{N_x M} F_i^{(k)}(x)\| \ge \eta. \quad (10) $$

We define a subsequence such that $(F_i^{(k)}, T_i^{(k)}, C^{(k)}) \to (F_i^*, T_i^*, C^*)$ for each $i$. Let $\phi_i^{(k)}$ and $\phi_i^*$ be the corresponding flows. By continuity of the flow map and the uniform implication built into the loss,

$$ (\phi_m^* \circ \cdots \circ \phi_1^*)(x) \equiv C^* \quad \text{for all } x \in M. $$

Define the intermediate images $M^{(0)} := M$ and $M^{(j)} := \phi_j^* \circ \cdots \circ \phi_1^*(M)$ for $j = 1, \dots, m$.

**Step B-1:** Each flow can reduce tangent rank by at most one. Fix $j \in \{1, \dots, m\}$ and consider $\phi_j^*$ acting on $M^{(j-1)}$. Lemma C.1 gives a constant $c > 0$ (depending only on the geometry and admissible class) such that $\max_{r \in \mathcal{R}_{M^{(j-1)}}^e} \text{lc}_2^{\phi_j^*}(r) \ge c$. If $\text{rank}(D\phi_j^*|_{TM^{(j-1)}})$ dropped by at least 2 on $M^{(j-1)}$, then $\phi_j^*$ would collapse two independent tangential directions, forcing the second intrinsic side length of every mapped $m$-rectangle to vanish, i.e. $\text{lc}_2^{\phi_j^*}(r) = 0$ for all such $r$, contradicting Lemma C.1. Hence, each $\phi_j^*$ can decrease the tangential rank by at most one.

**Step B-2:** Exact and controlled one-dimensional drop at every step. Since $(\phi_m^* \circ \cdots \circ \phi_1^*)(M)$ is a single point, the tangential rank of the full composition drops from $m$ to 0, i.e. by a total of $m$. By Step 1, each individual flow $\phi_j^*$ can reduce the tangential rank of $M^{(j-1)}$ by at most one. Consequently, each of the $m$ steps must induce an exact one-dimensional rank drop.

Moreover, this rank reduction cannot occur in a degenerate or vanishing manner. Applying the Two-direction non-collapse bound (Lemma C.1) to $M^{(j-1)}$, there exists a constant $c > 0$, independent of $j$, such that the second intrinsic side length of any Euclidean hyperrectangle in $M^{(j-1)}$ remains uniformly bounded below after applying $\phi_j^*$. Therefore, collapsing exactly one tangential direction at step $j$ forces the shortest intrinsic side to vanish, while all remaining directions retain a positive extent. In particular, the collapse at step $j$ satisfies the quantitative condition

$$ \max_{r \in \mathcal{R}_{M^{(j-1)}}^e} \text{lc}_1^{\phi_j^*}(r) = 0 \quad \text{and} \quad \max_{r \in \mathcal{R}_{M^{(j-1)}}^e} \text{lc}_2^{\phi_j^*}(r) \ge c, $$

ensuring that the one-dimensional rank drop is well-separated and occurs at every step in the composition.
**Step B-3:** tangency enforced at each step. Fix $j$. Because $\phi_j^*$ reduces the tangential rank of $M^{(j-1)}$ by one everywhere, the image of any Euclidean $m$-hyperrectangle $r \in \mathcal{R}_{M^{(j-1)}}^e$ under $\phi_j^*$ is an at-most $(m-1)$-dimensional set, so its shortest intrinsic side is collapsed:

$$ \text{lc}_1^{\phi_j^*}(r) = 0 \quad \text{for all } r \in \mathcal{R}_{M^{(j-1)}}^e. $$

In particular, $\max_{r \in \mathcal{R}_{M^{(j-1)}}^e} \text{lc}_1^{\phi_j^*}(r) = 0$, and Lemma C.2 implies

$$ \sup_{x \in M^{(j-1)}} \| P_{N_x M^{(j-1)}} F_j^*(x) \| = 0, $$

i.e. $F_j^*$ is tangent along $M^{(j-1)}$. Pulling back through $\phi_{j-1}^* \circ \cdots \circ \phi_1^*$ yields

$$ \sup_{x \in M} \| P_{N_x M} F_j^*(x) \| = 0. $$

Since $j$ was arbitrary, this holds for all $j = 1, \dots, m$.

**Step B-4:** contradiction. By convergence $F_i^{(k)} \to F_i^*$ and continuity of $x \mapsto P_{N_x M}$ on the smooth embedded manifold $M$, we obtain

$$ \sup_{x \in M} \| P_{N_x M} F_i^{(k)}(x) \| \to \sup_{x \in M} \| P_{N_x M} F_i^*(x) \| = 0 \quad \text{for each } i. $$

Hence, for $k$ large enough, $\max_i \sup_{x \in M} \| P_{N_x M} F_i^{(k)}(x) \| < \eta$, contradicting (10). This proves the claim.

**Step C (Length coordinates are charts).** Define $\ell_j(x)$ as the signed arc-length traversed along the integral curve of $F_j$ during its time horizon (measured intrinsically on $M$) at the $j$-th stage, pulled back to $M$. On a domain where $TM = E_1 \oplus \cdots \oplus E_m$, the differential of

$$ x \mapsto (\ell_1(x), \dots, \ell_m(x)) $$

has columns the unit vectors spanning $E_1, \dots, E_m$; hence its Jacobian is invertible. By the inverse function theorem, $(\ell_1, \dots, \ell_m)$ form local coordinates.

This proves achievability $\mathcal{L}_m = 0$ and the three structural statements at any optimizer. $\square$

**Proof of Theorem 3.2**

*Proof.* Let $\phi_j$ be the flow defined by $(F_j, T_j)$, and set

$$ M^{(0)} := M, \quad M^{(j)} := \phi_j \circ \cdots \circ \phi_1(M) \quad (j = 1, \dots, k). $$

By Theorem C.1, for each $j$ there exists a constant $c_j > 0$ (depending only on the geometry of $M^{(j-1)}$ and admissible bounds) such that $M^{(j)}$ contains an intrinsic $m$-dimensional hyperrectangle whose second shortest side is at least $c_j$; in particular, a single admissible map cannot collapse two independent directions at once.

Iterating this $k$ times with $k < m$ implies that after $k$ compositions there remains at least $(m-k) \ge 1$ intrinsic directions with nontrivial extent. More precisely, there exists an intrinsic $(m-k)$-dimensional hyperrectangle

$$ R_k \subset M^{(k)} = \phi_k \circ \cdots \circ \phi_1(M) $$

whose second side length is bounded below by

$$ \text{lc}_2^{\phi_k}(R_k) \ge v := \min\{c_1, \dots, c_k\} > 0. $$
In particular, $\mathrm{diam}(M^{(k)}) \geq \mathrm{lc}_2^{\phi_k}(R_k) \geq v$.

Consequently, no point $C \in \mathbb{R}^n$ can be at zero distance from all of $M^{(k)}$; the optimal choice of $C$ in a minimax sense must incur a radius at least $\mathrm{lc}_2^{\phi_k}(R_k)/2$, hence

$$ \inf_{C \in \mathbb{R}^n} \sup_{y \in M^{(k)}} \|y - C\| \geq \frac{v}{2}. $$

In particular, the squared error cannot vanish:

$$ \inf_{\{(F_i, T_i)\}_{i=1}^k} \sup_{C \in \mathbb{R}^n} \sup_{x \in M} \|C - (\phi_k \circ \cdots \circ \phi_1)(x)\|^2 \geq \frac{v^2}{4} > 0. $$

If $X \sim \rho$ is supported on $M$, then $Y := (\phi_k \circ \cdots \circ \phi_1)(X)$ is supported on $M^{(k)}$, so $\min_C \mathbb{E}\|Y - C\|^2 \geq 0$ cannot be 0. (Indeed, since $\mathrm{diam}(M^{(k)}) \geq v$, at least two points of the support are $v$ apart, forcing a positive mean squared deviation for every $C$.) This proves that $\mathcal{L}_k \geq c$ for some $c > 0$ depending only on $M$ and the admissible bounds (e.g. one may take $c = v^2/4$ in the minimax sense above). $\square$

## Proof of Theorem 3.3

*Proof.* Define the composed flow

$$ S(x) := \phi_{T_m(x)}^{F_m} \circ \cdots \circ \phi_{T_1(x)}^{F_1}(x), $$

and the autonomous vector field

$$ G(x) := \sum_{i=1}^{m} T_i(x) F_i(x). $$

We prove that $S(x) = \phi_{\log_2 2}^G(x)$ and then the loss equivalence follows immediately.

**Step 1: A homotopy between the identity and $S$.** For $s \in [0, 1]$, define

$$ H(s, x) := \phi_{sT_m(x)}^{F_m} \circ \cdots \circ \phi_{sT_1(x)}^{F_1}(x). $$

Then $H(0, x) = x$ and $H(1, x) = S(x)$.

Because the fields commute, $[F_i, F_j] = 0$, their flows commute and preserve each other: for all $i, j, t$,

$$ (\phi_t^{F_j})_* F_i = F_i. $$

Moreover, for each $i$ and any point $z$,

$$ \frac{d}{ds} \phi_{sT_i(x)}^{F_i}(z) = T_i(x) F_i(\phi_{sT_i(x)}^{F_i}(z)). $$

Differentiating the $s$-dependent composition defining $H(s, x)$ and using the chain rule, the contribution of the $i$th factor is exactly $T_i(x) F_i(H(s, x))$, since flowing by the other $F_j$ does not change $F_i$ thanks to $(\phi_t^{F_j})_* F_i = F_i$. Summing over $i = 1, \dots, m$ yields

$$ \frac{d}{ds} H(s, x) = \sum_{i=1}^{m} T_i(x) F_i(H(s, x)). \qquad (11) $$
**Step 2: Evolution of $T_i$ along the curve.** By the chain rule,

$$ \frac{d}{ds} T_i(H(s, x)) = \langle \nabla T_i(H(s, x)), \frac{\partial H(s, x)}{\partial s} \rangle. $$

Insert (11),

$$ \frac{d}{ds} T_i(H(s, x)) = \sum_{j=1}^{m} T_j(x) \langle \nabla T_i(H(s, x)), F_j(H(s, x)) \rangle. $$

Using the unit Jacobian/Lie-derivative condition

$$ L_{F_j} T_i = \langle \nabla T_i, F_j \rangle = -\delta_{ij}, $$

we obtain the constant derivative

$$ \frac{d}{ds} T_i(H(s, x)) = \sum_{j=1}^{m} T_j(x) (-\delta_{ij}) = T_i(x). $$

Since $T_i(H(s, x)) = -T_i(x)$, integration yields the affine identity

$$ T_i(H(s, x)) = -T_i(x) + s T_i(x) = (-1 + s) T_i(x). \qquad (12) $$

**Step 3: Rewriting the ODE as an autonomous flow by time change.** Using (12), each coefficient in (11) satisfies

$$ T_j(x) = \frac{1}{-1 + s} T_j(H(s, x)). $$

Thus

$$ \frac{d}{ds} H(s, x) = \frac{1}{-1 + s} \sum_{j=1}^{m} T_j(H(s, x)) F_j(H(s, x)) = \frac{1}{-1 + s} G(H(s, x)). $$

Introduce the new time variable

$$ t = \int_{0}^{s} \frac{dr}{1+r} = \log(1+s), \quad \frac{dt}{ds} = \frac{1}{-1+s}, \quad s = e^t + 1, $$

and write $K(t, x) := H(s(t), x)$. By the chain rule,

$$ \frac{d}{dt} K(t, x) = \frac{d}{ds} H(s, x) \frac{ds}{dt} = \left( \frac{1}{-1+s} G(H(s, x)) \right) \cdot (-1+s) = G(K(t, x)). $$

Also $K(0, x) = H(0, x) = x$. Hence

$$ K(t, x) = \phi_G^t(x) \quad \Rightarrow \quad H(s, x) = \phi_G^{\log(-1+s)}(x). $$

Evaluating at $s = 1$ gives

$$ S(x) = H(1, x) = \phi_G^\infty(x) \quad \text{where} \quad G = \sum_{j=1}^{m} T_j F_j. $$

**Step 4: Loss equivalence.** For every $X$,

$$ (\phi_m \circ \dots \circ \phi_1)(X) = \phi_G^\infty(X), $$

so

$$ \|C - (\phi_m \circ \dots \circ \phi_1)(X)\|_2^2 = \|C - \phi_{\sum_{i=1}^m T_i F_i}^\infty(X)\|_2^2. $$

Taking expectations and minimizing over the same parameters $\{T_i, F_i\}$, $C$ yields the stated reformulation of $\mathcal{L}_m$.

□
## D. Appendix - Unknown manifold

In theory, by applying Theorem 3.2, one can determine the intrinsic dimension of the manifold by inspecting the behavior of the loss function. Specifically, the intrinsic dimension corresponds to the smallest number of vector fields (or flow layers) $k$ for which the minimum of the loss $\mathcal{L}_k$ vanishes, i.e., $\mathcal{L}_k = 0$. For all smaller values $k < m$, Theorem 3.2 guarantees that $\mathcal{L}_k \ge c > 0$. Hence, the intrinsic dimension can be identified as

$$m = \min\{k \in \mathbb{N} | \mathcal{L}_k = 0\},$$

which provides a direct and constructive criterion for selecting the minimal number of vector fields to learn.

## E. Appendix - Experiment setup

**Synthetic datasets.** For all synthetic datasets, we uniformly sample 10,000 points from the underlying manifold for training and evaluate the models on an additional 2,000 test points. For the hyperbolic paraboloid dataset, we consider the two-dimensional case $m = 2$, yielding a surface embedded in $\mathbb{R}^3$. The manifold is generated according to

$$X_3 = X_1^2 - X_2^2,$$

which defines a saddle-shaped geometry with one positive and one negative curvature direction.

**Enforcing positive semidefiniteness of the Lie derivative.** To enforce the positive semidefiniteness of the Lie derivative of the metric, we penalize negative eigenvalues of the associated symmetric matrix through a spectral penalty function. Directly computing or storing this matrix is expensive in high dimensions, as it would require materializing the full Jacobian of the vector field. Instead, we rely on matrix-free techniques based on Jacobian-vector products (JVPs) and vector-Jacobian products (VJPs), which can be computed efficiently via automatic differentiation.

Specifically, let $A$ denote the symmetric matrix representation of the Lie derivative operator. We apply a scalar penalty function $v$ to its spectrum and estimate the resulting trace $\text{tr} v(A)$ without explicitly forming $A$. This is achieved using a stochastic Hutchinson trace estimator (Hutchinson, 1990), which approximates the trace via random probe vectors, combined with a differentiable Lanczos approximation of the matrix function. The Lanczos procedure requires only matrix-vector products, which are implemented implicitly through JVPs and VJPs. Our implementation leverages the `matfree` library (Krämer et al., 2024), enabling scalable and differentiable enforcement of the positive semidefiniteness constraint even in high-dimensional ambient spaces.

For synthetic datasets, we employ a smooth approximation of the minimum eigenvalue of the Lie derivative, which can be evaluated using two stochastic trace estimators. Let $\Lambda = (\Lambda_1, \dots, \Lambda_n)$ denote the eigenvalues of the associated symmetric matrix $A$. We define the penalty

$$h(\Lambda, T(x)) = T(x)^2 \left( \left( - \frac{\sum_{j=1}^n \Lambda_j e^{-\Lambda_j}}{\sum_{j=1}^n e^{-\Lambda_j}} \right)_+ \right)^2,$$

which corresponds to a smooth, soft-min approximation of the most negative eigenvalue, scaled by the squared time horizon.

This formulation requires estimating two spectral quantities, corresponding to the functions

$$v_1(\lambda) = \lambda e^{-\lambda},$$

$$v_2(\lambda) = e^{-\lambda},$$

applied to the spectrum of $A$. Both traces can be computed efficiently using matrix-free stochastic trace estimation.

For the CIFAR-10 dataset, we instead adopt a simpler and more robust penalty based on a ReLU-type function applied directly to negative eigenvalues:

$$h(\Lambda, T(x)) = T(x)^2 \sum_{j=1}^{n} (-\Lambda_j)_+^2.$$
In this case, only a single spectral function is required,

$$v(\lambda) = (-\lambda)_+^2,$$

which directly penalizes negative eigenvalues and was found to be sufficient and numerically stable for high-dimensional real-world data.

**Linear manifold sanity check.** All models are trained by minimizing the loss defined in Eq. 3.2, augmented with the lie derivative metric regularization $\sum_k (-\min_i \lambda(JF_k + JF_k^T)_+)$. Trajectories are obtained by numerically integrating the learned dynamics using the Dormand-Prince method (Dopri8), with relative and absolute tolerances set to $rtol = 10^{-8}$ and $atol = 10^{-8}$, respectively.

The vector fields are parameterized by a multilayer perceptron (MLP) with four hidden layers of width 32 and tanh activation functions. The output layer is followed by a spherical projection with radius $10^{-2}$ in order to control the minimum magnitude of the vector field. The time scalar functions are separate MLP with the same hidden architecture. Positivity is enforced by applying a softplus activation at the output. All parameters are optimized jointly using the SOAP optimizer (Jia et al., 2019) with learning rate $10^{-4}$, momentum coefficients $(\beta_1, \beta_2) = (0.95, 0.95)$, weight decay $10^{-2}$, and a preconditioning update frequency of 5, over 50 epochs. The regularization term is weighted by a coefficient of 1.0. We employ a parameter initialization scheme specifically tailored for tanh activations, following woo Lee et al. (2025).

**Synthetic datasets.** All experiments are conducted using a fixed dataset of 10,000 points sampled on the manifold. Training is performed with a mini-batch size of 100 on 200 epochs. All results are reported over 5 random seeds.

The models, vector fields, times predictor and non-autonomous vector field, are implemented as multilayer perceptrons with four hidden layers of width 32 and the LipSwish activation function (Chen et al., 2019). All networks are initialized using Glorot normal initialization. In addition, the learnable target vector $C$ in $\mathbb{R}^d$ is initialized from a uniform distribution on $[-0.5, 0.5]^d$. Optimization is carried out using the splus optimizer (Frans et al., 2025) with parameters $\beta_1 = 0.9$, $\beta_2 = 0.95$, weight decay $10^{-3}$, and an inverse update applied every 100 steps. The learning rate follows a linear decay schedule from $10^{-2}$ to $10^{-4}$ over 100,000 steps. We apply the Lanczos algorithm with 3 steps and 8 Monte Carlo samples.

The isotropic autoencoder (AE) is trained using the following architecture. The encoder and the decoder are multilayer perceptrons (MLPs) with layer widths [32, 32, 32, 2] and [32, 32, 32, 32, 3], respectively. Both networks use the LipSwish activation function and are initialized using Glorot normal initialization. The training objective consists of a standard autoencoder reconstruction loss, augmented with an isotropy regularization term applied to the encoder, defined as

$$\eta \| J_e J_e^\top - I_n \|^2,$$

where $J_e$ denotes the Jacobian of the encoder, $I_n$ is the identity matrix, and $\eta = 1$. The model is trained for 200 epochs using the Adam optimizer with a learning rate of $10^{-3}$ and a batch size of 100.

**CIFAR-10 dataset.** Training is performed with a mini-batch size of 100 on 200 epochs. The vector field, and non-autonomous vector field are implemented using fully-convolutional networks with 3×3 kernels and the LipSwish activation function. The vector-field network uses five convolutional blocks with channel sizes (64, 64, 64, 64, $C \times M$) where $C$ is the number of image channels and $M$ is the number of models. The non-autonomous vector field takes as input the image concatenated with time (channel-wise) and uses five convolutional blocks with channel sizes (64, 64, 64, 64, $C$). The time predictor is a convolutional feature extractor with channels (32, 64, 128) followed by flattening and an MLP with hidden sizes (64, 32, 32) producing an output of size $M$. We apply the Lanczos algorithm with 16 steps and 8 Monte Carlo samples.

All convolutional and MLP layers are initialized using Glorot normal initialization. In addition, the learnable target tensor $C$ (with the same shape as the input image) is initialized from a uniform distribution on $[-0.5, 0.5]$. Optimization is carried out using the splus optimizer with parameters $\beta_1 = 0.9$, $\beta_2 = 0.95$, weight decay $10^{-3}$, and an inverse update applied every 100 steps, with a constant learning rate of $2 \times 10^{-3}$.

The isotropic autoencoder (AE) is trained using the following architecture. The encoder is defined as a sequential model composed of convolutional layers followed by fully connected layers. The convolutional part consists of three convolutional layers with feature sizes [32, 64, 128] and kernel size 3. Each convolution is followed by the LipSwish activation
function. The output of the last convolutional layer is flattened into a one-dimensional vector. All convolutional layers are initialized using Glorot normal initialization. The flattened representation is then passed through a multilayer perceptron with layer widths [64, 32, 32, 20]. Each hidden layer uses the LipSwish activation function, and all fully connected layers are initialized using Glorot normal initialization. The generator is defined as a sequential model composed of fully connected layers followed by transposed convolutional layers. It begins with a fully connected layer that maps the latent representation to a tensor of size 128 × 4 × 4. This tensor is reshaped and then processed by a sequence of transposed convolutional layers with feature sizes [128, 64, 32, 3] and kernel size 3. Each intermediate transposed convolutional layer uses the LipSwish activation function, while the final layer uses the identity activation. All layers are initialized using Glorot normal initialization. The isotropic regularization coefficient is $\eta = 0.1$. The model is trained for 200 epochs using the Adam optimizer with a learning rate of $10^{-3}$ and a batch size of 100.

The classifier head is implemented as a multilayer perceptron (MLP) with layer widths [32, 32, 10]. The hidden layers use the Leaky ReLU activation function, while the output layer uses the identity activation. All weights are initialized using orthogonal initialization. The classifier is trained with softmax cross-entropy loss using the Adam optimizer with a learning rate of $10^{-2}$ for 200 epochs.
F. Regularization Coefficients sensitivity analysis

These plots illustrate, on the synthetic spherical dataset, the influence of the regularization coefficients on the angular error to the normal (see Figure 6). We observe that performance is not highly sensitive to these regularization coefficients. An appropriate choice of the performance can slightly improve performance. However, when the regularization coefficients are insufficiently large, performance can be drastically degraded due to mode collapse, as theoretically predicted. NB: For a metric coefficient value of 10 − 5, *Not a Number* occur during execution. An interesting direction for future work is to refine the regularization scheme, which may lead to improved empirical performance. In particular, while time regularization degrades training performance on the sphere, it remains beneficial on the torus (see Figure 7).

<figure><img src="image_8.png" /><figcaption>*Figure 6. Regularization Sensitivity Analysis on the Spherical Dataset*</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 7. Time Regularization Sensitivity Analysis on the Torus Dataset</figcaption></figure>