Bi-Lipschitz Autoencoder With Injectivity Guarantee
Authors: Qipeng Zhan
Zhuoping Zhou
Zexuan Wang
Qi Long
Li Shen
Venue: ICLR 2026 Poster (2026)
Keywords: Autoencoder
Injectivity
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Autoencoders are widely used for dimensionality reduction, based on the assumption that high-dimensional data lies on low-dimensional manifolds. Regularized autoencoders aim to preserve manifold geometry during dimensionality reduction, but existing approaches often suffer from non-injective mappings and overly rigid constraints that limit their effectiveness and robustness. In this work, we identify encoder non-injectivity as a core bottleneck that leads to poor convergence and distorted latent representations. To ensure robustness across data distributions, we formalize the concept of admissible regularization and provide sufficient conditions for its satisfaction. In this work, we propose the Bi-Lipschitz Autoencoder (BLAE), which introduces two key innovations: (1) an injective regularization scheme based on a separation criterion to eliminate pathological local minima, and (2) a bi-Lipschitz relaxation that preserves geometry and exhibits robustness to data distribution drift.  Empirical results on diverse datasets show that BLAE consistently outperforms existing methods in preserving manifold structure while remaining resilient to sampling sparsity and distribution shifts.

Published as a conference paper at ICLR 2026
BI-LIPSCHITZ AUTOENCODER WITH INJECTIVITY GUARANTEE
Qipeng Zhan", ZZuoping Zhou', Zexuan Wang', Qi Qi Long', Li Shen University of Pennsylvania
(qi pqige,,, zhuopinz, zxwang) @sas upenn. edu (qlongê, 1i, www.shee@eeean peemeee cine. upenn.edu
"Equal contribution. CCoresponniig authors
ABSTRACT
Autoencoders are widely used for dimensionality reduction, based on the assump- tion that nigh-dimensional data lies on low wow-imensional manifolds. Regularized autoencoders aim to preserve manifold geometry during dimensionality reduction, but existing approaches often suffer from non-injective mappings and overly rigid constraints that limit their effectiveness and robustness. In this work, we identify encoder non-injectivity as core bottleneck that leads to poor convergence and distorted latent representations. To ensure robustness across data distributions, we formalize the concept of admissible regularization and provide sufficient condi- tions for its satisfaction. In this work, we propose the Bi- Bi-Lipschitz Autoencoder (BLAB), which introduces two key innovations: (1) an injective regularization scheme based on separation criterion to eliminate pathological local minima, and a bi-Lipschitz relaxation that preserves geometry and exhibits robustness to data distribution drift. Empirical results on diverse datasets show that BLAE consistently outperforms existing methods in preserving manifold structure while remaining resilient to sampling sparsity and distribution shifts. Code is available at bttps:://gitub. com/ http//ggpengggBBBeee
INTRODUCTION
Autoencoders have been established as powerful tools for dimensionality reduction and visualization.
While theoretically promising due to their universal approximation capabilities, vanilla autoencoders
often fail to preserve the geometric properties essential for meaningful latent representations This
limitation has motivated two main approaches regularize the latent space: 1) Gradient-based
methods (Nazari et al. 2023 Salah et al.] [011 Lim et al., 2024a; Lee et al. 2022) constrain the
Jacobian of the encoder decoder promote smoothness and local geometrie preservation. 2) Graph-based methods align latent representations with distance structures derived from neighborhood
graphs (Moor al.) 2020 Schönenberger et al. 2020) Singh & Nag 2021 Zhan et al. 2025 or pretrained embeddings (Mishne et al. 2019 Duque et al. 2022).
Despite their distinct theoretical foundations, both approaches face substantial practical challenges. Graph-based methods depend heavily on the graph accuracy, limiting their effectiveness under sparse sampling conditions. Gradient-based methods exhibit robustness to sample size and yield smoother manifolds but frequently converge to local minima during optimization, resulting in compromised performance that falls short of theoretical expectations.
From differential topology perspective, gradient constraints typically encode local geometric proper- ties. For example, a mapping f :M - I=" satisfying J J, is an isometric immersion. With an additional global injectivity condition, f becomes an embedding. This motivates us to investigate the interplay between injectivity and optimization in gradient-based autoencoders. Through theoretical and empirical analysis, we identify the non-injectivity of the encoder as a key bottleneck in training autoencoders effectively. To address this limitation, we propose novel framework that guarantees injectivity through separation criteria in the latent space, helping to avoid pathological local minima that can trap standard gradient-baa based methods and enabling them achieve their theoretical potential.

---

Published as a conference paper at ICLR 2026
To create geometrically consistent latent embedding that maintains intrinsic manifold structures, robustness against distributional shifts is essential because our goal isn't tied to specific data distribu- tions. This requires the imposed regularization to be admissible specifically, the targeted geometric properties must be strictly satisfied. While prior work employs isometric constraints (Lee et al. [022] Gropp al. 2020: Lim al. 2024a), such embeddings typically demand ((m²) ) dimensions, where m is the dimension of the manifold. This severely impairs the efficiency of the autoencoder. To address this, we propose a principled relaxation through bi-Lipschitz regularization, achieving linear complexity in manifold dimension while maintaining robustness to distribution shifts.
To validate our theoretical insights, we develop the Bi-Lipschitz Autoencoder (BLAE). which com- bines injective and bi-Lipschitz regularization. Our experiments across multiple datasets demonstrate that BLAE preserves manifold structure with higher fidelity than existing methods while exhibiting significant robustness to distribution shifts and sparse sampling conditions.
The rest of this paper is organized follows. Section S discusses the limitations of existing methods. 3 introduues our proposed framework. Section 4 revieww related work. Section presents Section
experimental results, and Section concludes the paper.
BOTTLENECK OF AUTOENCODERS
Throughout this work, we assume the intrinsic data manifold M C I is compact and connected Riemannian manifold with Riemannian measure μ.M unless otherwise stated. We also assume the data distribution P on M is equivalent to μ.M, i... P < μ.M and pm < P.
2.1 AUTOENCODERS AND REGULARIZATIONS
Conventional manifold learning and dimension reduction frameworks typically assume that high- dimensional data resides a dimenniiiio manifold M C R". An autoencoder learns this intrinsic representation through a pair of parameterized mappings
via neural networks:
The encoder Eg R > R" (n < m) compresses data onto low- oow-iiensiional latent space; The decoder Dø TR" + Th reconstructs the original data from the latent representation.
Training optimizes this encoder-decoder pair by minimizing the expected reconstruction error:
Lreson = Eg~p||> (E+(())))
While vanilla autoencoders effectively learn compressed representations, they often fail to preserve essential data properties. Advanced variants have emerged to address requirements such as repre- sentation smoothness, model robustness, and geometric preservation. Based on their regularization mechanisms, we classify these variants into three categories.
Gradient-regularrzztton This category applies explicit constraints on the derivatives of network mappings. Formally, gradient regularization is expressed as:
Lynd Ep R (z))
(2)
where Rí is loss function, and (x(x) represents the Jacobian matrix of differentiable function f input æ. The function f can be instantiated as either the encoder &g or the decoder Da- A canonical example is the contractive autoencoder (Salah al. 2011). where R₁ = || || imposes Frohenius-norm constraints on the encoder's. 's Jacobian to promote local stability.
Geoometry-regularizztinn This category preserves distance relationships when mapping from the manifold M to the latent space N. Geometry regularization takes the form:
Lgo = Ex,u= [R2 v), (x))
where Ra is a loss function and d, denotes the geodesic distance on the corresponding manifold. Implementation typically involves approximating geodesic distances d.M through k-nearest neigh- bors (k-NN) graph construction and thortest-path computations. The latent space metric dy is conventionally defined as the Euclidean distance || ||I
Embedding- regularizatiion The last category directly guides the encoder match embeddings produced by classical dimension reduction methods. The regularization is formulated as:
Lumb Enwel (l, - =
44)
2

---

Published as a conference paper at ICLR 2026
where is the target embedding of obtained from methods such as Isomap or diffusion maps.
Since both embedding and geometry regularizations rely on graph construction procedures standard in conventional dimensionality reduction methods, we unify them under the framework of Graph-Regularizatiin
Remark 1.
2.2
WHY AUTOENCODERS AND GRADIENT VARIANTS FAIL?
The universal approximation theorem grants neural network-based autoencoders significant potential for learning effective latent representations. However, in practice, standard gradient descent optimiza- tion frequently converges to poor local minima rather than discovering globally optimal solutions. This convergence Dehavior-wheteee terminating in suboptimal local minima or achieving global optimality is fundamentally connected the concept of injection.
Definition (Injection). f is an injection (injective mapping) on if + f(v), vx + M.
Non-injective encoders create latent space collisions where distinct data points map identical or nearly identical codes, resulting in inevitable reconstruction errors and suboptimal model perfor- mance" In practical implementations with finite sample sets (.1,... .N C M,, the point-wise injectivity condition Ea;); * Exyy) ) for all * j typically holds (wc.p.1) when the encoder & avoids local constancy. However, this does not guarantee global injectivity. More precisely, even when Ex;z, * (x,) for distinct sample points, if disjoint neighborhoods U;, Uj of these points have overlapping encodes (Eg(U.) (U) # Ø), the injectivity property is violated.
Figure I iillustrates this non non-injective bottleneck using a toy example of 20 points sampled from V-shaped manifold. We analyze three autoencoder configurations with two hidden layers of varying capacities (hidden dimensions: 2, 16, 256). The condition U; n U, Ø and (U,) O (U,)) & 2 manifests when the encoder maps points from distant manifold regions to nearby coordinates the latent space. Figures
(h) (i) (I) demonstrate this distortion between red and blue classes.
To compensate for these latent space collisions, the decoder must generate sharp variations, appearing as high local curvature, within intersection regions to minimize reconstruction error, as shown in [ (k). The required network complexity for these variations scales polynomially with the density of encoded data points. When the decoder's capacity cannot accommodate these geometric demands, the optimization process becomes trapped in suboptimal local minima where gradient signals align with the network's expressivity boundaries (Figure I (g)).
Figure
Notably, gradient-based regularization schemes suffer from similar injectivity bottleneck. While these methods effectively constrain local nappinggrroperties through differential constraints, they remain insufficient for ensuring global injectivity- critical topological property that distinguishes proper embeddings from mere immersions. In approaches like (Lim al., [024aa] Lee al. (022). isometric constraints yield locally structure-presery encoders via isometric immersion but fail to satisfy the additional topological requirements for genuine manifold embedding.
2.3 GRAPH REGULARIZZTION VS. GRADIENT REGULARIZZTTONSS
Unlike gradient-based approaches, graph-based autoencoder variants inherently enforce injective mapping by requiring distance preservation between manifolds M and latent space N, where d. > 0 = dw 0. However, these graph-driven approaches have two fundamental limitations: i) Their discrete graph approximations become unreliable under sparse sampling conditions, as shortest-path distances increasingly deviate from true manifold geodesics; ii) The Euclidean metric assumption in latent space introduces systematic errors unless A satisfies strict convexity require- ments. Comparatively, when the injectivity of the encoder is guaranteed, gradient-based methods can achieve smoother embeddings with greater robustness to variations in sample density.
Formally, the reconstruction loss calculated in expectation, injective guarantees need only hold a.c..
3

---

Published as a conference paper at ICLR 2026
Grund Trush Manilaid
Learaed Masifuld
Training Sample
Recatntrucied Sample
(6) 3-D ground truth minnnng samples
(4) 16)
(6) (hid 256)
(b) Ideal 1-D bacst reprosntation
(i) BLAE I-D latent ropes. (bid. din. 16)
(1) BLAE 1-D baient repea. (lid. dim. 256)
Learrad Manined
Recemntenctdd Samplo
Vanilla (hid,
Vanills
ropes, (hid, 2) Vanilla repra. (hid. 16)
ropes, (hid. 256)
Toy example demonstrating the non-injective encoder bottleneck. (a) 20 training points sampled from a V-shaped manifold with two classes (red and blue). (b) Ground truth: an ideal encoder separates the two classes in ID latent space. (c) (e) Reconstructed manifolds by BLAE (ours) with hidden dimensions 16 and 256, respectively. (d) (f) Corresponding latent representations learned by BLAE showing proper class separation. (g) (i) (k) Reconstructed manifolds by vanilla
Figure 1:
autoencoders with hidden dimensions 2, 16, and 256. (h) (j) (I) Corresponding latent representations
showing non-injective collapse, where distant manifold regions (red vs. blue) map overlapping latent codes, resulting in pathological local minima.
GRADIENT AUTOENCODER WITH INJECTIVE CONSTRAINT
3.1 SEPARATION CRITERION AND INJECTIVE REGULARIZATION
Our theoretical analysis in Section 2.2 yields two critical insights: i) Imposing injectivity constraints on the encoder can effectively eliminate pathological local minima that trap optimization trajectories. ii) Sample-level condition Eo (xx) * (2,)) ) is insufficient for global injectivity. To address these issues, we must prevent distant manifold neighborhoods from collapsing into proximal latent clusters, leading us enforce metric separation:
Definition ((ō, separrrtii,, Given 8, 0, mapping f :MM - > is (6, s)-sarrtte if for all x, E M satisfying (M y) 5:
dx(((x), >. dM(x,
(5)
This separation criterion provides a sufficient condition for f to be injective: if is (6, e)-separated for any e 0, then must be an injection. Indeed, under some mild assumptions, this condition serves as an equivalent characterization of injection:
Theorem Suppose f : M - N is continuous and M is compact, then f is injective if and only for any 8 0, there exists O such that f is (5, -separrated.
To address the limitations of naive injectivity constraints, we propose regularization that penalizes sample pairs violating the separation condition:
(Log cdv(x,y)
_ 1d
Cin = Ex,y
ReLU
(6)
dx(Eo(r))
All proofs deferred to Appendix
4

---

Published as a conference paper at ICLR 2026
However, this penalty permits trivial optimization path to zero loss: simply scaling the encoder by dx((),))))
a factor h c max (x((a(z)) To prevent this, we additionally constrain the encoder to be non-expansive, meaning Vz, y E M, d S dm(r,y) Our final regularization term combines both constraints:
( d dd)(E(()), Eo(o)) -1) ddn(s)sd
dM( (W
c) = Lin Ex.yorr ReLU
(7)
where a (default: 5) a weighting factor calibrating the strength of the non-expansive constraint.
Remark 2. While Theorem requires validating the separation condition for all 5 0, in practical implementations, it suffices to validate at threshold 6min = min (
Similar to graph-based approaches, we use Fuclidean distance for dy and approximate d.M through graph construction. Although this approximation introduces some systematic error discussed in Section 2.3 our separation criterion proves remarkably resilient such approximations compared to other geometry-basee regularizations. This robustness allows effective combination with gradient regularization techniques (see Appendix [..4for details).
The proposed injective regularization systematically mitigates pathological local minima in the loss landscape. Figure visualizes this effect through 2D loss landscapes comparing a standard autoencoder with its inggllive counterpart. Both models were initialized at O and optimized to 01 (vanilla autoencoder) and 02 (our regularized autoencoder), respectively. The contour maps represent loss values across the parameter subspace spanned by vectors 01 and 0g 02- Figure 2/a) reveals how the reconstruction loss landscape contains local minima that trap the vanilla model during optimization. In contrast, Figure illustrates how our combined loss (reconstruction plus injective regularization) reshapes the landscape to provide smoother optimization paths toward superior global minima.
Lond Moaro
Log Log-scaled reconstruction contour map
Log-scaled
regularization contour map
Figure 2: Loss landscapes of autoencoders on Swiss roll data. Warmer colors indicate lower loss. (a) Log-scale contour of reconstruction loss, showing the presence of local minima. (b) Log-scale contour of reconstruction loss combined with regularization, where local minima are eliminated.
3.2 ROBUSTNESS TO DISTRIBUTION SHIFT AND ADMISSIBLE REGULARIZATION
When training autoencoders, both reconstruction error and regularization terms are computed as expectations over specific data distribution IP on M. This typically causes the resulting latent space embedding A to influenced by P. However, our fundamental goal is to learn a low-dimensional embedding of the manifold structure itself, independent of any particular data sampling distribution. To achieve robustness against distribution shifts, we introduce the concept of Admissibility:
Definition (Admissibility). For a regularization term Ex-p [)-+[[((e((x)) where Rt is loss function and fe belongs parameterized smooth function class Fe (which may represent the encoder, decoder, or their Jacobians), let
Sp : arg min Ex=s(R((f(z))) JoEFa
(8)
The regularization is admissible if for any probability measures P and Q which are both equivalent to μ.M, Sp = Sq. i.e., the set global minima is independent of the probability measure.
The following theorem provides constructive approach for designing admissible regularizations:
5

---

Published as a conference paper at ICLR 2026
Theorem 2. Let R. be loss function that has a global minimum, and if
min foeJ [=-r[[((f(=])] = min R(u),
(9)
then the corresponding regularization Eg (R({(((((( is admissible.
Admissible regularizations typically enforce specific geometric functional properties uniformly across the manifold. Consider, for example, a regularizer of the form R(f(x)) ll(f(z)) f(z)
Al1?. where represents the Jacobian of the decoder. This formulation imposes isometric constraint, and achieving its minimum ensures the decoder maintains local isometry at each input point x. The admissibility of such regularization guarantees that this isometric property is preserved regardless of how data points are sampled from the manifold.
The standard reconstruction error can itself be viewed as a special case of admissible regularization, where R = || 2 and Fe {Do o Eg- id 0, with 'id' representing the identity mapping. This reconstruction term is inherently admissible.
Remark 3.
3.3 BI-LIPSCHITZ RELAXATION
The injectivity property enables the integration of complementary gradient regularizations into autoencoders, enhancing smoothness and geometric fidelity. For robustness against distribution shifts, we need these gradient regularizations to be admissible. Previous research has explored isometric constraints for geometric preservation, but these formulations are proven overly restrictive in practice.
According to the Nash embedding theorem (Nash, 1956), an k-dimensional compact Riemannian manifold requires a latent dimension of O(k2) to guarantee an isometric embedding This quadratic scaling contradicts the fundamental purpose of autoencoders as tools for efficient dimensionality reduction. For example, even a modest manifold intrinsic dimension 10 would theoretically require over 50 dimensions for isometric embedding, introducing substantial representational redundancy.
Furthermore, when the latent dimension fails to meet the requirements for isometric embedding, the corresponding regularization becomes non -on-aiiiss,,le, which motivates to introduce bi- bi-Lipschitz regularization, a principled relaxation scheme that balances geometric preservation with admissibility.
Definition (Bi-Lipschitz). A mapping f M + N xiBi--ipschhtz, where K 2 1, if
1 K
dm((, S dx(f(=), f()) S dm(z,y) Vz,y E M.
(10)
While geodesic distance approximation introduces estimation errors, we can establish the bi-Lipschitz property through differential analysis using the gradient of f. Let I'M denote I restricted to M, and (M(x)) be the Jacobian of f restricted to the tangent space T.M. We then have:
Theorem 3. Let fM + R" be smooth mapping. If M is connected and f is diffeomorphism, then f is Lipi--ips if and only if
1
S m (J, (J) S K, E
(11)
where a muin (J)M))) max (JM(2)) are the minimum and maximum singular values of (xx).
Theorem 3 provides a principled approach to rigorously analyze bi-Lipschitz properties without
requiring explicit geodesic distance computation. However, computing singular values of J.M (x) requires knowledge of the tangent space, and existing methods (Lim et al. 2024b) Zhang & Zha 2003) for estimating tangent spaces from empirical data introduce additional errors.
Two key insights help address these challenges: i) When f is bi-Lipschitz on RR" it is naturally bi-Lipschitz on any sub-manifold M. This allows substituting A with R.T in Theorem 3 ii When m > n (as in typical encoding scenarios), f cannot be diffeomorphism due to dimensional incompatibility. Therefore, we apply condition equation II to the decoder Dø R. - Rd rather than the encoder Eg T + R". since ff is ri-bi-Lisschizz if and only if f 1 is xiii--ips
'The required latent dimension n satisfies k(k+1) n 1(3++11)

---

Published as a conference paper at ICLR 2026
Our proposed bi-Lipschitz regularization is formulated as:
Loi-Lip(R) Eewp [eeLU K
(x))² -i (eLU (x) x))],
(12)
where min (1:) and F (1)) represent the smallest and largest singular values of Ja, respectively. This regularization retains admissibility even with relatively low embedding dimensions (O(m))
Theorem 4. Suppose M C _ is connected compact k-dimensional Riemannian manifold. Then there exists k-bi-Lipschitz mapping that embeds M into IR" for some and k S n S 2k.
When the manifold M admits k- k-bi-LLpsshitzz embedding in R", the corresponding loss function achieves its minimum (zero) and hence satisfies the assumption of Theorem 2 Consequently, the bi-Lipschitz regularization is admissible.
We refer to an autoencoder regularized by both bi-Lipschitz and injective constraints as Bi-Lipschitz Autoencoder (BLAE):
CBLAE = Croeo + Areg Creg + Abi-Lii Loi-Lip
(BLAE)
where Areg and Ani-Lip are weighting factors. The injective term eliminates local minima caused by non-injective encoders, while the bi-Lipschitz term ensures consistent geometric mapping regardless of data distribution. Together, these constraints enable BLAE to preserve manifold structure with higher fidelity and robustness than existing methods.
Remark 4. While Theorem assumes smoothness (satisfied by neural networks using activation functions like tanh, sigmoid, ELU), our framework naturally extends to continuous piecewise-smooff mappings. This preserves the theoretical conclusions even with non-smooth activation functions like ReLU,, demonstrating the universal applicability of our results across neural network architectures.
4 RELATED WORK
Standard autoencoders often exhibit geometric distortion in the latent space. To address this, regular- ized methods have been proposed to preserve geometric structure, which can be grouped into three main paradigms based on their regularization mechanisms:
baabeddiig--reulariize autoencoder. Conventional dimensionality reduction techniques such as ISOMAP (Tenenbaum et al. 2000). LLE (Roweis & Saull 2000), t-SNE (Van der Maaten & Hinton 2008), and UMAP (Meinnes et al. 2018) effectively preserve geometric structures through neighborhood graphs. However, these methods lack explicit mappings between the original and latent spaces, limiting their ability to generalize to new data points. To overcome this, Duque et al. (2022) introduced the Geometry Regularized Autoencoder, unified framework that integrates autoencoders with classical dimensionality reduction methods to enable robust extension to unseen data. Similarly, Diffusion Nets (Mishne et al. 2019) enhances autoencoders by learning embedding geometry from Diffusion Maps (Coifman & Lafon 2006) through additional eigenvector constraints.
reometry-regullriize autoencoder. This approach enforces geometric properties in the latent space via graph-based regularizations. Neighborhood Reconstructing Autoencoders (Lee et al. 202I) reduce overfitting and connectivity errors by enforcing correctly reconstructed neighborhoods. Structure- Preeturr-rreervingg Autoencoders (Singh & Nag 202I) maintain consistent distance ratios between ambient and latent spaces. Topological Autoencoders (Moor al., 2020) capture data's topological signature through homology groups. While initially developed for Euclidean spaces, these methods can adapted to non-Euclidean manifolds by constructing neighborhood graphs and computing geodesic distances through shortest paths.
Gradient-regullrizzed autoencoder.
This paradigm directly constrains on the derivatives of the network mappings. Contractive Autocncoder (Salah et al. 2011) penalizes the Frobenius norm of the encoder's Jacobian matrix, enforcing local stability and enhancing feature robustness. [Cenn et al. (2020) proposed regularizing the decoder's induced metric tensor learn flat manifold representations. Geometric Autoencoders (Nazari et al., 2023) preserve volume form in latent space by regularizing the determinant of the decoder's metric tensor, improving data visualization.
Recent advances focus on learning isometric embeddings: Gropp et al. (2020) enforces identity metric tensors stochastically, while lee al. (2022) achieves coordinate invariance through spectral
7

---

Published as a conference paper at ICLR 2026
constraints. Graph gradient-based approaches by leveraging graph Laplacian spectral properties to approximate the underlying Riemannian metric.
Autoencoders (Lim al. 2024a) bridge graph- based and
EXPERIMENTS
Experimental setup. We evaluate our approach against nine baselines: (1) geometry-basedr autoen-
coders (SPAE (Singh & Nag 2021). TAE (Moor et al. 2020 (2) gradient-based autoencoders
(IRAE Lee 2022), GAE Nazari et al. 2023). CAE (Salah al. 201T)) D. (3) embedding-
autoencoders (GRAE (Duque et 2002). Diffusion Net (DN) (Mishne et al. 2019)). (4) hybrid
approach combining graph and gradient regularization (GGAE (Lim al., 2024a)), and (5) vanilla
autoencoder. We conduct grid search over hyperparameters for each model-datasee combination and report the best performance. Detailed implementation settings are provided in Appendix [B.1
Evaluation metric. We assess model performance through both reconstruction accuracy and geo- metric preservation. Reconstruction fidelity is quantified by Mean Squared Error (MSE) between original samples and reconstructions, Geometric preservation is evaluated using two metrics: (I) k-NN recall (Sainburg et al. 2021 Kobak et al. 2019), measuring neighborhood correspondence between latent and original spaces, and (2) KL divergence (Chazal et al. (011)) with bandwidths a E {0.01, 13, assessing similarity of distance distributions across scales. To better evaluate manifold structures, adapt these metrics to use geodesic distances derived from similarity graphs rather than the original Euclidean formulations. See Appendix B.2 for details.
Table 1 reports the average ranks of all methods across metrics and datasets, providing compact overview of overall performance that complements the per-dataset comparisons. BLAE achieves the highest average ranking on key metries, reflecting superior performance in graph geometry preservation, reconstruction fidelity, and downstream task accuracy.
Table 1: Average ranks of evaluation metrics across all datasets (lower is better). Detailed metric values for each dataset are provided Appendix B.4 and Appendix C.6
The best shown in bold.
Measure
TAF
GRAF
GGAR
IRAF
Vanilla AE
KNNN
18 :: 1.3
3.2
3,8
1: 0.8
4.5
4 2.7
4.0
1 2.7
5.8 4
1.8
7.5
2.1
7.2
1 0.4
9.0
1,7
7.8 0.8
KLa.01
10
0.0
2,5
+
0.9
6.
4,5
+
1.5
7. 0.7
822
2.5
688
+ 24
6.5
1.1
9.2 2 0.4
Kl
10 0.0
3.2
2,8
1 0.8
5.5
22
3,5
0.
7.5
4 0.9
9.0
7.2
1 1.6
6.5
0.9
8.8
1 0.4
KL,
1.0
4.2 23
3.
5.2
4.2 19
70
8.0 1.6
722
1.3
6.0
22
4.2008.
MSE
0.
4.3
4.0
3.2
3.0
55
2.
7.0
8.0
+.
Accuracy
7
2
10
8
3
9
Swiss Roll. The Swiss Roll dataset consists of synthetie 2-dimensional manifold embedded in R33 We construct it by uniformly sampling points from 2, 10] x [0, 6] and isometrically mapping them to RR. To create meaningful contrast between Euclidean and geodesic distances, we remove strip [1.5, x [2.5, 3.5] from the data (see Appendix B.3 for details).
As shown in Figure 3 our BLAE method correctly preserves the geometric structure in latent space. Graph-based architectures (SPAE. TAE, GRAE, DN) successfully unroll the manifold but distort its geometry due to discrepancies between geodesic and Euclidean distances near the removed strip. All gradient-driver models without injectivity constraints fail to preserve the topological structure of the manifold, stemming from their non-injective encoders. Further analysis (Appendix C.I and C.2) reveals that gradient-based baselines exhibit strong dependence on the Swiss roll's geometry (curvature and axis length), while graph-based methods vary with sample size. contrast, BLAE consistently preserves topology across both geometric and population variations.
dSprites. The dSprites dataset (Matthey et al. 2017) serves as a benchmark for evaluating disen- tangled representations. It contains 64 X64 binary images of three geometric primitives (squares, ellipses, and hearts) generated through systematic variation of five factors: shape, color, orientation, scale, and position (x. y). In our experiments, fix color, scale, and orientation (white, 1, 0) and select squares and hearts with all possible positions except a cross-shaped region in the center.
We conduct supe-supe autoencoder training on this partiippati dataset, creating two distinct data clusters by adding a constant value to all pixels in square images. Figure 4 shows that only BLAE,, SPAF, TAE, CAE, and IRAE successfully reconstruct the topological structure
8

---

Published as a conference paper at ICLR 2026
Swisscal
BLAE
Dillaiot
GBAE (MIAAT)
Figure 3: (a) 3-D Swiss roll data. (b) Ground truth: 2-D) latent representations generate Swiss
Roll. Others: 3-D reconstruction and 2-D latent representations learned by AE methods.
of both clusters, with BLAE exhibiting the least geometric distortion between the parallel planes representing the two shape classes.
BLAK (ours)
SPAE
TAB
Dicfusion Net
GRAB (PHATE)
repeeszntation
CAE
GGAE
IRAE
GAE
Vanilla AE
Figure 4: (a) Two parallel planes: 3-D latent representation of square (blue) and heart (red) clusters. Others: 2-D latent representations learned by BLAE and other baseline methods.
MNIST. To evaluate robustness against distribution shift, train models using two different sampling distributions from the same underlying manifold. We generate these datasets using distinct rotation strategies 28 x 28 handwritten digit* '3":
Rotate the image in 1 increments over the range (30%) and uniformly sample 25% of these rotations the training set (Figure The remaining used the testing set. 2. Non-uniform: Apply 1 rotation steps within the ranges (0", 30%] ] U (180 210%), and 10" steps
1. Uniform:
within (30', 180%] (210%, 360"] to obtain the training set, as illustrated Figure 5(c). The
remaining rotation angles constitute the testing set.
Each rotated image is zoomed five scales (0.8x, 0.9x, 1.0x, 1.1x, 1.2x), generating 450 samples distributed across annular manifold (Figure 5b (c)). Figure 5 illustrates the results. Gradient-based models without injectivity constraints struggle capture the manifold topology. while graph-based methods demonstrate better performance. Notably, although Diffusion Net and TAE preserve topological structure their latent representations, only BLAE achieves consistent embedding struturre-formin concentric across both training distributions, demonstrating its invariance to sampling density variations.
For completeness, Appendix B.4 presents the numerical results for all experiments, Appendix B.5 analyzes the computational complexity, and Appendix C includes extended experiments such as sensitivity analysis, ablation studies, and downstream classification additional real-world datasets.
CONCLUSION
This work provides both theoretical and empirical foundations for understanding optimization bottlenecks in autoencoders. We demonstrate that the non-injective nature of standard encoders

---

Published as a conference paper at ICLR 2026
Renstton
Scaled Rotated MNIST desa
(6) Uniferm
Non-anifeem
BLAE (ours)
SPAE
TAB
Diftaion Net
GRAB (MHATE)
CAB
OGAE
IRAE
GAE
Vaiila AE
Figure 5: (a) Digit "3' at various scales and rotations. (b) (c) Ground truth: 2D concentric circle latent representation for uniform and non-uniform training sets. Others: 2D latent representations learned by BLAE and baseline methods on uniform (left) and non-uniform (right) training sets.
fundamentally induces local minima entrapment, explaining the suboptimal convergence observed in conventional formulations. To address this fundamental limitation, we introduce two key innovations. First, novel injective regularization framework based separation criteria that enforces topological consistency between input and latent spaces. Second, a bi-L.ischhitz geometric constraint that ensures admissible latent space construction even under aggressive dimensionality reduction. Our Bi-Lipschitz Autoencoder (BLAE) demonstrates state-of- satte-of-thh-arr the-art performance in geometric structure preservation across multiple datasets. Importantly, BLAE exhibits significantly enhanced robustness to distribution shifts and low-sample regimes compared to existing approaches, validating our theoretical analysis.
Although this paper primarily compares deterministic autoencoders for manifold learning, it is closely related to deep generative modeling. Many commonly used deep generative models struggle to learn distributions supported on unknown low-dimensional manifolds due to dimensionality mismatch and the phenomenon of manifold overfitting, where likelihood- Loaiza
models fail to capture the true distribution on the intrinsic manifold Loaiza- Ganem et al. (2024). One common strategy to introduce
manifold awareness is to adopt two-step generative architectures, which first perform generative modeling in a low-dimensional latent space and then map samples back to the data space. A variety of existing works Li al. (2015): Dao al. (2023) ;; Xiao al. (2019) provide empirical and theoretical evidence that training generative models in a learned latent representation improves performance compared to training directly in the ambient data space, and it is expected that generative models built on better latent representations achieve even stronger performance. In Appendix C.6.2, we integrate our BLAE framework with variational autoencoder (VAE), which improves performance compared to standard VAE, further validating that our method is effective for latent generative models as well.
One limitation training BLAE, common to graph-based methods, lies in the need to precompute the geodesic distance matrix, whose time and space complexities grow quadratically with the number of data points. As result, on larger-scale datasets (e.g., ImageNet), techniques are needed to mitigate this overhead. For example, one can construct the distance matrix only subset of points, and approximate the geodesic distance between any two points using the distance between their nearest neighbors within this subset.
ACKNOWLEDGMENTS
This work supported in part by NIH grants UOI AG068057, UOI AG066833, and RO1 EB037101 The content solely the responsibility the authors and does not necessarily represent the official views of the NIH.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Frédéric Chazal, David Cohen-Steinee and Quentin Mérigot. Geometric inference for probability measures. Foundations of Computational Mathematics, 11:733-751, 2011.
Nutan Chen, Alexej Klushyn, Francesco Ferroni, Justin Bayer, and Patrick Van Der Smagt. Learning flat latent manifolds with vaes. arXiv preprint 2rX2i:2002.04888 2020.
Ronald R Coifman and Stéphane Lafon. Diffusion maps. Applied and computational harmonic analysis, 21(1):5-30, 2006.
Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint 0xXi:23307.08698, 2023.
Andres Duque, Sacha Morin, Guy Wolf, and Kevin R Moon. Geometry regularized autoencoders. IEEE transactions pattern analysis and machine intelligence, 45(6):7381-7394, 2022.
Amos Gropp, Matan Atzmon, and Yaron Lipman.
Isometric autoencoders.
arXiv preprint
arXiv:2006.09288, 2020.
Dmitry Kobak, George Linderman, Stefan Steinerberger, Yuval Kluger, and Philipp Berens. Heavy- tailed kernels reveal finer cluster structure t-sne visualisations. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 124-139. Springer, 2019.
John M Lee. Smooth manifolds. In Introduction to smooth manifolds, pp. 1-29. Springer, 2003.
Yonghyeon Lee, Hyeokjun Kwon, and Frank Park. Neighborhood reconstructing autoencoders. Advances in Neural Information Processing Systems, 34:536-544 546, 2021.
Yonghyeon Lee, Sangwoong Yoon, MinJun Son, and Frank C Park. Regularized autoencoders for
isometric representation learning. International Conference on Learning Representations, 2022.
Yujia Li, Kevin Swersky, and Rich Zemel. Generative moment matching networks. International conference on machine learning, pp. 1718-1727. PMLR, 2015.
Jungbin Lim, Jihwan Kim, Yonghyeon Lee, Cheongjae Jang, and Frank C Park. Graph geometry- preserving autoencoders. In Forty-first International Conference on Machine Learning, 2024a.
Uzu Lim, Harald Oberhauser, and Vidit Nanda. Tangent space and dimension estimation with the wasserstein distance. SIAM Journal on Applied Algebra and Geometry, 8(3):650-66 685, 2024b.
Gabriel Banzz, Brendan Leigh Ross, Rasa Hosseinzadeh, Anthony Caterini, and Jesse C Cresswell. Deep generative models through the lens of the manifold hypothesis: A survey and new connections. arXiv preprint arXiv:2404.00 02954, 2024.
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement testing sprites dataset, 2017.
Leland McInnes. John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03422 2018.
Gal Mishne, Uri Shaham, Alexander Cloninger, and Israel Cohen. Diffusion nets. Applied and Computational Harmonic Analysis, 47(2)2259-285, 2019.
Michael Moor, Max Hor,, Bastian Rieck, and Karsten Borgwardt. Topological autoencoders. In International conference on machine learning, pp. 7045-7054. PMLR, 2020.
John Nash. The imbedding problem for riemannian manifolds. Annals of Mathematics, 63(1):20-63, 1956. ISSN 0003486X. UtLh:t www. http:///coooo/aaaa/////////3332
Philipp Nazari, Sebastian Damrich, and Fred A Hamprecht. Geometric autoencoders- you see is what you decode. arXiv preprint arXiv:2306.17638, 2023.
Sam Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally linear embedding. science, 290(5500) 2000.
11

---

Published as a conference paper at ICLR 2026
Tim Sainburg, Leland Melnnes, and Timothy Q Gentner. Parametric umap embeddings for represen- tation and semisupervised learning. Neural Computation, 33(11):2881-290) 2021.
Rifai Salah, P Vincent, X Muller, X Gloro, and Y Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In Proc. of the 28th International Conference Machine Learning, pp. 833-840, 2011.
Simon Till Schönenberger, Anastasiia Varava, Vladislav Polianskii, Jen Jen Chung, Danica Kragic, and Roland Siegwart. Witness autoencoder: Shaping the latent space with witness complexes. In TDA & Beyond, 2020.
Ayushman Singh and Kaustuv Nag. gruuctre-preseevvin deep autoencoder-based dimensionality reduction for data visualization. In 2021 IEEE/ACIS 22nd International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), pp- 43-48. IEEE, 2021.
Joshua B Tenenbaum, Vin de Silva, and John C Langford. A global geometric framework for nonlinear dimensionality reduction, science, 290(5500):2319 2000.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine
learning research, 9(1)), 2008.
Zhisheng Xiao, Qing Yan, and Yali Amit. Generative latent flow. arXiv preprint 1054v.990..10485. 2019.
Qipeng Zhan, Zhuoping Zhou, Zexuan Wang, and Li Shen. Multi-scale geometric autoencoder. arXiv preprint arXiv:2509.24166 2025.
Zhenyue Zhang and Hongyuan Zha. Nonlinear dimension reduction via local tangent space alignment. In International Conference on Intelligent Data Engineering and Automated Learning, pp. 477-481. Springer, 2003.
12

---

Published as a conference paper at ICLR 2026
A THEORETICAL PROOFS
A.1 PROOF OF THEOREM 1
Proof. (z Yy y E M, choose ā = (I, there exists > such that f is (6, -eparrtted
then
dx(((z), >.
(13)
dm(x, v)
Therefore, dy(f(x), f(v)) > c. d.((z,)) c é > 0, i.. f(x) + f(v). So f an injection. Note that the sufficiency does require any assumption.
(+) Because M is compact, M x M is compact as well. For any > 0, let
dm(x, v) 2 63. C. = (
(14)
The continuity and injectivity of f imply that dx(f(x)), f()))/d(zz y) is continuous and positive on Cs. Note that Cs is a closed subset of M x M, and hence also compact. Therefore, there exists (N.,V+) Cs such that
dx(((zz))
inf dx(f(z),{())
(15)
dy (2..).))))
(x.0)ECs
(2
Let E = -d)((((())) y.)), then f is (6, -espprrteed
A.2 PROOF OF THEBRREM 2
Proof. Because R has global minimum, we know that
Unin = arg min R(u) # Ø. u
(16)
Yuu Umin, we have R(f(x)) R(u,) 2 0 for any x E M. Let f be a minimizer of
min feEFs
Eger [R(fe()))]
(17)
then
0 0= Ezup (R((x))) R(u.) = BEAP [R(fe(x)) - R(u.)] 2 0.
(18)
This implies that R(f(=)) R(u,) 0 as..P, which means f(x) Umin
M, since IP is strictly positive. Therefore, fxx E Umin a.s. on M. On the other hand, if fxx) E Umin a.s. on M, then since IP is absolutely continuous, we have that (x) E Umin a.s.-P. Hence, is obvious that is a minimizer of equation 17 so
Sp = Lffe E Fe fe(x) Umin a.s. on M}.
(19)
Similarly, for another strictly positive and absolutely continuous probability measure Q
Se t/e E Fe fz( E Unin a.s. on M}.
(20)
i.e. Sp = Sq which proves the admissibility.
A.3 PROOF OF THEOREM 3
Before proving Theorem 3 we first introduce auxiliary lemma:
Lemma 1. Given the assumptions in Theorem 3 we have
(x)
mmax (J)M
VET_MMX(0)
max
|
(21)
J
Omin (J)M(:)) =
min (
(22)
||a|||
13

---

Published as a conference paper at ICLR 2026
Proof. Let k = M)), choose fei.-.. u& orthonormal basis of M.M. Then have the following (raisedep)nnden) representation:
(x) = (1
Jy()).
(23)
Note that (x) is an n x matrix and - n because flas is a diffeomorphism, so it has exactly singular values. We denote them 01 2 2 ok. By singular value decomposition
J (x) Paxn Enx Q
(24)
where P.Q are orthonormal matrices and E = diag(or, ... ,kk)- Then
V (z (x) Jj (x) (x) QET EQ_
(25)
So by eigenvalue decomposition,
u T Eu | 1 w QΣ JQ w
(J = max
MER*\
max WER* (10
|
(a(x) Ja(D)V
max WER (10)
(26)
T f (x))
max (ET
|
max
||1||2
i..
|
O max (J)
max VET.M
||a||
(27)
Similarly,
[x)
Tmin (J)(x)) =
min
(28)
|||||
Proof of Theorem 3 (=) Suppose is k-bi-Lipschitz, Vz E int M,, consider unit vector e E T.
Let y (-s,E) + M be a smooth curve with 7(0) = x and (0) = v. By the chain rule:
(f 7) (0) (z)e.
(29)
For [t] < e, the bi-Liischitz condition implics that
1
=d) S dx((((()) <x-x>((()))
(30)
K
Through dividing by |t| and taking + 0, we obtain:
U|l < (z)ell S <|lll| Yo E T.M.
(31)
For 0, dividing equation 31 by ||>||| and taking maximum (minimum) give:
||
[
min MT..M
S k.
(32)
|el|
WET_Mi{0
|
By lemma
< Jmin (minn(J (Jy(z)) S mmax (J,(z)) S n, Vr i..
(33)
By Weyl's inequality, equation 33 holds for all x E int M M.
14

---

Published as a conference paper at ICLR 2026
Let M and suppose y [0, - M smooth path from æ to y with ||'ll 0 (such path must exist because M path-connected), Then the image of y is path from f(z) to f(y).
(flggh()l = ! 1101 - (s)
[J (s)l (+)(s) TTM)
[ [s)(s)
[n'(,)
(34)
max vETiMiio
|ll|l
(s)llds
k Length(7)-
Because J'M is diffeomorphism, fMM exists and is smooth, hence for any smooth path 8: [0, - f(M) from f(x) to f3y). I geodesic distance,
e o 8 must be smooth path from x to So, by the definition of
dx(((z),, inf (e)
inf Length( flm B)
inf Length(flm 07)
(35)
x. inf Length(7)
dm(
Similarly, we can prove that dx-(f()), f()) 2 1/2 dm(xz)) which completes the proof.
A.4 PROOF OF THEOREM
Although this theorem can be proved just few lines using Proposition C.29 and the Inverse Function Theorem in (Lee 2003), for the sake of completeness we present here a more detailed proof.
Proof. Suppose / embeds M into Rt The Whitney's embedding theorem guarantees that such an embedding exists and < 2k. On the other hand, the embedding dimension obviously cannot be less than k. remains to show that f bi-Lipschitz. Since f an embedding, we know that the linear mapping (x) TyM + f is injective at every point M. Thereby, for any unit vector u E T,M,
| 0,
(36)
Because V == {o: UE T.M, |o|| 13 is compact, there must exist u. E such that
[z))l) min (z)|| (,,(()),)1 >
mmin (z)(
min (ET
(37)
|||||
wev
By Weyl's inequality (which implies the continuity of singular values of Jy), there exists a neighbor- hood Un of v, such that Vy E Un.
Omin (Jin( 2 miin ( (JMM
(38)
2
Tmax Omax( (J)(2)) < 20 max (J/M (x)).
(39)
Choose x(x) max 1200mam (J (x)), 2/0 2 (J+ (x)) it easy to see that x(x) 2 have:
1
(Jin (y)) (g)) S mmax (JM (1)) S x(x), Vy U..
(40)
k(x)
15

---

Published as a conference paper at ICLR 2026
Notice that {U2|2 x M is open cover of M, since M is compact, find finite sub-cover {U2,:''' Uux Choose = max(n((1); ". [K(NN)) then,
min mi - max (J) Ay M.
(41)
By Theorem 3 I s-bi-Lipschhiz.
B EXPERIMENTAL DETAILS
B.1 HYPERPARAMETER
We implemented sppeccc- autoencoders tailored to the structural characteristics of each dataset. For the Swiss Roll and ssREAD datasets, used fully connected autoencoder with two hidden layers of 256 units and ELU activations in both the encoder and decoder. The encoder projected the input-either 3D coordinates (Swiss Roll) or 50-dimensional PCA-reduced features (ssREAD) a 2-dimensional latent space, which the decoder then used to reconstruct the original input. For Swiss Roll, models were trained for 3000 epochs using the Adam optimizer with weight decay 1 10-5 and an initial learning rate of 2 x 10 3 reduced by a factor of 0.1 every 1000 epochs. For ssREAD, models were trained for 1500 epochs with Adam, weight decay x 10-5 and an initial learning rate of 10-3, reduced by factor of 0.1 every 500 epochs.
For the dSprites dataset, which detailed Appendix C.6) we adopted convolutional autoencoder based on the ConvNet64 and DeConvNet64 architectures. The encoder consisted of five convolutional layers with increasing channel widths- from nı to n, nn-followed by mapped the feature maps to 2D latent representation. The decoder mirrored this structure using transposed convolutions to reconstruct the original 64 x 64 binary image. Training was conducted for 1000 epochs using Adam with weight decay 1 10-5 and an initial learning rate of 1 10 10-3,
convolution that
For the MNIST dataset, we employed convolutional autoencoder optimized for 28 x 28 grayscale images. The encoder consisted of two convolutional layers with ReLU activations and max pooling, followed by fully connected layers that compressed the input into a 2D latent vector. The decoder performed the inverse, using fully connected layers and transposed convolutions to reconstruct the image. Models were trained for 3000 epochs with Adam, weight decay x 10-5, and an initial learning rate of 10 10-2, reduced by factor of 0.5 every 300 epochs.
All models were designed to produce 2-dimensional latent space to facilitate direct visualization and consistent comparison.
Table Hyperparameter settings for baselines across all evaluated datasets.
Dataset
Parameter
SPAE
TAF
DN
GRAE
CAE
GGAE
IRAE
GAE
À
2
5
100
100
0.1
0.01
0.01
Swiss Roll
yj
0.001
n_neighbor bandwidth
10
5
/
/
A
10 /
0.01
0.001
0.1
0.1
0.1
0.1
0.01
0.1
/
0
dSprites
n_neighbor bandwidth
1000
6
/
À
5
10
0.01
0.01
0.1
0.1
0.1
0.01
MNIST
T n_ n_neighbor bandwidth
/
50
5
15
/
/
0.01
A
10
100
0.01
0.01
0.01
1
/
0.1
/
0.2
s&READ
neighbor bandwidth
5
5
10
/
0.01
16

---

Published as a conference paper at ICLR 2026
The hyperparameters used for the baseline models each dataset are listed in Table These selected via grid search: À E (0.01, 0, 2, 5, 10, 100). 1/ E {0.001, 0,01, 0.1.0......... 0.2, 0.5. 1), number of neighbors {5, 6, 7, 8, 9, 10, 15, 50, 100, 1000), and GGAE bandwidth E {0.01, 0.1, 1, 2,5, 10).
In our method, A is treated as tunable hyperparameter. A simple approach to apply a standard grid search, but more efficient alternative to use binary search the trained model produces a nonzero Bi-Lipschitz loss (above a small and should be increased; if the
given interval (e.g.. [1,
Starting from the midpoint,
tolerance such as 10 10-4), this indicates that the current loss is effectively zero, the condition is satisfied and & can be accepted or further reduced. In practice, the guiding principle is to select the smallest g for which the Bi-Lipschitz loss remains below the threshold. Alternatively. if one wishes to enforce a specific distortion bound K between the latent representations and the input space, can be directly set K, thereby ensuring the constraint while maintaining robustness to distributional shift.
The hyperparameters used for our model (BLAE) are provided in Table
Table 3: Hyperparameter settings for BLAE across all evaluated datasets.
Datasets
Swiss Roll
dSprites
MNIST
sRRAAD
Xueg
1
2
30
2
Au Lip
0.3
0.1
0.1
0.1
1
1.1
2
1.2
0.3
0.3
0.6
0.6
B.2 EVALUATION METRICS
We evaluate the performance of each model using three metrics: mean squared error (MSE), A-NN recall (Sainburg et al. 202T Kobak al. 2019), and K L., divergence (Chazal al. 201T). where 0.01, 1. While these metrics are traditionally defined using Euclidean distance, we adapt them to better capture the underlying manifold structure. Specifically, we compute distances using geodesic metrics: d' on the data manifold and dv in the latent space. These geodesic distances are approximated by first constructing a neighborhood graph and then computing the shortest paths between node pairs.
B.2.1 N-NN RECALL
The NNNN recall metric quantifies how well the local neighborhood structure is preserved in the latent space. It measures the proportion of k-ncarest neighbors the data manifold that remain among the k-nearest neighbors in the latent space. For the Swiss Roll dataset, we report the average k-NN recall over h (5, 10....,.00, For MNIST and dSprites, we use f2,44.... f2,4....,110 10).
B.2.2 KL, DIVERGENCE
The K L metric computes the Kullback-Leible divergence between the normalized density estimates on the data manifold and the latent space. Let X = {.1....; ... ØN) denote the original data and Z (z) (,, their corresponding latent representations. The density at each point is defined by:
(x4) (x.) Σ, Ex,x(()))'
Pz.a(z1)= 12.0
(42)
S2,00
where the unnormalized densities computed as:
d (Di, 2gg)
fx.o(1s) = Σ exp
max,, d. (2 xzy)
(43)
fz,o(z)= = Σ exp
dx((2
maxx, dy (z)
The final metric is given by KL, = DKL. (px.. ll2..)) Smaller values emphasize local structure, while larger values reflect global geometry.
17

---

Published as a conference paper at ICLR 2026
B.3 GENERATION OF SWISS ROLL DATA
We use the logarithmic spiral r -= 60 (b + 0) to construct the Swiss Roll dataset. The are length of the spiral from angle 0, to 02 is given by:
02
s(02) s(01)
1 ds
01
(r² + (0)dd
6,
(44)
02
V 52d0
+ b2
(e.
he>
201)
Fixing the starting point at 01 and allowing the negative are length to be negative, we obtain the are length as function of #:
s(0) = vT
00 1),
(45)
b
which leads to the inverse function:
bs
((s) log(
1). v1 + b²
(46)
This yields an isometric parameterization the logarithmic spiral over the interval (- -1+ +00):
r(s) = 10(a)
(47)
To generate the Swiss Roll, we first uniformly sample points (s, E 2-2, 10] x 6]. then re- move points within the rectangular strip [1.5, 6.5) [2.5, 3.5). The remaining points embedded isometrically into R3 via:
(8, + (0))) cos(((s))) (0(() sin)s)
(48)
In Section 5
we set 0.1.
B.4 QUANTITATIVE RESULTS
In this section, present quantitative results, including mean squared error (MSE), k-nearest- neighbor (k-NN) recall, and KL divergence (KLg) under multiple bandwidths o {0.01, 0.1, 1}. for the Swiss Roll (Table 4). dSprites (Table 5). and MNIST datasets(Table 6 and Table 7). For dSprites, evaluation is performed specifically on the withheld cross-shaped regions to assess model generalization
-idtt--diittibttion samples. For each cluster, we compute MSE, k-NN recall, and K K and report the results as the mean + standard deviation over five independent runs, with the final score for each model given by the average across clusters. The best performance for each metric is highlighted in bold.
The Swiss Roll results show that BLAE attains the lowest reconstruction error and KL divergence across all bandwidths, together with the highest k-NN recall, indicating both accurate geometry recovery and strong neighborhood preservation. across metrics, with particularly large margins on K LL., demonstrating generalization to out-of- distribution regions. For uniform MNIST, BLAE remains competitive on all measures, leading in MSE, k-NN, and KI. at three bandwidths. On the more challenging non-uniform MNIST, although some basclines approach similar k-NN performance, BLAE secures the best results on three of the five metrics, and the embeddings in Figure 5 show markedly greater robustness to distributional shift.
dSprites, BLAE also achieves the best scores
B.5 COMPUTATIONAL COMPLEXITY
BLAE, along with other graph- based autoencoders, begins by constructing a neighborhood graph and computing pairwise shortest paths to approximate geodesic distances. This preprocessing step has
18

---

Published as a conference paper at ICLR 2026
Table Evaluation metries (mean + standard deviation over runs) on the Swiss Roll dataset. For MSE and KL metrics, lower values are better; for k-NN, higher values are better.
Model
(-NN(1)
Klla.01(4)
Kloa(()
Kl,())
MSR(()
BLAE
9.61c-01
3.59c-03
1.28c-04
2,95c-05
1.78c-05 + 5,98c-06
1.39c-06 = 1.02-066
9.M6e-05 =: 1,92c-05
SPAE
8.95e.01
290e-02
7.95e003
1.13e-02
5.15e-03 + 8.95e433
4.73e-04
8.81e44
2.10e-03 3.25e-03
TAE
8.14c-01
1.4c-02
4.28c-33
7:20--04
1.57c-03 + 6.38c-04
5.64c-05
1.72c405
6.94c-03
5.07c-03
DN
7.22e-01
1.58e-02
5.26e + 7.95e-03
1.32e-02 + 7.50e-03
7.43e-04 + 7.03e-04
1.02e011
1.73e-02
GRAB
5.70e-01
3.03e-02
4.09e-02 1.26e-02
9,94g-03 + 4.36e-03
8,34e-04
4.11e-04
7.72c-02
2.16c-02
CAF
6.14c-01
1.71e-02
6.11e-02 1.66e-02
4.10c-02 + 7.76e-03
2.31e-03
4.27e-04
5.37e-02
7.41e-03
GGAE
6.61c-01
= 1.22-011
4.04c-02 + 2.52e-02
1.80e-02 + 1.09e-02
1.18c-03 = 7,70c-04
7.28e-02
4.96c-02
IRAE
5.88c-01
12
2155e 2.15e-002
6.75e-02 8.34e-01
2.31e-03 4.82e-04 04
4.06e-02
2.08e-03
GAE
5.03c-01
3.21e-02
6.96e-02 + 3.56e-02
2.73e-02 + 1.28e-02
1,92e-03
+ 4.27c-04
4.80e-02
6.22e-04
Vanilla AB
5.18e-01
2.90e 02
2.00e 9.144-02
5.41e-02 +: 2.07e-02
2.05e-03
6.25e-04
0.14e-02
3.09e-03
Table 5: Evaluation metrics (mean + standard deviation over runs) on the dSprites dataset. For MSE and KL metrics, lower values are better; for k-NN, higher values are better.
Model
k-NN(1)
Kla.a1())
(l)
(L₁
MSB())
BLAE
7.39c-01 5.17-433
6.42e-03
9.5
3,98c-03 +: 1.84c-04
3.79c-05 = 1.74c-06
1.69c-02 =: 1.46e-03
SPAE
7.24c-01 = 4.60e-03
2.95e402
4.67e-04
9.1 + 5.36e-05
7.83e-05 = 4.92e007
1.72e-02 1.02e-03
TAE
5.58c-01
5.09c-02
3.15c-02
2.44e-02
1.46c-02 5.37e-03
1.25c-03 2.23c-03
2.84c-02
4.65c-03
DN
4.8le-11 4.59c-02
8.41e402
8.1cc02
4.45e-02
7.32c-02 + 5.16e-02
6.38c-03 = 5.67e-03
3.47e402
3,55e-03
GRAE
5,23e-01 = 1.99--22
2.9c-02
4.97c-04
9.10c-03 +: 5.59c-05
7.84e-05 = 4.726-07
3.06c-02
8.94c-03
CAF
6.79c.01 6.31e-02
3.99e-02
2.38e--02
2.05e-022 + 2.54e-02
1.79c.03 3.83e-03
2.59e-02
8.76e-03
GGAE
4.36c-01 = 1.00-01
1.85e-01
5.64c-00
2,95e-01 t 1.33e-01
2.72c-03
= 1.57e-03
5.19e-02 = 6.98c-03
4.89e-01
2255-022
3.51e-02
3.13e-02 1,98e-02
3.32e.03
1.84e-03
5.57e-02
3.09e-03
GAE
4.99c-01
1.12-001
3.73e-02
2,35e-02
1.79e-02 + 1.846-02
1.68c-03
= 3.57e-03
4.36c-02
8.35e-03
Vanilla AE
4.89c-01
6.01e-02
1.21e-01 7.22e-02
5,23e-02 + 3.44e
4.71e:03 5.16e-03
4.79e.02 4.05e-03
Table 6: Evaluation metrics (mean standard deviation over 5 runs) on the (Uniform) MNIST dataset. For MSE and KI. metrics, lower values are better; for k-NN, higher values are better.
Model
A-NN(t)
KLa.m()
Kl)
KL₃(4)
MSE())
BLAE
9.03e-01 9.38c-33
4.79e-02
+: 5,32e-03
4.01e-02 1: 1,39e-02
1.22e-02 = 6,28e-03
2.92e-03
1.53e-03
SPAE
8.59e-01
3.10e
7.50e-02
1.71e-02
6.35e-02 4 1.42e
1.58e-02
:: 7.63e-03
6.91e:03
2.29e
TAE
8.29c-01
2.68c-02
6.69c-02 4: 1.99--02
1.64e-02 8.08--03
4.93c-03
6.78c004
DN
8.
3.17e-02
1.33e-01 4.488--02
9.15e-02 1 3.60e-02
1.49e-02 1 6.72e
3.43e-03
: 9.977--00
GRAE
8.70c-01 = 5.67e-02
1.1e-011
5.63e-02
6.87e-02 :: 2.43e-02
1.4le-02 = 7.06c-03
3.51e-03 1.32c-03
CAE
7.69c-01 3.23e-02
2.58e-01
3.32e-002
1.62e-01 1: 4.66e-02
1.93e-02 8.74e-03
5.54e-03 8.10--04
GGAE
8.11e-01 = 4.09c-02
4.95e-11
+ 1.54c-01
1.95-001 4 4,04e-02
1.96c-02 = 8.65e-03
4.33e-03 = 1.69c-03
IRAE
7.82c-01 4.04c-02
1.04c-01
3,58e-02
9.41e-02 4: 2.13e-02
1.65-002
8.70c-03
6.17e-03 4.27e-04
GAE
7.03c-01
3.81e-02
133e-01
44.99-002
1.31e-01
: 4,31e-02
1.96e-02
9.24c-03
6.59e-03 1.27e-03
Vanilla AE
7.69c-01
6.32e-02
4.27c-01
1.65-001
1.76e-01 i 4.96e-02
1.85-02 8.32e-03
6.58c-03 4.79c-03
Table Evaluation metrics (mean + standard deviation over 5 runs) on the (non-uniform) MNIST dataset. For MSE and KI. metrics, lower values are better; for k-NN, higher values are better.
Model
k-NN(1)
Kl.a.m())
kl)
Kl,{()
MSR())
BLAE
8.65e-01 = 4.50c-02
2.89c-02
1.80-02
1.66-02 :: 2.60c-03
1.19c-03 = 7.91e-05
3.88c-03 # 7.87c-04
SPAE
8.22c-01
1.05e-01
5.59c-02
3.13e002
6.11e-02 4 4.39e-02
5.44e-03
5.09-03
7.78e-03
2.90e-03
TAE
8.77e-01
1.22c-02
3.35e-02
8.59--03
2.23c-02 + 1.01e-03
1.32c-03
1.66c-05
5.6le-03
1.88c-03
DN
X81e-00
6.48e-02
7.19c-02
+ 0.5
4.10e-012 + 1.74e-02
4.23e-03 + 2.03e-03
5.48e-03
1.65e-03
GRAB
9,14e-01 2.22e-02
1.06c-01
1.57e-02
4.84e-02 + 1.72e-02
5.23e-03 + 1.30e-03
3.08e-033
8.84c-04
CAF
7.58c-01 4.05e 02
1.4e-011
1.95e-02
1.12e-01 1: 5.71e-03
4.55e-03 # 2.4e 04
8.50e-03
1.33e 03
GGAE
7.15e-01 4.59e-22
2.63e-01
9.
1.52e-01 4 1.06e-01
8.92e-03 # 1.03e-02
9.75e-03
2.48c-03
IRAE
7.36c-01
7.25e-02
1.85e-02
7,93e-02 2.35e-02
5.17e-03 3.55e-03
8.75e-03
2.46e-03
GAE
6.82c-01
194c-01
1.48-011
+ 6.116e-02
9,22e-012
4 2.45e-02
4.13e-03
+ K.02e-04
1.45-022
1.90e-02
Vanilla AB
7.49c001
4.75e-02
2.76e-01 1.07e-01
1.48e-01 4: 3.72e-02
7.74e-03 : 3.39--03
9.70c-03
1.75e-03
19

---

Published as a conference paper at ICLR 2026
time complexity of On² log n). but it is performed only once and thus does not impact training efficiency. During training. each mini-batch only requires slicing the precomputed geodesic distance matrix to extract the relevant submatrix. For instance, in the Swiss Roll experiment, computing the full geodesic distance matrix took only 0.3 seconds.
Although BLAE integrates both graph- base- (injective) and gradient-basse (bi-Lipschitz) regular- ization mechanisms, its computational complexity does not significantly exceed that of employing either regularization approach in isolation. This efficiency stems from the fact that only small subset of samples activate the regularization terms during practical training. Specifically, for data pair (2r, z²) satisfying (z, z') /dd (z, z') E (c, 1) or data point where 1/1G < T (JD, < maax (JD, (a:) < K, the corresponding gradients of regularization vanish identically.
Table 8: Runtime for training BLAE and other baseline models. All experiments were conducted on a Mac Mini equipped with an Apple M4 chip (16GB RAM).
Model
BLAE
SPAE
TAE
DN
GRAB
CAE
GGAE
IRAB
GAE
Vanilla AE
Runtime (s)
170.9
162.1
601.3
204.4
147.3
199.6
415.8
189.3
207.6
136.2
Consequently, these inactive components are naturally excluded from backpropagation computations. To benchmark, we measured the runtime for training the Swiss Roll dataset in 3000 training steps with 1500 samples divided into three batches (batch size=500). Notably, considering the robustness of crudient-based regularization under limited sample sizes, we implemented partial sampling strategy where only 10% of data points within each training batch are utilized for computing the Jacobian matrix and the corresponding regularization. As shown in Table 8 BLAE exhibits comparable time complexity to other baselines.
C EXTENTED EXPERIMENTS
C.I SENSITIVITY ANALYSIS OF b AND LENGTHI
In this section, conduct sensitivity analysis the Swiss Roll dataset with respect to two key parameters: the spiral factor and the manifold length, both of which influence the geometric complexity of the data manifold.
BLAE
GGAE
CAE
IRAE
GAE
Vanilla
b=1.1
0=1.2
b=0.15
Figure 6: 2-D latent representations of Swiss Roll data learned by BLAE and gradient-hased based baselines trained with different values (0.1, 0.12, 0.15) on the Swiss Roll data. All models were trained using 1500 sample points, and all figures were plotted using the entire 10,000 sample points.
In logarithmic spirals, the curvature is inversely related to |b|: smaller values of [6] correspond to tighter coiling of the spiral. During experiments, observed that gradient-based autoencoders, such
20

---

Published as a conference paper at ICLR 2026
TAE, SPAE, and GGAE, more prone to converge to suboptimal local minima as |b| decreases. To assess this behavior systematically, we varied b while keeping all other parameters fixed. As shown in Figure 6 when b 0.1, the baseline models struggle to fully unfold the manifold structure. Interestingly, at = 0.12, GGAE shows earlier signs of escaping non-injective regimes, likely due to its hybrid design that combines gradient-hasee regularization with structural information from graph. Full unfolding of the spiral is achieved by all models once b increases to 0.15.
BLAB
GGAE
CAE
IRAE
GAE
Vanilla
[-2, 1o1
[1. 10]
Figure 7: 2-D latent representations learned of Swiss Roll data by BLAE and graph-based baselines trained with different lengths (I-2, 10], [1. 10]) on the Swiss Roll data. All models were trained using 1500 sample points, and all figures were plotted using the entire 10,000 sample points.
The manifold length is another important factor influencing unfolding quality. Empirical evidence suggests that gradient- based models tend to perform better on shorter Swiss Roll configurations. To evaluate this, fixed b 0.1 and generated two versions of the Swiss Roll: (1) a longer manifold defined over [-2, 2, 10] x [0, 6) [1.5, 6.5] x [2.5, 3.5]. and (2) shorter manifold over
[1, 10) [0, [3.5, 7.5] [2.5, 3.5). As shown in Figure 7 the shorter configuration leads to more
effective unfolding for most models, particularly those graph-based baseline methods.
C.2 SENSITIVITY ANALYSIS OF SAMPLE SIZE
BLAE
GGAE
SPAE
TAE
Diffusion Net
GRAE
size 300
Sizee 1000
size = 3000
Figure 8: 2-D latent representations learned by BLAE and graph-based baselines trained with different sample sizes (400, 1000, 3000) on the Swiss Roll data. All models were trained on the indicated sample sizes, while visualizations use the full set of 10,000 data points.
The performance of graph-based methods is highly sensitive to sample density, as the quality of the neighborhood graph- and hence the accuracy of geodesic distance estimation sriimatio--direelly directly depends on the number of training points. In this section, we systematically analyze how the training sample size affects the behavior of graph-based autoencoders.
21

---

Published as a conference paper at ICLR 2026
Table 9: Evaluation metrics (over 5 runs) of BLAE and graph-based baselines trained with different sample sizes (400, 1000, 3000) on the Swiss Roll data. For MSE and KL metrics, lower values are better; for k-NN, higher values are better. The best performance for each metric is shown in bold.
BLAB
GGAB
SPAE
Diflasion Net
GRAB
Sample size = 400
MSEL)
1.22-0511,07--0
9.696-0217.998
L86x-02.16.77c 03
5.39g 23966-03
L3Mc-0112..tee
1.80c00 33
E-NN(T)
9.19-01 1.310
4.55c-01 1:2 89c402
7. 7,30-401 + 38c-02
6.81e-01:4
1:4 18c-03
5.73c-01
1:3.5e-022
4,76p-01
-1666. 31e-33
KI Kl.amm())
2.225-04
1.14
2. 2.300-02:4444
3.01e-2241
52a-01
4.4e-42411
1.23e-22
1.01e-01
M6--0
3.550-04:3380--6
1.81-0241
84c-03
3.85c-03+9..6 6.4444
1.37--0343
3.33-0224400ee
2.29p.02=11 56e-04
(L)
1778:005:2.22e006
132
2.57e 94c
2.56e-0426 34
2.21e-05:7.78--00
2.30e-03--7 55
Runsime (5)
4.2.2-01 01
L.3c+0229.880 SSc-01
7.79c=01 10c+00
1.3e+02211.6
9.25=0111..77+00
7.55e+01a
Sumple size 1000
MSEL))
8.550-05-7.640-06
3.96e-12-11
18c-112
1.726-01+461e--0
2.42e-01+33.566
5.64e-01-11500--
A-NN(()
9.Nco01-22.66-01
4.876-01
9:35e.01 4-000-003
7.16g-01+2
5.03e-01 +4441ee002
6.25e-01 =33 242-02
KLe.osf.)
1.72-04:4.22c05
1.59-00
1.2e2-22
1.29g (13+ 95e 04
2.22e-03=7 23c 044
3.75e-02+
6.31e-02=22 97e-02
KLa
L14c-0513,96e00
1288-02233:3e-000
2.00c 100e-0418.880 05
2.22e-03::7223e
04
3.05e-024:470e
1,24 02---22-18.16 8.1688
KL,(1)
863e-07.3.322--
1/35-0319:01.-5
6.82e-0611.886000
2.22e-03117
23c-01
2.0tc-0311.556
1.00-03.16.55 5.55-6
Rumtine 64)
13--0221411
1774402.1.445-0000
1000-002.13370---
3.05p402.13.6860
1.18:=00214.39e-e
9.85+40 2:4 94499--01
Sumple size 3000
MSBL)
6.25-0511
1.06-025:111--22
1.29-00122.66---
5.72e-01:1.28:
1966-0111.001e:
3.166-03-13.B 15:1:
B-NN(C)
9.81-01 33.32e-03
6ttc-011:: 41556-412
9.53e-01 1-406cc-01
9.31c-01 14c-03
4.37c401 +:4
6.79p-01 40e-03
4.99c-04-.22500
8.66-012-77700
7.0e-04+1
1,964444
2.496-03422:55--0
207e-42+1
9.30c-02-45 89e-03
(()
1.75c-05=1.001c-66
2.70s-02:8.P5
78e-03
3.55--04=33890-
1.77-0221.76ee
8.81e-03--9 79e-04
(1)
L988-06211888 088c-07
1.31--33: 1.
04296-04:6.7 1e-05
Runtime
1.97::01--17775
N22e-422211 42o4000
2.16e403-1000sle
1944402+ 1.750000
3.37ex02 +.000400
1.57
0.370
3.00
3.00
0.30
3.50
6.30
10
0:30
10 12
28
10
2.0
160
04
07
(a) Effect on different metries
(b) Effect of e on different metrics
Figure 9: Performance evaluation across different hyperparameter settings. (a) shows the impact of varying on k-NN accuracy and error metrics. (b) demonstrates the effect of к on the same metrics. Error metrics are displayed on a logarithmic scale.
We generated 10,000 Swiss Roll samples using fixed parameters (b = 0.15, latent domain [-2, 10] x [0, 6)), and trained models using subsets of 400, 1000, and 3000 samples. To maintain consistency and isolate the effect of sample size, we avoided removing any subregions from the latent space, ensuring smooth geodesic-Euclinns
correspondence.
As shown in Figure g graph-based based baselines exhibit increasing distortion in the latent representa- tions as the sample size decreases, reflecting their reliance on high-resolution graphs for structural accuracy. In contrast, BLAA consistently preserves the underlying geometry across all sample sizes, demonstrating strong robustness to data sparsity and limited supervision.
C.3 HYPERPARAMETER SENSITIVITY ANALYSIS
To assess robustness to hyperparameter choices, we vary the bi-Lipschitz constant k and separation threshold e on Swiss Roll, keeping other settings fixed. Detailed discussion and ablation studies on individual regularization terms and latent space visualizations provided in Appendix C.4 and C.5
Bi-Lipschitz constant k. Figure 9a shows performance as k E {1.0, 1.1, 1.2, 1.5, 2.0, 5.0, 10.0). k-NN recall peaks at Æ = 1.2, demonstrating optimal balance between geometric preservation and flexibility. Overly strict constraints (N- = 1.0) enforce near-isomery, but slight relaxation maintains high fidelity while improving robustness. Excessive relaxation approaches unconstrained behavior and causes significant deterioration. Notably, MSE remains stable across all x, confirming that geometric preservation drives the performance trade-off. [1.0, 1.2) provides robust performance.
22

---

Published as a conference paper at ICLR 2026
Separation threshold Figure 9b shows results for {0.2, 0.3,......... 0.8). k-NN recall peaks small e values and declines gracefully as e increases. This behavior reflects two competing effects: smaller provides flexible separation that tolerates geodesic approximation enrors, while larger enforces increasingly strict separation constraints. As approaches 1, the constraint increasingly resembles rigid tistance-preserving requirements similar SPAE, which suffers from vulnerability to geodesic approximation errors and conflicts with bi-Lipschitz relaxation. KL divergence increases approximately one order of magnitude across this range. The optimal range E [0.2, 0.4] balances effective topological separation with robustness to distance estimation errors.
C.3.1 LATENT VISUALIZATIONS FOR HYPERPARAMETER SENSITIVITY
Bi-Lipschitz constant K. Figure 10 shows latent embeddings across different K. For small values (x E [1.0, 1.2)), the Swiss Roll structure is excellently preserved with smooth rectangular boundaries and uniform point density. At moderate values (ı E [1.5. 2.0)), the overall structure remains intact, but subtle irregularities begin to appear along the manifold boundaries. As x. increases beyond 2.0, the geometric distortion becomes progressively more pronounced: the rectangular boundaries lose regularity and the manifold exhibits significant warping with visible curvature in regions that should be flat. Notably, even at extreme values, the topological correctness is maintained the manifold remains properly unfolded without collapse. This visual degradation directly corresponds to the quantitative decline in k-NN recall shown in Figure Da The visualizations confirm that while the bi-Lipschitz constraint is admissible (allowing geometric relaxation), tighter bounds (x closer to 1) better preserve the intrinsic manifold geometry.
(a)
1.0
(b)
1.1
(c) 1.2 (d) 1.5
(e) 2.0
(I) 5.0
(g) 10
Figure 10: Sensitivity analysis of k: 2-D latent representation of Swiss Roll data learned by BLAE with different values.
Separation threshold Figure visualizes how latent structure evolves varies from 0.2 to 0.8. Low values (0.2-0.4) produce consistently high-quality embeddings that faithfully preserve the rectangular structure of the Swiss Roll's ground truth parameterization. The manifold boundaries smooth and regular, and the three embeddings are visually nearly indistinguishable. This stability confirms that performance is robust within this range. Moderate values (0.5-0.6) show the emergence of subtle geometric distortions. The rectangular boundaries become slightly less regular, though overall topology remains correct. High values (0.7-0.8) exhibit significant geometric irregularities. The rectangular structure is distorted, and local smoothness is compromised. However, critically, even 6 = 0.8, the manifold topology still preserved-no collapse occurs, and distant manifold regions remain separated. This graceful degradation aligns with the quantitative k-NN decline shown in Figure 9b] The visualizations confirm that smaller e values provide flexible separation that tolerates geodesic approximation errors, while larger values enforce increasingly rigid constraints. As c grows, the requirement (x((a(())
for points satisfying d. y) 2 becomes more restrictive, approaching strict distance preservation that is vulnerable to estimation errors, leading to the geometric drift visible at larger values.
dm(a)))
(a) 0.2
(b) 0.3
(c) 0.4
(d)
0.5
c=)= 0.6
(f) 0.7
(g) 0.8
Figure 11: Sensitivity analysis of e: 2-D latent representation of Swiss Roll data learned by BLAE with different values.
23

---

Published as a conference paper at ICLR 2026
C.4 COMPARISON OF INJECTIVE REGULARIZATION AND SPAE REGULARIZATIONS
Our injective regularization shares structural similarities with SPAE's regularization in their formal of geodesic distance ratios between the data manifold M and the latent space. .V. Both frameworks impose constraints involving dm/dy- However, their underlying philosophies differ significantly.
SPAE enforces strict constraint of dm//dx constant, relying on graph-based approximation d that introduces systematic bias under finite sampling. Additionally, SPAE substitutes dy with the Euclidean norm -I |12, which inherently underestimates geodesic
in non-convex latent spaces, where equality || l|2 = dy holds only in convex settings. For example, in the Swiss Roll dataset, points located opposite sides of the removed strip exhibit large geodesic distances, yet are deceptively close in Euclidean space.
In contrast, injective regularization selectively penalizes extreme distortions in the dm//dv ratio, and is inherently more tolerant to approximation errors. Rather than enforcing global constraint, it activates only when specific geometric violation is detected, enabling targeted regularization through appropriate gradient signals.
Notably, combining SPAE with gradient-based (bi-Lipschitz) regularization results in inferior perfor- mance. This degradation arises from the compounded effects of distance approximation errors and the rigid constraints enforced by SPAE.
C.5 ABLATION STUDIES
C.5.1 COMBINING INJECTIVE REGULARIZATION WITH GRADIENT-BASER METHODS
To demonstrate that injective regularization alleviate pathological local minima in existing gradient-based autoencoders, we combine our separation criterion with two representative gradient- based regularizers: CAE and GAE. We conduct experiments on the Swiss Roll dataset with 1,500 training samples and visualize using the complete 10,000-point dataset.
Figure 12 shows the results combining injective regularization with CAE and GAE. Panels (a)-(b) show that CAE and GAE alone struggle to properly unfold the Swiss Roll manifold, exhibiting non-injective collapse similar the vanilla autoencoder. However, when combined with our injective regularization (panels (c)-(d)), both methods successfully preserve the manifold topology and properly unfold the structure without collapse. While the unfolded representations show some geometric irregularities along the boundaries compared to the ideal rectangular structure, the critical topological property, preventing distant manifold regions from collapsing into nearby latent codes, is successfully enforced. This demonstrates that our injective regularization acts complementary constraint that helps gradient- bassen methods escape pathological local minima caused by non-injectivity. For optimal geometric preservation, including smooth boundaries, combining both injective and bi-Lipschitz regularization (as in full BLAE) is necessary, which we demonstrate throughout the next section.
CAR injective regularization
(d) GAB injective regularization
(4) CAE regularization
(b) GAB regularization
Figure 12: 2-D latent representation of Swiss Roll data learned by autoencoders with (a) only CAE regularization, (b) only GAE regularization, (c) CAE injective regularizations, (d) CAE + injective regularizations.
C.5.2 COMPLEMENTARY ROLES INJECTIVE AND BI-LIPSCHITZ REGULARIZATION
To validate that both regularization terms serve distinct and complementary purposes, we conduct ablation experiments the Swiss Roll dataset. Figure [3 presents 2-D latent representations learned under four different regularization schemes, all trained with 1500 samples and visualized using the complete 10,000-point dataset.
24

---

Published as a conference paper at ICLR 2026
(a) With A > and AU--LiP 0, the model successfully unfolds the Swiss Roll topology, conhrming that the separation criterion eliminates non-injective collapse. However, the embedding exhibits geometric irregularities and uneven point density. This demonstrates that injectivity alone necessary but insufficient for smooth geometric preservation.
Injective regularization only. (Fig. 13
SPAE regularization only. (Fig. 13 (b)) SPAE's preseeving constraint produces cleaner rectangular embedding with more uniform point distribution. However, as discussed Appendix CC44 SPAE's rigid constraint d /dy- constant vulnerable to geodesic distance approximation errors, especially near the removed strip region.
Injective bi-L bi-Lipschitz regularization (BLAE). (Fig. 13 (c)) Our full framework combines both terms with x 1, achieving: (1) topological correctness from the injective term, and (2) geometric smoothness from the bi-Lipschitz term. The embedding closely resembles the ground truth parameterization with a uniform point distribution and regular boundaries.
SPAE with bi-Lipschitz regularization. (Fig. 13 (d)) This combination yields inferior results with visible distortion and irregular geometry. This validates our analysis in Appendix C.4 SPAE's strict global constraint conflicts with bi-Lipschitz relaxation, and approximation errors compound. In contrast, our injective regularization only penalizes extreme violations, making it naturally compatible with bi-L.isschiz constraints.
Bi-L Bi-Lipschitz regularization only. (Fig. 13 (e) The model fails to unfold the Swiss Roll.
(6) Injective I-bi-Lipschitz regularizations
(4) SPAE 1-bi-Lipschitz regularizations
Injective regularization
regularization
(c) 1-bi-Lipsschite regularization
Figure 13: 2-D latent representation of Swiss Roll data learned by autoencoders with (a) only injective regularization, (b) only SPAE regularization, (c) injective 1-bi-Lipschitz regularizations, (d) SPAE + I-bi-LLipschitz regularizations, (e) only I-bi-L-pschitz regularization.
C.6 EXTRA DATASETS AND EXPERIMENTS
C.6.1 SSREAD DATABASE
BLAE (ours)
SPAE
TAE
Diffusion Net
GRAE (PHATE)
Cell type
CAE
GGAE
IRAE
GAE
Vanilla AE
Figure 14: Left: Single cell type in the AD00109 dataset from the ssREAD database. Others: 2-D latent representations learned by BLAE and other baseline methods.
We utilize the AD00109 dataset from the ssREAD database, which provides single-nucleus RNA sequencing (snRNA-sen -ee)) data from brain tissue of Alzheimer's disease (AD) patients. This dataset contains transcriptomic profiles from the prefrontal cortex of several female AD patients, aged between 77 and over 90. Sequencing was performed using the 10x Genomics Chromium platform.
25

---

Published as a conference paper at ICLR 2026
Standard preprocessing steps applied, including quality control, normalization, dimensionality reduction, and unsupervised clustering. The resulting dataset consists of 9,891 cells and 27,801 genes, annotated into seven distinct cell types.
For downstream analysis, we apply principal component analysis (PCA) and retain the top 50 principal components. We split the data by using 25% of the cells for training and the remaining 75% for evaluation. Figure [1 shows the 2D latent representations learned by BLAE and several baseline methods. Among these, BLAE, SPAE, TAE, and GAE better preserve the distinct boundaries between different cell types. To quantitatively assess representation quality, we train a Support Vector Machine (SVM) classifier on the latent representations (latent dimension 2, 32) of the training set and report classification accuracy on the evaluation set. Results are summarized in Table where BLAF achieves the highest aceuracy on both latent dimensions, indicating that learns the most discriminative latent representations for single-cell type identification.
Table 10: Classification aceuracy (over 5 runs) of SVM trained on latent representations learned by different models. BLAA achieves the highest accuracy, demonstrating better preservation of sell-fy information the latent space. The best performance is shown bold.
Model
Latent dimension =
Latent dimension 32
BLAE
0.9250 0.0033
0.9626 + 0.0013
SPAE
0.9146 0.0206
0.9576 = 0.0019
TAE
0.9107 + 0.0121
0.9524 = 0.0026
DN
0.9167 + 0.0064
0.4200 = 0.0821
GRAE
0.9131 + 0.0119
0.9476 + 0.0022
CAE
0.9222 + 0.0030
0.9517 + 0.0025
GGAE
0.8855 + 0.0161
0.9556 + 0.0017
IRAE
0.9066 = 0.0200
0.9605 = 0.0016
GAE
0.9185 = 0.0113
0.9575 = 0.0022
Vanilla AE
0.8996 = 0.0172
0.9557 = 0.0017
C.6.2 EXTENSION To VARIATIONAL AUTOENCODERS
To demonstrate that our regularization framework generalizes beyond deterministic autoencoders, we extend BLAE to variational autoencoders (VAEs), creating Bi-Lipschitz VAE (BL-VAE). This validates that our theoretical insights apply probabilistic latent variable models. We evaluate the Swiss Roll dataset from Section DeetionnSiiie with identical settings, applying our regularization (equation performance using four metrics: Reconstruction MSE
BLAE) the VAE decoder. We (fidelity), Fréchet Inception Distance (FID) (generation quality via feature distribution comparison), KL Divergence (posterior-prio divergence in VAE objective), and Mutual Information Gap (MIG) (disentanglement quality).
Table 11 shows BL-VAE achieves substantially lower reconstruction error (7.09e-2 1.42e- 1), confirming our regularization helps escape pathological local minima in probabilistic settings. FID scores are comparable. While BL-VAE exhibits higher KL divergence due to geometric constraints, achieves dramatically better disentanglement (MIG: 0.828 vs. 0.534), indicating the latent space better aligns with true underlying factors of variation. These results confirm our framework successfully extends to variational settings while preserving its core benefits.
Table 11: Swiss Roll Results (Mean Std over 5 seeds)
Model
Recon MSE
FID
KL
MIG 1
VAE
1.42c-1 + 3.65e-3
1.23e-2
2.32e-3
3.06c+0 + 1.33e-2
5.34c-1 + 1.96c-1
BL-VAE
7.09e-2 + 1.41e-3
1.15e-2 1.15e-3 5.58c+0 + 4.06e-2
8.28e-1 + 1.93e-2
C.6.3
TOY EXAMPLE WITH INCREASED SAMPLE DENSITY
To address concerns about whether non-injective embeddings reflect genuine inadequacy merely insufficient data, we extend the toy example from Figure by increasing the sample size from 20 to 200 points while maintaining the same V-shaped manifold structure.
26

---

Published as a conference paper at ICLR 2026
We sample 200 points uniformly from the V-shaped manifold and train both vanilla autoencoder and BLAE with identical architectures (hidden dimension = 16). Both models are trained under the same conditions to isolate the effect of injective regularization. As shown in Figure [5] even with 10x more training samples, the vanilla autoencoder (panels c-d) continues to exhibit non-injective collapse: the latent representation shows severe overlap between the red and blue classes, resulting in distorted reconstruction with an irregular curve. In contrast, BLAE (panels e-f) achieves proper class separation in the latent space and accurately reconstructs the V-shaped structure.
Truth Manifold
Manifold
Training Samples
Reconstructed Samples
(A) ground craining samples
(6)
16)
(hid. 16)
(b) Ideal I-D lateni reprosmtations
Vanilla AE (hid. dim BLAE I-D latent (hid. dim.
Figure 15: Extended toy example demonstrating the non-injective encoder bottleneck. (a) 200 training points sampled from a V-shaped manifold with two classes (red and blue). (b) Ground truth:
an ideal encoder separates the two classes in ID latent space. (c) Reconstructed manifolds by Vanilla AE with hidden dimension 16. (d) Corresponding latent representations learned by Vanilla AE. (c) Reconstructed manifolds by BLAE with hidden dimension 16. (f) Corresponding latent representations learned by BLAE show proper class separation.
D USAGE OF LARGE LANGUAGE MODELS (LLMS)
We used large language models as general-purpose writing assistant. Its role was limited to grammar checking, minor stylistic polishing, and improving the clarity of phrasing in some parts of the manuscript. The authors made all substantive contributions the research and writing.
27


Reviews:
[{"review_id": "iG4lNOXP7b", "reviewer": "Reviewer_JaoM", "summary": "This paper proposes Bi-Lipschitz Autoencoders (BLAE), which is an autoencoder with additional regularization terms to make it injective and robust to distribution shifts. The authors argue that non-injectivity is a bottleneck for autoencoders and leads to them being trapped in suboptimal local minima. By enforcing injectivity through regularizations, the learned manifold more closely matches the ground truth manifold of the data.", "questions": "- I may be mistaken, but could the proposed BLAE method be applied to the toy data shown in figure 1? If so, does it succeed in learning the true manifold? I was surprised to not see the proposed method included here, since this example helps justify the weakness of standard AEs and the need for the proposed BLAE.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The discussion on why AEs and gradient based methods tend to get trapped in local minima, and how even though gradient-based regularization methods maintain local properties they are not globally injective, is very interesting \n\n- The injectivity regularization loss term seems to be novel, and a well-justified and reasonable solution to the issues faced by AEs identified by the authors in section 2\n\n- The focus on making the admissible regularization, and making the global minima be independent of probab ility measure, is very interesting\n\n- The proposed bi-lipschitz regularization for admissibility is again novel and well-justified \n\nThe performance against the compared methods is good; the proposed approach consistently has the best quantitative performance, and qualitatively the learned embeddings appear to be much closer to the ground truth manifold", "weaknesses": "- The evaluation is done on simple toy datasets: Swiss Roll, dSprites, MNIST. I don't believe the paper *needs* to include results on larger benchmarks, but the lack of these datasets does lead one to question the practical utility of the proposed approach. \n\n- It would be nice to see a quantitative runtime comparison with other approaches", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "liVINYqI7Q", "reviewer": "Reviewer_4NEa", "summary": "In this paper, the authors emphasize the importance of constructing an injective mapping and propose two regularization techniques which, when combined, are claimed to improve the embedding. Both regularizations are based on distance constraints: one enforces that two separate points in the original space should be guaranteed to be separated in the embedding space, and the other imposes a margin (Hinge-type loss) on the pairwise distances that the embedding data must satisfy. In the experiments, the embeddings produced by the proposed method more closely resemble the ground truth or exhibit a perfectly circular shape.", "questions": "Can you provide even a simple theoretical guarantee that the proposed method can achieve?\n\nDoes the second regularization L_{bi-Lip} already encompass the first regularization L_{reg}? If so, please clarify why L_{reg} is needed in addition, and include the explanation on their distinct role.\n\nThe results shown in the experiments appear to be nearly trivial embeddings (essentially, copies of the original data). Is this interpretation correct? What would be the pragmatic utility of such embeddings? In general, trivial embeddings are not necessary in real-world applications. \n\nTypically, embeddings are designed to reduce dimensionality. Therefore, constructing injective mappings usually requires bijective mappings or diffeomorphism. Could you include a discussions on this relationship?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The authors’ discussion of the importance of injective embeddings is largely accurate and significant. Overall, the paper is well-written, and the experimental results on simple benchmark examples convincingly highlight the distinction between the proposed approach and other methods used for comparison.", "weaknesses": "The importance of injective mapping is emphasized in Section 2, and I enjoyed reading the discussion. However, the algorithm proposed in Section 3 is not innovative in achieving injective mappings; it appears to be a minor engineering modification of previously established methods, such as Isomap, Unfortunately, the proposed method does not directly address the construction of injective mappings compared with other well-known approaches. For example, Variational Autoencoders embed distributions that represent points to ensure separation in the latent space. Normalizing flows successfully learn bijective neural networks. It is difficult to identify any substantial or nontrivial derivations that would motivate the use of the proposed method.\n\nIn terms of discussion, while identifying injective mappings is indeed important, it should be noted that, since the neural network does not have access to the ground truth manifold in advance, we cannot conclude that the embeddings in Figure 1(e) or1 (g) are entirely incorrect when data are insufficient. Most of the embedding methods include hyperparameters that control manifold complexity, and with appropriate choice of hyperparameter, other methods could likely produce embeddings that are between the results in (c) and (e).\n\nThe authors proposed distance preservation in the latent space. However, under dimensionality reduction, distance preservation cannot generally be expected, even for successful and practically useful embeddings. Useful embeddings typically impose nontrivial mappings, and the authors does not acknowledge this point in their discussion and experiments. Overall, the paper falls below the level of acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nFR1cb0gik", "reviewer": "Reviewer_1zwJ", "summary": "The given paper addresses the issue that, when extracting latent dimensions through an auto-encoder, the mapping between certain samples and their latent representations is not one-to-one, leading to degraded overall reconstruction performance. The main contributions of the paper are as follows:\n1.\tA regularization loss is introduced to ensure that samples satisfying a specific separation criterion remain distinguishable from each other even after being embedded into the latent space.\n2.\tA Bi-Lipschitz-based loss term is additionally incorporated to preserve geometric information and achieve robust embedding performance under variations in the data distribution.", "questions": "1.\t I am wondering why the paper does not include experiments with a Variational Autoencoder (VAE). Was this decision made because the VAE inherently relies on probabilistic distributions to some extent?\n2.\tIn Table 1, the paper reports state-of-the-art (SOTA) performance. Could you clarify under what specific conditions these experiments were conducted?\n(For example: experiments were performed with random seeds ranging from 0 to 5, and the reported results represent the mean and standard deviation across these runs.)\n3.\tIn Figure 2, I would like to know what the x-axis and y-axis specifically represent.\n4.\tDo you have any plans to release the code on GitHub?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1.\tOriginality:\nThe paper is original in that it applies mathematical concepts - specifically the Bi-Lipschitz condition and the separation criterion - to the existing deep learning model of an auto-encoder. This approach preserves the geometric information of the original input, demonstrating a creative integration of mathematical theory into deep learning. Moreover, the work contributes to the development of interpretable deep learning models, which further enhances its originality.\n2.\tQuality:\nThe definitions and theoretical explanations are presented with precision and clarity, supported by detailed discussions in the Appendix, which reinforces the overall quality of the work.\n\n3.\tClarity:\nThis aspect is sufficiently addressed through the paper’s quality, as the theoretical framework and methodology are clearly explained.\n\n4.\tSignificance:\nTraditional auto-encoders have shown limitations in providing interpretable representations of the latent space. This paper overcomes such limitations by introducing a regularization loss term and a Bi-Lipschitz relaxation loss term, enabling analytical interpretation of the latent space. In doing so, it offers a new research direction beyond previous works that primarily focused on reconstruction performance or clustering in the latent space.", "weaknesses": "1.\tThe proposal of an interpretable latent embedding method is commendable. However, demonstrating that the model merely achieves “better latent embedding” in the experiments may not be sufficient to establish the paper’s novelty. It would strengthen the contribution if the authors could visually illustrate - through figures as well as tables - that improved latent embedding leads to better reconstruction of the original data, thereby providing more intuitive evidence of the method’s effectiveness.\n\n2.\tThe roles of the regularization loss and the Bi-Lipschitz loss term should be highlighted more clearly through experiments. To substantiate the claim that “the injective term eliminates local minima caused by non-injective encoders, while the bi-Lipschitz term ensures consistent geometric mapping regardless of data distribution,” it would be valuable to conduct ablation studies. Specifically, experiments should be performed with (i) the injective term coefficient set to zero, (ii) the Bi-Lipschitz term coefficient set to zero, and (iii) both terms included with appropriate regularization weights, in order to empirically demonstrate the distinct contribution of each term.\n\n3.\tI suspect that the Bi-Lipschitz Autoencoder might require longer training time compared to geometry-based or gradient-based autoencoders. Since the loss function involves the Jacobian and the separation criterion requires pairwise computations, the computational cost could increase significantly as the dataset grows. I am curious about the authors’ perspective on this issue and whether they have considered strategies to mitigate the potential increase in training time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5QDwL9czVH", "reviewer": "Reviewer_aWuy", "summary": "This paper proposes a novel autoencoder regularization method with an injectivity guarantee. The authors identify the non-injectivity of encoders as a bottleneck that causes existing gradient-based regularization methods to suffer from pathological local minima. To address this issue, the paper introduces a separation criterion and an injective regularizer. Furthermore, the notion of admissibility of a regularizer is proposed as a condition for distributional robustness, which motivates the use of a Bi-Lipschitz regularization. Empirical results demonstrate that the Bi-Lipschitz autoencoder captures the manifold structure of data more effectively than several state-of-the-art methods.", "questions": "-  Can the authors provide an example where the proposed injectivity regularizer is combined with existing gradient-based regularizers to alleviate their pathological local minima? Such experiments would considerably strengthen the paper’s contribution.\n-  What are the detailed outcomes of the minima shown in Figure 2? Visualizing the corresponding latent representations, as in Figure 3, would help in interpretation.\n-  How does the method behave as the latent dimensionality increases? Since the regularizer primarily enforces local and global topology preservation, it might lead to overexpansion of latent space volume. It would be informative to compare with methods that constrain latent volume, such as:\n\n    Chen, Qiuyi, and Mark Fuge. Compressing Latent Space via Least Volume. ICLR 2024.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper presents an original and interesting regularization framework for autoencoders, designed to preserve the topological structure of the data manifold.\n- The issues with existing regularization methods are clearly analyzed, providing a natural motivation for introducing the injective regularization.\n- The proposed framework is evaluated through comprehensive empirical comparisons.\n-  The proposed regularizer is relatively simple to implement, which can be advantageous for practical applications.\n-  The paper is generally well-written.", "weaknesses": "- The **practical relevance of the admissibility condition** remains unclear. In realistic settings, $\\mathbb{P}$ and $\\mathbb{Q}$ in Eq. (8) would correspond to empirical measures supported only on training samples. Under such circumstances, it is doubtful whether their equivalence with $\\mu_\\mathcal{M}$ (the data manifold measure) can hold. It is also unclear how admissibility is concretely related to distributional robustness. In particular, the reconstruction error (claimed to be admissible) is not robust outside the training region, as illustrated in the toy example in Figure 1.\n- The **motivation for the Bi-Lipschitz regularizer** is not sufficiently persuasive. Based on the current exposition (e.g., L274–L276), it appears to be one among many possible regularizers that satisfy the admissibility condition. The specific reason for selecting the Bi-Lipschitz form should be better justified, perhaps by emphasizing its relation to local topology preservation.\n- The method introduces **many hyperparameters**, but the paper lacks ablation studies. In particular, varying $\\epsilon$ and $\\kappa$ could have significant effects, and removing or changing relative weights of the injectivity or Bi-Lipschitz components would likely alter results considerably. Without such analyses, it is difficult to grasp the sensitivity and interpretability of the proposed regularization.\n\n**Minor comments**\n-  Figure 1(h) contains an error.\n-  The notation $m$ is used both for manifold and ambient space dimensions, which reduces readability.\n-  The correct reference for LTSA should be:\n\n    Zhang, Zhenyue, and Hongyuan Zha (2004). Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment. SIAM Journal on Scientific Computing, 26(1): 313–338.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)