Isometric Regularization for Manifolds of Functional Data
Authors: Hyeongjun Heo
Seonghun Oh
Jae Yong Lee
Young Min Kim
Yonghyeon Lee
Venue: ICLR 2025 Poster (2025)
Keywords: Isometric regularization
Geometric reularization
Implicit Neural Representation
Manifold Learning
Neural SDF
Neural BRDF
Neural Operator
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We present isometric regularization for manifolds of functional data, leading to robust data representation learning.

Abstract: While conventional data are represented as discrete vectors, Implicit Neural Representations (INRs) utilize neural networks to represent data points as continuous functions. By incorporating a shared network that maps latent vectors to individual functions, one can model the distribution of functional data, which has proven effective in many applications, such as learning 3D shapes, surface reflectance, and operators.
However, the infinite-dimensional nature of these representations makes them prone to overfitting, necessitating sufficient regularization. Naïve regularization methods -- those commonly used with discrete vector representations -- may enforce smoothness to increase robustness but result in a loss of data fidelity due to improper handling of function coordinates. 
To overcome these challenges, we start by interpreting the mapping from latent variables to INRs as a parametrization of a Riemannian manifold. We then recognize that preserving geometric quantities -- such as distances and angles -- between the latent space and the data manifold is crucial. As a result, we obtain a manifold with minimal intrinsic curvature, leading to robust representations while maintaining high-quality data fitting. Our experiments on various data modalities demonstrate that our method effectively discovers a well-structured latent space, leading to robust data representations even for challenging datasets, such as those that are small or noisy.

Published as a conference paper at ICLR 2025
ISOMETRIC REGULARIZATION FOR MANIFOLDS OF FUNCTIONAL DATA
Hyeongjun Heo', Seonghun Oh", Jac Yong Lee', Young Min Kim**, Yonghyeon Lee
Seoul National University Yonsei University
SChung-Ang University
4 Korrea Institute for Advanced Study
(heo0224, youngmin. sendell@yonsed ac.kr ac ylee@kias.re. re.kr
ac. kr
ABSTRACT
While conventional data are represented as discrete vectors, Implicit Neural Rep- resentations (INRs) utilize neural networks to represent data points continuous functions. By incorporating shared network that maps latent vectors individ- ual functions, one can model the distribution of functional data, which has proven effective many applications, such as learning 3D shapes, surface reflectance, and operators. However, the infiniteddimensional nature of these representations makes them prone to overfitting, necessitating sufficient regularization. Naïve regularization methods those commonly used with discrete vector representa- tions - may enforce smoothness to increase robustness but result loss of data fidelity due improper handling of function coordinates. To overcome these challenges, we start by interpreting the mapping from latent variables to INRs as parametrization of Riemannian manifold. We then recognize that preserving geometric quantities - such as distances and angles - between the latent space and the data manifold is crucial. As result, we obtain manifold with minimal in- trinsic curvature, leading to robust representations while maintaining high-quality data fitting. Our experiments on various data modalities demonstrate that our method effectively discovers well-structured latent space, leading robust data representations even for challenging datasets, such those that are small or noisy.
INTRODUCTION
Implicit Neural Representations (INRs), often referred to Neural Fields, functional representations of data points typically expressed as fe X' + V with net- work parameters 0, where X' is input space such spatial or temporal coordinates and is an output vec- tor space (Xie et al. 2022). One significant advantage of this representation is that, since individual data points continuous functions, one can sample values of V an arbitrary resolution from X. Moreover, given multiple instances of functions, has recently been shown that, in- stead of learning separate network weights for each data instance, shared neural network can constructed by conditioning on an auxiliary latent variable 2:
4
2
F
X
V
(c.
2)
(w أوس
Figure 1: Illustrations of manifolds of functional data.
F
X
x -
where, for an instance of a latent vector E F. F(.2) corresponds to data point of functional representation. This latent variable model has been widely applied to tasks such as data
compression, generation, prediction, and solving differential equations, across various data types,
including images (Sitzmann et al. 2020: Tancik et al., 2020). 3D shapes (Park et al. 2019), tex-
tures (Fan et al. 2022), and functions for differential equations (Lu et al. 2019 2021
*Corresponding authors.
Project wehsite: 1oo0024..itthb.io//oeo projectpage

---

Published as a conference paper at ICLR 2025
To achieve strong performance in downstream tasks such interpolation and reconstruction, the la- tent variable model F must satisfy two essential criteria: (i) it should accurately fit the data instances, resulting in high-fidet fidelit reconstructions, and (ii) the latent space Z should be well-behaved, ensur- ing that small changes in the latent space lead to gradual and predictable changes in the output. With- out proper regularization, particularly when the training dataset is small noisy, simply fitting the data instances can cause the latent space to become ill- ill-behaved, leading to poor representations and weaker performance in downstream tasks. Recently, Lipschitz regularization (Liu et al. (002)) pro- poses to learn a well-behaved latent space with a continuous function by promoting smoothness in F. However, they regularize across both input domains, X' and Z, and may result in over wver-smoohinng of data instances in X. failing to meet the aforementioned two criteria simultaneously.
In this paper, we propose balanced regularization method that produces a well-hehaved latent space 2 without overly smoothing the data in X. Our approach begins with geometric interpretation of the latent variable model F as a Riemannian manifold of functional data, where the latent variable corresponds to the coordinates of the manifold, as depicted in Figure I Based on this interpretation, we regularize F in the z component to produce a smooth and parsimonious manifold - resulting in small intrinsic curvatures (Lee& & Park 2023) and well-behaved latent spaces while leaving the component unregularized, thus preserving the fidelity of individual data instances.
Specifically. we adopt the recent isometric regularization method developed for mnite-dimensional data (Lee et al.] 2022b), which preserves geometric quantities such as distances and angles between the latent space and the manifold, and extend it to infnite-diimensioall functional data. While regu-
larizing the manifold of discretized data on a grid, like images, has been widely studied (Chen et al.,
2020
Lee al. 2021 2022b Jang al., 2023 Lee al., 2022a] Nazari al. 2023 Lee & Park 2023 applying these methods to latent variable INRs is challenging due to variability in sample locations and numbers, making consistent regularization difficult across different data instances. To address this, we propose a seaeetiizaaionnaanostt approach to isometric regularization for F. One notable difference from the finite-dimensional cases is that this approach involves integrating with respect to a positive measure in X, which requires significant computation. To make this practi- cal, we develop an approximate yet efficient algorithm, using methods such as Hutchinson's 's trace estimator and Monte Carlo approximation.
Through extensive experiments, we validate the effectiveness of our isometric regularization for functional data various tasks with INRs including neural Signed Distance Functions (SDFs) (Park)
jt al. 2019), neural Bidirectional Reflectance Distribution Functions (BRDFs) (Fan et al. 2022). and Deep Operator Networks (DONets) (Lu et al. 2019) showing that our method is modality- independent. Further, we illustrate that isometric regularization guides the model F to learn an accurate manifold with smooth latent space leading good generalization performance and robust- ness to noise in data.
2 RELATED WORK
Latent Variable Models for Functional Data. Recently, many works have shown that continuous signals can be efficiently modeled as function parameterized by a neural network, referred to as Implicit Neural Representations (INRs). INRs directly map the input variable (i.e., spatial coordinate or time index) into the corresponding value and efficiently represent broad class of high-resolution data including images (Ha) (0166), audios (Sitzmann et al., (2020), videos (Chen et al. [022] Li et al.) 2021), 3D shapes with SDF (Park et al. 2019), occupancy (Mescheder al. 2019). and radiance fields (Mildenhall et al. 2020). INRs are also actively employed to represent solutions to diverse differential equations in physics-informed machine learning (Raissi et al., 2019 Karniadakis al.,
2021
Lu et al. 2019 2021). Instead of representing a single function, recent INR works have shown learning the distribution of range of functional data by conditioning a shared neural network on a latent variable for each functional data (Chen & Zhang 2019 Park et al.) [219 Mescheder et al.)
2019 He et al. 2022 Fan et al. 2022 Du et al. 2021). There are two approaches to constructing a latent variable model. One straightforward way to condition is to concatenate input coordinate
with x (Park et al. 2019: Fan et al. 2022 He et al. (022). Alternatively, novel network architectures used condition the latent variable such as hypernetworks (Du et al. 2021 Gokbudak et al.) 2024| Lee et al.] 2223)) or attention networks (Rebain et al. 2022). We emphasize that our framework
generic and can be applied to both architectures.
2

---

Published as a conference paper at ICLR 2025
Geometric Regularization for Manifold Learning. For finite- miite-dimensional data, pret the learned latent space of generative models as an explicit parametrization of the data mani- fold (Arvanitidis et al. 2017: Lee 2023). It has been widely demonstrated that proper regularization on latent space can significantly improve the performance on downstream tasks for generative mod-
inter-
els, such clustering, interpolation, retrieval, and more (Chen et
2020 Lee al. [02T] 2022b
Jang et al. 2023 Lee et al. 2022a; Nazari et al., 2023
Lee & Park 2023 Lim et al. 2024 Lee
2024).
However, it is less explored to extend such manifold interpretation of latent variables beyond finite- dimensional data. While Du et al. (202T) view the latent variable model as manifold of INRs and presents local isometry loss, they require functions to be sampled from fixed points in coordinates with a fixed resolution. However, functional data often needs different sampling strategies for each data instance as it considerably affects the fitting quality (Park et al., 2019, SStrajman et al., 2021 Furthermore, is impossible to sample from fixed points with fixed numbers for real data in cer- tain scenarios, for example, mobile sensors such as floating buoys change their location over time and some sensors can malfunction (Luo et al., 2024). Instead, we design geometric regularization method specifically for functional data without fixing sampling condition.
3 RIEMANNIAN MANIFOLD OF FUNCTIONAL DATA
In this section, we introduce a geometric view that considers latent variable INRs as Riemannian manifold of functional data, laying the foundation for subsequent isometric regularization. We start with a formal definition of the space of functional data ʃ X' - V, where X' = Ih and V is vector space with the standard inner product, such that the inner product between u, w E v w. Throughout, we will consider a set of square-integrable mappings
F / f(x)" f(x)
f(x) de < 20)
(2)
as a functional data space, which is an infinite dimensional vector space
3.1 INNER PRODUCTS ON FUNCTIONAL DATA SPACE
First, we define the inner product on the functional data space F. Let d1 62 be square-integrable functions with a countably additive measure μ (e.g., probability measure) in X. A standard way to define inner product between them is as follows:
/ 8, (x) dp(x)).
(61 52)
This inner product does not depend a functional data point f E F. However, in some cases, need to consider different measures for each functional data point, as important regions X' could be different depending on f. For example, DeepSDF (Park al.) 2019) utilizes truncated SDF (TSDF) that clamps distance values of regions far from the surface to be constant, which means that function values only from areas close the surface have meaningful information. In such case, we would like to use a measure μ concentrated near the surface depending on f. Therefore, in this work, we consider the inner product depend on the function f by employing a function -ependett measure Jt
(81, = / 61(())T (2)x) d2y(x) for fEF.
(4)
3.2 RIEMANNIAN MANIFOLD EMBEDDED IN FUNCTIONAL DATA SPACE
Given our definition of the functional data space F in Equation 2
we can interpret the model F X V as mapping from the latent space 2 the functional data space F:
(5) where F3 is a functional data and F3 (x) = F(x, 2). Taking DeepSDF as an example, a latent code mapped to 3D shape represented by SDF F. R R. which takes spatial point
h 2 F s.t. + h(2)) := Fe,
F is Banach space, complete normed vector space, with respect the Lebesgue measure.
3

---

Published as a conference paper at ICLR 2025
E X' = R input and outputs the signed distance value d E V. Moving 2 in the latent space leads to a continuous change in the output signed distance of F. (z).
If Z is m-dimensional manifold (e.g., IR,",, then the image of h, denoted by = F(., 2), is an m-diimensiona manifold embedded in the functional data space F under some mild condition as illustrated in Figure [ We note that, although FF is anfiniteddimensione is dimen-iimessional. This manifold interpretation offers a geometric understanding of the model F. allowing us leverage tools from differential geometry develop regularization in a principled manner, as we demonstrate in the subsequent section.
the embedded manifold
We conclude this section by proposing a Riemannian geometric structure for the embedded man- ifold. Given the function-dependee
inner product (,-), for the functional data space F in Equa- tion (3) standard way to define a Riemannian metric for the embedded manifold via projecting ( f to the manifold. If 2 = IR" is treated as a local coordinate space, then the projected Rieman- nian metric can be expressed in Z as follows:
m
/
(OF(2, Oz
2F(
ds²
Σ
duF, do)'
(6)
Ozi
W-
dur, (I) forms an X m positive-definite matrix H(z)= = (hig Y. called Riemannian metric. We note that, with this metric, given an infinitesimal change the latent value 2 H ++ dz, the length of dz computed via 2))ddF(())))) dF F,, (2), where dF((x)) = F(2, 2 + dz) - F(x,2).
where the integral (z) = I @F(a,))
@F(2,e) Oz
dz H(2)dz = [dFF(x, 2,),
4 ISOMETRIC REGULARIZATION
Without proper regularization, the latent space can become ill-behaved, overfitting to the data in- stances, which is exacerbated by the infinite dimensionality of the function space. Our goal of finding a well-behaved latent space without compromising data fidelity can be restated in geometric terms: the given data instances lie in the manifold h(2) while the mapping h from Z to the manifold h(Z) should preserve geometry as much as possible. The former condition of fitting data instances enforced by a task-relevant loss function, which varies depending on the application. Examples in the experiment section include the signed distance and the reflectance fitting loss in deep generative models and the regression loss in operator learning. The latter soometry-preservinn condition can be achieved through our proposed isometric regularization.
By 'geometry-pressee preserving,'' we mean that distances and angles measured in the manifold should be preserved in the latent space. Consequently, slight changes in latent values will not result in dramatic changes the output data but will cause changes of a consistent magnitude, leading to latent space. Formally, let the latent space Z I be assigned with the identity metric and the manifold of functional data be assigned with the projected Riemannian metric in
Equation (6) Throughout, we will denote the Jacobian of F by Jxz #F(2.2) d= E dim
and the projected metric
H()) {(,)=)J( 2)x
2)dur, (x) E mm
A mapping h is called isometry if T H+)) for all E Z. If F satisfies the above condition, then for infinitesimal changes in the latent value dz and the corresponding change in the data dF 2) m
F(x, 2 d2) F(x, 2), the norm of dz in the latent space is equal the norm of d.F. i.e., dz dz dz H()ddz = f d dF duF, (r), meaning that F preserves local distances and angles. If this holds for all z, then F preserves the global geometry. As discussed in Lee al. (2022b), encourage scaled isometry of F that preserves angles and scaled distances, such that = (2) for some positive scalar and all
It is important to note that enforcing this scaled isometry condition does not impose a smoothness requirement on the z-coordinates, unlike Lipschitz regularization. For instance, consider the limiting
'The mapping h : Z F is an embedding if is injective immersion. In other words, (i) (injectivity) if F(z, =1) = F(x, =2), then 22 for all X' and (ii) (immersion) if @F(z.)), y for E mR"", then () for all 2,
4

---

Published as a conference paper at ICLR 2025
case where (i) I
(2) is exactly satisfied for all and (ii) duF, dj is independent of z. In this scenario, a trivial solution is that [ 2) remains constant over 2, Crucially, no conditions are imposed on the z-coordinates. This is one of the key properties of our regularization, which results in well-behaved latent space without compromising data fidelity.
Relaxed Distortion Measure. We introduce a functional called the relaxed distortion measure to impose isometric regularization for manifolds of functional data. It quantifies how far a mapping F from being scaled isometry. To handle the continuous data space, we apply an approximation based on efficient sampling algorithm. Here, we provide formulas for the relaxed distortion measure with minimal details: please refer to Appendix for more information and proofs. Throughout, we will use a probability measure for μF, in Equation 77) and denote its density function by p(x; Fs). Then the Riemannian metric can be written as (2) TE Eepp((;FF)) (=)
[JT (x, 21, 2)] E Rmxm, Let
be the eigenvalues of (z) for i 1, ,,...,
An equivalent characterization of the scaled isometry condition exists: if (x()) = e for some posi- tive scalar e and all and 2, then F' is scaled isometry. Enforeing this condition for all R" is unnecessary since latent points only occupy a specific region. We introduce latent probability den- sity Pz and focus on its support. Then, a coordinate-invarión relaxed distortion measure G(F, Pz)
for F with respect to Pz can be defined as follows:
(z) EvnPs E Ax(2/m]]
G(F. P2) == E Eg
1)
which is minimal if and only if F meets the scaled isometric condition in support of Pz. Ignoring a constant additive term, GFF, Pz) becomes scalar multiple of
E Px (IT((Be-p(=: (x, ((,,)),(z ExwP2 (T((E>-p(=:,,)) 2x,,))((z2))) 2)]]]²
When training manifolds of functional data F. we add the term above the original loss term to incorporate isometric regularization for the latent mapping. The expression in Equation (9) is similar to the one introduced in Lee et al. (2022b), but ours involves (e in the traces, and the Jacobian also depends on æ. This difference makes applying existing implementations non-trivial, motivating us to develop a new approximate and efficient distortion computation algorithm.
The computation of Equation 9 involves (i) cal- culating the Jacobian [xx and (ii) comput- ing the expectation of JJJ with respect to p(a; F.). These calculations must be performed at each training iteration, which significantly slows down the overall process. To address this, we introduce several practical techniques for approximate computation. First, esti- mate the trace terms using Hutchinson's 's stochas- tic trace estimator (Hutchinson 1989). Tr(A)
Efficient Approximation of Eq.
Algorithm 1: Efficient approximation of Eq. (9)
Precondition: input concatonation (F : :B*tm - R')
Input: latent codes {Zp...., #xy inpul coordinate samples
{(x) (0) (K)) b.... [x (0) (K)
Output: Relaxed distortion measure G
1 G1. G2 +
Augment with modified upx-up duta--uggentittion
3 forall x; do
4
x, [x([) ........)) (80
Sample vector vs N'C,, lmxm)
Expand by repeating K times
Eu-A((,,)) [v Av). which allows us to bypass 7 the need to compute the full Jacobian 2(x,z) and instead focus on Jacobian-vector and vector- Jacobian products. Second, to compute the ex- pectation of J with respect to p(z; Fz), we typically need to sample from p(a; F2). How- ever, the sampling distribution p(=; Fz) evolves
Augment vector vn by concatenating 16 0t] Compute G J(x;, 2:)u; with Jacobian-vector
8
product
Gi
G1 R. [G]
Compute G (F(x.. 2.)/ (2) with vector Jacobian product
Slice index by taking the m-th components
Go + 92 + B. E. [D]
during the training of F, making the online sam- pling infeasible. To mitigate this, we use offline samples from p(æ; F.)) where F; represents the functional data for training 2;-
9 return 9
9./91
We also apply the latent augmentation method from Chen et al. 22000; Lee et al. (022b) to define
Pz. A pseudocode for the approximate computation of the relaxed distortion measure is provided in A Algorithm]] 1 more details can be found in Appendix B
5

---

Published as a conference paper at ICLR 2025
Latent Space
Linear Interpolation
0.1
0.5
0.5
G1
N/A
A0
0.05
0.2
0.04
IsoAD (Ouns)
ISOAD (Ours)
-0.2
0.02
-0.4
0.01
Figure 2: Toy example of neural SDFs with five circles. (Left) Two-dimensional latent space of auto-decoder trained with and without isometric regularization. (Right) SDFs decoded from linear interpolants & LI errors on SDFs.
5 EXPERIMENTS
In this section, conduct extensive experiments to show the effect of isometric regularization three data modalities: neural SDFs, neural BRDFs, and neural operators.
5.1 NEURAL SDFs
SDF is a function f F + R that maps a spatial point to its signed distance to the surface of the shape. The surface points the shape are extracted from the zero-level of f. We train the neural SDFs with the latent variable model, denoted by Fg X' 2 - R, where A represents spatial coordinates, 2 Z is shape latent vector, and Fo(-, 2) is a SDF value, following the auto-decoder in DeepSDF (Park et al. 2019). We provide experimental details in Appendix C.I
5.1.1 TOY EXAMPLES ON SIMILAR SHAPES
Table 1: LI errors (x10-3) on lin- ear interpolation.
We first demonstrate the effect of isometric regularization with simple 2D shapes. We train auto-decoders using two- dimensional latent space = E R.2 without any regularization (AD) and with isometric regularization (IsoAD), and compare the results. Appendix D.I contains additional analysis on the probability density p(z: Fz).
IsoAD (Ours)
N
AD
36,5885
2.7497
Circle
10
10.1056
2.5622
20
9.5139
1.9630
Dataset. We make training datasets with three kinds of sim- ple shapes: circles, squares, and equilateral triangles. Each dataset contains N similar shapes whose parameters change linearly, for example, circles with radius r from 0.1 to 0.5 or squares with side lengths from 0.1 0.5. Then, train the network with the SDF values of each shape sampled from 32 32 grid.
13.1302
6.7625
Square
10
2,2039
1.9842
20
2.7227
1.4770
21,8526
4,4000
Triangle
10
2.0579
1.7629
20
2.7957
1.3722
Results. Given the smallest and the biggest shapes for each model, we generate intermediate shapes by linearly interpolating the latent space. We evaluate the decoded intermediate shapes against reference shapes created by interpolating the shape parameters. We compute the I.1 errors at 256 x 256 grid points. Quantitative results in icantly improves the interpolation results, especially when the number of training data N small. the latent space of auto-decoders trained with five circles. When we linearly increase the circle's radius, the most intuitive latent manifold would be a straight line with equidistant latent codes. While duto-decoder without regularization leads to distorted latent space resulting in unde- sirable interpolation results even with extremely simple shapes, isometric regularization encourages the output shape expand with constant velocity leading predictable interpolation results.
show that isometric regularization signif-

---

Published as a conference paper at ICLR 2025
Table 2: Quantitative results on surface reconstruction.
Dataset
MNIST
ShapeNet
N
100
500
271
542
Metrics
error average
L1 error
error
LI error median
CD
CD
median
average
average
median
average
median
DeepSDF
0,0366
0.0355
0.0288
0.0285
0.002202
0,001812
0.001588
0.001207
LipDeepSDF
0.0351
0,0341
0.0279
0.0255
0.001999
0,001606
0.001361
0.001054
IsoDeepSDF (ours)
0.0283
0.0265
0.0267
0.0245
0.001915
0.001472
0.001263
0.000925
5.1.2 SURFACE RECONSTRUCTION
We now train neural SDFs with variety of more complex
GT
DeepSDF
Uip.
Iso. (Ours)
2D & 3D shapes from MNIST (Deng 2012) and ShapeNet (Chang et al. 2015). Similar to Section 5.1.1 we auto- decoder architecture for training 2D SDFs. For training the autoencoder with 3D SDFs,
b
6
6
N=300
use encoder from Point- net Qi et al. (2017) and a decoder from DeepSDF, following the experimental setup from Liu et al. (0022).
We can then reconstruct the surfaces by finding latent codes that minimize the output SDF values the surface points pro-
== 1500
vided at the test time. Test time optimization of a latent code is commonly used auto-decoders and autoencoders to recon-
8
struct surface from the point cloud data (Park et al. 2019
Gurumurthy & Agrawal 2019 Liu et al.] 2022). In particu- lar, latent code optimization is needed when simple pass for- ward through the autoencoder leads to unsatisfying results. We compare method, referred to as IsoDeepSDF, with vanilla DeepSDF without regularization and DeepSDF with Lipschitz regularization (LipDeepSDF) (Liu et al. 2022).
Qualitative results of surface reconstruction given zero- level set on MNIST.
Figure 3:
MNIST Dataset. We make two datasets with different numbers of training data N = 300, 1 1500. The datasets contain 100 & 500 images randomly chosen from each of three digits [6, 8. 9]. The images are transformed to 2D signed distance fields on a 64 x 64 grid, where the contour of the digit the zero-level set. For test time optimization, we randomly sample 256 points from zero-level surfaces of the test dataset with 100 images from each digit [6, 8, 9].
ShapeNet Dataset. We randomly choose 5% (N = 271) and 10% (N 542) of shapes from the chair category of ShapeNetV2 for training datasets. As we observe that the choice of training dataset strongly impacts the reconstruction results, we make five different datasets for each N random choice. Quantitative results the average of the metrics evaluated from each dataset. The test- time optimization reconstructs the full 3D shape from partial point clouds obtained by deleting the right half of the surface point cloud.
Results. We compute the LI error for 2D SDFs on a 256 x 256 grid and Chamfer distance (CD) for 3D shapes with 30,000 points on the surfaces for evaluation metrics. Quantitative results are summarized in Table 2 IsoDecpSDF quantitatively outperforms the others for all datasets, demon- strating that isometric regularization helps learn a well-behaved latent space, leading better reconstruction results. While decreasing N severely deteriorates the reconstruction quality on the MNIST dataset, isometric regularization significantly reduces the performance degradation, provid- ing reliable regularization despite scarce data points. Figure results of surface reconstruction on the MNIST and ShapeNet datasets. Our method qualitatively shows better reconstruction results for both cases with 2D point clouds and 3D partial point clouds. Especially for 3D shapes, DeepSDF fails to fully reconstruct the unseen parts the chairs, while method can better reconstruct the overall shape given partial observations.
Figure/eaac each show the qualitative

---

Published as a conference paper at ICLR 2025
N 271
N 542
Input
GT
DeepSDF
Lip.
Iso. (Ours)
Input
GT
DeepSDF
Lip.
Iso. (Ours)
Figure 4: Qualitative results of surface reconstruction on ShapeNet chair dataset. Our method helps to learn better latent space, leading to better reconstruction results from partial observation.
5.2 NEURAL BRDFs
The bidirectional reflectance distribution function (BRDF) is function that takes the incoming and outgoing directions of light input and outputs the reflected radiance the surface, which used to render the appearance of materials. Recent works have proposed generalizable BRDF representations with latent variable models to reconstruct a BRDF of unseen materials from its samples (Sztrajman et al.) 2021 Fan et al. 2022 Gokbudak et al.) 2024). We show that isometric regularization complement those methods to learn better latent space for BRDFs, resulting in better reconstruction quality.
We train neural BRDFs with the auto-decoder architecture by concatenating latent variable = with input directions, similar to Fan et al. (2022). We then optimize the latent codes to reconstruct the en- tire BRDF of unseen material from BRDF samples. To evaluate the results of BRDF reconstruction, we render simple scene with the reconstructed BRDF with Mitsuba 3 renderer (Jakob et al. 2022) We compare the results of our method (IsoAD) with vanilla auto-decoder (AD) and auto-decoder with Lipschitz regularization (LipAD). We provide experimental details in Appendix
Dataset. We use the MERL dataset (Matusik al. 2003), common BRDF dataset measured from 100 real isotropic materials. We split the dataset into 80 materials for training and 20 materials for the test dataset. We make five different datasets for N 20, 40, 60 with random choice of materials from the full training dataset (N = 80) and compute the average metrics for evaluating BRDF reconstruction.
train the model with various numbers of training data: N = 20, 40, 60, 80,
Results. Figure 6 shows the reconstruction accuracy measured in PSNR and SSIM (Wang al. 2004). Isometric regularization improves the BRDF reconstruction by large margin for both met- rics. The effect of regularization is prominent when the training data N is greatly reduced to 20, maintaining robust results while the reconstruction results of AD drastically degrade. In particular, IsoAD trained with 20 materials shows higher PSNR and SSIM than the baselines trained with the full training dataset of 80 materials. This result demonstrates that isometric regularization enhances the generalization performance without compromising the fidelity of reconstruction. Qualitative re- sults on BRDF reconstruction are shown in Figure 5 The quality of reconstructed materials aligns with what we expect from quantitative metrics.
5.3 NEURAL OPERATORS
The Deep Operator Network (DONet) (Lu et al. 2019), originally introduced for solving PDEs, is standard architecture for constructing neural operators that map between functions. In this work, we demonstrate that incorporating isometric regularization of operator learning. The neural operator is a regression task that aims to learn a mapping G from an input function u X + IR to output function y - R, denoted by & u(xx) o(v), using pairs of input-output functions. Specifically, we represent the input as vectorized function Uvee = (u(æ1),... ...., u(xM)) E 24 for set of fixed points (xx, { We denote the training dataset of size N by {( 0, (1)))
significantly improve the performance
8

---

Published as a conference paper at ICLR 2025
N = 20
= 80
GT
LipAD
IsoAD (Ours)
GT
LipAD
IsoAD (Ours)
Figure 5: Qualitative results on BRDF reconstruction.
1.000
PSNR
SSiM
0.975
AD
0.960
UpADD
IsoAD (Curs)
30
0.825
60
N
Figure 6: Average PSNR (left) and SSIM (right) BRDF reconstruction.
Our model consists of two networks: an encoder E U + Z and a decoder F x R, where the output function is modeled as o(y) = F(E(uvec) y) for an input tive. When Uiee is high-ding nigh-dimensional, E' encodes it into -i-demnnsional vector that captures the essential informa- tion required predict the output function. This structure allows F be interpreted as latent variable model for functional representations, enabling the effective application of isometric regu- larization. We train and by minimizing Ei, (1, F(F((uee,)) U, 2 for set of query
points {yj 1M and evaluate the regression performance the test dataset. To assess the robustness of the model, we introduce varying levels of noise to the output functions in the training dataset, while keeping the input functions clean, simulating real-world uncertainties such as measurement errors numerical approximation inaccuracies. We compare the performance of three models: the unregularized DONet, the model with Lipschitz regularization (LipDONet), and the one with isometric regularization (IsoDONet).
Dataset. This study focuses on two types PDE datasets: the reaction-diffusion equation, as discussed in Yang et al. (2022), and the Darcy flow problem, based on et al. (2022). The reaction- diffusion equation describes how solution u(t, æ)) evolves over time and space under the effects of diffusion and reaction forces:
i u(t,z))
02
Ox. ultt,, kur f(x), (t,z) e [0,1] [0,
(10)
Øt
with some initial and boundary conditions. The operator aim to learn mapping 9 f z), where f(z) represents the forcing term.
The Darcy flow problem models fluid flow through a porous medium, of which steady state on a unit square is given by:
(a(x, Vul = f(x y), (a, y) e [o, 112
(11)
with some boundary conditions. The operator we aim to learn is a mapping : a(x, y) + u(I y) where a(x, y) is the diffusion coefficient and u(z, y) is fluid density, while assuming fixed external force f (z, y). These datasets are generated by solving the PDEs; details regarding the datasets can be found in Appendix C.3

---

Published as a conference paper at ICLR 2025
Input
DONet
LipDONet
IsoDONet (Ours)
Reaction Diffusion
Darcy
0.1
0.2
0.5
0.1
0.2
0.5
0.1
0.2
0.5
Noise Level
Figure 7: Qualitative results on neural operator. In the output images of reaction-diffusion, the horizontal axis represents and the vertical axis represents x.
Reaction Diffusion
Darcy
Relative Error
Relative Error
10
LipDONet
IsoDONel (Ours)
0.2
0.4
1.0
0.0
0.1
0.3
Noise Level
Noise Level
Figure 8: Relative errors as functions of the noise levels added to the training output functions.
Results. Figure shows the relative errors (or root mean square percentage errors) measured on clean test datasets as functions of the noise levels added to the training output functions. First, it can be observed that the errors increase for all three methods as the noise level increases. Impor- lantly, for IsoDONet, the rate of increase is much smaller compared to the other methods, and it consistently shows lower errors. This demonstrates that isometric regularization a highly effective regularization technique. Figure 7 shows some prediction results given input functions from the trained operator models that were trained with corrupted training datasets. DONet produces highly distorted predictions, severely overfitting the noise in the data, and expected, the distortion be- comes greater as the noise increases. LipDONet, while performing much better than DONet, tends overly regularize the output function in the spatio-temporal domain, especially observed in the Darcy flow problem, when compared to the ground truth (GT) output function on the left. It can be observed that IsoDONet produces the best qualitative results, predicting the output function with relatively less distortion and without overfitting to the noise.
CONCLUSION
In this work, we have proposed the isometric regularization for manifolds on infiite-dimensionall functional data space. We define inner products of functions with probabilistic weights and adapt Riemannian manifold learning within the formulation of latent variable models of INR. With ex- tensive experiments various modalities of functional data, we have confirmed that our isometric regularization can balance data fidelity and generalization, demonstrating improved performance on downstream tasks including interpolation and reconstruction. However, there still arbitrary choices in the process of mapping the infinite dimension into the latent space, such as the function- dependent inner product p(æ; F,) in the latent variable Z. An interesting future direction is to design a more systematic formulation suitable for downstream applications according the data characteristies.
function data the dimension of the
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We would like thank Clément Jambon for his valuable help with Mitsuba 3 renderer. This work
was supported by Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) [No. No.2021-0-0134 (Artificial Intelligence Graduate School Program (Chung-Ang University)), No. ASS-202-1122133 (Artificial Intelligence
Graduate School Program (Seoul National University)), No.2021-0-02068 (Artificial Intelligence
Innovation Hub)], CAINS through the Center for AI and Natural Sciences at Korea Institute for
Advanced Study (KIAS), and by KIAS Individual Grant No. AP092701.
REFERENCES
Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: on the curvature of deep generative models. arXiv preprint arXiv: 2017.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song. Hao Su, et al. Shapenet: An information 3d model repository. arXiv preprint arXiv: 2015.
Nutan Chen, Alexej Klushyn, Francesco Ferroni, Justin Bayer, and Patrick Van Der Smagt. Learning flat latent manifolds with vaes. arXiv preprint 20Xi::2002.0488/ 2020.
Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, and Xiaolong Wang. Videoinr: Learning video implicit neural representation for continuous space-time super-resolution. In Proceedings of the IEEF/CVF Conference Computer Vision
and Puttern Recognition, pp. 2047-2057, 2022.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 141-1444 2012. doi: 10.1 199 2211477.
Yilun Du, M. Katherine Collins, B. Joshua Tenenbaum, and Vincent Sitzmann. Learning signal- agnostic manifolds of neural fields. In Advances in Neural Information Processing Systems, 2021.
Jiahui Fan, Beibei Wang. Milos Hasan, Jian Yang, and Ling-QU Yan. Neural layered brdfs. In
ACM SIGGRAPH 2022 Conference Proceedings, SIGGRAPH '22, New York, NY, USA, 2022. Association for Computing Machinery, ISBN 9781450393379,
doi: 10.1145/522223333337733
URL /ttpps://oo..org/111111/// htt.://///...111535555522323333 3530732
Fazilet Gokbudak, Alejandro Sztrajman, Chenliang Zhou, Fangcheng Zhong, Rafal Mantiuk, and Cengiz Oztireli. Hypernetworks for generalizable brdf representation. 2024.
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometrie regu- larization for learning shapes. arXiv preprint arXiv:2002. 10099, 2020.
Swaminathan Gurumurthy and Shubham Agrawal. High fidelity semantic shape completion for point clouds using latent optimization. In 2019 IEEE Winter Conference on Applications of Com- puter Vision (WACV), pp. 1108. IEEE, 2019.
David Ha. 2016.
blog.otoro.net, generation i ing large images from- iitent-vectors. ///00//000000///////////////////
Generating large
images from latent vectors.
URL
Chengan He, Jum Saito, James Zachary, Holly Rushmeier, and Yi Zhou. Nemf: Neural motion fields for kinematic animation. In NeurIPS, 2022.
Dan Hendrycks and Kevin Gimpel. arXiv: 10tv:1606.0840 08415, 2016.
Gaussian error linear units (gelus). arXiv preprint
11

---

Published as a conference paper at ICLR 2025
Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in PoatitiissSimullant and Computation, 18(3):15591076, 1989.
Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zelt- ner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. Mitsuba 3 renderer, 2022. hutps:/miisuba--eneeeer..gg
Cheongjae Jang, Yonghyeon Lee, Jung-Kyun Noh, and Frank C Park. Geometrically regularized autoencoders for non-euclidean data. In The Eleventh International Conference on Learning Representations, 2023.
George Em Karniadakis, loannis Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422-440, 2021.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6900, 2014.
Jae Yong Lee, Sung Woong CHO, and Hyung Ju Hwang. HyperDeepONett. learning operator with complex target function space using the limited resources via hypernetwrk. In The Eleventh International Conference on Learning Representations, 2023. net/Eorum?id 02W6V32AhSd
Yonghyeon Lee. A geometric perspective on autoencoders. arXiv preprint arXiv:2309.08247, 2023.
Yonghyeon Lee. Mmp++: Motion manifold primitives with parametric curve models. IEEE Trans- actions on Robotics, 2024.
Yonghyeon Lee and Frank C Park. On explicit curvature regularization in deep generative models. In Topological, Algebraic and Geometric Learming Workshops 2023, pp. 505-518. PMLR, 2023.
Yonghyeon Lee, Hyeokjun Kwon, and Frank Park. Neighborhood reconstructing autoencoders. Advances in Neural Information Processing Systems, 34:536-544 2021.
Yonghyeon Lee, Seungyeon Kim, Jinwon Choi, and Frank Park. A statistical manifold frame- work for point cloud data. In International Conference on Machine Learning, pp. 12378-12402. PMLR, 2022a.
Yonghyeon Lee, Sangwoong Yoon, MinJun Son, and Frank C. Park. Regularized autoencoders for isometric representation learning. International Conference on Learning Representations, 2022b. URL htt.:httss///ppeeeeeeee net. wtte/eeo/rriiteeeeeeeeeeeeeeeeee forum? 1d=
Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space- time view synthesis of dynamic scenes. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Jungbin Lim, Jihwan Kim, Yonghyeon Lee, Cheongjae Jang, and Frank C Park. Graph geometry- preserving autoencoders. In Forty-first International Conference on Machine Learning, 2024.
Hsueh-Ti Ti Derek Liu, Francis Williams, Alec Jacobson, Sanja Fidler, and Or Litany. Learning smooth neural functions via lipschitz regularization. In ACM SIGGRAPH 2022 Conference Proceedings, SIGGRAPH '22, New York, NY. USA. 2022. Association for Computing Machinery. ISBN URL
9781450393379. doi: 10.1145/35222333330713
0128233.3330777
Lu Lu, Pengzhan Jin, and George Em Karniadakis, Deeponet: Learning nonlinear operators for iden- tifying differential equations based the universal approximation theorem of operators. arXiv preprint arXiv:1910.03/2 2019.
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Kariiadais.. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-229 2021.
12

---

Published as a conference paper at ICLR 2025
Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineer- ing, 393:1 14778, 2022.
Xihaier Luo, Wei Xu, Balu Nadiga, Yihui Ren, and Shinjae Yoo. Continuous field reconstruction from sparse observations with implicit neural networks. In The Twelfth International Confer- ence on Learning Representations, 2024. URL kuTZMZdCPZ
net. formm?
Wojeiech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. A data-driven reflectance model. ACM Transactions on Graphics, 22(3):759- 769, July 2003.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. In ECCV, 2020.
Philipp Nazari, Sebastian Damrich, and Fred A Hamprecht. Geometric autoencoders-winne you see is what you decode. 2023.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Charles R Qi. Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-666 6600, 2017.
M. Raissi, Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: deep learning framework for solving forward and inverse problems involving nonlinear partial differential equa- tions. J. Comput. Phys., 378:686-7011 2019. ISSN 0021 -9911. doi: j0.1016/fjcp.2018.. 1000045
URL https: /ol http:///..../00111111111111111/1 jcp. 2018 10.08.....0 045
Daniel Rebain, Mark Matthews, Kwang Moo Yi, Gopal Sharma, Dmitry Lagun, and Andrea Tagliasacchi. Attention beats concatenation for conditioning neural fields. arXiv preprint arXiv:2209.1068/ 2022.
Szymon M Rusinkiewicz. A new change of variables for efficient brdf representation. In Rendering Techniques' 98: Proceedings the Eurographics Workshop in Vienna, Austria, June 29 29-July 1, 1998 9, pp. 11 11-22. Springer, 1998.
Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wet- zstein. Implicit neural representations with periodic activation functions. In arXiv, 2020.
Alejandro Sztrajman, Gilles Rainer, Tobias Ritschel, and Tim Weyrich. Neural BRDF representa- tion and importance sampling. Computer Graphics Forum, 40(6):332-344 3333446,,,66 September 2021. doi: URI. https::/ inel 1 Sibrrrrrr...r wi ley. - com/doi
https///doi.or//10..1 10. 1//g.. 14335.
abs/10.. htc././.............1111
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Pridovich-Keil. Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Process- ing Systems, 33:7537-7547, 2020.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from enror visibility to structural similarity. IEEE transactions on image processing, 13(4))600- 612, 2004.
13

---

Published as a conference paper at ICLR 2025
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural fields in visual comput- ing and beyond. In Computer Graphics Forum, volume 41, pp. 641-676. Wiley Online Library, 2022.
Yibo Yang, Georgios Kissas, and Paris Perdikaris. Scalable uncertainty quantification for deep operator networks using randomized priors. Computer Methods Applied Mechanics and Engineering, 399:115399, 2022. ISSN 0045-7825. doi: https://doi.org//00001////
2022. 115391 URL https:/ /ttp:://wwww.oo sci enceidii /ett.c/om//cceeeeee/aiiic/l///// 506455622522045593
14

---

Published as a conference paper at ICLR 2025
APPENDIX
A PROOF OF RELAXED DISTORTION MEASURE
Denote ((x, 2)
OF (x, 2) E Rlxm for E R and E X. In this section, show that F(r, 2) Oz
a scaled isometric function generative model in the support Pz, i.c., for some positive scalar e,
(=)) = / (x, 2)1(2; F2) dr = el for all z 2 (F)
(12)
if and only if F is any minimizer of
Eg Pz (T (2))
(13)
E Tr H (=))
This proof is based on the Proposition 2 in Lee et al. (2022b). Denoting the eigenvalues of (z) by A=(=), 1,... m, can re- write the above equation follows:
Eg~P3 (x)
Ex Pa
TM
(=)
E E.-rz
(T)(2
E Pz
{x)]]
m
2 E BanPa Ei=l {x(2)]
(x
E ExSPa
i=1
m
{1
2
{x(2)
Eg~Pa
m )
E Exn Pæ D>mmm (x
m
E Ee~ 12 As(2)]
m
m
Ax(z)
1 )
1
Ps
(14)
Ez~Pz >E {x(2]]
i=1
Therefore, Equation 13 is minimal if and only if
A1(2)
0 for all 2 E supp(Pz).
(15)
Rg- >> {=(=)]
m
Now, we prove both directions.
suppose H(=) = el at all E supp Pz) for some positive
scalar c, then A=(2) for all E supp Pz) and Thus Equation 155) holds true and Equation
13 minimal. (=) suppose Equation (15) holds true, then (x E Ex- D2 Xx(=)] that
some positive constant for all z E (Tpp(?z)). which ends the proof.
B APPROXIMATE COMPUTATION OF RELAXED DISTORTION MEASURE
In this section, we explain details for approximately computing Equation 13) during the training of F(x, 2). To further elaborate on the computation of Jacobian-vector and vector-Jacobian products, first consider the Tr. (=))) = [r",, J]) term in the denominator. Using the Hutchinson's estimator, this trace term can be approximated with N E (v. E. J(uk) for eg (O, 4m)
where Im is an m x m identity matrix. Since of Ex [J JJ Ex (v J Jux), we need to compute [z, 2)04 = 8z Uk.
Removing the averaging term of trace estimator for simplicity, the denominator becomes
E (2))2
JT Jug]
= =Eg-Pg.m- Jo
(16)
Since F' is a function of (x, z). the conventional Jacobian-vector product implemented via automatic differentiation operates on vector of size (x. 2). Therefore, to compute Juk, we augment the vector AF (0, v). Similarly,
o with zero vector of size æ, denoted by 0, and compute the J (x, z z)UR as Ø(.,)) we examine the numerator,
Ey-Pz [T(H+ =
Jr JJJ Jeal
-E Jualla)
(17)
15

---

Published as a conference paper at ICLR 2025
Let the output of Jacobian-vector product Juh w E V. we need to compute J' ar We compute this by taking the last m-th components from the conventional vector-Jacobian product wT OF 0(,,2) E {ddi(())+
8s w.
We note that if F has certain specialized architectures, such neural network where parameters are not shared between 2 and 2, more efficient implementations of these products may be feasible without the need for zero vector padding.
C EXPERIMENTAL DETAILS
C.1 NEURAL SDFs
C.I.1 TOY EXAMPLES ON SIMILAR SHLAPES
Training, We train multi- llyer perceptron (MLP) with fully connected layers with sequence of nodes (4, 128, 128, 128, 128, 1) with ReLU activation functions. The input is the 2-dimensional latent code concatenated with 2D spatial point and the output is the signed distance value of the corresponding shape. We initialize the latent codes 2 ~ (0, 0.13) and optimize the network pa- rameters and the latent codes jointly with MSE loss between the output of the network and the ground truth SDF samples. We use Adam (Kingma 2014) with a learning rate of le-4 for network parameters and 1e-3 for the latent codes. We uniformly sample 512 points on the unit square for in- put coordinate samples for isometric regularization. We perform parameter sweeping for the weight of isometric regularization on log scale and report the best case for each model regarding test accuracy on linear interpolation in Table 3
Table 3: Weights of isometric regularization on toy examples.
N
Circle
Square
Triangle
0.001
0.001
0.0001
10
0.001
le-5
0.0001
20
0.001
le-5
0.0001
C.1.2 2D SURFACE RECONSTRUCTION
Training. We train neural SDFs with the MNIST dataset for 2D surface reconstruction, sim- ilar to Appendix C.I.I We use fully connected neural network with a sequence of nodes (4, 256, 256, 256, 256, 256, 1) with ReLU activation functions. The input is the concatenation the 8-dimensional latent code and the 2D spatial point. We initialize the latent codes (O, 0.1²) and optimize the network parameters and the latent codes jointly with MSE loss between the output of the network and the ground truth SDF samples. We use Adam with a learning rate of le-4 for network parameters and 1e-3 for the latent codes. We uniformly sample 4096 points on the unit square for input coordinate samples for isometric regularization. We perform parameter sweeping for the weight of each regularization (Lipschitz and ours) on a log scale and report the best case for each model regarding the accuracy of test time optimization in Table 4
Table 4: Weights of regularization terms 2D surface reconstruction.
N
Lipschitz
Isometrie (Ours)
100
1e-7
0.01
500
1c-7
0.001
Reconstruction. We reconstruct the 2D surface given 256 points randomly sampled from the zero- level set of digits from the test dataset by optimizing the latent code during test time. The test dataset consists of 100 images for each digit. We randomly initialize the latent code z ~ AN 12) and optimize the latent code minimizing the I.1 norm of the output, the SDF values of the input points
16

---

Published as a conference paper at ICLR 2025
should be zero. We use Adam with learning rate of le-4 and iterate until the loss converges. As the initialization fairly affects the reconstruction results, we repeat the optimization process twice with different initializations for each test data and report the result with lower loss convergence.
C.1.3 3D SURFACE RECONSTRUCTION
Training. We train autoencoder for 3D surface reconstruction, following Liu et al. (2022). The encoder is Pointet (Qi et al. 2017), which gets 3D spatial points for input. The encoder con- sists of two blocks of MLP with fully connected layers. The first block has sequence of nodes (3, 256, 512). After the hidden layer of 256 neurons, the input passes through the layer normaliza- tion and tanh activation before the 512-dimensional output feature vector. As the input points are x 3 matrix, the output of the first block is the x 512 feature matrix. Then, we obtain global fea- ture vector Eglobal E I212 by max- max- paollion the feature matrix. We expand the global feature vector by repeating times and concatenate it with the feature matrix (output of the first block). The second block has the sequence of nodes (1024, 256). Same as the first block, we pass through the layer normalization and tanh activation after the hidden layer. We then perform another max-pooling to get the final global feature vector. We apply the sigmoid function to the final global feature vector get the latent code lying between 0 and I.
The decoder follows the architecture of DeepSDF (Park et al. 2019). The input of the decoder is the latent code concatenated with the spatial query point. The decoder is fully connected neural network with a sequence of nodes (259, 1024, 1024, 1024, 512, 256, 128, 1), each hidden layer fol- lowed by layer normalization and leaky ReLU activation. Also, we perform dropout with probability 0.2 for each hidden layer and skip connection by concatenating the input latent vector to the output of the fourth hidden layer.
For the training dataset, we sample SDF samples following(Parks et al. (2019). For input point clouds to the encoder, we randomly sample points on the surface. We use I.I loss between the predicted SDF from the decoder and the ground truth SDF We optimize the network parameters with Adam with a learning rate of 5e-3 3 for 2000 epochs. We adjust the learning rate by half per 500 epochs.
As we train the truncated SDF (TSDF) with clamping distance of 0.1, we sample spatial points near the surface for isometric regularization. Please refer to Appendix[D.]] for the choice of sampling randomly sample 4096 points with a distance smaller than 0.1. We perform parameter sweeping for the weight of each regularization (Lipschitz and ours) on a log scale and report the best case for each model regarding the accuracy of test time optimization in Table 7
strategy for p(2: F.). Specifically,
Table 5: Weights of regularization terms on 3D reconstruction
Lipschitz
Isometric (Ours)
Dataset
1
2
3
1
2
4
5
= 5
le-9
1e-10
le-11
1e-9
le-11
0.0001
le-5
0.0001
0.0001
0.0001
N 10
le-11
le-9
le-11
le-11
1e-11
0.0001
0.001
0.001
0.001
1e-5
Reconstruction. We reconstruct the full 3D surface given partial observations of zero-level set by optimizing the latent code during test time. The latent code is initialized by passing forward through the encoder. Then we optimize the latent code parameters (parameter before applying the sigmoid function) by minimizing the loss function. Our loss function for the test time optimization is as follows: L Li + Aeikona where Li is the LI norm of the output of the network, Leikonal the eikonal term, and Aeikonal is the weight for eikonal term. Leikonal is the regularization term to force the norm of the gradient of the output to be 1 (Gropp et al. al. 2020). We use Adam with a learning rate of le-4 and iterate until the loss converges. We set Aeikonal = 0.01.
C.2 NEURAL BRDFS
Training. We train neural BRDFs with the auto-decoder architecture similar to Fan et al. (2022). However, we simplify the network architecture to MLP with fully connected layers, as
17

---

Published as a conference paper at ICLR 2025
BRDF dataset from simple isotropic materials instead of complex materials with multiple layers. The network consists of 8 hidden layers of 256 neurons with GELU activation (Hendrycks Gim- pel 2016). We add a final exponential layer to the MLP's output BRDF values have high dynamic range. The network input is a 12 dimension latent vector z and 6 dimension input direction vectors. The network output is 1-dimensional scalar, the BRDF value of each RGB color channel.
Thus, each BRDF is represented by three latent vectors optimized for each RGB channel.
Following Sztrajman et al. (2021), we parametrize the input directions the Cartesian vector h and d in the Rusinkiewicz parametrization (Rusinkiewicz, 1998). As BRDF values have high dynamic for training, similar to Sztrajman
range, we use logarithmic loss L loog(1 f+) log f |
al. 2021).
We randomly initialize the latent vectors ~ .V 0.13). Then, we jointly optimize the latent vectors and network parameters with Adam by minimizing the training loss L. The learning rate Se-4 for the network parameters and le-4 for the latent codes. We trained the models for 200 epochs. We decreased the learning rates by half after 100 epoch.
For input coordinate samples for isometric regularization, uniformly sample from the angle space of input directions (0n, oh, and da). We perform parameter sweeping for the weight of each regu- larization (Lipschitz and ours) log scale and report the best case for each model regarding the accuracy of test time optimization in T
Table 6: Weights of regularization terms on BRDF reconstruction.
Lipschitz
Isometric (Ours)
Dataset
2
3
4
5
2
3
4
5
N 20
1e-8
le-9
le-8
1e-8
1e-8
0.0001
0.001
le-5
1e-5
1e-5
N 40
le-10
le-8
le-9
le-10
le-10
le-5
le-5
0.0001
0.001
0.001
N 60
le-1l
le-11
le-10
le-10
le-9
0,001
0,001
0,0001
le-5
0,0001
N 80
le-9
N/A
N/A
N/A
N/A
le-5
N/A
N/A
N/A
N/A
Reconstruction. We reconstruct the BRDF from samples by optimizing the latent codes during
test time. We randomly initialize the latent codes z ~ (O, 0.1²). Then we optimize the latent codes by minimizing the logarithmic loss between the network output and the BRDF samples. We use Adam and set the learning rate as le-4. As the initialization fairly affects the reconstruction results, we repeat the optimization 3 times with different initializations and report the best regarding the convergence loss.
C.3 NEURAL OPERATORS
Datasets. First, we consider the reaction-diffusion equation covered in Yang et (2022) The ceaction-diffustio equation is written as
i ul1, x)) = " ul, kud [(x),
82
(1, E [0, x (0. 1].
Hr
(18)
a(t 0, 0,
x E [0,1].
=(t,== w(t,
E [0, 1),
where is the diffusion coefficient and ls is the reaction rate. We v 0.01 and k - 0.01. We then consider the operator G f(x)) + u(t, 2) where fxx is generated from a Gaussian process prior with an exponential quadratic kernel using length scale parameter / 0.2. For the projection of input and output functions, the time domain / E [0, 1] and spatial domain x: E [0, 1] are uniformly discretized into 100 points each. Standard Gaussian noise with varying standard deviations is added to the training output function to generate corrupted datasets.
For Darcy flow, we regenerated the data using the same procedure as in Lu et al. (2022). This problem involves diffusion equation with an external force, modeling fluid flow through a porous medium. The steady state on a unit square is given by:
V (a(x, 2a Vu(x, w)) [(,,), u((x,)) y) = O,
(x.y) [0, (a,y) E 0 1)2,
(19)
18

---

Published as a conference paper at ICLR 2025
where u(,, y) is the fluid density, a(x, 3) is the diffusion coefficient, and f(z y) is the external force.
Our goal is to learn the mapping 9 a(2; a, y) + u(x,3 y/ with a fixed external force f(x,y) = 1. The input a(z, 3) and the output u(z, y) are uniformly discretized to resolution of 20 x 20. Standard Gaussian noise with varying standard deviations is added to the training output function to generate corrupted datasets.
Training. We train neural operators consisting of E - 2 and F: 2 x + R. where F is further composed of branch and trunk networks, as adopted from Lu et al. (2019):
No.
F(z.D) = Σ Bo(r)T((2), b=1
(20)
where B, y - R and T3 - T= (Ti....,Ty............. TN.) are referred to as the branch and trunk networks, respectively. For the Reaction- Diffusion problem, E' is set as the identity map, i.c., Uyec. The map B is fully connected neural network with a sequence of nodes (100, 256, 256, 256, 256, 100) and GELU activation functions, while T is fully connected neural network with sequence of nodes (2, 256, 256, 256, 256, 100) and GELU activation functions, where No 100. For Darcy Flow, E' is set as the resnet18 followed by the linear layer that maps 512-dimensional vector to 32-dimensional vector a fully connected neural network with sequence of nodes (32, 1024, 1024, 1024, 1024, 100) and GELU activation functions, while T is fully connected neural network with a sequence of nodes (2, 256, 256, 256, 256, 100) and GELU activation functions, where No,- = 100.
1, ....., No. The vectors B (Bi..... Bx.) and
The map B is
The learning rate is set 1e-4. We trained the models for 200,000 epochs with a batch size of 1. 1,000 for the reaction-diffusion datasets and for 55,000 epochs with a batch size of 200 for the Darcy flow datasets. We perform a parameter sweep for the weight of each regularization (Lipschitz and ours) for each corrupted dataset separately and report the best case for each model based on the test set error.
Table 7: Weights of regularization terms on Neural Operator leraning.
Lipschitz
Isometric (Ours)
Noise level
0.01
0.1
0.2
0.5
1.0
0.01
0.1
0.2
0.5
1.0
Reaction Diffusion
0.0001
0.0001
0.0001
0.0001
0.001
1
10
30
100
Darcy Flow
0.0001
0.0001
0.0001
0.001
N/A
0.1
10
30
100
N/A
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 ABLATION ON p(:; F2)
In this section, perform ablation study the probability density p(1; F2) with neural SDFs. We compare two sampling strategies for p(æ; Fz): uniform sampling and near-surface sampling- We train the models with SDF samples from the toy example on five similar circles with radius 0.1 to 0.5. To consider various data characteristics, we also train TSDF samples by clamping the distance of SDF samples with +0.1. We follow the experimental details of toy examples on similar shapes described in Appendix C.1.1 We visually analyze the effect of p(æ: F,) on the latent space according to the training data (SDF & TSDF).
Figure 9 shows the latent space of neural SDFs trained with SDF and TSDF samples from five circles. We show decoded SDF&&TSDES and L.1 errors of linear interpolants in Figure TO and Figure 11 While other cases show nearly the same results expected with isometric regularization, showing straight equidistant latent space, isometric regularization with uniform sampling for TSDF straightens the latent space with "non-equidistant" latent codes. Specifically, the distance between the optimized latent codes increases as the radius increases. These results are due to the differences in data characteristics between SDF and TSDF. TSDF values from the points far from the surface than 0.1 are constant with the change of the latent code. As samples from those points do not affect the computation of Equation (12), shapes with small surface areas should be closer the
19

---

Published as a conference paper at ICLR 2025
AD
IsoAD (Uniform)
IsoAD (Near-surface]
SDF
TSDF
Figure 9: Latent space trained with SDF & TSDF from 5 circles.
Linear Interpolation
L1 error
01
G1
0.06
0.4
0.05
0.2
AD
0.04
0.0
0.03
130A
-0.2
0.02
IsoAD
0.01
0.00
Figure 10: Linear interpolation on latent space trained with SDE.
Linear Interpolation
L1
0.12
GT
N/A
0.06
0.06
0.04
0.00
IsoAD IsoAD (Neer- -uutr- (unform)
0.03
0.04
-0.08
0.01
-0122
0.00
Figure 11: Linear interpolation on latent space trained with TSDF.
latent manifold than those with larger surface areas when uniform sampling is used for training TSDF. Therefore, concentrating the samples near the surface for isometric regularization achieves the same expected results for both cases trained with SDF and TSDF. We note that there could be numerous choices of p(x; F2) depending on the nature of the data and the expected effect of isometric regularization.
20

---

Published as a conference paper at ICLR 2025
D.2 REGULARIZATION WEIGHT ANALYSIS
In this section, we provide an analysis on weights of regularization terms. The scale of the relaxed distortion measure varies across datasets, necessitating weighting factor tuning for each dataset. Empirically, we start by setting the weighting factor to match the scale of the isometric regularization term similar the original training loss term and then change the weighting factor with parameter sweeping. Tables 8 9 and T0 show the results with various weighting factors on the toy dataset, MNIST dataset, and MERL dataset, respectively. The results of N 20, 40 and 60 in Table [T] the average PSNR and SSIM over five datasets with the same weights on regularization term. When the weights on the isometric regularization are too large, the network may struggle to fit the training data, resulting in reduced performance compared to results without regularization. However, our method consistently outperforms those without isometric regularization across most weighting settings, demonstrating sufficient robustness to the choice of the weighting factor.
Table 8: Results of regularization weight analysis on toy dataset (N 5).
AD
IsoAD (Ours)
weight
N.A.
0.1
0.01
0.001
0.0001
1e-5
1.1 error
36.5885
4.3318
2.6013
2.7497
9.7079
20.5224
Table 9: Results of regularization weight analysis on MNIST dataset (N 100).
DeepSDF
LipDeepSDF
IsoDeep (Ours)
weight
N.A.
le-06
lc-07
le-08
0.01
0.001
0.0001
Average L1 error
0.03656
0.03511
0.03507
0.03854
0.02829
0.03302
0.03032
Median LI error
0,03551
0.03401
0.03413
0.03755
0.02654
0.03133
0.02933
Table 10: Results of regularization weight analysis on MERL dataset.
AD
LipAD
IsoAD (Ours)
weight
N.A.
Ic-8
lc-9
lc-10
lc-11
0.001
0.0001
le-5
PSNR
35.9776
35.5008
36.6982
33.9015
35.4429
32.7568
39.0047
40.621
N 80
SSIM
0.9544
0.9402
0.9720
0.9582
0.9326
0.9426
0.9767
0.9882
PSNR
33.5426
32.5683
32.6521
33.2074
34.0972
35.8485
37.4657
35,3189
N 60
SSIM
0.9466
0.9483
0.9504
0.9459
0.9561
0.9585
0.9691
0.9557
PSNR
33.3672
34,4997
31.8454
33.8662
31.4173
36.4007
34.2912
35,9980
N 40
SSIM
0.9383
0.9613
0.9381
0.9591
0.9237
0.9635
0.9502
0.9590
PSNR
31.0048
33.0056
31,9782
31.0097
29.4476
33.7202
34.9357
34.6490
N 20
SSIM
0.9286
0.9450
0.9336
0.9253
0.9155
0.9429
0.9550
0.9557
D.3 COMPUTATIONAL TIME
In this section, we provide the per-epoch runtime of AD, LipAD, and IsoAD during the training of neural SDFs with toy examples and the MNIST dataset. We use N - 5 circles for toy examples and N 100 for the MNIST dataset with the same experimental settings in Appendix C.I We use a single NVIDIA GeForce RTX 2080 Ti GPU for training. Table TT shows the per-epoch runtime.
D.4 REGULARIZATION INTERVAL ANALYSIS
While Algorithm avoids costly Jacobian computations, method still requires considerable additional computation regarding training batch size and number samples on X. One simple yet effective way to accelerate training is add the isometric regularization term specific intervals
21

---

Published as a conference paper at ICLR 2025
Table 11: Averages and standard deviations of the per-epoch runtimes (s) during training for each dataset.
Dataser Models
Toy Example
MNIST LipDeepSDD
AD
IsoAD (Ours)
DeepSDF
BoDeepSDF (Ours)
per-epoch runtime (s)
0.0049 (# 0.002)
0.0152 (# 0.0064)
0.8793 (#: 0.0543)
0.9256 (#: 0.042) 2,0819 (#: 0.0735)
rather than every epoch. To show the robustness of our regularization method the regularization interval, we conduct additional experiments on neural SDFs and neural BRDFs. We use the toy example dataset with circles (NV =5), MNIST dataset (NV 100), and MERL dataset (N 20).
We use the same experimental settings in Appendix [CI for neural SDFs and Appendix C22 for neural BRDFs. Tables 12
13 and show the results of regularization interval analysis on the toy
example dataset, MNIST dataset, and MERL dataset, respectively.
Results of regularization interval analysis on toy example dataset (circles: N 5). We show average L1 errors (x x0003) on linear interpolation.
Table 12:
AD
IsoAD (Ours)
interval
N.A.
2
5
10
weight
N.A.
0.01
0.01
0.01
0.01
L1 error
36.5885
2.6014
1.8571
3.2573
3.1487
Table 13: Results of regularization interval analysis on MNIST dataset (N 100). average and median LI errors on reconstruction from zero-level sets.
We show
DeepSDF
LipDeepSDF
IsoDeepSDF (Ours)
interval weight
N.A.
5
10
N.A.
1.00F-07
0.01
0.01
0.05
0.05
Average LI error
0.0366
0.0351
0.0283
0.0330
0.0335
0.0359
Median L1 error
0.0355
0.0341
0.0265
0.0321
0.0322
0.0338
Table 14: Results of regularization interval analysis on MERL dataset (N 20). We show average PSNR and SSIM over five datasets of N -20 BRDF reconstruction.
AD
LipAD
IsoAD (Ours)
interval
N.A.
5
10
PSNR
31,0048 0.9286
33,9142 0.9568
37.3545 0.9731
36.4126
36.3188
36.0928
SSIM
0.9498
0.9514
0.9471
D.5
ADDITIONAL RESULTS
D.5.1 TOY EXAMPLES ON SIMILAR SHAPES
We show additional results of neural SDFs trained with simple 2D shapes. Figure 12 shows the latent space of AD and IsoAD each trained with five squares and five triangles. Figure [3] and Figure [4] show generated intermediate shapes by linearly interpolating the latent space given the smallest and the biggest shapes of each model.
22

---

Published as a conference paper at ICLR 2025
AD
IsoAD (Ours)
Square
Triangle
Figure 12: Latent space trained with five squares & triangles.
Linear Interpolation
L1 error
(= 0.1
== 0.5
0.6
GT
N/A
0.035
0.2
0.025
AD
0.0
0.020
0.015
0.2
0.010
IsoAD
0.4
0.005
0.6
0.000
Figure 13: Linear interpolation on latent space trained with five squares.
Linear Interpolation
L1 error
0.1
GT
0.4
N/A
0.06
0.05
0.04
AD
0.0
0.03
0.02
IsoAD
0.4
0.00
Figure 14: Linear interpolation latent space trained with five triangles.
D.6 SURFACE RECONSTRUCTION
We show additional qualitative results of surface reconstruction MNIST and ShapeNet datasets. Figure [5]shows the reconstruction results of 2D shapes given zero-level set input. Figure 16 shows the reconstruction results of 3D shapes given partial observations.
D.6.1 NEURAL BRDFS
We show additional qualitative results of BRDF reconstruction in Figure 17 Ours reconstructs correct BRDFs while other baselines fail to reconstruct correct BRDFs resulting in severe color differences in the rendered images. The details discernible when the figure zoomed in.
23

---

Published as a conference paper at ICLR 2025
N 300
1500
GT
DeepSDF
Lip.
Iso. (Ours)
GT
DeepSDF
Lip.
Iso. (Ours)
8
Figure 15: Qualitative results of surface reconstruction given zero-level set on MNIST.
N 271
N 542
Input
GT
DeepSDF
Lip.
Iso. (Ours)
Input
GT
DeepSDF
Lip.
Iso. (Ours)
Figure 16: Qualitative results surface reconstruction on ShapeNet chair dataset.
D.6.2 NEURAL OPERATORS
We show additional qualitative results of neural operator with reaction-diffusion and Darcy problem datasets in Figure 18 and Figure 19
24

---

Published as a conference paper at ICLR 2025
IsoAD (Ours)
addaaaaoo aaaaaaa
N 80
UpAD
AD
0 aanaaaa Canaaaa
IsoAD (Ours)
N 60
LipAD
Figure 17: Qualitative results of BRDF reconstruction.
AD
IsoAD (Ours)
N 40
UiAAD
AD
IsoAD (Ours)
0 a
N = 20
LipAD
AD
25

---

Published as a conference paper at ICLR 2025
Input
GT
DONet
LipDONet
IsoDONet (Ours)
0.1
0.2
0.5
0.1
0.2
0.5
0.1
0.2
0.5
Noise Level
Figure 18: Qualitative results on neural operator with reaction-diffusinn dataset.
Input
GT
DONet
LipDONet
IsoDONet (Ours)
0.1
0.2
0.5
0.1
0.2
0.5
0.1
0.2
0.5
Nolse Level
Figure 19: Qualitative results on neural operator with Darcy problem dataset.
26


Reviews:
[{"review_id": "0fW0HFWm4Z", "reviewer": "Reviewer_qgrQ", "summary": "This paper proposes an isometric regularization technique for manifolds embedded in an (infinite-dimensional) function space. The goal is to enforce the learned operator $h: Z \\to \\mathcal{F}$ as an isometric embedding. Here, $Z$ represents a (finite-dimensional) latent space with the standard metric, and $\\mathcal{F}$ is the space of functions on $X$, equipped with an appropriately defined metric. A distortion measure that quantifies the lack of isometry (up to rescaling) is introduced, along with an efficient method for its estimation. Numerical experiments validate the effectiveness of the proposed method.", "questions": "Please refer to the previous section", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, well-organized, and easy to follow.  \n2. The concept of encouraging isometric embedding from $z$—the latent variable—to $h(z) = F(\\cdot, z)$, a functional representation, is interesting.  \n3. The efficient estimation of a distortion measure that quantifies the lack of isometry is also novel.  \n4. Extensive experiments, including applications to neural SDFs and DeepONets, effectively demonstrate the robustness of the proposed method", "weaknesses": "1. Algorithm 1 and its explanation in Appendix B should be expanded. For instance, further elaboration is needed on estimating the numerator in Equation 9. Additionally, in line 14 of Algorithm 1, should $G$ be given as $G = G_2 / G_1$?  \n2. Given the significant computation involved in estimating the distortion measure, especially when computing gradients, a comparison of computational time should be included in the paper.  \n3. While the method promotes an isometric embedding between the latent variable $z$ and its functional embedding $h(z) = F(\\cdot, z)$, it remains unclear if an autoencoder mapping the original input $u \\in \\mathcal{U}$ to its latent code is isometric. For example, in DeepONet, where $u$ represents the input function, if the encoder from $u$ to $z$ diverges substantially from an isometric mapping, enforcing isometry in $z \\mapsto h(z)$ may have limited impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KIOtD8A8Ri", "reviewer": "Reviewer_efT4", "summary": "This paper proposes a new regularizer for implicit neural representations (INRs). Motivated by the fact that existing methods such as Lipschitz regularization result in overly smooth neural representations with respect to the spatial coordinates, the authors propose isometric regularization. This regularizer encourages the image of the latent variable model F(x,z) with respect to the latent z variable to form a well-behaved manifold, so that small changes in the latent variable lead to small changes in the resulting INR. As their regularizer requires expensive Jacobian computations at many spatial and latent inputs during training, the authors employ the Hutchinson trace estimator to avoid materializing full Jacobian matrices during training, and pre-sample input points to avoid the need to draw fresh samples during training. They then demonstrate that their method improves on an unregularized baseline and Liu et al’s Lipschitz regularization on surface reconstruction, BRDF learning, and neural operator learning tasks.", "questions": "- How does the computational cost of the proposed method compare to competing methods like LipDeepSDF? I’m particularly interested in understanding how the marginal cost of training the latent variable model with isometric regularization compares to the marginal benefit relative to cheaper baselines.\n- Why is it insufficient to enforce the smoothness of F with respect to the latent coordinates alone? Is this especially challenging from a technical standpoint? More challenging than isometric regularization?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "- The paper is well-written, and the authors do a good job of introducing the mathematical preliminaries needed to describe their method.\n- The experiments persuasively make the case that isometric regularization empirically improves on the Lipschitz regularization baseline.\n- The numerical scheme the authors propose in Algorithm 1 to avoid costly Jacobian computations and sampling of spatial coordinates at training time is a reasonable strategy for approximating their regularizer.", "weaknesses": "I’m not sure if I understand the motivation for isometric regularization relative to simpler alternatives. The authors state that existing works such as Liu et al (2022) apply Lipschitz regularization to the latent variable models F to make them smooth across both the input space X and the latent space Z; this is problematic because for any fixed latent z, the function h(z) is overly smooth in X-space. While I agree that this is a notable deficiency in existing methods, why can one not simply enforce the smoothness of F with respect to the latent coordinates alone?\n\nThis question is especially salient in light of the high cost of isometric regularization relative to simpler baselines like Lipschitz regularization. I wonder whether the undeniable improvements arising from using isometric regularization outweigh its costs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oecvRvNNR7", "reviewer": "Reviewer_Awog", "summary": "The paper proposes a regularization method, Isometric Regularization, for handling manifolds in implicit neural representations (INRs) designed for infinite-dimensional functional data. It suggests that existing regularization methods often over-smooth data, leading to a loss of fidelity. To address this, the authors introduce a Riemannian manifold-based regularization that minimizes curvature and preserves the geometric consistency between the latent space and data manifold. Experiments across multiple data modalities demonstrate that this approach enhances the structure of the latent space, providing better generalization, especially for noisy and small datasets.", "questions": "1.\tIn Line 416, it says “The effect of regularization is prominent when the training data N is greatly reduced to 20”. However, it is difficult to object the significant improvement of IsoAD with N=20 from Fig. 6? Could you clarify this argument?\n2.\tHow does the strength of regularization choose for different datasets in the paper?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe approach is grounded in differential geometry, offering a mathematically rigorous framework for managing functional data representations. \n2.\tIt demonstrated effectiveness on various data types, including neural signed distance functions (SDFs), bidirectional reflectance distribution functions (BRDFs), and neural operators, showcasing its versatility. \n3.\tThe method preserves data fidelity while regularizing, resulting in better interpolation, reconstruction, and generalization across different types and qualities of data.", "weaknesses": "1.\tThis work uses Hutchinson’s stochastic trace estimator to approximate the trace terms, but it is unclear how much additional computational cost this method incurs compared to the baselines. It is better to include a discussion of the computational requirement.\n2.\tIt utilizes offline samples from the \"ground truth functional data\" to compute the expectation of $J^TJ$. It should provide the justification of accessing the ground truth data.\n3.\tThe regularization process should involve parameter tuning, which might be non-trivial. The paper lacks discussion of how to choose the strength of isometric regularization when applying the method to novel datasets or configurations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9xJprrhplv", "reviewer": "Reviewer_JUq7", "summary": "This paper proposes a way to conditional implicit neural models that may have been conditioned on additional information, such as a class label, or shape vector information. The core idea is to create a mapping from latent-codes to generated output in a way that preserves isometry – that is changes in input and changes in output should be related in a way that preserves distances and angles in the input and output spaces, subject to certain scaling parameters encoded by the function Jacobian. The paper shows experimental results that show the favorable properties of this approach.", "questions": "Please clarify what is the total loss function that is being optimized for, and what parameters are being learnt, and what are being kept fixed?\n\nHow is the latent-space itself being learnt?\n\nDiscussion about training convergence and model size comparison (without needing new experiments).\n\nClarify ‘auto-decoder’ terminology.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Geometry-preserving and isometry-preserving loss functions are a novelty, and are a growing area of work in generative models. \n\nThe approach to add an isometry-preserving loss to conditional INRs is novel. Approximate algorithms to compute this isometry measure are proposed which enhance the strengths of the paper.\n\nMultiple types of results for proposed INR learning are shown including on 2D, 3D, and neural operator learning.", "weaknesses": "Notation is unnecessarily complicated and hinders readability. For instance in the definition of the INR itself, \\mathcal{X} is more often than just \\mathbb{R}^m with m = 1, 2, or 3. Similarly, V is simply  \\mathbb{R}. The lean toward generality isn’t helping with comprehensibility, although everything could be described without loss of generation for f: \\mathbb{R}^2 -> \\mathbb{R}. I would recommend taking such an approach and moving the generalized framework for an appendix instead. We suffer from the same issue once again when defining the structure of the latent space \\mathcal{Z} which always happens to be simply a vector space of the type \\mathbb{R}^m.\n\nThe premise of section 4, which is the core starts of in a confusing way by stating “Without proper regularization, the latent space can become ill-behaved, overfitting to the data instances, which is exacerbated by the infinite dimensionality of the function space” – here it is unclear what is meant by ‘latent space can become ill-behaved’ – because until this point   \\mathcal{Z} is referred to as the latent-space, which is more or less given and not learnt end-to-end. E.g as I understood it,  \\mathcal{Z} can refer to class categorical labels, or shape-features that are learnt elsewhere. So I am not at all clear what is meant by ‘latent space can become ill-behaved’.\n\nAfter reading section 4, we are presented with a way to compute a measure of isometry – as I understand it, this measure really applies to the core INR map, but it is not clear how this measure is actually used in a loss function. No loss function is described from what I can tell. It is unclear thus where latent-codes themselves are being optimized for, or is only the INR mapping portion being optimized. On reading the appendix, it is found that network parameters and the latent codes are both being optimized. I would recommend clarifying this upfront, as we do not see any loss function that suggests latent-code optimization, and this makes it hard to understand how this is being done. This also means that the latent-codes cannot be class categorical labels, as I had initially thought they could be. Overall, this leads to multiple types of confusion which I would suggest really clearly describing.\n\nThe paper also uses the term auto-decoder multiple times, which is unclear what that means. Do they mean ‘decoder-only’ architecture, or ‘auto-encoder’. I did additional digging among the references cited to see if I am missing something, but I could not find what an ‘auto-decoder’ means anywhere.\n\nMany conditional INRs also include an element of randomness that creates variation in output, which could be easily included in this model, but was less clear if this was already considered. \n\nDetails about training convergence, comparison with other approaches in terms training times, model size would help position the experiments more clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Cp9I7P2r9S", "reviewer": "Reviewer_PybU", "summary": "The paper proposes a novel isometric regularization technique for Implicit Neural Representations (INRs) aimed at regularizing learned functions, a crucial component of optimizing over a theoretically infinite-dimensional space. This regularization is interpreted through the lens of Riemannian geometry over functional spaces, which helps maintain a well-behaved \"latent space\" while reducing overfitting, even for small or noisy datasets. The proposed method is validated across multiple scenarios, including synthetic, 2D, and 3D visual data.", "questions": "1. How does the proposed isometric regularization scale with respect to the dimensionality of the latent space and the number of data points? Have you tested its performance on more complex datasets or tasks beyond those mentioned in the paper?\n2. How sensitive is the approach to the choice of hyperparameters, such as the weight of the isometric regularization? Is there a systematic way to select these parameters for different types of data?\n3. Can you provide more insights into the computational costs relative to other regularization methods, such as Lipschitz regularization or weight decay, especially for larger datasets?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The proposed method is tested across various types of functional data, including neural SDFs, BRDFs, and neural operators. This cross-domain applicability highlights the generality and versatility of the approach. Experimental results show significant improvements in the quality of latent space, interpolation, and reconstruction tasks. The breath of synthetic to realistic tasks also gives a degree of insight both into the underlying functionality of the method as well as its practical use. The authors also address the computational complexity of infinite-dimensional regularizations and use practical approximations for computational feasibility. Finally, the authors provide an interesting theoretical perspective by interpreting the regularization problem through a Riemannian lens, allowing future researchers to leverage differential geometry principles to derive further regularization algorithms over functional spaces.\n\nIn general, this was a concrete, interesting problem with a fresh take that was presented well and demonstrated very well. I enjoyed reading this paper and believe it will make a positive impact on the field.", "weaknesses": "1.  While the paper demonstrates robustness and generalizability across several datasets, it does not provide sufficient exploration into the scalability of the approach for very high-dimensional latent spaces or large datasets. An obvious interesting application is in NeRFs, but in my view it is reasonable to delegate this to future work -- there is enough for the paper itself to stand.\n2. The paper lacks comprehensive ablation studies on the choices of parameters and effect of the individual components (e.g. the approximations), which may help future researchers in applying and developing from this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)