Beyond Uniformity: Regularizing Implicit Neural Representations through a Lipschitz Lens
Authors: Julian McGinnis
Suprosanna Shit
Florian A. Hölzl
Paul Friedrich
Paul Büschl
Vasiliki Sideri-Lampretsa
Mark Mühlau
Philippe C. Cattin
Bjoern Menze
Daniel Rueckert
Benedikt Wiestler
Venue: ICLR 2026 Poster (2026)
Keywords: Implicit Neural Representations
Neural Fields
Lipschitz
Regularization
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We take a Lipschitz perspective to study and regularize INRs in inverse problems.

Abstract: Implicit Neural Representations (INRs) have shown great promise in solving inverse problems, but their lack of inherent regularization often leads to a trade-off between expressiveness and smoothness. While Lipschitz continuity presents a principled form of implicit regularization, it is often applied as a rigid, uniform 1-Lipschitz constraint, limiting its potential in inverse problems. In this work, we reframe Lipschitz regularization as a flexible *Lipschitz budget framework*. We propose a method to first derive a principled, task-specific total budget $K$, then proceed to distribute this budget *non-uniformly* across all network components, including linear weights, activations, and embeddings. Across extensive experiments on deformable registration and image inpainting, we show that non-uniform allocation strategies provide a measure to balance regularization and expressiveness within the specified global budget. Our *Lipschitz lens* introduces an alternative, interpretable perspective to Neural Tangent Kernel (NTK) and Fourier analysis frameworks in INRs, offering practitioners actionable principles for improving network architecture and performance. Code and experimental results are available at: [https://lipschitz-inrs.github.io](https://lipschitz-inrs.github.io).

Published as a conference paper at ICLR 2026
BEYOND UNIFORMITY: REGULARIZING IMPLICIT NEURAL REPRESENTATIONS THROUGH A LIPSCHITZ
LENS
Julian McGinnis
Suprosanna Shit
Florian A. Hölz| Paul Friedrich
Paul 3
Vasiliki Sideri-Lampretsa
Mark Mühlau' Philippe C. Cattin Benedikt Wiestler
Bjoern M
Daniel Rueckert 1,2,7,
TTechniaal University of Munich 'ETH AI Centre
Munich Center for Machine Learning University of Basel
University of Zurich 'Imperial College London
Ciasso-Plattter-nee
"Equal contribution
Equal senior contribution
os(Wa)
4((W,]
feles)
Total budget
K= s IT Ris
aut Aolo)
Uniform allocation
As
k; Tip(de) Lip(W.)
Non-umiform allocation
Figure 1: Bridging the gap between theory and practice in Lipschitz regularization. While Lipschitz continuity offers a principled form of implicit regularization, selecting and distributing the budget K remains non-trivial. We propose novel, data-driven approach derive K from interpretable, signal- or domain-specific properties, such as tissue compressibility in deformable registration, and strategically distribute this budget across layers. This methodology allows for better balance of smoothness and expressiveness compared to uniform allocation strategies.
ABSTRACT
Implicit Neural Representations (INRs) have shown great promise in solving in- verse problems, but their lack of inherent regularization often leads to a trade-off between expressiveness and smoothness. While Lipschitz continuity presents a principled form of implicit regularization, it is often applied as rigid, uniform I-Lipschitz constraint, limiting its potential in inverse problems. In this work. we reframe Lipschitz regularization as a flexible Lipschitz propose a method first derive a principled, task-specific total budget K, then proceed to distribute this budget non-uniformly across all network components, including linear weights, activations, and embeddings. Across extensive experi- ments on deformable registration and image inpainting, show that non-uniform allocation strategies provide a measure to balance regularization and expressive- ness within the specified global budget. Our Lipschitz lens introduces alterna- tive, interpretable perspective to Neural Tangent Kernel (NTK) and Fourier analy- sis frameworks in INRs, offering practitioners actionable principles for improving network architecture and performance. Code and experimental results are avail- able at: https: /tctps://1iscchttee inrs. ww.rr.ggthuss github.
framework, We

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Sitzmann et al. 2020) have emerged as a promising coodality-agooste modeling framework, with applications ranging from data compres- sion (Strümpler et al., 2022; Dupont et al. [202bb] Wu et al. 2025), representation learning (Dupont
Implicit neural representations (Tancik et al. 2020
al. 2022a Bauer et al. 2023] Friedrich et 2025), novel view synthesis (Mildenhall al. 2021
Barron et al. 2021). to inverse problems including deformable registration (Wolterink et al. 2022
Sideni-Lamprets. et al. 2024). MRI reconstruction (Corona-Figueroa al., 2022 2022 McGinnis al.
2023 Huang et al. 2023), , and population modeling (Bieder et al. 2024 Dannecker et al.) 2024).
Despite their widespread success, significant gap remains in our understanding of the generaliza- tion mechanisms underlying INRs. Notably, studies by (Fridovich- Keil et al. (022) and (Kim & Fridovich- Keil (025) reveal that regularized grids can match or even outperform non-regularized INRs, challenging the presumed superiority of INRs in computer vision. These findings may be intuitively explained by the absence of implicit regularization expressive INR architectures (Ra-) masinghe et al. 2022) and may be addressed by incorporating explicit regularization into the learn- ing process (Gropp et al., 2020 Niemeyer et al.] 2022). Altematively, one can employ spectrally
constrained architectures (Liu et al. 2022] Coiffier & Béthune 2024 Zhang et al. 2023 implicit
regularization, which we systematically study in this paper.
In these applications, the network is typically constrained to be I-Lipschitz, which refer to Lipschitz Budget of one. This choice of unit Lipschitz constant is theoretically well-founded: the Wasserstein distance can be reformulated as maximization over 1-Lipschitz functions through the Suntorovich-Rubinster
duality (Arjovsky et al. 2017). while for SDFs, the constraint naturally emerges from the triangle inequality and Eikonal equation (Coiffier & Béthune 2024). To globally fulfill the Lipschitz Budger of one, current methods typically set all network component bounds to 1-Lipschitz, thus allocating the total Lipschitz Budger] uniformly across all network components, including layers, activations, and embeddings. However, we question whether this uniform distribu- tion of the budget is optimal.
In this work, we propose lens of Lipschitz continuity. We advocate for shifting the paradigm from a rigid, uniform Lipschitz constraint to more flexible, budgeted strategy. This approach directly addresses two fundamental, open problems in the Lipschitz regularization literature (Gouk et al. 2021 Newhouse et al. 2025): (1) how determine a principled total Lipschitz budget K for given task, and (2) how this budget should be optimally allocated among the network's 's components to maximize performance. Our primary contributions are as follows:
perspective on the regularization of INRs by analyzing it through the
1. We derive induced Lipschitz constants for INR components providing framework move beyond rigid 1 I-Lipschitz constraints. This allows for task-specific regularization strength.
2. We propose to estimate the Lipschitz budget guided by domain knowledge (deformable image registration) or via an interpretable oracle based on expected signal variation within the target domain (inpainting), and set it as an upper Lipschitz bound of the network.
3. Proposing a non-uniform budget allocation strategy, distributing the K-Lipschitz budget across network components based on their intrinsic Lipschitz characteristics, allows for balancing expressiveness and smoothness within the network.
4. Finally, we show how viewing empirically effective strategies, such as weight scaling, through a Lipschitz lens can provide an important, complementary perspective to Fourier analysis and the Neural Tangent Kernel theory in explaining their effectiveness.
2 BACKGROUND AND RELATED WORK
INRs and Regularization: Implicit representations have seen increasing adoption since neural ra- diance fields (Mildenhall et al. 202I). with coordinate encodings (Rahimi & Recht, 2007; Tancik
ct al. Müller et al. [2022) activation functions (Sitzmann al. [220 Ramasinghe & Lucey)
2022
Saragadam et al.) 2023). and optimization (MeGinnis et al.) (025) being crucial for network
expressivity to enable detailed modeling. Recent architectural works target the inherent trade-off
Total ipschitzz of neural network the product of each component's Lipschitz, defined in subsection 3.1
2

---

Published as a conference paper at ICLR 2026
of high expressiveness and smoothness (Chen et al., 2023 Kazerouni al. 2024 Liu et al. 2024
Yeom et al. 2024), but so far, no principled approach to network design exists. To balance expres- siveness and generalization, explicit regularization has been extensively studied (Niemeyer et al.
2022 Yang et al., 2023 Wynn & 'urrukhhabeetvy 2023), especially in shape modeling (Gropp) al. 2020: Atzmon & Lipman [2200] Ben-Shabat et al. [022). While explicit regularization can be beneficial, it only rigidly constrains model capacity and does not inherently limit the network's susceptibility to input perturbations. We deem this particularly relevant in the case of INRs, where early works have shown implicit regularization to be largely absent [Ramasinghe & Lucey 2022 Ramasinghe et al.
(022). putting this at the core of our study.
Lipschitz Constraints: The use of Lipschitz constraints has been widely studied in both the- ory (Szegedy et al., 2013 Bartlett et al., 2017 Gouk et al. 2021) and application (Arjovsky et al.) 2017 Gulrajani et a 2017). However, within neural fields, it remains relatively underexplored. We believe systematically studying this for INRs timely, especially as spectral regularization has
recently gained traction in computer vision (Kim et al., 2021: Qi et al. 2023) and large language models (Kim et al., 202T: Large et al. 2024 Newhouse et al., 2025). While Liu et al. (2022) reg- ularizes the upper Lipschitz bound in conditional shape modeling
enable meaningful inter- and
extrapolation, most other works are limited to I-Lipschitz networks. Coiffier & Béthune (2024) have introduced 1-L-isschtzz neural distance fields, MMujkanovic et al. (2024) for neural Gaussian scale- space fields, and /Zhng et al.] (2023) for vision applications. However, solely relying on uniform allocation of I-Lipschitz layers, without task-specific budgets or non-uniform allocations, severely limits a network's 's expressivity.
To enable more expressive Lipschitz bounds requires enforcing constraints in the INRs' linear lay- ers. Most commonly, this is achieved with spectral normalization (Golub & Van Loan 2013), or
efficient approximations thereof using power iteration methods (Miyato al.) [2018]. For stricter constraints, Björck orthonormalization (Björck & Bowie 1971; et 20.1 iteratively refines
matrix orthogonality, which inherently maintains unit spectral norm. Alternatively, directly devel-
oping Lipschitz constrained layers is topic of active study (Serrurier et 2022 Prach et al. 2024 Araujo et al. 2023).
2020 Prach & Lampert
Gradient pres--res activations (Anil et al.) 2019; Singla al., 202 Ducotterd et al. 2024)
are designed to approach the upper bound of a network's Lipschitz capacity, in contrast to standard activations like ReLU. Complementary methods aim to approximate the actual Lipschitz constant, both for ReLU-based MLPs (Virmaux & Scaman 2018 Fathony et al. (2020) and, more recently, for networks with gradient-preserving activations (Pauli al.] 2024a b). This work builds upon previous work by providing a unified framework for INR components. We investigate the critical questions of deriving task-specific Lipschitz budgets and non-uniform allocation for INRs tackle the
trade-off. In the following, we provide the theoretical background on Lipschitz theory for this work.
3 LIPSCHITZ COMPOSITION OF INRS
3.1 LAYER-WER WISE LIPSCHITZ CONSTANTS
function f Rdd -> IR' is Lipschitz-continnne with constant K if (21) f(x2)) S K||æ æ2ll for all 11, 22 E Rdd This Lipschitz constant K bounds the maximum rate of change of the function. the context of INRs, we argue that controlling the Lipschitz constant crucial for balancing smoothness and expressiveness. For network fo with L layers, the composition property of Lipschitz continuity dictates that the overall Lipschitz constant K is bounded above by the product of the Lipschitz constants of individual layers and activations o:
K = Lip(Jo) S II Lip(ds) Lip(W.). ==1
where W. represents the weight matrix of the i-th linear layer and Φ; is the activation function applied after it. This compositional property represents the comeestonn of our framework. It reveals that a global budget K can be achieved through a vast combination of layer-wise constants, which motivates our central exploration of different budget allocation strategies.
3

---

Published as a conference paper at ICLR 2026
Linear layers are the fundamental building blocks of INRs. The Lipschitz constant of a linear layer determined by its weight matrix W, specifically through the induced operator norm. The spectral norm, defined as the largest singular value of W., is common choice: Lip(W.) Tmax W) Constraining the norm of a linear layer is achieved through techniques like spectral normalization, e.g. using power iteration (Miyato al. (018)) to estimate the singular value decom- position, and orthogonalization (Björck & Bowie 1971 Anil et al. 2019), which forces the weight matrix to be orthogonal, such that W; W.=1.
Will2
For some INRs, the first layer is a feature projection r(-) with an associated Lipschitz constant. For positional encodings (Mildenhall al. 2021). given mapping 7p(P)
sin...........(..-.............. (sin(2 Tp), cp) ..., sin(2n-1 Tp), cos(22- Tp)) of ID input p to vector of sinusoidal functions, the Lipschitz constant is Lip(7p) = / & Recht 2007 Tancik et al. 2020) similarly map imputs using random projections with a Lipschitz
-يي Random Fourier Features Yy(v) (Rahimi
constant of Lip(7s) = 2T Amax
- bzb,) We refer to Appendix E for the exact derivation.
Lastly, we analyze activation functions as a crucial non-linear building block of INRs. Activations can be broadly categorized into two groups. 1-L-isscitzz activations, like ReIUU (6(x)= = max and GroupSort (Anil et al. 2019), as well related MaxMin and FullSorr, inherently bound the change in output relative to the imput, simplifying Lipschitz control. Learnable activation func- tions can be constrained to maintain Lipschitz constant of through parameter regularization. Non- I-Lipschitz activations usually have byperparameter-depenee frequency w or the scale parameter a. For sinusoidal activations, we have Φ(x) sin(war with For zu², the derivative is d' (z) - - a. To find the maxi- mum of (z))),, we solve d (z)) = O, which occurs at x = aa. Evaluating at (o'()a) _e yields Lip( (GGasssian) = ave. Non- Won-s-Lipsehitz activations typically require careful scaling or regu- larization to prevent unbounded growth in the overall Lipschitz constant. The choice of activation function and its parameterization significantly impacts network behavior.
Lipschitz bounds, such as the
derivative o' (ar) = cos(war). Since max,, cos(waz) = 1, we obtain Lip( (Simusidaa))
Gaussian activations with ((x)
3.2 LIPSCHITZ. BUDGET ALLOCATION
Given the total Lipschitz budget KB. we need to allocate this to individual network components. Let's consider that we have total of M network components. For this, Eq. is used for com- puting the total Lipschitz of the network. The allocation problem is to find [ which satisfy II K, KB where individual K, > 0. This product allocation can be thought of as a total sum allocation in logspace. Besides uniform allocation (A), we also study four non-uniform allocation strategies (B-E):
(A) Uniform allocation. In this work, we consider a baseline uniform allocation strategy, where we allocate each component the same Lipschitz contribution. This results in K; VKB.
(B) All-first allocation. We allocate K, = KB and K; = for = 2: M. This is motivated by the intuition that the first layer often plays critical role in feature learning for neural networks, where early weights have been shown to critically influence network sensitivity (Raghu et al.) 2017).
Additionally, we consider monotonically decreasing parametric allocation using linear, exponential, and cosine-annealed strategies constrained by the minimal Lipschitz of the last layer KM = Kmin- Let 4 = M-1 for 1: M with 4 and IM and define u, log K₁-
(C) Linear allocation. We impose straight line in budget space from unknown K; Su to KM = K Kmin K; = So + (Kmin so) 44 = 1: M, and choose 8o > 0 as the unique solution of Eid log( 5o + (Kmin 4.) log K B. using a one-dimensional root finder such as the Nisection/Newton method.
(D) Exponential allocation. We use front-heavy ramp in log space follows
u; = log minn + log KB - log Kmin (1 tz),
K, = cxp(u:).
Σ} - 11)
4

---

Published as a conference paper at ICLR 2026
(E) Cosine-annealed allocation. We consider the following g(t;)
1+000(m7)))
S0 g(ti)
1, g(a() = 0. We seek a single amplitude a > -1 such that
M M-1
K2 Kmin (1 a g(t))) for = 1: M, subject to KM
II
+ (1+ag(tt)) = KB.
Equivalently, is the unique solution to the 1-D monotone equation E log(1 + agg)t)
M
log
Kn Rr
-1. Exemplary allocation strategies provided in the supplementary Fig.
K min
4 REVISITING 1-LIPSCHITZ UNDER FLEXIBLE BUDGET ALLOCATION
4.1 I-LIPSCHITZ SIGNED DISTANCE FIELDS
Definition: Let consider SDF learnalee, implicit neural function fo which maps each coordinate of an input space Ω to its shortest orthogonal distance from that point to the boundary, i.e., the surface of geometric object. The sign of the function's output encodes spatial relationship: negative values indicate that the coordinate resides inside the represented object, while positive values indicate that the coordinates are outside the object's boundary. We can thus express the shape's boundary as the function's 's zero-level set (Park et al., 2019) Coiffier & Béthune 2024).
1-I I-Lipschitz: Since SDFs should represent the shortest, orthogonal distance to a shape's 's boundary, the gradient magnitude is required to be || = almost everywhere (Coiffier & Béthune 2024)
This property is known as the eikonal equation and holds for a true SDF throughout its domain, except at points where it is not differentiable, i.e., at the non-smooth boundary points. Adhering to this property ensures the function is I-Lipschitz, which is critical for algorithms like ray marching (Hart 1996) and further also enables the of specific losses such the ninge-Kantorovites Rubinstein loss for training neural distance fields (Coiffier Béthune 2024).
4.2 ON CAPACITY IN 1-LIPSCHITZ SDFs
I-Lipschitz SDFs can be achieved through various architectures and components; therefore, first objective is to systematically evaluate the impact of two key design choices: (1) different spectral normalization techniques applied to linear layers, and (2) the incorporation of 1-Lipschitz activation functions, which, to the best of our knowledge, have not been explored in the neural SDF literature. Following the approach Anil et al. (2019) for Wasserstein GANs, we investigate how achieving tighter Lipschitz bounds influences the learned SDF.
Experiment Lipschitz Capacity: Given the widespread use of ReLU activation functions in implicit neural representations for SDF' (Park al. 2019 Gropp et al. 2020 Davies et al.) 20020] Liu et al. 2022 Coiffier Béthune 2024), we investigate how imposing spectral normalization as a 1-Lipschitz constraint affects SDF learning quality. We evaluate this approach by training neural representations of the Stanford bunny model, comparing reconstruction quality against non- regularized SDFs using Chamfer Distance. To address the potential limitations of ReLU activations under Lipschitz constraints, we further examine how gradient- norm preserving activation functions can improve representational capacity by evaluating common preeeenn- functions, includ- ing MaxMin (Anil et al.) 2019). FullSort (Anil et al. 2019). and Householder activations (Singla al. 202T) implemented Serrurier al. (020)) against I-Lipschitz ReLU-based SDF repre- sentations. To substantiate our claims, we also measure the network's 's Lipschitz constant using the empirical method presented in (Wang & Manchester 20236).
Our results in Figure 2 demonstrate that Björck orthonormalization (Björck & Bowie 1971 Anil
al. 2019) and SI.L (Araujo al. 2023) achieve sharper reconstruction quality compared to stan- dard spectral normalization (Miyato et al.) 2018). Similarly, mradient-preserving activation func- tions, particularly MaxMin and Householder, also enable more effective utilization of the network's representational budget, allowing for finer geometric detail preservation. Notably, we find that the concept of Lipschitz capacity usage with respect to its empirically measured Lipschitz constant, proves intuitively interpretable for SDFs, correlating strongly with perceptual quality metrics: The closer a network is to utilizing the entire I-Lipschitz budget, the better the image quality. Remark- ably, with suitable spectral regularization, observe that certain 1-Lipschitz activations provide
5

---

Published as a conference paper at ICLR 2026
CR
CD 0--2-779
CoL4330
CDILLOR
CD 18.27
ReLU
PullSert
MaxMin
Hounsholder
Figure 2: Learning the classical Stanford bunny shape with different spectral normalization tech- niques (first row) and different gradient- preserrvng activation functions in combination with a Björck normalized linear layer (second row) demonstrates that approaching the upper Lipschitz bound in SDFs correlates with perceptual quality. We report the Chamfer Distance (4) and the empirically es- timated Lipschitz constant Km employing the estimation strategy in Wang & Manchester (202a))oo all reconstructions. Please refer to the Appendix for experimental details and results on other shapes. All shape visualizations are rendered using the excellent tools provided in Liu (018)).
MO
6,000
31300
630012
5:00775
02:00
0:00k
UAL3
Figure 3: Budget allocation experiments for 1 I-Lipschitz SDFs. We train ten SDFs with Householder (HH) and GorupSort (MaxMin) activations, using Björck and SLL normalization. Non-uniform allocation offers negligible advantages over uniform constraints, confirming that standard strategies suffice for shape representation. Experimental details are described in (subsection D.T
strong alternatives to the widely established ReL.U activation in shapes (Park et al.
2019
Davies
al. 2020 Coiffier & Béthune 2024).
Up to this point, we have applied uniform 1 l-L-ischitz con- straints across all layers of the neural network. However, the potential benefits of non-uniform constraint allocation remain largely unexplored (Gouk et al. (021). This gap is particularly in- triguing given the theoretical insights of Raghu et al. (217), who demonstrate that neural network behavior exhibits high sensitivity to initial weight distributions, suggesting that layer-wise constraint variation could yield improved reconstruction capabilities. Motivated by this theoretical foundation, we investigate strategic non-uniform allocation approach: we relax Lipschitz constraints in early layers to enable more expressive feature transformations, while imposing progressively tighter con- straints in later layers. This allocation strategy maintains the global 1-Lipschitz budget through the composition property of Lipschitz functions outlined in (subsecttion 3.1] ensuring theoretical guaran- tees while potentially enhancing representational capacity. The results presented in Figure 3 show that non- uniform allocation strategies perform on par with the standard uniform approach.
Experiment Budget Distribution:
This result suggests that the benefits of non- mon-uniform allocation may be stifled by the overly restrictive nature of a unit budget. This motivates our extension to the general K-Lipschitz setting, where the budget itself is meaningful, task-dependent parameter.

---

Published as a conference paper at ICLR 2026
5 BUDGET ESTIMATES & ALLOCATION STRATEGIES IN K-LIPSCHITZ INRs
In this section, we extend our investigation to a broader class of K -Liischhitz INRs for inverse prob- lems, with an exemplary study of budget estimation and allocation within the context of deformable image registration for medical imaging using a knowledge-driven estimate, and an oracle-driven es- timate for the classical computer vision application of inpainting. We include third experiment for single-image super-resolution in Appendix B
5.1
DOMAIN-DRIVEN BUDGETS THE K-LIPSCHITZ SETTING
INRs within the registration landscape: While deep learning models have been proposed to ac- celerate medical image registration from pairwise to cohort-level settings (Balakrishnan al. 2018 Dalca et al., 2019),, conventional optimization- based methods (Rueckert et al.) 2002] Avants 2011; Yushkevich et al. 2016) remain the gold standard for pairwise registration. Recently, INRs have emerged as a promising learning-basee -ased alternative, showing strong potential in lung CT and brain MRI registration (Wolterink et al. 2022] (Sideri-Lampreessa et al., 2024). However, due to the lack of implicit regularization (Ramasinghe et al. 2022), explicit constraints such as bending energy (Rueckert et al. 2002), elastic (Burger al., 2013), and curvature- based regularizers (Fis- cher & Modersitzki 2003 are often required. Even then, implicit deformation fields tend to fold (Sider--Lampretsa et al. 2024), motivating our investigation of spectral regularization as a form of implicit regularization within our K K-Lipschitzz framework.
2019
et al.
Lipschitz budget in K-Lipschitz displacement fields: Accurate deformable registration of lung images is essential for monitoring COPD (Murphy et al.] 2012: Galbán et al.) [2012] Castillo et al. 2013), but respiratory motion and disease-related changes make this task challenging (Wolterink et al. 2022). While lung tissue is deformable, it preserves structural integrity and cannot undergo extreme distortions. Enforcing a Lipschitz constraint provides a principled upper bound on local stretching compression, ensuring anatomically plausible displacement fields and preventing fold- ing, tearing, or excessive compression. Clinical evidence suggests that a strain approaching 2.0 marks a threshold for tissue fallure (Chiumello al. 2008: Brower al., 2008 Fung 2013), tivating our choice of KB = 2. Guided by this principle, we study allocation strategies on a lung dataset (Castillo et al.) 2013) using SIREN (Wolterink al. 2022 Sideri-Lamppon Lampeeesa et al. 2024) and
ReLU FFN, both shown to be promising for modeling continuous displacement field, and evaluate performance via target registration error (TRE) and folding ratio of deformation field.
While we observed no significant differences between allocation strategies in SDFs, we discovered important insights re- garding training dynamics in deformable registration tasks that lead to concrete rec- ommendations for practitioners. Our sys- tematic evaluation reveals that the com- bination of network architccture and Lip- schitz normalization method significantly impacts training stability and registration quality. We found that FFN using Björk SLL normalization (Araujo et al. 2023), as well SIREN with standard spectral nor- malization (Myatoo et al. 2018), can exhibit training instabilities when applied regis- tration tasks. Based on these findings, we recommend two stable and effective config-
Results:
Pobling
2.000
tesat rPs (Soecrad)
5R0W SuP 1943
Nesti FP (spectad)
Figure 4: Comparison of smoothness (Folding Ra- tio (()) and expressiveness (TRE (()) across three Lipschitz-eeuulaized INR architectures and alloca- tion strategies. Non-uniform allocations (e.g., expo- nential) can improve TRE while maintaining a com- parable folding ratio uniform allocation.
urations: spectral ReLU FFN and SIREN networks with Björk or SLL normalization (c.f. Fig 4 These architectures consistently achieve stable training while maintaining the intended benefits of Lipschitz regularization, delivering balance between TRE and folding ratio. Furthermore, we observe that the choice of allocation strategy (c.f. Fig provides control over this trade-off: non- uniform strategies enhance model expressiveness and reduce TRE, while maintaining comparable folding ratio to uniform strategies. These insights may enable practitioners to select configurations that best match their specific accuracy and smoothness requirements.
7

---

Published as a conference paper at ICLR 2026
ReLU
0.9
UN
Drst
1.0
25.0
077
0.8
17.5
0.5
6.5
Uni
0.3
6.4
10.
0.2
100
20D
300
400
500
102
200
300
500
0
130
200
300
400
500
Distance to Oracle
Distance to Oracle
Distance 10 Gradie
Figure 5: Results for different allocation strategies in the inpainting experiment using FFNs with K Minn 1.0 and SLL-normalization layers.
5.2 DATA-DRIVEN BUDGET FOR K -IPSSCCHI SETTINGS
well-studied application of INRs in computer vision is image inpainting (Pathak et al. 2016 Yu jet al., 2018), which aims to fill in missing or corrupted regions of an image. INRs are well-suited for this task (Sitzmann et al. 2020) as they represent images as continuous functions that can be queried at any spatial coordinate, enabling natural interpolation of missing pixel values.
Let's consider image inpainting as the task of reconstructing a complete image i from corrupted image I with missing or masked regions M. The goal is to fill in the missing pixels with plausible content. Given input coordinate (æ,, y) 52 where is the 2D image domain, an INR fo learns to map each spatial coordinate to its corresponding RGB color value: fe (z, y) - (R. G. B) For coordinates in the known regions, i.e., (Ir, y) & M, the network is trained to reproduce the orig- inal pixel values: [ar, y) R [xx,y) for (v., y) & M For masked coordinates, i.e., (x, E M, the network infers appropriate color values based on the learned continuous representation, effectively completing the image by querying fo at the missing coordinate locations. The reconstructed image is then given by: [zz y) = fo (z, y) for all (z, v) E Ω
Definition:
We begin by discussing two useful oracles in INRs, For image recon- struction, i.e., inpainting experiments, we assume the image to have bandwidth B, and assume that Lipschitz signal observation may follow this expectation. this case, we can then state that (1) the Lipschitz constant is upper-bounee bounded by the sampling distribution over bounded input and output domain. (2) We decide to use an L2-norm-based approach (since we are spectrally constraining with L2), to estimate a lower-bounded Lipschitz constant using gradient-based estimate. We refer to Appendix for the oracle estimate.
Lipschitz oracle for INRs:
For the inpainting experiments, several common INR architectures, including SIREN (Sitzmann et al.) 2020). FFNs (Tancik et al.) 2020), and Caussiinnactivvated INRs (Ramasinghe & Lucey 2022). using the allocation strategies presented earlier. We evaluate performance using standard metrics: PSNR (t ). SSIM (t) (Wang al. 2004), and LPIPS (L) (Zhang al. 2018). We present exemplary quantitative (Fig. 5 and
Experiment:
the CelebA dataset (Liu et al. 2015) and train
qualitative (Fig. 6 results and refer to Appendix for setup details, G.4 for statistical significance
tests and G.5 for additional results.
As shown in Fig non-uniform budget allocation strategies yield statistically significant improve- ments in reconstruction (c.f. G.4), peaking near the oracle estimate. Conversely, as shown in Fig. 5 performance degrades when the Lipschitz budget deviates from the oracle, particularly when using higher computational budgets (shown here for FFNs). This demonstrates that the oracle provides meaningful approximation of the upper Lipschitz bound for the inpainting task. Moreover, as shown in G.5 different INR architectures exhibit varying degrees of self-regulation when deviating from the estimated oracle budget, as evidenced by the steepness of their performance decline curves. We interpret this behavior as an inductive Lipschitz regulation bias that corresponds to how suboptimal budget allocations (i.e., greater deviation from the oracle estimate) affect each architecture's 's abil- ity to relf-tegulate local Lipschitzness. Observations for other architectures, including SIREN and Gauss, are provided in Appendix G.5
8

---

Published as a conference paper at ICLR 2026
Distance--50
Distance--20 15300
Distance-0 503508 00333
Distance=20
Distance-50
13360 6000
103308 9033
Uniform
Input Image
Ground Truth
30
2744 8000211
2236
Linear
23 110
28
10015
2430 (88 505
Exponential
Figure 6: Qualitative examples from image inpainting experiments on the CelebA dataset using Lipschitz FFN with Kmin 0.5 and SLL-normalization layers.
PRACTICAL GUIDELINES FOR LIPSCHITZ REGULARIZATION
While the experiments in section demonstrate the efficacy of our framework in inverse problems, the core principles of the proposed Lipschitz framework are broadly applicable. In this section, we distill our findings into generalized methodology for practitioners, addressing (1) deriving principled global budget K, and (2) selecting an optimal allocation strategy.
Strategies for Estimating the Budget K: We propose three distinct methods for estimating Lipschitz budget, using available information:
Domain-driven estimate: Beyond the tissue stretch constraints used in our experiments, domain- specific priors (e.g., max cardiac contraction) or intensity bounds can serve as interpretable upper bounds. For instance, in CT reconstruction, the maximal plausible gradient between adjacent voxels, such as the transition from air to tissue Hounsfield Units, can directly used to set an informed budget K relative spatial resolution.
Data-driven estimate: When representative samples are available (e.g., high-resolution reference images for super-resolution tasks), they may serve meaningful reference signal for the pro- posed oracle to estimate Lipschitz budget as described in A dataset of representative images can quantify local variations, effectively transferring the smoothness prior from the observed domain to guide budget selection.
Signal-theorert estimate: Lacking strong priors or reference data, we recommend conservative estimation based signal processing fundamentals (see Appendix C). Known bandlimits or sam- pling rates, such as those standard in audio (44.1 kHz) or electrocardiograas (x150 Hz), provide robust, noise- suppressip basclines for regularization.
Strategies for Allocating the Budget K: Distributing the global budget remains an open chal- lenge (Gouk et al. 2021). We recommend treating allocation hyperparameter search centered on network expressivity. Specifically, practitioners should analyze the performance of different alloca- tion strategies introduced in Section with respect to Kmans i.c., the minimum imposed Lipschitz bound of the networks' components. This allows for systematic exploration of the trade-off be- tween rigid, uniform regularization and flexible, non-uniform budget distributions.

---

Published as a conference paper at ICLR 2026
Post than
30
Peat Nest POOR
Network 12-Spestea harms
25
23
20 35
70
7800
1000 1754
Figure 7: Visualization of the upper induced Lipschitz bound of the method proposed by Yeom al. (224)) for SIREN. Scaling the initialization leads to direct scaling of the Lipschitz bounds of the linear layers, which allows the network increase its capacity overfit high-frequency content. We report laser-wise spectral norms for the experiment in subsectiin G.3
7 How CAN LIPSCHITZ THEORY PROVIDE NOVEL PERSPECTIVE?
So far, we have employed Lipschitz regularization as measure to improve smoothness in learned neural representations, while setting a global constraint on allowed perturbations, thus directly guid- ing implicit regularization strength. In this section, we want to highlight the appeal of using Lips- chitz theory to explain concurrent advances neural field architectures, and provide an outlook for the community on how Lipschitz theory may complement and even go beyond NTK literature (Jacot ict al., 2018 Yuce et al. 2022) and Fourier analysis (Benbarka et al. 2022 Ramasinghe et al., (023) in providing unifying paradigm. Recent work by Yeom et al. (2024) shows that scaling the initial and hidden layer weights of SIREN networks by a constant factor a improves both accuracy and convergence speed. While their explanation relies on optimization and NTK theory, we argue that Lipschitz theory offers a complementary perspective.
To test this, performed inpainting experiment with three-layer, 256-neuron SIREN (w 30), trained on 25% of the pixels and evaluated using PSNR on training and test sets. Following Yeom al. (2024), we uniformly scaled the first and hidden layer weights by E [1.0, 4.0] in steps of sin(w(a:W Wa + b)), the induced Lipschitz constant is Lip(f) |a||| Wll, showing that the upper Lipschitz bound scales linearly with a. Empirical spectral analysis of the scaled layers and the full network confirms this relationship. Scaling in- creases the spectral norm, enabling the network to capture more strongly. Moreover, layer-wise analysis shows that with small a, the Lipschitz bound grows during training, while with large a it remains constant, indicating a self-regulating capacity when weights are sufficiently scaled.
0.2. For standard SIREN layer f(x)
components and overfit
DISCUSSION AND CONCLUSION:
In this study, we proposed shift in how Lipschitz regularization is applied to INRs: from a rigid, global constraint to more flexible Lipschitz budget. Using an extensive set of experiments from shape representation, lung registration, to image inpainting and super-resolution, we have demon- strated that successful Lipschitz regularization critically depends on (i) selecting meaningful global budget, and (ii) utilizing appropriate allocation strategies.
To achieve this, we first introduced systematic framework for deriving task-specific budgets, in- formed by data, domain, or signal-theoretic properties. Second, we showed that non-uniform budget allocation enables balancing regularization with expressiveness under the same global budget. After demonstrating the proposed framework across different modalities and tasks, we have provided a practical guide for adapting it to novel problems and applications.
In future work, we aim to extend task-specific budgets to novel domains and explore how budget allocation might generalize within and beyond spectral regularization.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS:
We thank Patricia Pauli for helpful discussions. This work is funded by the Munich Center for Machine Learning. Julian McGinnis and Mark Mühlau are supported by Bavarian State Ministry for Science and Art (Collaborative Bilateral Research Program Bavaria - Quebec: AI in medicine, grant 2.4V0134.KS. 34..6..////// Suprosanna Shit supported by the UZH Postdoc Grant (K-74851-03-011 Suprosanna Shit and Bjoern Menze acknowledge support by the Helmut Horten Foundation. This work was supported under project ID a135 as part of the Swiss AI Initiative, through a grant from the ETH Domain and computational resources provided by the Swiss National Supercomputing Centre (CSCS) under the Alps infrastructure.
REFERENCES
Cem Anil, James Lucas, and Roger Grosse. Sorting out lipschitz function approximation. In Inter- national conference on machine learning, pp. 291-301. PMLR, 2019.
Alexandre Araujo, Aaron Havens, Blaise Delattre, Alexandre Allauzen, and Bin Hu. A unified algebraic perspective on lipschitz neural networks. arXiv preprint urXiv:230.003169, 2023.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks In International conference on machine learning, pp. 214-223. PMLR, 2017.
Matan Atzmon and Yaron Lipman. Sal: Sign agnostic learning of shapes from raw data. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2565,2574, 2020.
Brian Avants, Nicholas Tustison, Gang Song, Philip A Cook, Amo Klein, and James C Gee. A reproducible evaluation of ants similarity metric performance in brain image registration. Neu- roimage, 54(3):2033-20) 20444444,,4,,,,,, 2011.
Guha Balakrishnan, Amy Zhao, Mert Sabuneu, John Guttag, and Adrian V Dalca. An unsu- pervised learning model for deformable medical image registration. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 9252-9260, 2018
Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Voxelmorph: learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8): 1788--8088, 2019.
Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5855-5866 2021.
Peter L Bartlett, Dylan J Foster, and Matus Telgarsky. Spectrally- normalized margin bounds for neural networks. Advances in neural information processing systems, 30, 2017.
Matthias Bauer, Emilien Dupont, Andy Brock, Dan Rosenbaum, Jonathan Richard Schwarz, and Hyunjik Kim. Spatial functa: Scaling functa to imagenet classification and generation. arXiv preprint arXiv:2302.0311 2023.
Yizhak Ben-Shabat, Chamin Hewa Koneputugodage, and Stephen Gould. Digs: Divergence guided shape implicit neural representation for unoriented point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2022.
Nuri Benbarka, Timon Höfer, Andreas Zell, al. Seeing implicit neural representations as fourier series. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2051-2050, 2022.
Florentin Bieder, Paul Friedrich, Hélène Corbaz, Alicia Durrer, Julia Wolleb, and Philippe C. Cattin. Modeling the neonatal brain development using implicit neural representations. In International Workshop PRedictive Intelligence In MEdicine, pp. 1-11. Springer, 2024.

---

Published as a conference paper at ICLR 2026
Åke Björek and Clazett Bowie. An iterative algorithm for computing the best estimate of orthog- onal matrix. SIAM Journal on Numerical Analysis, 8(2)358- 1971.
Roy G Brower, Rolf D Hubmayr, and Arthur S Slutsky. Lung stress and strain in acute respiratory distress syndrome: good ideas for clinical management?, 2008.
Martin Burger, Jan Modersitzki, and Lars Ruthotto. A hyperelastic regularization energy for image registration. SIAM Journal on Scientific Computing, 35(1):B132 B148, 2013.
Richard Castillo, Edward Castillo, David Fuentes, Moiz Ahmad, Abbie M Wood, Michelle $ Lud- wig, and Thomas Guerrero. A reference dataset for deformable image registration spatial accuracy evaluation using the copdgene study archive. Physics in Medicine Biology, 58(9):2861, 2013,
Zhang Chen, Zhong Li, Liangchen Song, Lele Chen, Jingyi Yu, Junsong Yuan, and Yi Xu. Neurbf: A neural fields representation with adaptive radial basis functions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4182-4194, 2023.
Davide Chiumello, Eleonora Carlesso, Paolo Cadringher, Pietro Caironi, Franco Valenza, Federico Polli, Federica Tallarini, Paola Cozzi, Massimo Cressoni, Angelo Colombo, et al. Lung stress and strain during mechanical ventilation for acute respiratory distress syndrome. American journal of respiratory and critical care medicine, 178(4):346-35) 355, 2008.
Guillaume Coiffier and Louis Béthune. 1-lipschitz neural distance fields. In Computer Graphics Forum, volume 43, pp. e15128. Wiley Online Library, 2024.
Abril Forura-Figueroa, Jonathan Frawley, Sam Bond-Taylor, Sarath Bethapudi, Hubert PH Shum, and Chris G Willcocks. Mednerf: Medical neural radiance fields for reconstructing 3d-aware ct-projections from a single x-ray. In 2022 44th annual international conference of the IEEE engineering in medicine & Biology society (EMBC), pp. 2843-3848, IEEE, 2022.
Adrian V Dalca, Guha Balakrishnan, John Guttag, and Mert R Sabuncu. Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Medical image analysis, 57: 226 236, 2019.
Maik Dannecker, Vanessa Kyriakopoulou, Lucilio Cordero-Grande, Anthony N Price, Joseph V Hajnal, and Daniel Rueckert. Cina: conditional implicit neural atlas for spatio-temporal represen- tation of fetal brains. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 181-191. Springer, 2024.
Thomas Davies, Derek Nowrouzezzhraai, and Alec Jacobson. On the effectiveness of weight-encoded neural implicit 3d shapes. arXiv preprint arXiv:2009.0995 2020.
Stanislas Ducotterd, Alexis Goujon, Pakshal Bohra, Dimitris Perdios, Scbastian Neumayer, and Michael Unser. Improving lipschitz-constrraiedd neural networks by learning activation functions. Journal of Machine Learning Research, (5(65): 2024.
Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From data to functa: Your data point is a function and you can treat like one. arXiv preprint arXiv:2201.12204, 2022a.
Emilien Dupont, Hrushikesh Loya, Milad Alizadeh, Adam Golinski, Yee Whye Teh and Amaud Doucet. Coin++: Neural compression across modalities. arXiv preprint P:Xiv:2201.12900 2022b.
Rizal Fathony, Anit Kumar Sahu, Devin Willmott, and J Zico Kolter. Multiplicative filter networks. In International Conference on Learning Representations, 2020.
Bernd Fischer and Jan Modersitzki. Curvature based image registration. Journal of Mathematical Imaging and Vision, 18(1):81-85, 2003.
Sara Fridovich-Keie Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings the the IEEE/CVF conference on computer vision and pattern recognition, pp. 5501-5510, 2022.
12

---

Published as a conference paper at ICLR 2026
Paul Friedrich, Florentin Bieder, and Philippe C Cattin. Medfuneta: Modality- representa- tions based on efficient neural fields. arXiv preprint arXiv:2502.14401, 2025.
Yuan-cheng Fung. Biomechanics: mechanical properties of living tissues. Springer Science & Business Media, 2013.
Craig J Galbán, Meilan K Han. Jennifer L Boes, Komal A Chughtai, Charles R Meyer, Timothy D Johnson, Stefanie Galbán, Alnawaz Rehemtulla, Ella A Kazerooni, Fernando J Martinez, et al. Computed tomography-base biomarker provides unique signature for diagnosis of copd pheno- types and disease progression. Nature medicine, 18(11):1711- 1111171111111,,11 2012.
Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013.
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael Cree. Regularisation of neural net- works by enforcing lipschitz continuity. Machine Learning, 100()):333 -16, 2021.
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regu- larization for learning shapes. arXiv preprint arXiv:2002.10099 2020.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
John C Hart. Sphere tracing: A geometric method for the antialiased ray tracing of implicit surfaces. The Visual Computer, 12(10):527-545 1996.
Wenqi Huang, Hongwei Bran Li, Jiazhen Pan, Gastao Cruz, Daniel Rueckert, and Kerstin Ham- mernik. Neural implicit k-space for binning-free non-cartesian cardiac mr imaging. Interna- fional Conference on Information Processing in Medical Imaging, pp. 548-560. Springer, 2023.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. Advances in neural information processing systems, 31, 2018.
Amirhossein Kazerouni, Reza Azad, Alireza Hosseini, Dorit Merhof, and Ulas Bagei. Incode: Implicit neural conditioning with prior knowledge embeddings. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1298-1307, 2024.
Hyunjik Kim, George Papamakarios, and Andriy Mnih. The lipschitz constant of self-attention. In International Conference on Machine Learning, pp. 5562-5571. PMLR, 2021.
Namhoon Kim and Sara Fridovich-Keii. Grids often outperform implicit neural representations. arXiv preprint arXiv:2506. /339, 2025.
Tim Large, Yang Liu, Minyoung Huh, Hyojin Bahng, Phillip Isola, and Jeremy Bernstein. Scalable optimization in the modular norm. Advances in Neural Information Processing Systems, 37: 73501-73548, 2024.
Hsuch-Ti Derek Liu. Blender Toolbox, 12 2018. URLJatpps//ggthub..o////tDrrLLLD BlenderToolbon
Hsuch-Th Ti Derek Liu, Francis Williams, Alec Jacobson, Sanja Fidler, and Or Litany. Learning smooth neural functions via lipschitz regularization. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-13, 2022.
Zhen Liu, Hao Zhu, Zhang, Jingde Fu, Weibing Deng, Zhan Ma, Yanwen Guo, and Xun Cao. Finer: Flexible spectral-bias tuning in implicit neural representation by variable-periods activa- tion functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2713-2722, 2024.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) December 2015.
13

---

Published as a conference paper at ICLR 2026
Julian McGinnis, Suprosanna Shit, Hongwei Bran Li, Vasiliki Sideri-Lamprettas Robert Graf, Maik Dannecker, Jiazhen Pan, Nil Stolt-Ansó, Mark Mühlau, Jan S Kirschke, et al. Single-subjer multi-contrast mri super-resolution via implicit neural representations. In International Confer- ence on Medical Image Computing and Computer-Asiitted Intervention, pp. 173-183. Springer, 2023.
Julian McGinnis, Florian Hölzl, Suprosanna Shit, Florentin Bieder, Paul Friedrich, Mark Mühlau, Björn Menze, Daniel Rueckert, and Benedikt Wiestler. Optimizing rank for high-fidelity implicit neural representations. arXiv preprint arXiv:2512. 14366, 2025.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization 2018
for generative adversarial networks. arXiv preprint
Felix Mujkanovic, Nuuma Elie Nsampi, Christian Theobalt, Hans-Peter Seidel, and Thomas Leimkuchler. Neural gaussian scale-space fields. arXiv preprint 2xXiv:2405.20980 2024.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with a multiresolution hash encoding, ACM transactions on graphics (TOG), 41(4):1-15, 2022.
Keelin Murphy, Josien PW Pluim, Eva M Van Rikxoort, Pim A De Jong, Bartjan De Hoop, Hester A Gietema, Onno Mets, Marleen De Bruijne, Pechin Lo, Mathias Prokop, al. Toward automatic regional analysis of pulmonary function using inspiration and expiration thoracic ct. Medical physics, 39(3):1650-166) 2012.
Laker Newhouse, R Preston Hess, Franz Cesista, Andrii Zahorodnii, Jeremy Bernstein, and Phillip Isola. Training transformers with enforced lipschitz constants. arXiv preprint arXiv:2507.13338. 2025.
Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan. Regnerf:: Regularizing neural radiance fields for view synthesis from sparse inputs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 5480-5490, 2022.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 165-174, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer 2016.
vision and pattern recognition, pp.
Patricia Pauli, Dennis Gramlich, and Frank Allgöwer. Lipschitz constant estimation for general neural network architectures using control tools. arXiv preprint arXiv:2405.01 025, 2024a.
Patricia Pauli, Aaron Havens, Alexandre Araujo, Siddharth Garg, Farshad Khorrami, Frank Allgöwer, and Bin Hu. Novel quadratic constraints for extending lipsdp beyond slope-restricted activations. arXiv preprint arXiv:2401. 14033, 2024b.
Bernd Prach and Christoph H Lampert. Almost-orthogonal layers for efficient general-purpose lipschitz networks. In European Conference on Computer Vision, pp. 350-365. Springer, 2022.
Bernd Prach, Fabio Brau, Giorgio Buttazzo, and Christoph H Lampert. 1-lipschitz layers com- pared: Memory speed and certifiable robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
14

---

Published as a conference paper at ICLR 2026
Xianbiao Qi, Jianan Wang, Yihao Chen, Yukai Shi, and Lei Zhang. Lipsformer: Introducing lips- chitz continuity to vision transformers. arXiv preprint arXiv:2304.09856, 2023.
Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. On the ex- pressive power of deep neural networks. In international conference on machine learning, pp. 2847-2854. PMLR, 2017.
Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007.
Sameera Ramasinghe and Simon Lucey, Beyond periodicity: Towards a unifying framework for acti- vations in coordinate-mlps. In European Conference Computer Vision, pp. 142-158. Springer, 2022.
Sameera Ramasinghe, Lachlan E MacDonald, and Simon Lucey. On the frequency-bias of coordinate-mlps. Advances in Neural Information Processing Systems, 35:796-809, 2022.
Sameera Ramasinghe, Lachlan Ewen MacDonald, Moshiur Farazi, Hemanth Saratchandran, and Simon Lucey. How much does initialization affect generalization? In International Conference on Machine Learning. pp. 28637-28655. PMLR, 2023.
Daniel Rueckert, Luke Sonoda, Carmel Hayes, Derek LG Hill, Martin O Leach, and David J Hawkes. Nonrigid registration using frre-form deformations: application to breast mr images.
IEEE transactions on medical imaging, 18(8)7712-72, 2002.
Vishwanath Saragadam, Daniel LeJeune, Jasper Tan, Guha Balakrishnan, Ashok Veeraraghavan, and Richard G Baraniuk. Wire: Wavelet implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 18507-18516, 2023.
Andreas Schuh, Huaqi Qiu, and Heartflow Research, deepali: Image, point set, and surface registra- tion in pytorch, July 2025. URL / wttp:///11...21///eeeeeee
16420530
Mathieu Semurier, Franck Mamalet, Alberto Gonzále7-Sanz, Thibaut Boissin, Jean-Michel Loubes, and Eustasio del Barrio. Achieving robustness in classification using optimal transport with hinge regularization, 2020.
Vasiliki Sideri-Lampretsa, Julian McGinnis, Huaqi Qiu, Magdalini Paschali, Walter Simson, and Daniel Rueckert. Sinr: Spline-enhanced implicit neural representation for multi-modal registra- tion. Medical Imaging with Deep Learning, pp. 1462-1474, 2024.
Sahil Singla, Surbhi Singla, and Soheil Feizi. Improved deterministic 12 robustness on cifar- 10 and cifar-100. arXiv preprint arXiv:2108. 04062. 2021.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in neural information processing systems, 33:7462-7473, 2020.
Yannick Strümpler, Janis Postels, Ren Yang, Luc Van Gool, and Federico Tombari. Implicit neural representations for image compression. In European Conference on Computer Vision, pp. 74 74-91. Springer, 2022.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keis Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Neural Information Processing Systems, 31, 2018
Ruigang Wang and lan Manchester. Direct parameterization of poaschitz-boundee deep networks. In International Conference Machine Learning, pp. 36093-36118 10. PMLR, 2023a.
15

---

Published as a conference paper at ICLR 2026
Ruigang Wang and Ian R. Manchester. Direct parameterization of lipschitz-bounded deep networks, 2023b.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE transactions on image processing, 13(4):600- 612, 2004.
Jelmer M Wolterink, Jesse C Zwienenberg, and Christoph Brune. Implicit neural representations for deformable image registration. In International Conference on medical imaging with deep learning, pp. 1349-1359, PMLR, 2022.
Haotian Wu, Gongpu Chen, Pier Luigi Dragotti, and Deniz Gündüz. Lotterycodec: Searching the implicit representation in a random network for low-complexity image compression. arXiv preprint arXiv:2507.0 0.24444 2025.
Jamie Wynn and Daniyar Turmukhambetov. Diffusionerf: Regularizing neural radiance fields with denoising diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
41 1180 4189-418,, 2023.
Jiawei Yang, Marco Pavone, and Yue Wang. Freenerf: Improving few-shot neural rendering with free frequency regularization. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 8254-8263, 2023.
Taesun Yeom, Sangyoon Lee, and Jaeho Lee. Fast training of sinusoidal neural fields via scaling initialization. arXiv preprini 1rX1v:2410.04779, 2024.
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas Huang. Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5505-5514, 2018.
Gizem Yüce, Guillermo Ortiz-Jiménez, Beril Besbinar, and Pascal Frossard. A structured dictionary perspective implicit neural representations. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 19228 19238, 2022.
Paul A Yushkevich, John Pluta, Hongzhi Wang, Laura EM Wisse, Sandhitsu Das, and David Wolk. Ic-p-174: fast automatic segmentation of hippocampal subfields and medial temporal lobe subre- gions in tesla and 7 tesla t2-weighted mri. Alzheimer's 's & Dementia, 12:P126-P121 2016.
Roman Zeyde, Michael Elad, and Matan Protter. On single image scale-up using sparse- representations. In International conference on curves and surfaces, pp. 711-730. Springer, 2010.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 586-595, 2018.
Zicheng Zhang, Yinglu Liu, Congying Han, Yingwei Pan, Tiande Guo, and Ting Yao. Transforming radiance field with lipschitz network for photorealistic 3d scene stylization. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pp. 20712-20721, 2023.
16

---

Published as a conference paper at ICLR 2026
A STATEMENTS
Reproducibility Statement: The complete implementation, including data preprocessing, model training, and evaluation scripts, is available at https: httt::://lipchatti inrs. github. We further describe details regarding hyperparameters in Appendix D
Ethics Statement: Our research advances the theoretical foundations of implicit neural represen- tations through novel regularization techniques. While our work focuses on fundamental method- ological contributions without direct harmful applications, we recognize that implicit neural repre- sentations potentially be applied to sensitive domains. We encourage practitioners to carefully evaluate privacy, security, and fairness implications when deploying such models in real-world ap- plications, particularly those involving personal or sensitive data.
Usage of Large Language Models: We utilized large language models as auxiliary tools during various phases of this paper while maintaining full responsibility for validating all outputs. Specif- ically, LLMs assisted with literature search, writing refinement, ideation and brainstorming during the initial development stage and brain-storming storming and were utilized during the implementation of source code and and visualization utilities. All LLM-generated content was thoroughly reviewed and modified to ensure accuracy and appropriateness.
B ADDITIONAL EXPERIMENT: SINGLE-IMAGE SUPER- RESERRRROOONOON
We study the applicability of the oracle estimate and the Lipschitz budget allocation for the task of single-image super-resolution, following the approach and experimental setup presented in Sara- gadam et al. (2023).
Definition: Let us consider single-image super-resolution (SISR) as the task of reconstructing a super-resolved, high-resolution image IHR from the observations of a low-resolution image ILR- Here, the goal is to recover high-frequency details while reconstructing an image that is also consis- tent with I.R-
Given input coordinate (z, y) E St where Q is the continuous 2D image domain, define Q and QHR as the coordinate grids for IL and IHR respectively. Following (Saragadam et al.) (2023), train the INR on the full, high-resolution grid HH and compute the pixel-wise loss between the degraded, downsampled intensities ((f((S (Q)n)) and TLn, where in our experiments, 'D represents a downsampling operator with a scale factor of four.
Budget estimate: In light of super-resolution applications, we would like to discuss two useful oracles.
Conservative estimate: Assuming that a representative set of natural images is not available for deriving signal-based estimate, we can set the conservative. upper Lipschitz bound by considering the worst-case Lipschitz constant. (c.f. section
Signal-based estimate: For the sake of evaluation, let us consider that we have access to both the LR and HR images, and that can measure the estimate directly using proposed signal- based estimate (c.f. section C).
Experiment: For the SISR experiments, we use the Set14 dataset Zeydee et al. 2010) and train several common INR architectures, including SIREN (Sitzmann et al., 2020). FFNs (Tancik et al. 2020), and Gaussian-activated INRs (Ramasinghe & Lucey| (022). using the allocation strategies presented earlier. We evaluate performance using standard metrics: PSNR (t), SSIM (t) (Wang et al. 2004), and LPIPS (+) (Zhang et al. 2018). The quantitative results are shown in Fig. 37 38
41 for spectral norm and Kmin = 0.25, 0.5, 1.0, respectively. The statistical results are shown in
Fig. 36 39 40 respectively. We also provide qualitative examples in Fig. 44
17

---

Published as a conference paper at ICLR 2026
C DATA DRIVEN ORACLE ESTIMATION FOR 2D RGB IMAGES
C.I SIGNAL-BASED ORACLE DERIVATION
Let f C R² + g denote the RGB image-valued function with f(x, y)
(Jn(æ, v), y) (n(r, T With Euclidean norms input and output, the (global) Lips- chitz constant of f is
L inf l 2 0: [l((:)) - f(p2)l2 I Elp1 - Pall P2 EΩ}.
If f is C²,, then
= sup (a.y)e
|, 1)1
J5(0, E ER
where Jy the Jacobian
Sn 8,Sn
Jy
0zJa 0B
0,f0 0yJB_
||/|| mmax(()
Equivalently, if we write
[
g)( (,
b
d with Σ
(aafc), d= [(0gfe) b= E (O,f)))
cE(R.G,B)
then
[5 8)ll2 = (J V (a 412
Discrete estimation on a rectangular grid. Let the image be sampled on a grid {(z,, 9:) with spacings AT and Ay. Using centered finite differences for interior points (with one-sided differ- ences at boundary).
Oxfc(zz.16) fe(x)+1.81) fe(zz-1. fe(ee)---)))
0yfe(15,01) fc(I). fe(Ø,, 91-1)
2 2Ar
2
where Ar (W and Ay s (]-1) for an x W size image assuming coordinate range in [-1, 1]. Define aij. big, di from these discrete partials above and set
2
Va + dij + V(as dzy)² -42%)
Our spectral (Euclidean) Lipschitz estimator then
Fist = max Oij-
i.j
C.2 NOISE SENSITIVITY ANALYSIS FOR SIGNAL-BASED ORACLE
Following the common denoising setup in Saragadam et al. 2023) we evaluate the robustness the signal-based oracle estimator under simulated photon shot noise. We vary the peak photon counts y E {30, 90, 120} with a fixed readout noise of 2, utilizing examples of our inpainting dataset (CelebA).
As shown in the plots, the oracle estimator is only moderately sensitive to noise. Although noise induces slight shift in the gradient distribution, particularly in the tail, leading to increased max- imum estimated Lipschitz constant, the overall distributional structure remains largely preserved. This effect becomes more pronounced high noise levels (e.g., 7 30). In such cases, where noise
18

---

Published as a conference paper at ICLR 2026
Original
Noisy (v= 30)
Original Gradients
Noisy Gradients (y=30)
10:14
10"11
12:19
1010
Original
Nolsy
Original Gradients
Gradients (yw90)
UP
63
Original
Notay 1201 1201
Original Gradients
Noley Gradients (y=1200)
13.1
14:14
10
12
14
Original
Nolsy (v=30)
Original Gradients
Nolsy Gradients (y=30)
13-
150
11
140
20
60
130
340
Original
Notsy (v=90)
Original Gradients
Nolsy Gradients (y=90)
14-11
16
10114
1970
1pm
1pm
10mm
Original
Noisy 1ym120)
Original Gradients
Gradients (ym120)
13
10
12
10
110 238
140
30
308
130
348
Figure 8: Sensitivity Analysis to simulated photon shot noise of various levels. We recommend switching to percentile- based estimates in noisy images instead of utilizing the absolute maximum estimated Lipschitz constant.
artifacts distort the extreme values, percentile-based thresholds provide a robust alternative to rely- ing on the absolute maximum. In particular, find that the 99.9% and 99.99% percentiles reduce the gap between estimates obtained from the original and noisy images.
19

---

Published as a conference paper at ICLR 2026
C.3 CONSERVATIVE ORACLE ESTIMATE:
In applications where access to representative samples is not possible, or the sample-based esti- mates exhibit high variability, recommend considering conservative global upper bound for the Lipschitz constant that can be set from the input sampling resolution and the output amplitude range.
Let fa be the sampling frequency, fN f2/2 the Nyquist frequency, R the peak-to-peak output range, and C' the number of output channels. For a continuous, bandlimited signal (worst-case sinusoid at fN) with amplitude R/2, the derivative magnitude per channel bounded by
ã. [c(1) S 2T INA Rfe
2
Bounding the operator norm of the Jacobian by its Frobenius norm yields the conservative global Lipschitz upper bound:
L VC Rs VCNRSN.
Let R be the output range and 8x, By the grid spacings along and y. For each channel
10.fel S π
10ySel S R 8y
8
Let us define:
(0 S C T2 R2
d (00sco) S C T2 Re sCT
4 22
4 s
Using the norm incquality ||lll2 - || ||1 = Va + d, the global bound becomes:
π R L ~
2
2
y
In the isotropic case, where x-/y- spacing are identical, i.c. S. = Sy = this simplifies to
V2C
L R
2
s
D DETAILS REGARDING EXPERIMENTS
D.1 SHAPES
For all shape experiments, we used the data from the Common 3D Test Model + namely armadillo, beast. bimba, cow, homer, ogre, spot. stanford-bunny suzanne, teapot. Following (Davies et al. 2020), we employ an 8-layer multi-layer perceptron for all experiments with 256 hidden dimensions, and no skip connections. We train all models for 1000 epochs, with learning rate of 5e batch size of 16384, and a combination loss À = 0.05 (Gropp al.) 2020). For 1-Lipschitz neural networks, regularization implemented in PyTorch (Paszke et al. (019). Björck normalized layers implemented in Serrurier al. (2020) and SLL layers from Araujo (2023). All I-Lipschitz activations implemented using Serrurier et al. (2020).
SDF loss À = 1, normal loss X = 0.05, and eikonal the native spectral
20

---

We use the repository of Wolterink et al. 2022 and use the identical setup in terms of dataset and hyperparameters. To have comparable setup, we restrict the registration, similarly Wolterink et al. (2022) to the lung area. Since Wolterink et al.] (2022) do not report the folding ratio, we complement the existing metries computation by implementing the folding ratio within the masked Published as a conference paper at ICLR 2026
D.2 DEFORMABLE REGISTRATION
region of Deepali (Schuh et al. (2025) and erode the deformation field by three voxels based on boundary effects imposed by the masked registration.
D.3 INPAINTING
For the inpainting experiments, we choose 25 random images the CelebA (resolution: 218x178) dataset (Liu al. 2015), and 4-layer MLPs with hidden dimension of 256. We interpret positional encoding and Fourier Features as a non-learnable, fixed first layer, and thus allocate one layer less for all models employing embedding. in comparison to e-g. SIREN. We keep the learning rate I, = le -3 3 and number of epochs Repochs = 5000 constant across all experiments.
D.4 SINGLE IMAGE SUPER-RESOLUTE
For the super-resolution experiments, we select all images from the Set14 super-resolution dataset (Zeyde al., 2010) and use 4-layer ML.Ps with hidden dimension of 256. We interpret positional encoding and Fourier Features as non-learnable, fixed first layer, and thus allocate one layer less for all models employing an embedding, compared to, c.g., SIREN. We keep the learning rate 4, = le 3 and number of epochs Nepochs 5000 constant across all experiments.
D.5 WEIGHT SCALING FROM A LIPSCHITZ LENS
For this experiment, we choose 10 random images of the CelebA (resolution: 218x178) dataset (Liu ct al. 2015), and use a 3-layer SIREN with a hidden dimension of 256. We scale alpha in steps of 0.2 within the interval [1, 4).
21

---

Published as a conference paper at ICLR 2026
E ADDITIONAL DETAILS REGARDING LIPSCHITZ DERIVATION
E. DERIVATION OF LIPSCHITZ IN INDIVIDUAL COMPONENTS
In this section, detail our derivation of Lipschitz constants with respect the results presented in subsectiion 3.1
E.2 POSITIONAL ENCODING
To compute the Lipschitz constant of the positional encoding function 7(),, we must find the supre- mum of the norm of its derivative. Since the derivative's magnitude is constant, the supremum is simply this constant value.
The given positional encoding function RR - R. is defined as:
(p)) (sin(2" πD), cos (p) .p....., sinn Tp), cos(22-1
This function maps a scalar value p to a vector dimension 2L.
The Lipschitz constant of differentiable function is the supremum of the norm of its derivative. We first compute the derivative of each component of the vector 7(p) with respect to For each integer l E (0, 10. L 1}, the derivatives are:
d sin(2 Tp) = -2 cos22 cos(2
dp
cos(2* mp) = -2 sin( mp) dp The derivative of the entire vector function, 7p)), vector composed of these derivatives:
(p) (2" i cos -2" sin(2"TT) xp) 2 π cos(2 1-1 Tp), -2" sin(22-11m)) Tp)
The Lipschitz constant K is given by K supp (p))2, We compute the squared Euclidean norm of the derivative vector (p):
2 + (iin((2p)))
(4 cos((2 )) dp
11)(2)
dp
((2 cos mp)))))))))))))))))))))))))))))) + (- 2 क sin (2* mp))²
Σ
(2 mp) (2s
m)² sin 32n?(2t
k=
Factoring out (2 2² m2 and using cos? sin²(0)=
12)(p)11 Σ (2" (co mp) sin (2 (p)
L-1
1-1
1-1
(2 ( Σ
)
Σ
(2
Σ 4
k=
k=0
The sum Eh-1 4h is a finite geometric series with first term a 1, common ratio terms. The sum is given by the formula St, a d-1
4, and L
L-1
44 -
4h
Σ
4"=1
4-1
3
0=
22

---

Published as a conference paper at ICLR 2026
Substituting this result back into the expression for the squared norm:
|n)())) π
3
The Lipschitz constant Lip(n) = K is the square root of this value. Since the norm is constant, it is equal to its supremum:
Va² 44
K (n((p)
K == 44-1
Now, if we want to specifically set it, we can solve for giving:
In In(3(/mT) 1)
L loga (/(K/)+++))
ln4
E.3 RANDOM FOURIER FEATURES
Similarly, for Random Fourier Feature (RFF) encoding, we consider
T0) [coo(2mb]]) v), simbh v). .... cos(2mb)) v), sin (2nb2 v)) E Rmm
where e E Rd and bj Rd.
The Lipschitz constant K of 7(v) is defined as the supremum of the spectral norm (largest singular value) of its Jacobian matrix (v):
K sup (w)]2 = sup max (mx (J)(
The Jacobian (u) is 2m d matrix. Its entries given by the derivatives:
cos [Tb] v) = - 2n sin (2mb) bJ,
sin(2 v) 2T cos(2 v) b)
Oc
dv
The squared spectral norm is the largest eigenvalue of the Gram matrix (v) (T()).
(b) (v()) = Σ
(- 2n sin(2 e- sim62 22nn u)6,) + (2iT cos(22b) v)by) (2T cos(2 o)65))
1=1
= (2m²) )2
sin (2xb] b)bbb + cos (2mb) o)bbb,]
j=1
(2i²) )2
[sin (2mb, + (2mb) b))b,b]
(2x)
byb
Since this result is independent of v, the Lipschitz constant is also independent of
m
K 111((1))12= = / (L (y())) = 2n
Amax
Σ j=1
b,b
where AA) denotes the largest eigenvalue of the matrix A.
More generally. the encoding uses amplitudes aj. i.e.
(u) (a) cos u), a sin (2mbb v), .... am (os(2 am sin m2
23

---

Published as a conference paper at ICLR 2026
Then the Lipschitz constant generalizes
2n
Amax
ogpb,0] 5=1
If the frequencies are sampled from a Gaussian distribution, by ~ (O, (O), then Σラ agb1b]] a random matrix. The expected Lipschitz constant is then given by:
E
K
2T
Amax
Σ obbbb}
For common RFFs proposed by Tancik al. 2020 expression becomes:
where all amplitudes equal, a, the
m Σ
E [ = 2
Amax
j=1
E.4 LIPSCHITZ BUDGET ALLOCATION
We visualize different budget allocation strategies for the budget KB = 2 as used in the deformable image registration experiment in 5.1
Na 025
0.50
K.. 0.35
K.m 1.00
Cumponent
Component
Component FREER
Componest
Figure 9: Exemplary Lipschitz budget allocation strategy for a budget Kg = 2 for a 9 component (5-layer) network.
F ADDITIONAL REGISTRATION BASELINES
To contextualize the deformable image registration results, we evaluate two additional baselines to benchmark the trade-off between target registration accuracy (TRE) and topological preservation (folding ratio).
While Lipschitz INRs are not specifically designed to avoid folding, but rather to model anatomically plausible transformations by constraining the maximum stretch, they remain particularly effective in these experiments and strike good balance between TRE and the folding ratio. For the folding ratio, they outperform SIREN, with Jacobian regularization, chitz SIRENs remain robust across all cases and do not fluctuate strongly SIREN or NODEO. NODEO, as expected, yields the ffate-of-the-art folding ratio, but results in higher TRE given that its transformations are strictly constrained, posing challenge the learning of the transformation.
out of 10 cases. Importantly, Lips-
24

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of TRE and Folding Ratio for SIREN, Lipschitz SIREN, and NODEO.
SIREN
Lipschitz SIREN
NODEO
Case
TRE (
Folding Ratio (4)
TRE (U)
Folding Ratio (1)
TRE (C)
Folding Ratio (?)
I
0.87 + 0.95
2.43 10
0.78
+ 0.93
2.13 10
1.20 = 1.69
6.61 10
2
0.76 + 0.92
3.46 10-3
0.79 + 0.92
3.54 X 10
2.13 = 3.61
2.94 x 10
3
1.03 + 1.04
4.63 10-3
0.98
+ 1.07
4.84 x 10"3
2.60 = 3.87
1.08 x 10
1.46
4.96
1.42 = 1.33
4.50
4.72 5.14
4.82 x 10
1.22
4.01 10-3
1.58
10-3
4.37
10
1.20 + 1.06
1.93 10-3
1.26
+ 1.08
1.92 x 10
7.95 + 8.43
7.54 10
7
9.39 + 10.48
5.07 10
1.41
1.09
1.80 10
8.57 = 9.52
2.22 x 10
8
1.22
+ 1.26
2.65 10
1.52
+ 1.27
2.45 X 10
12.55 + 11.10
5.63 X 10
9
21.69
15.52
7:21
1.30 + 1.01
9.90 10
4.78 = 4.79
2.63 X 10-6
10
10
Ke 0.575
Kuy 0.725
Kiy 0.883
K.217,877
Björck
20:0.522.
CD:0.117
0.975
ReLU
FullSort
MaxMin
Householder
Figure 10: Learning Spot with different normalization techniques (first row) and different gradient- preserving activation functions combination with a Björck normalized linear layer (second row) demonstrates that approaching the upper Lipschitz bound in SDFs correlates with perceptual quality. We report the CD, Chamfer Distance (L) and the empirically estimated Lipschitz constant Km for all reconstructions.
G ADDITIONAL RESULTS FOR EXPERIMENTS
G.1 SHAPE EXPERIMENTS
G.2 REGISTRATION EXPERIMENTS
We exemplarily visualize our registration setup in 14
We ablate the importance of the lower Lipschitz bounds in the allocation experiments for deformable image registration in Fig. 15
G.3 SCALING WEIGHTS IN SIREN
We provide additional plots for the inpainting experiments in section
25

---

Published as a conference paper at ICLR 2026
13:00
Spectral Norm
CD 3.a
CD 0--
Bjorck
c0:04884
CB 11,56
CD-0.333
SLL
MoxNin
Learning Stanford bunny with different gradient- peeieetttpreevvingg activation functions (columns) in combination with different normalized linear layer (rows) demonstrates that approach- ing the upper Lipschitz bound in SDFs correlates with perceptual quality. report the CD, Cham- fer Distance ()) and the empirically estimated Lipschitz constant Km for all reconstructions.
Figure 11:
Spectral Norm
PullSort
MaxMin
Householder
CD Ne
CD-BA177
10.222
SLL
RALU
PultSort
MaxHiin
Householder
Figure 12: Learning Spot with different gradient- preserving activation functions (columns) in com- bination with different normalized linear layer (rows) demonstrates that approaching the upper Lip- schitz bound in SDFs correlates with perceptual quality. We report the CD, Chamfer Distance (J) and the empirically estimated Lipschitz constant K,, for all reconstructions.
26

---

Published as a conference paper at ICLR 2026
02380
MDD
00303
030023
02.113
GROOK
0:00013
0:000
Figure 13: Visualization of the budget allocation experiment for SDFs using k = 0.85. Results similarly indicate clear trend for improvements with non-uniform allocation strategies.
Figure 14: Visualization of warped deformation field and TRE before/after registration. Notably, Lipschitz INRs naturally enforce anatomically plausible transformations without other forms of aux- iliary regularization, striking good balance between registration accuracy and folding ratio.
TRE
Folding Ratio
0.004
Kmin 0.75
0.003
0.002
Allocation
Allocation
Uni
0.001
Uni
Exp
0.000
ReLU FFN (Spectral)
SIREN MLP (Björck)
SIREN MLP (SLL)
ReLU FFN (Spectral)
SIREN MLP (Björck)
SIREN MLP (SLL)
Figure 15: Visualization of the deformable image registration results of the budget allocation exper-
iments using K Kmin 0.75.
G.4 STATISTICAL SIGNIFICANCE TEST
We perform statistical significance test to determine the effect of different non-uniform allocation strategies against a uniform one. the inpainting experiments, we select distance-to-oracle value at -20, 0, 20 and perform the Wilcoxon Signed rank test between three non-uniform (First, Linear, Exponential) with the uniform one. The results for Spectral Norm, Björck, and SLL normalization presented in Fig. (3 We employ the test for all three metrics, PSNR, SSIM, and LPIPS. We observe that non-uniform strategies consistently yield significant results 0.05) compared uniform strategies.
32 35
28 19 20
23 respectively with different Kmin value.
Similarly, we perform a Wilcoxon test in registration experiments to determine whether non-uniform allocation is statistically significant than uniform. The results are provided in Fig. 16 We observe
27

---

Published as a conference paper at ICLR 2026
Fau 05
Foiding Ratto
Uipooits 11% (Sparioi)
8.0013
0.0010
0,0010
LONcOIe Stren (Speecc))
0.0010
0.0000
0.0020
Liseenna Siren (SUL)
PANGO
Lipocrics TTN (ppcctin)
0.2158
0.4229
0.3542
0.422
Unschit Sizen (Bonece))
0.2558
0.052
6:2159
0.2461
LIBGCHAE (863)
0:0010
6.0000
Figure 16: Heatmap for statistical significance test for registration experiment with Kmin 0.5 and Kwin 0.75. Green indicates that non-uniform allocation is statistically better (p < 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results for both TRE and Folding Ratio in Lipschitz FFN (Spectral) and Lipschitz Siren (Björck) for Kmin 0.5 and in Lipschitz Siren (SLL) for K Kmán 0.75.
1500
1000
1502
1200
Figure 17: Visualization the upper induced Lipschitz bound of the scaling initialization method proposed by Yeom et al. (2024) for SIREN. Scaling the initialization leads to direct scaling of the Lipschitz bounds of the scaled linear layers, which allows the network to increase its capacity to overfit high-frequency content. Please note that following (Yeom al. 2024), only scale the initial and hidden layers, but not the final layer (i.e. layer in this case.)
that, for each architecture, non-uniform strategy with an appropriate Kmin achieves significantly better results than uniform allocation.
We also provide similar statistically significant results on super-resolution experiments in Fig. 36 39 40 for different Kmán respectively. We observe that a non-uniform strategy yields statisti- cally better results in most cases.
G.5 ADDITIONAL INPAINTING RESULTS
Due to space constraints, we provide the remaining results of the budget allocation experiments for the inpainting application here. While FFNs demonstrate limited self-regulation capabilities,
SIREN, for instance, exhibits superior self-regulation properties. We also provide statistical test results for each experiment setup. Further, we provide qualitative examples Fig. 42 and
28

---

Published as a conference paper at ICLR 2026
ReLU
UN
1.0
25.0
0.
0.8
17.5
0.5
6.6
03
6.4
10.0
0.2
100
200
300
400
500
102
200
300
100
100
0
120
200
300
400
500
Dedaace to Drwele
Gauss FFN
UN
1.0
Firtt
LeT
Exp
0.8
0.5
03
CA
12.5
0.
200
1M1
300
150
201
Distance
Oracie
Distance
ReLU PE
0.8
Uni
lint
74
Dyp
22
6.6
0.5
055
25
0.4
0.4
Unl
24
63
200
500
Distance
Distance
Distance
Geuss MLP
Uk
0.55
0.6
0.50
0.25
0.5
P5N0
0.55
0.50
Uni
Tiret
0.55
29
Un
Exp
0.50
02
Sistance acla
Listance
Uistance Uracle
0.7
0.8
28
0.5
1530
5SW
2
6.4
Uni Brst
Un
Ant
Firt
Us
Exp
0.3
Pxp
6.2
100
305
300
102
912
402
520
150
200
300
Dislance bn Dinerie
Thecenre to Diree
Disteeere in Crreele
Figure 18: Budget Allocation for inpainting experiments for SLL with Kmin = 0.25
29

---

Published as a conference paper at ICLR 2026
PSNR
SSM
LPIP5
Lieu
Rests
caenss rrs
Kbril
Livit
Link
Besu PR.
Gaanis MAL
SHREE
0.0.5
Distance Oracle:d
BeLli 9PW
Causs PPP.
Recid
Gourss
Madi
Max
0.45
199
Graas PFN
14/1
05:15
WALD
Gaans MLD
196
SIAEE MLD
Flack
Risz
Exp
Un
Figure 19: Heatmap for statistical significance test for inpainting experiment with SLL normaliza- tion with Kmin 0.25. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results in most cases across PSNR, SSIM, and LPIPS.
PSNR
555M
IPI5
best IYN.
RAxe7
daens rrss.
Bee ye.
Lince
Gaviss saue
SHREN SALe
0.05
Diatance Dracle:d
Ralui PF
Rave
Gaomo PPns
Pecu Pe
Kivc
Kiver
<00.5
Beild 9994
Glesss PPN
Gaons MLP
DINDS MLD
KS/C7
Rover
Fin
D4p
Rnt
Lin
Figure 20: Heatmap for statistical significance test for inpainting experiment with SLL normaliza- tion with K Kmi.. 0.5. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results in most cases across PSNR, SSIM, and LPIPS.
30

---

Published as a conference paper at ICLR 2026
ReLU
UN
Rrst
1.0
25.0
07
688
17.5
0.5
03
CA
10.0
6.2
100
200
300
400
500
102
200
330
100
100
0
120
200
300
400
500
Dedtacee Drwele
Gauss FFN
UN
1.0
Fixtt
Len
077
Exp
0.8
20.0
S3M
0.5
17.
CA
15.0
12.1
0.2
200
150
200
ReLU PE
20
Uni
lint
20
Dyp
25
Unl
0.4
6A
200
Distance
Distance
Geuss MLP
0.30
Uke
0.55
C.8
26
0.75
PSN0
3
0.30
C4
22
Uni
0.50
63
Tiret
0.35
Find
20
Un
Exp
0.50
62
stsnnce acla
Listance Urac
Uistance Uracle
30
35
c3
Un
First
Us
30
0.4
0.2
100
300 Disbance kn Dinarie
305
102
315
402
520
150
200
300
Tlecharre Diree
Disteeeee io Crance
Figure 21: Budget Allocation for inpainting experiments for SLL with Kmin 0.5
31

---

Published as a conference paper at ICLR 2026
ReLU
UN
Drst
1.0
25.0
0.7
6.8
17.5
0.5
03
6.4
10.0
6.2
100
200
300
400
500
102
200
330
100
100
0
120
200
300
400
500
Destance Drwele
Gauss FFN
27.3
1.0
UN
Tixt
25.0
Lan
Exp
22.5
0.8
20.0
S5M
6.6
17.5
15.0
12.1
0.2
200
1M
150
200
ReLU PE
0.8
Uni
24
lint
0.7
22
Dyp
20
0.6
65
Uni
6.4
0.3
200
Distance
Distance
Geuss MLP
Uk
0.30
20
0.75
P5NB
0.70
0.55
0.50
Uni
Uti
Rist
Find
633
20
Um
13p
0.50
62
Distance
Uistance
Uistance Uracke
20
0.7
24
SSM
Uhi First
First
Un
30
Eap
044
100
325
300
102
335
402
520
150
320 Distaeee Crraee
200
Dishance tn Diseree
Diree
Figure 22: Budget Allocation for inpainting experiments for SLL with Kmin 1.0
32

---

Published as a conference paper at ICLR 2026
PSNR
SSI
LPI55
Tlxc3
Rests
caesss rrs
Livit
Besu PR.
Gaanss MAL
SHREE
-.0.05
Distance Oracle:d
BeLli 9PW
Causs PPP.
Recid
Gourss
Maxck
0.45
199
Graas PFN
14/1
05:55
Gaans MLD
196
SIAEE MLD
Flack
Risz
Exp
Un
Figure 23: Heatmap for statistical significance test for inpainting experiment with SLL normaliza- tion with K Kiin 1.0. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results in most cases across PSNR, SSIM, and LPIPS.
PONR
S55M
IPB5
best IYN.
11400
daens rrss.
2
Beeu re.
Gaviss soLe
SHREN SALE
0.05
Distance Draclecd
Ralui PF
Liiit
Line
Gaomo PPns
Pecu Pe
Kiver
<.0.5
Beild 1994
Glesss PPN
1949
17400
Gaons Mip
DIADS MEL
Rind
D4p
Rnt
Lin
Figure 24: Heatmap for statistical significance test for inpainting experiment with Björck normaliza- tion with Kmnn 0.25. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results in most cases across PSNR, SSIM, and LPIPS.
33

---

Published as a conference paper at ICLR 2026
ReLU
28
0.9
Rrd
03
22
PS40
8
6.5
0.6
0.4
0.4
24
300
30D
300
120
302
100
200
300
400
100
100
200
300
400
500
Dotance Dharle
Gauss FFN
Uri
0.8
Fird
Exp
0.5
0.6
200
0.5
173
15.0
8.1
0.4
12.1
X00
430
902
100
200
100
Distance Oracee
Oracie
ReLU PE
26
Uni
Uni
25
limt
0.5
First
0.7
Exp
Evi
0.6
23
22
80
0.5
20
0.5
0.4
24
0.3
200
200
Distance Oracie
Distance
Geuss MLP
0.7
25
2xp
Di
Dep
P5N0
55M1
0.6
0.6
20
0.3
0.2
Listance Draacle
Lissaace
Ciittar Uracle
0.9
27.5
22.5
0.6
20.0
0.4
0.6
123
0.2
300
3CD
420
503
100
300
400
550
100
200
300
500
Dalanre to Dinerie
(Teceanra Diree
Cidterre In Pleerle
Figure 25: Budget Allocation for inpainting experiments for Björck with Kmin 0..5
34

---

Published as a conference paper at ICLR 2026
ReLU
28
03
Rrd
03
22
P540
LR1S
6.5
0.6
0.4
0.4
24
300
30D
300
120
302
100
200
300
400
100
100
200
300
400
500
Ditance Dhwrle
Gauss FFN
UTi
0.8
Find
0.7
Exp
0.6
20.0
0.5
0.6
17.5
15.0
8.3
87
12.1
0.2
200
130
902
100
200
100
Orace
Oracie
RoLU PE
26
Uni
Uni
limt
First
0.T
Eip
Evp
0.7
0.6
23
81
0.5
20
8.5
0.3
200
Distance
Distance
Geuss MLP
0.7
Dep
PSN0
55M
0.7
U.6
Uni
Un
20
Ridt
Epp
0.3
Lin
Birt
Dx
0.2
Listance Draccee
Lisoaace
Gisttrr Uracle
0.9
202
0.8
24
X0
0.5
0.4
35
Un
Firt
0.3
Us
Eag
3CD
3CD
420
502
100
300
400
550
100
200
300
Dadanre to Dinerie
[Tectanra Diner
Cideerre In Clrerle
Figure 26: Budget Allocation for inpainting experiments for Björck with Kmin 0.5
35

---

Published as a conference paper at ICLR 2026
PSNR
SSI
LPIP5
Rests 19N
Caenss rrs
Gaanss MAL
SHREE
-.0.55
Distance Oracle:d
BeLli 9PW
Hlack
Causs PPP.
Recid
Gourss
Maxhk
0.45
199
16/00
05:15
Mhr
Gaans MLP
196
SIAEE MLD
Flack
Riacc
Eep
Un
Figure 27: Heatmap for statistical significance test for inpainting experiment with Björck normaliza- tion with K Kmin 0.5. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results in most cases across PSNR, SSIM, and LPIPS.
PONR
555M
IPI5
nest IYN.
14400
L14C
danns rrss.
Bee re.
Gavis soLe
SHREN SALE
0.05
Distance Draclecd
Ralu PF
Chick
Gaomo PPns
Pecu Pe
Kiver
<00.5
Beild 9914
Glesss PPN
1949
17400
Gaons Mip
DIADS MEL
Kivo
Rind
D4p
Rnt
Lin
Figure 28: Heatmap for statistical significance test for inpainting experiment with Björck normaliza- tion with K Kmi.. 1.0. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results in most cases across PSNR, SSIM, and LPIPS.
36

---

Published as a conference paper at ICLR 2026
ReLU FFN
79
69
Uh
12
2.6
64
2.4
14
52
0.2
100
200
300
100
500
100
200
302
100
300
100
200
300
120
120
Distarre Oharle
Destarmm
Destamee Onwrle
Gauss FFN
Uh
26
1.0
Find
Lan
Exp
8.8
0.5
8.6
10
9.4
202
200
100
RaLU PE
Uni
Drtt
0.7
Exp
5.7
23
8:6
0.5
71
0.4
18
300
Distance
Gauss
20
877
Uk
8.0
0.7
8.5
22
0.6
0.4
Uni
Uti
Un
Teet
0.5
first
Lin
0.3
Exp
Exp
Distance Oracl
Listance Cradl
Dista Gracl
3.8
877
22
0.3
0.5
16
Uhi D1st
L1
11:7
Un
Un
8.3
Us
Eap
Fap
100
.02
350
400
503
100
200
505
500
300
400
Dastatrn Onarle
Deshnree in Cearle
Destamie In Osarle
Figure 29: Budget Allocation for inpainting experiments for Björck with Kmin 1.0
37

---

Published as a conference paper at ICLR 2026
ReLU FFN
UHI
Rrst
03
0.6
20.0
06
0.4
15.0
12.5
022
0.2
300
250
120
902
300
400
100
200
300
Gauss FFN
Uv
Finst
0.5
25.0
DJ
Fieh
22.5
20.0
0.4
0.6
84
Distance Oracle
Distance
0.8
0,7
15
35
0.5
First
20
Unl
Un
lint
0.5
lint
0.3
Lin
28
ba
0.4
Exp
300
200
Distanca Oradi
Distance Orac
Distar de Oracle
0.5
0.F
0.6
1500
0.3
0.5
0.4
20
Uni
0.5
Uti
0.3
Frd
0.2
30
Exp
0.4
Exg
3CD
430
302
300
400
100
200
200
Diblance Dracle
Uistence to Uniec
Gistance to Unacle
SIREN MLP
27.5
0.9
0.9
0.8
Fink
25.0
Pxp
0.6
0.5
0.4
125
300
XCD
300
420
502
100
200
300
400
120
100
200
300
400
500
Dittnce Dracie
Distance to Drace
Cistance to Oracle
Figure 30: Budget Allocation for inpainting experiments for spectral normalization with Kmin 0.25
38

---

Published as a conference paper at ICLR 2026
PSNR
SSI
LPI55
1100
Rests
Flacc
Gaanss MAL
SHREE
-.0.05
Distance Oracle:d
BeLli 99W
Hlack
Causs PPP
poees
Recid
Gounss
Maxr7
Maxck
0.45
199
16/00
03:15
Mhr
Gaano MAP
196
SIAEE MLL
Flack
Eep
Un
Figure 31: Heatmap for statistical significance test for inpainting experiment with spectral nor- malization with Kmin 0.25. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results most cases across PSNR, SSIM, and LPIPS.
PGNR
555M
IPI5
best ITN
11445
Ganns 199.
11.05
1.4m
Rew PR.
Gaves MaLe
SHREK SLL
0.05
Distance Dracledd
Rall PF
ANQ
Gasmo PPn
PecU
14/04
<00.5
Belld 9994
Glssss PPN
Gaons MLP
6044
11/-4
18:45
14/B
D40
Ent
Lin
Figure 32: Heatmap for statistical significance test for inpainting experiment with spectral nor- malization with Kmin 0.5. Green indicates that non-uniform allocation is statistically better (p < 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results most cases across PSNR, SSIM, and LPIPS.
39

---

Published as a conference paper at ICLR 2026
ReLU FFN
28
0.9
25
Rrst
07
0LB
0.8
20
0.5
18
8.4
0.3
0.2
300
250
120
902
300
400
:00
200
300
Gauss FFN
0.
Uv
Findt
5.7
Fieh
22.5
0.6
40.0
0.6
0.4
0.6
84
Distance Oracle
Distance
0.8
0.7
12
35.1
0.5
First
20
lint
Unl
0.3
lint
Un
Lin
28
ba
9-4
Exp
0.2
300
200
Distanca Orade
Distance Oracc
Distarce de to Oracle
0.5
70
0.F
25
0.8
0.6
25
0.7
0.5
PSN0
0.6
Uni
0.5
Uti
30
Exp
0.4
Exg
3CD
430
302
300
100
200
200
Dibtance Dracle
Uistence to Uniec
Cistance to Unacle
SIREN MLP
27.5
0.9
0.9
25.0
0.8
0.7
0.7
Un
hind
0.5
Exp
244
0.3
125
0.1
0
300
XCD
300
420
502
100
200
300
400
100
100
200
300
400
500
Detance Drocie
Distance Orace
Cistance to Oracle
Figure 33: Budget Allocation for inpainting experiments for spectral normalization with Kmin 0.5
40

---

Published as a conference paper at ICLR 2026
ReLU FFN
28
0.9
20
Rrst
07
0.8
0.8
20
0.5
0.6
18
8.4
0.4
0.2
300
250
120
902
300
400
100
200
300
Gauss FFN
Un
Findt
24
5J7
Fieh
0.5
0.6
0.3
12
61
Oracle
Distance
25
0.8
35.1
0.5
First
20
lint
Unl
0.3
lint
Un
Lin
28
ba
9-4
Exp
0.2
300
200
Distance Oradi
Distance Oracc
Distarce de Oracle
20
0.7
0.3
1500
Uni
0.5
Una
0.4
23
Rirt
35
Exp
0.4
Dx
3CD
420
302
300
400
200
200
Dibtance Dracle
Uistence to Unrec
Gistance to Unacle
SIREN MLP
0.9
Uni
0.8
20.0
17.5
Exp
15.0
0.4
first
0.4
125
0.1
300
2CD
300
420
502
100
200
300
400
100
100
200
300
400
500
Dittnce Drocie
Distance Orace
Cistance to Oracle
Figure 34: Budget Allocation for inpainting experiments for spectral normalization with Kmin 1.0
41

---

Published as a conference paper at ICLR 2026
PSNR
SSI
LPI55
1100
Rests
Flacc
14:00
2004
Gaanss MAL
SHREE
-.0.05
Distance Oracle:d
BeLli 99W
Hlack
Causs PP
Recid
Gounss
Maxr7
Maxck
0.45
P99
16/00
05:15
Mhr
Gaans MLP
SIAEE MAL
44/
Flack
Eep
Un
Figure 35: Heatmap for statistical significance test for inpainting experiment with spectral nor- malization with Kmin 1.0. Green indicates that non-uniform allocation is statistically better (p < 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results most cases across PSNR, SSIM, and LPIPS.
PONR
555M
IPI5
best IYN.
12:03
daens rrss.
CANO
1140
2443
Beeu yR.
Gavess saue
SHREN SAL
0.05
Diatance Dracle:d
Ralli PF
Lavel
Call
Li:00
14/00
61/03
Gaono PPns
Pecu Pe
Kird
Crrl
<.0.5
Reild 1994
Glesss PPN
1999
Gaons Mip
KV/O
64/0
65/03
Rind
D4p
Rnt
Uin
Figure 36: Heatmap for statistical significance test for super-resolution experiment with spectral normalization with Kmnn 0.25. Green indicates that non-uniform allocation is statistically better (p < 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results most cases across PSNR, SSIM, and LPIPS.
42

---

Published as a conference paper at ICLR 2026
ReLu FFN
0.8
3.0
20
0.9
24
5.8
06
22
9.7
2.6
8.5
Unn
Uni
18
Firt
0.4
Rist
5.4
Fist
14
5.3
c20
1200
1000
Destance Oracle
Distance
stance Orocie
Gauss
0.9
18
3.7
2.8
4
0.5
0.7
13285
12
551
8.6
20
Un
UN
18
Find
hol
Ln
0.4
Exp
16
200
400
900
1010
1200
407
600
1000
3250
200
400
F00
1003
1200
Distance Oracle
Distance to Oratie
D stance to Oracie
20
0.9
0.8
8.7
SSM
20
0.5
Uni
Uni
125
lint
lint
9.4
Exp
Exp
16
200
400
600
930
1030
1200
200
403
600
B0C
1005
1230
200
400
600
500
2003
1200
Distance To Oracie
Disranca to Oracle
D stance to Oracie
Gauss MLP
5.8
25
3.9
Fist
24
58
0.7
P5NR
0.5
0.5
18
Un
53
0.4
16
Exp
53
20D
.003
650
900
1030
1200
D
200
401
620
190
1000
1100
200
(20
0C0
200
1002
2200
Distance to Onacle
Uistance to Onatle
stance to Dracle
SIREN MLP
0.2
1.0
UN
25
First
5.7
8.0
Exp
0.7
5.6
18
0.5
02
84
8.3
200
.02
600
920
1030
3200
200
403
620
1000
3200
200
400
000
100
1002
1200
Distance Oracle
Distance to
stance Dracle
Figure 37: Budget Allocation for super-resolution experiments for spectral normalization with K Kinn 0.25
43

---

Published as a conference paper at ICLR 2026
ReLu FFN
0.8
UN
1.0
20
Antt
0.9
24
Exp
5.8
06
22
9.7
SEM
0.6
8.5
Uhn
18
Firt
0.4
5.4
Fist
Us
14
5.3
c20
1200
1000
Destance Oracle
Distancc
stance Orocie
Gauss
0.9
18
3.7
2.8
4
0.5
0.7
13285
12
551
20
Un
UN
18
Find
hrl
Ln
0.4
Exp
16
200
400
900
1010
1200
407
600
1000
3250
200
400
F00
1003
1200
Distance Oracle
Distance to Oratie
D stance to Oracie
0.9
24
0.8
22
8.7
SSR
9.5
Uni
Uni
lint
lint
Exp
Exp
16
200
400
600
930
1030
1200
200
403
600
B0C
1005
1230
200
400
600
500
2002
2200
Distance To Oracie
Disranca to Oracie
D stance to Oracie
Gauss MLP
2.8
5.
Fist
24
5.8
0.7
158R
20
2.5
0.6
18
Un
53
9.4
16
Exp
0.3
20D
.003
650
900
1030
1200
0
200
401
620
190
1000
1100
200
(20)
0C0
200
1002
2200
Distance to Oracle
Uistancc to Onatle
stance to Dracle
SIREN MLP
1.0
UN
25
First
3.9
Exp
0.8
5.5
8 0.7
044
9.6
18
03
8.5
53
8.4
84
200
920 Distance Oracle
.02
600
1030
3200
200
403
620
00C
1000
3200
200
400
000
100
1002
1200
Distance
stance Dracle
Figure 38: Budget Allocation for super-resolution experiments for spectral normalization with K Kiin 0.5
44

---

Published as a conference paper at ICLR 2026
PSNR
SSI
LPIP5
Rests
caesss rrs
CAR
CDP
1140
Besu PR.
Gaanis MAL
SHREE
-.0.05
Distance Oracle:d
BeLli 9PW
Causs 19P.
Recud PR:
Gounss
Call
0.45
199
16/00
43/C3
Gaans MAP
19/
SIAEE MLL
Call
Lock
Lock
Eep
Un
Figure 39: Heatmap for statistical significance test for super-resolution experiment with spectral
normalization with K 0.5. Green indicates that non-uniform allocation is statistically better (p 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results most cases across PSNR, SSIM, and LPIPS.
PSNR
S51M
IPB5
nest IYN
Airc
Kard
Lo0
Lid
daens Fre..
Med
CAN
Lewil
Beeu FR.
Gaviss saLe
SHREN SAL
0.05
Diatance Draclecd
Ralli PF
Kir
Call
Livk
Gaomo PPns
Pecu Pe
Livee
Cerl
0.05
Beild 9994
Glesss PPN
Gaons MAP
SIADS MEL
Kavel
14/0
Livoe
Rind
Dep
Rnt
Uin
Figure 40: Heatmap for statistical significance test for super-resolution experiment with spectral normalization with K 1.0. Green indicates that non-uniform allocation is statistically better (p < 0.05) than uniform allocation. We observe that non-uniform strategies offer significant results most cases across PSNR, SSIM, and LPIPS.
45

---

Published as a conference paper at ICLR 2026
ReLu FFN
Un
5.8
U11
3.0
20
R15
0.9
Un
24
Pxp
Exp
5.8
066
22
9.7
0.5
8.5
0.4
5.4
Fist
14
5.3
620
1200
1000
Destance Oracle
Distancc
stance Orocie
Gauss
0.9
25
0.7
0.8
0.6
0.7
0.5
Uni
18
hist
0.4
16
Exp
200
600
R00
1200
401
610
1000
3250
200
400
F00
1003
Distance Oracle
Distance to Oratie
D stance to Oracie
0.8
22
PONR
S5
20
8.0
20
Uni
5.4
lint
Uni
8.5
lint
U1
Exp
9.4
200
400
600
930
1030
1200
200
403
600
B0C
1005
1230
200
400
600
500
1002
2200
Distance To Oracie
Disranci to Oracle
D stance to Oracie
Gauss MLP
24
Uni
Rist
8.9
bxp
0.5
15RR
SSM
3.4
10
Un
83
9.6
21D
.003
650
900
1900
1200
0
200
403
C20
1902
1000
3100
200
(20)
8C0
200
1002
2200
Distance to Oracle
Uistannc to Onatle
stance to Drscle
SIREN MLP
100
UN
3.7
First
24
3.9
5.6
Exp
0.7
8.6
34
82
8.5
5.4
82
200
.02
600
920
1030
3200
200
403
620
D0C
1000
3200
200
400
000
100
1002
1200
Distance Oracle
Distance
stance
Figure 41: Budget Allocation for super-resolution experiments for spectral normalization with Kmin
1.0
46

---

Published as a conference paper at ICLR 2026
Distance--0
Distanzee
Detanca-50
oround FUT!
011101111
Expomental
(a) Spectral Normalization
Oistonte 50
Distancee
Sttance-o
Distacc-220
Ditanco-500
nout mage
RIL
Ground Burt
(b) Björck Normalization
Delamoen-30
aatersam20
Detencen20
RIST
(c) SLL Normalization
Figure 42: Qualitative examples from super-resolution experiments.
47

---

Published as a conference paper at ICLR 2026
Distancee-00
G4sanca
Disanneed
Sesancewa0
Deranca-50
Unilform
1
oround FUT!
2011 01001111
L1A 1000
Expomental
(a) Spectral Normalization
Oktonnee 50
Distancee
Ditancewo
Distnce-22
Ditanco-500
nout mage
Rizz
(b) Björck Normalization
Delanoew-50
Dallanaam2
Detencent0
Rist
Expanendal
(c) SLL. Normalization
Figure 43: Qualitative examples from super-resolution experiments.
48

---

Published as a conference paper at ICLR 2026
Distanco
Distancowo
Distanco-80
Discance=200
Unifom
First
Ground fruth
Linear
Exporeentall
Distance- 50
Distanee 20
Distance00
Distance-B0
Distance=200
Unifom
Image
First
Ground
27.0 WCONND
Linear
Experental
Figure 44: Qualitative examples from super-resolution experiments using spectral normalization.
49


Reviews:
[{"review_id": "dCjn2mIBUZ", "reviewer": "Reviewer_aw4e", "summary": "This paper proposes a K-Lipschitz budget framework for implicit neural representations, deriving task-specific budgets and exploring non-uniform allocation strategies across network components. The work provides theoretical foundations and demonstrates applications in deformable registration, signed distance fields, and inpainting.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Writing is clear.", "weaknesses": "* The paper claims uniform 1-Lipschitz allocation is suboptimal but Figure 3 shows minimal differences between allocation strategies at K=1. Why is non-uniform allocation necessary if uniform allocation of a different budget K could achieve the same effect?\n\n* Results conflate budget value K, allocation strategy, and\n  normalization method, which factor actually drives performance?\n\n* The inpainting oracle estimates Lipschitz constant from finite differences on the discrete image. Why should a continuous neural representation mimic the Lipschitz constant of its discrete sampling?\n\n* The paper positions Lipschitz constraints as addressing \"lack of implicit regularization\" but never compares to actual implicit regularization from optimization dynamics or simply training longer. Does explicit Lipschitz regularization outperform other forms of regularization or architectural constraints?\n\n* Only 5 hand-designed parametric schedules are explored with minimal\n  justification. Given that optimal allocation appears task-dependent.\n  The paper promises \"actionable principles\" but provides no mechanism\n  for deriving allocations for new tasks.\n\n* Experiments use small samples without confidence intervals or significance testing. Claims like \"significantly sharper reconstruction\" rely on visual inspection despite high variance in plots, making it difficult to assess whether observed differences are meaningful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0toIRuKO6e", "reviewer": "Reviewer_m2hg", "summary": "This paper focuses on Lipschitz-constrained implicit neural representations (INRs). The authors propose to estimate a Lipschitz bound K using domain knowledge and explore different ways to distribute K across the components of the underlying neural network. They validate their framework on several tasks, including learning signed distance functions for object representation, image registration, and image inpainting. Overall, the paper builds on relatively simple and nice ideas and presents them clearly. The results are consistent with known trade-offs in Lipschitz-constrained networks.", "questions": "Could you include a few lines on how one should proceed with estimating the K bound? I know it is mentioned for each task, but I want a more general paragraph that tells what domain knowledge to look for and how to process estimating K when facing a new task. Could it be embedded into the training or form an optimization scheme to estimate it? \n\nCould you give an intuition why the budget distribution is not that effective in the SDF example, but it makes a difference for the image inpainting task? \n\nIn Figure 2, why is the second bunny in row one different from the first of row two? Aren't they both with Bjoerick and ReLU? \n\nIn the description of Figure 4 is written \"the Lipschitz-regularized model exhibits significantly reduced folding artifacts while maintaining comparable TRE and the anatomical plausibility of the deformation\"; however, in the Figure, there is no indication of Lipschitz / non-Lipschitz model, which confuses me. Could you clarify? \n\nCould you include visual examples (images) for the image inpainting? \n\nI suggest moving the citations in the conclusion to other parts and trying to summarize your own work there. \n\nI suggest more visual results (for the images) to help understand the effect of the methods.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Overall, the paper is easy to follow. There are extensive numerical experiments on different tasks that investigate the effect of different components. The integration of Lipschitz constraints into INRs seems like a nice line of research to explore.", "weaknesses": "Some parts could be improved (see questions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6t8nOWghId", "reviewer": "Reviewer_apk7", "summary": "This paper reframes Lipschitz regularization for Implicit Neural Representations (INRs) as a Lipschitz budget allocation problem. Rather than enforcing a strict 1-Lipschitz constraint, the authors treat the global Lipschitz constant K_B as a finite budget to be distributed non-uniformly across layers, activations, and embeddings. They derive layer-wise Lipschitz constants for common encodings, propose practical allocation strategies (uniform, linear, exponential, cosine), and demonstrate the approach on signed distance field reconstruction, deformable registration, and image inpainting. Empirically, non-uniform allocation improves the expressiveness-smoothness trade-off, and a data-driven oracle for estimating K_B yields interpretable, task-specific regularization.\nThe theoretical framing is sound: the derivations for per-component Lipschitz constants and the multiplicative composition rule are correct. The oracle-based Lipschitz budget estimation is mathematically consistent, though potentially sensitive to noise. Experiments are well designed and align with the hypotheses, though the product bound on Lipschitz constants is known to be loose, and the oracle lacks robustness analysis.\nThe paper is clearly written and well organized, with strong visual explanations and thorough appendices detailing hyperparameters, derivations, and code. Overall, the work unifies several regularization heuristics under a single “Lipschitz budget” framework. While the underlying tools are known, the reinterpretation and empirical synthesis are novel and practically valuable. The insights on how non-uniform allocation shapes expressivity and how Lipschitz analysis connects to NTK and Fourier perspectives provide novel intuition.", "questions": "1.\tCould the authors provide empirical estimates of the actual Lipschitz constant of trained models and compare them to the theoretical budget?\n2.\tHow sensitive is performance to noise or discretization in the oracle K_B estimation?\n3.\tWould a bilevel optimization approach that learns K_B automatically yield comparable results?\n4.\tFor registration, have the authors compared with diffeomorphic baselines that explicitly constrain Jacobian determinants?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Conceptually elegant reframing of Lipschitz regularization as a quantitative resource-allocation problem.\n2. Derivations for sinusoidal and random Fourier feature encodings connect spectral and geometric views.\n3. Extensive appendices with reproducible setups (SDFs, registration, inpainting).\n4. Empirical results consistent across modalities: tighter Lipschitz enforcement improves fidelity up to a task-dependent optimum.", "weaknesses": "1. The oracle estimator of K_B is a finite difference approximation, which is highly sensitive to discretization and noise. The paper lacks a rigorous analysis of this sensitivity\n2. The product bound on Lipschitz constants is conservative; no tighter or data-dependent alternative has been explored.\n3. The validation is on a small-scale dataset, e.g., Bunny SDFs, 256×256 inpainting, and a single lung registration pair.\n4. No adaptive or learning mechanism for choosing K_B, for example, bi-level optimization \n5. The method does not guarantee a diffeomorphism resulting in folds; a comparison with diffeomorphic baselines would have been helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ssF4dE79kW", "reviewer": "Reviewer_ane9", "summary": "This paper revisits Lipschitz regularization in Implicit Neural Representations and introduces a K-Lipschitz budget framework that generalizes the traditional 1-Lipschitz constraint. The authors argue that uniform Lipschitz constraints (applied equally to all layers and components) are overly rigid and propose distributing a global Lipschitz budget non-uniformly across network components to balance smoothness and expressiveness. Experiments on signed distance fields (SDFs), medical image registration, and image inpainting demonstrate that flexible K-Lipschitz budgets improve stability, interpretability, and generalization.", "questions": "- I have some questions about the fairness of comparison. The authors mention that all models are trained for a fixed number of epochs. However, since different architectures (e.g., SIREN vs. FFN vs. Gaussian-INR) and normalization methods may converge at different rates, a uniform epoch budget could introduce bias — some models might underfit or overfit relative to others. A convergence-based or early-stopping criterion would be a fairer comparison.\n- What is the actual runtime or GPU memory overhead introduced by enforcing non-uniform Lipschitz constraints compared to the standard 1-Lipschitz setup?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The idea of framing Lipschitz regularization as a budget allocation problem is fresh and theoretically well-motivated. It bridges a gap between rigid Lipschitz constraints and flexible, task-adaptive regularization schemes.\n- The paper validates the theory across diverse INR applications—SDFs (geometry), deformable registration (medical imaging), and inpainting (vision)—showing generality.\n- The Lipschitz lens provides a unifying view that complements NTK and Fourier analyses, contributing to a deeper understanding of implicit regularization in INRs.", "weaknesses": "- While the qualitative results are compelling, quantitative improvements (e.g., PSNR/SSIM/Folding Ratio) over strong baselines are small and sometimes marginal, with statistical significance not rigorously reported.\n- I have some concerns regarding the oracle- and domain-driven estimates of K, but the estimation procedures are only sketched conceptually. Appendix references are insufficient for reproducing real-world tasks.\n- Although the paper proposes an interesting conceptual framework (the Lipschitz-budget idea), most figures are purely experimental, focusing on results such as SDF reconstructions, lung deformation fields, and metric plots. The lungs in Fig.1 are quite confusing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)