Understanding Bias Terms in Neural Representations
Authors: Weixiang Zhang
Boxi Li
Shuzhao Xie
Chengwei Ren
Yuan Xue
Zhi Wang
Venue: NeurIPS 2025 poster ()
Keywords: Implicit Neural Representation
INR Classification
Neural Field
Coordinate-based Network
Primary area: applications
TL;DR: In this paper, we examine the impact and significance of bias terms in Implicit Neural Representations (INRs).

Abstract: In this paper, we examine the impact and significance of bias terms in Implicit Neural Representations (INRs). While bias terms are known to enhance nonlinear capacity by shifting activations in typical neural networks, we discover their functionality differs markedly in neural representation networks.
Our analysis reveals that INR performance neither scales with increased number of bias terms nor shows substantial improvement through bias term gradient propagation. We demonstrate that bias terms in INRs primarily serve to eliminate \textit{spatial aliasing} caused by symmetry from both coordinates and activation functions, with input-layer bias terms yielding the most significant benefits.
These findings challenge the conventional practice of implementing full-bias INR architecture. 
We propose using freezing bias terms exclusively in input layers, which consistently outperforms fully biased networks in signal fitting tasks.
Furthermore, we introduce Feature-Biased INRs~(Feat-Bias), which initialize input-layer bias with high-level features extracted from pre-trained models. This feature-biasing approach effectively addresses the limited performance in INR post-processing tasks due to neural parameter uninterpretability, achieving superior accuracy while reducing parameter count and improving reconstruction quality.

Understanding Bias Terms in Neural Representations
Weixiang Zhang, Boxi Li, Shuzhao Xie, Chengwei Ren, Yuan Xue, Zhi Wang' Shenzhen International Graduate School, Tsinghua University zwx wwx.haalth@@gmii. com, 1ibx220mails tssinguua..... edu. wangzhi @sz. tsinghua. edu.
Abstract
In this paper, examine the impact and significance of bias terms in Implicit Neural Representations (INRs). While bias terms are known to enhance nonlin- capacity by shifting activations in typical neural networks, discover their functionality differs markedly in neural representation networks. Our analysis reveals that INR performance neither scales with increased number of bias terms nor shows substantial improvement through bias term gradient propagation. We demonstrate that bias terms in INRs primarily serve eliminate spatial aliasing caused by symmetry from both coordinates and activation functions, with input- layer bias terms yielding the most significant benefits. These findings challenge the conventional practice of implementing full-bias INR architecture. We propose using freezing bias forms fully biased networks in signal fitting tasks. Furthermore, we introduce Feature- Biased INRs (Feat-Bias), which initialize input-layer bias with high-level features extracted from pre- pre-trained models. This feature- feature-biasing approach effec- tively addresses the limited performance in INR post-procees pescessoessing tasks due to neural parameter uninterpretability, achieving superior accuracy while reducing parameter count and improving reconstruction quality. Our code is available this link
exclusively in input layers, which consistently outper-
Introduction
represents novel paradigm in signal repre- sentation, utilizing Multilayer Perceptrons (MLPs) to establish continuous mappings from spatial coordinates (e.g., (I y) for images) their corresponding attributes (e.g., RGB values of pixels). Despite this methodology enables compact and resolution -agnostic representation diverse natural
Implicit Neural Representation (INR)
48
45 30
30 [8], traditional ReLU- based networks suffer from spectral bias (37]. which impedes the fitting of high-frequency signal components during INR encoding. Various approaches have been proposed to address this limitation [48
signals 2 45
3 35
45
42
21.
60) 61, with the SIREN 45 and
its variants (27) emerging as the most widely adopted solutions. These networks implement peri- odic activation functions, such as sin(x) and sin + 1)x). aligning with formulation of Fourier series (3] and achieving state-of-the-are performance in various signal reconstruction tasks. Lever- aging these design principles, INRs have demonstrated significant potential, particularly in medical
imaging [31] 52 51]. data compression I 13 46), and inverse problem solving 30 25 40).
Although periodic activation functions in INRs have demonstrated exceptional interpretability and performance, the role of bias terms (i.c., trainable constants added to the weighted sum of inputs for neuro) in such special networks remains unexplored. In traditional deep learning practices, it acknowledged that bias terms can enhance model performance by enabling activation function shifts [16]. and empirical studies indicate that they significantly enhance classification accuracy, often serving as crucial contributors to final logits [9] Current research in INRs [45 27 17] [3] adheres
this conventional design, incorporating bias terms in each MLP layer. However, upon closer
Conresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

35-95 dB
36.32 dB
34-75 dB
34400
16:34
16.21 dB
full bias terms
w/ input layer
w/ hidden
w/ hidden #3
w/ output layer
w/o bias terms
traditional
proposed
"spatial allasing"
Figure I: Impact of bias terms in INRs. Pink and Ashen in the top row indicate layers with and
without bias terms, respectively. Applying bias terms (without gradient propagation to the input layer achieves optimal performance, while lacking bias terms or only applying them to output layer results in "spatial aliasing" manifested central symmetry in failed reconstructions.
examination of periodic activation functions, the shifting capacity of bias terms should intuitively be significantly diminished due to the boundedness and periodicity sine functions, in contrast to the linear growth of ReLU activation. In response to it, propose critically reassess bias terms in INRs by examining two questions: (1) How significant is the impact of bias terms on INRs? and (2) What unique functions do bias terms serve in INRs?
To answer these questions, we conduct empirical studies and make several findings. Consistent with intuition, bias terms b have significantly smaller effects than weight parameters W, and increasing their magnitude does not consistently enhance performance. However, we unexpectedly find that optimizing bias terms through gradient descent has negligible impact reconstruction quality, Instead, the mere presence of properly distributed bias terms proves enough critical, particularly benefiting the input layer. Based these observations, in INRs is not to shift activation functions as in traditional MLPs, but rather to eliminate spatial aliasing caused by symmetry from both coordinates and activations. Without bias modulation, input coordinates symmetric about the center (c.g., (m, n2) and (-m, -n) would produce identical outputs, leading to failed reconstructions with central symmetry (see the right part of Fig. D.. phenomenon we term spatial aliasing. Since this symmetry issue originates in coordinate space, bias terms in the input layer yield the most significant benefits. Moreover, given that bias terms have limited impact on shifting activation functions in periodically activated INRs, their presence in non-input layers, along with gradient propagation, proves negligible even detrimental to INR encoding.
uncover that the primary role of bias terms
Based on these findings, we challenge the conventional practice of full-bias architecture INR signal fitting, demonstrating that applying frozen bias terms exclusively to input layers consistently outperforms fully biased networks, achieving optimal performance relative to parameter count, Beyond enhancing INR's 's expressive capability, post-processing, i.e., developing neural networks to process INR parameters 62 23 19 15 While this emerging task has gained importance with the proliferation of INR represeeddd data (also known as functa [12). its performance remains limited by the inherent interpretability challenges of neural parameters. For instance, even the recently proposed end-to-end INR classification framework (15) achieves only approximately 60% accuracy on the CIFAR- 10 dataset. Unlike existing approaches 19 or designing task-specific end-to- end frameworks [15], we propose Featuring INRs with Bias Terms Fiaa),Bias,, which directly embeds high-level features from pre-trained encoders (c.g., ViTs [10
analysis reveals an opportunity to improve INR
that focus on parsing intractable neural parameters 62 2
34) into input-layer bias space, maintaining these values throughout INR encoding. This design is motivated by our key findings that bias term gradient propagation does not affect INR reconstruction quality, thus making bias terms ideal candidates for storing well-established features. Through this approach, we achieve substantial performance improvements using only a lightweight MLP as the neural parameter processor, offering an elegant solution that simultaneously enhances reconstruction quality, reduces model parameters, and improves INR post-process processoc performance.
Bias terms with gradient propagation yields nearly identical results (0.00d difference for each case).

---

2 Impact of Bias Terms in INRs
2.1 Intuitive Analysis
Given a typical |layer multilayer perceptron, the output value of the 1-th layer can be formulated as:
(I) where Wi and by denote the weights and bias at layer I, handling linear transformation from the previous layer's output Zt-1, and (-) represents the nonlinear activation function. In common practice, piecewise linear functions such as ReLU are employed as activation functions, owing to their computational efficiency and proven effectiveness across various learning- models. However, such networks demonstrate limited performance when applied to Implicit Neural Represen- tations (INRs), which aims to represent natural signals through coordinate- -based neural networks. Formally, an INR can be defined as Fo x E R'' + E R", mapping an ž-dimensional coordinate to o-dimensional signal y. Given signal set S = [xi Vi)==1 comprising N pairs of coordinates X; and their corresponding signal values y1. the objective is to fit an L-layer MLP (x to the ground truth y with minimal loss.
2 z- f( = oWWiz[1 bi),
In this context, Rel.U-based MLPs often yield suboptimal results due to spectral bias [37 char- acterized by preferential fitting of low-frequency over high-frequency components. To address this limitation, periodically activated representation network 45
27 employ sine-based activa- tion functions, aligning with courier-based signal decomposition 3 and demonstrates superior performance across various signal fitting tasks [6]]. In these architectures, Eq. [ be rewritten as f(2=-) sin (wea' (ioa'(Wizz- (Wiz--1 + by)), where wi controls the network frequency, and a' determines the activation function's periodicity. SIREN 45 sets a² 1 for static period of 2m, while FINER (77 employs of = W by] 1.
Intuitively, the effect of bias terms differs significantly between ReLU and sine activations. In ReLU-based networks, bias terms can provide more direct and substantial shifting of activated values, thus exerting greater influence on the network's 's nonlinear behavior. We can define 0(z b) ox + b) - (1) as an indicator to represent the effect of bias terms on activation function. For ReLU activation, where (xx) max(0, .r), obtain 8(x, b) max(0, + b) max(0, .7.. Its value is bounded by b: [8(z, b) I 1b), achieving this bound under the condition x 2 0nb 2 22 In networks with periodic activation, where a (xx) (x)( we derive:
2 )) sin( 2wa
6(x, b) = 2 cos(wa +
(2)
with 18(x,b) S 2 sin 2wa) (2nn)
< 2. Comparing 5(x, b) between these activations reveals that bias terms provide unbounded control in ReLU networks, whereas in periodically activated networks, their modulation capability is inherently bounded by the sine function, constraining their effect on nonlinear modeling-
Except for unbounded magnitude, bias terms in ReLU activation provide stronger threshold control capabilities, a feature absent in sine activation. With ReLU activation a oz + b) = max(0, x + b). neurons activate (a + b) > 0) when x > -b, where bias 6 directly modulates this activation threshold. Minor adjustments in b can toggle neurons between active and inactive states, establishing an effective gating mechanism. In contrast, bias terms in sine activation merely induce phase shifts rather than implementing hard gating states. Furthermore, the periodic nature of sine functions constrains the effect of bias terms to a 2 period, further diminishing their contribution to the network's expressiveness.
Based on the preceding analysis, we can hypothesize that the shifting effect of bias terms in INRs is significantly diminished. To validate this hypothesis and understand its implications for signal fitting tasks, comprehensive empirical studies are necessary. Therefore, we propose to critically reassess bias terms in INRs by examining two fundamental questions: (1) How significant is the impact of bias terms on INRs? and (2) What unique functions do bias terms serve in INRs? These questions will be investigated in Sec. 2.2 and Sec. 2.3 respectively.
2.2 Empirical Studies
To validate our analysis from Sec. regarding bias terms' impact in INRs, we conduct empirical studies under various bias-related configurations. Following prior work [1 27 61 48), implement
3

---

(1) Linear Weights
(2) Blas Terms
(3)
Training Dynamics for Different Bias Settingss
e/ grad
grad
35
wio gred
35
wio grad
35
full bies
30
30
Aug PSNR
PSWR
W/ e1
25
W/ 43
20
20
w/b blas
15
15
leyers
layers
leres lever
w/b weights
10
nio 1ols
10
1k
2k
3k
4k
5k
heations
Settings
Settings
Figure 2: Left: (1) Comparative analysis of INR performance with varying numbers of weights; Middle: (2) Empirical studies on the impact of bias terms in INRs. Right: (3) Training dynamics
under different bias configurations. Results demonstrate that applying bias terms exclusively to input layers yields optimal performance.
2D image fitting tasks the Div2K dataset [ using 3 x 256 SIREN network (detailed settings and alternative backbones are provided in the appendix).
We first examine whether model performance scales with the number of bias terms. For comparison, we conduct parallel studies with varying configurations of weights Wi across different layers. For weights, we systematically retain different numbers of layers to investigate their effects (Fig. (1)) For bias, we selectively preserve bias in specific layers while eliminating it in others for finer- grained control (Fig (2)).. Results shown in Fig. demonstrate that weights are significantly more crucial for network performance, exhibiting a strong linear relationship between parameter count and reconstruction quality. In contrast, bias terms show minimal impact; even reducing bias terms to approximately 25% of their original number (by retaining only one layer's ' bias) maintains nearly consistent reconstruction quality, supporting our intuitive analysis:
(FigZ-(2)).
(Observation #1) Compared to weights, bias terms exhibit substantially smaller effects on INRs' performance, and increasing their magnitude does not consistently lead to performance improvements.
To investigate bias effects further, conduct experiments by blocking gradient backpropagation. Fig. (1) shows that weight optimization is essential for signal reconstruction, as networks fail without their gradients. In contrast, we observe an unexpected phenomenon for bias terms:
(Observation #2) The gradient optimization of bias has negligible impact on INRs.
Our observations reveal that the impact of bias terms on INRs is even more limited than initially hypothesized, with their gradient updates having minimal effect on reconstruction quality. This suggests structural redundancy in current INR architectures, where computational resources are inef- ficiently allocated to storing and updating largely ineffective bias parameters, However, completely eliminating bias terms restricting them to the output layer significantly degrades performance, indicating that strategic bias placement is more fundamental than optimization. Through compre- hensive analysis of bias configurations, we discovered that bias terms in the input layer achieve optimal performance relative to parameter count. Notably, architectures with frozen input-layer bias consistently outperform fully biased configurations. In summary:
(Observation #3) The existence of bias terms plays a more essential role than their optimization, with neurons in the input layer benefiting most significantly from this property.
2.3 Bias can Eliminate Spatial Aliasing
Although empirical studies in Sec. 2.2 demonstrate that bias terms have minimal yet indispensable effects on INRs, the underlying mechanism remains unclear: What unique functions do bias terms serve in INRs? As revealed Observation #2, the negligible impact of bias term optimization suggests that their conventional role in enhancing nonlinear representational capacity through ac- tivation value shifting, fundamental to Rel_U- networks, no longer applies to INRs. Upon examining reconstruction failures without bias modulation, we observe consistent phenomenon:

---

Figure 3: Visualization for spatial aliasing: reconstructed signals demonstrate central symmetry, manifesting as aliasing artifacts between distinct regions of the input. Green and Pink denote the ground truth and reconstructed signals with spatial aliasing, respectively.
pixel intensities in the reconstructed image exhibit symmetry around the image center, resulting in aliasing effects between different regions of the input image, which we term spatial aliasing. We propose that the primary impact of bias terms in INRs is to climinate this spatial aliasing.
Why does spatial aliasing happen in bias-free INRs? We attribute this phenomenon the combined effect of activation function symmetry and input coordinate space structure. In INR training. it is common practice rescale the coordinate
as: max ()) minxx))))xxx) 2' which normalizes
& to [-1,1]. This normalization has been proven optimal for INR overfitting [45] 60). Under this configuration, coordinates that are symmetric about the image center generate values of equal magnitude but opposite signs, which, in the absence of bias modulation, directly induces spatial aliasing. Specifically, consider the input layer of bias-free INR with centrally symmetric coordinates (m, n) and (-m, -n), where m, n [] (0}. The activated value of the i-th neuron can be formalized
= Sm n.) = (inwwg/aaWWW + w!n)).
This function exhibits central symmetry. satisfying f(-m, = -m(m n), property that extends to all neurons in the bias-free network. While removing symmetry from either the coordinate space or activation function would eliminate spatial aliasing, it would significantly degrade reconstruction
quality compared to the default symmetric configuration (see Sec. 4.3) In summary:
The primary role of bias in periodically activated INRs serves as to eliminate spatial aliasing resulted from the the symmtery of both coordinate space and activation function, with negleciable effect enhance the nonlinear expressiveness of model with activation shifting.
Proposition 1
Discussion.
The distinct role of bias terms in INRs, as established in Proposition 1, aligns with both our intuitive analysis (Sec. 2 and empirical observations(See 2.2). This finding provides additional explanations for the training dynamics under various bias configurations in Fig. (3):: (1) Bias terms achieve maximal effectiveness in the input layer due to direct coordinate processing. While hidden layers can also mitigate spatial aliasing, their efficacy gradually diminishes with increasing depth from the input layer. (2) The output layer's limited neurons (matching the output signal dimension, three for images) provide only marginal improvement over bias-free networks. This minimal bias modulation proves insufficient to fully address spatial aliasing due to the extremely constrained number of bias. (3) More bias terms do not necessarily enhance performance: INRs with bias terms exclusively in the input layer outperform fully- billed configurations. This suggests that while input layer bias effectively climinates spatial aliasing, bias terms in deeper layers provide negligible activation shifting and potentially act maggituie noise, slightly compromising network overfitting capacity.
3 Application: Featuring INRs with Bias Terms
In this section, we introduce Featuring INRs with Bias (Feat-Bias), a method that leverages our analysis of bias terms enhance INR post-processing tasks, which aim develop neural networks for processing INR parameters (e.g., INR classification task) [22 has gained significance with the proliferation performance remains limited due to the inherent interpretability challenges of neural parameters. For instance, even the recently proposed end-to-end INR classification framework ([5) achieves only approximately 60% accuracy on the CIFAR-10 10 dataset. Current methods aiming on INR post- processing broadly fall into two categories: semmetry-equivarinnn parsers for neural parameters and task-speciff end-to-end frameworks. The former focuses designing parameter parsers that
19 15). While this emerging field reprrsesedd data (also known as functa [[2),
23
5

---

Neural Prama. Parser
End-to-End Manner
Feat-Bias (Proposed)
Learned Initialization
blas space of Input layer
down-sampler
Pre-tralned Encoder
Symmetric Neural Parameter Parser
Reconstruction Parameters
Downstream
Downstream NN
End-to-End Framework
Downstream NN
loss
loss
loss
Figure 4: Comparison of mechanisms in INR post-postprocessing. Left: Neural parameter parsers that address the inherent permutation symmetries of neurons. Middle: End-to-end frameworks that integrate INR encoding directly into task-specific networks. Right: Feat-Bias (proposed) that embeds high-level features into input-layer bias initialization, Our method leverages the understanding of bias terms in INRs, addressing neural parameter complexity through a straightforward approach that surpasses existing methods.
consider the inherent permutation symmetries of neurons, while the latter integrates INR encoding directly into task-specific networks, demonstrating rather promising results.
Our Feat-Biat method differs by directly embedding high-level features into neural parameters (specifically bias terms of the input layer) at the initialization step, maintaining these values fixed throughout overfitting. This design motivated by our empirical observations and Proposition since the backpropagation of bias terms in INRs does not affect reconstruction, these values remain constant throughout the INR encoding process. This stability makes bias terms ideal storage spaces for high-level features extracted from well cell-establishe pre- pre-trained encoders (e.g., VITs [10] (34]). By utilizing a lightweight MLP to process these bias values, we achieve excellent performance while cireumventing the challenging process of extracting features from intractable weights.
Specifically, we find that feature vectors extracted from pre-trained encoders approximately follow a uniform distribution. By rescaling these features to < V7l
I (where n denotes the number of input neurons) match default bias initialization, we maintain reconstruction performance while significantly enhancing INRs' feature representation capabilities. Moreover, while INR input layer dimensions can be flexibly adjusted, pre-trained models produce fixed-dimension features (c.g.. 768 dimensions for ViT-Base). This dimensional mismatch can be efficiently resolved through linear interpolation. Due to the high expressiveness of the encoded features, we achieve state-of-the-ard performance in INR post-pree post-processing tasks even with significant dimensionality reduction. While Feat- Bias offers limited insights into permutation -onn---yymmeeeeee neural parameter processing, it demonstrates practical application of our understanding of bias terms in neural representations, providing an efficient solution for INR poot-procsssing.
4 Experiments
4.1 INR Representation Task
Experimental Settings. In this section, we demonstrate how our findings regarding bias terms can enhance INR representation performance, serving as detailed extension of our empirical studies (Sec. 2..2.. Following prior work [4] 27 61 48), we conduct 2D image fitting tasks on the DIV2K dataset M (additional datasets are detailed in supplementary materials). We employ 3x256 MLP with SIREN (45 and FINER [27) architectures, setting the total number of iterations T to 5000. The reconstruction quality is evaluated using PSNR, SSIM (50, and LPIIS (58) metrics. All experiments in this section were performed using the PyTorch framework 36 on NVIDIA RTX 3090 GPU with
24.58 GB VRAM.

---

Quantitative Results. The quantitative results are presented in Tab. [ Building upon our findings in Sec. [22 we challenge the traditional INR architecture that applies bias terms in all layers (Standard) by proposing a model that utilizes bias terms solely in the input layer (Input Bias). We evaluate this against alternative bias configurations: a network without any bias terms (Bias Free), and networks with bias terms only in the first hidden layer (H, Bias) or output layer (Output Bias). Except for the Standard configuration, all variants operate without bias term backpropagation, as gradient propagation through bias terms yields negligible differences (+ 0.01dB for each case). As shown in Tab. [] our proposed Input Bias consistently enhances reconstruction quality across both SIREN and FINER architectures, achieving maximum PSNR improvements of 0.88 dB and 1.13 dB, respectively. The Bias Free and Output Bias configurations exhibit spatial aliasing artifacts due to the absence of appropriate bias terms, confirming our analysis in Sec. 2.3 Furthermore, our mechanism not only (1-1)) improves performance but also reduces the parameter count of each INR by approximately (n=+dn* where denotes the number of hidden layers and n represents the neuron count per hidden layer.
Table 1: Results of 2D image fitting tasks with varying bias configuration
1k Iterations
3k Iterations
5k Iterations SSIM
Settings
PSNR f
SSIM
PSNR1 SSIM
PSNRt
LPIPS
Standard
30.30
0.88120002
34.691007
0.948
36.1910.05
0.959
0.023_2000
Bias Free
13.98
0,556
14.00
0.574_10001
14.03
0.57720000
0.415=0000
H, Bias
29.87
0.870
10:00
34.00 1000
0.938
10.001
35.3410.4
0.950_0001
0.035 L.005_0001
Output Blas R
14.491001
0.562 10.001
14.96 10.01
0.583
10.000
15.13
0.588 10000
0.407 10002
Input Bias
1101
0.902 10.001
35.3210.0
0.953 0.002
36.63 10.01
0.962
0.019 0.0001
Standard
32.25
0.916
36.83
0.962=
38.47
0.971
0.014
Bias Free
13.
0.568 +0,000
14.01 +0.00
0.581
-00003
14.03 +0.01
0.583 +000
0.415 00000
Hi Bias
31.08
0.894
35.23 +0.01
0.949
0.949.0.000 =00000
36.62 +o.uz
0.958
0.026 -0.00
Output Bias @
14.51 14.51 +am
0.573..00.00
14. 98 +oum
0.590 =5.000
15. +
0.594
0.406m..om
Input Bias
33.38
0.932=00001
37.90
0,968_u..m
39.29
0.974=u000
0.010
4.2 INR Classification Task
Experimental Settings. In this section, evaluate the effectiveness of our proposed Feat-Bias (Sec.3) on INR downstream tasks. Following prior work 662 19 15), we focus on INR classification as the primary task. We compare Feat-Bias against -fthe-of-th methods including NFN (62). neural parameter parsers and end-to-end approaches. Due to code availability constraints, results marked with i are quoted from original papers. The downstream network of Feat-Bias consists of a lightweight 3 x 256 MLP classifier, trained for 1000 iterations using a cosine scheduler with a learning rate of le-3. All experiments are repeated three times, reporting both mean and standard deviation. Additional experiments regarding this section can be found in supplementary material.
ScaleGMM 19), Zoo 22). and MWT (15). encompassing both symmetry-equivariee
Dataset of Neural Representations. Beyond standard image datasets, establishing a large-scale dataset of neural representations is crucial. While Implicit-Zoo (9] provides CIFAR-10 in neural representation form, their architecture is incompatible with Feat-Bias, which requires specific bias initialization in the input layer. Therefore,
re-train the CIFAR-10 neural representation dataset using 1 x 64 SIREN network trained for 1000 epochs, maintaining alignment with Implicit- Zoo 229 parameters except for the bias initialization scheme. Our implementation achieves an average reconstruction PSNR of 38dB across the CIFAR-10 10 dataset, with an average training time of 3.11 seconds per image versus Implicit-Zoon reported 10 seconds.
Quantitative Results, Tab. 2 shows the classification results for CIFAR-10 dataset [24], where adopt Implicit-Zoo's 's default configuration with a x 64 SIREN network to ensure consistent experimental conditions across all comparisons. For all baselines except end-to-end MWT [5] and method, directly implement the released checkpoints from Implicit-Zoo as input to the neural parameter parser network. As evidenced by the results, our method achieves superior performance across accuracy, precision, and Fl score metrics, demonstrating Feat-Bias's effectiveness in INR post-processing tasks. While our reported performance is lower than the original MWT paper due to
'Settings without underlines represent standard SIREN implementations, while underlined settings corre- spond to FINER; @ represents reconstructions exhibiting spatial aliasing.
7

---

utilizing smaller network architecture (1 x 64 versus their 3 x 256), it noteworthy that even their state-of-the-an results on CIFAR- classification peaked at 60% accuracy. substantially below the performance of our method. Additionally,, our approach maintains high reconstruction quality without the eccuracy-reconstructiioo trade-off commonly observed end-to-end methods. Furthermore, by implementing lightweight MLP processor, we significantly reduce both computational overhead and parameter count, bringing execution time from minutes (min) to seconds (sec)
Table 2: INR classification on CIFAR-10 Datasets
Classification Task
INRs
Method
Accuracy (%) f
Precision (%)
F1
Time
Params (kilo #)
PSNR
SSIM
(dB) f
NFN
62
24.62=1a3 26.19
23.30
25.02=a0x (min)
544
34,59
0,968
NENNNP
62
25.67
28. 28.10=13
23.89
23.01
(min)
1672
34.59
0.968
ScaleGMMN
19
55.31+004
56,10=1x
55.25
215.86+0.2
(min)
397
34.59
0.968
ScaleGiMN-B
19
56. 14.00
57.67=053
56. 12:079
256.32 (min)
492
34.59
0.968
WT 15
39.3442.13
38.55
38.47
71.75 (min)
261
29.08
0.887
MWTMa
15
13.38
42.90
42.95
79.74 (min)
261
27.53
0.868
MWT [5
46.94.0.37
46,48_050
79.49_24 (min)
261
23.23
0.695
Zoo VIT]
29
80.82+
80.76=037
80.75
~55
34.59
0.968
29
S
80.24.1.47
80.49 00:3
80.44
55k
34.59
0.968
VIT
29
+ LC
81.33
81.29
81.30+0.3
~55
34.59
0.968
Zoo VTT
29
+ LP
+9.51
79.37=034
79.37+0.35
555
34.59
0.968
Vio
LP
11:57
81.53_0030
81.51
~55
34.59
0.968
Feat- -iat-Biasyyii
91. 00.18
91,70=a01
91.68+0.19
64. 14.96411.34 (sec)
151
37.99
0,991
Feat-Bie BiasIDINov3)
93.7840:21
93.80=a12
93.78+0.2
64.35 (sec)
151
38.02
0.991
denotes Woekk reported MWT [5]
4.3 Ablation Study: Additional Verification for Spatial Aliasing
Motivations and Settings. In this section, further investigate the source of spatial aliasing and elucidate why bias terms cannot be entirely climinated from INR architectures. Consistent with the settings in Sec. 4.1 conduct ablation study examining the symmetry of coordinate space and activation functions to explore their impact on the reconstruction quality of INRs. We denote the elimination of coordinate space symmetry as w/o coord. sym., achieved by rescaling coordinates to [0, rather than [-1, 1]. Similarly, w/o act. sym. (+:1) indicates the removal of activation function symmetry by modifying the activation function from (in)waa(-) sin(waa((-) + 1.
Results and Analysis. The results are presented in Tab. 3 where each block reports reconstruction metrics (PSNR/ SSIM / LPIPS). Our experiments demonstrate that spatial aliasing occurs exclusively in bias-free networks that maintain symmetry in both coordinate space and activation functions, providing strong evidence that this source of spatial aliasing is unique to INR architectures. While removing symmetry from either component eliminates spatial aliasing, it significantly compromises reconstruction quality, yielding results substantially inferior to conventional practice. Therefore, em- ploying bias terms solely in the input layer can effectively eliminate spatial aliasing while maintaining the symmetrical properties of INR, thus achieving optimal reconstruction quality.
Table 3: Ablation study for spatial aliasing
Settings
standard
w/o coord. sym.
w/o act. sym. (+11)
w/o act. sym. (-1)
Full-Bias
36.19/ / 0.023
33.09/0.922 0.069
30.86 / 0.892 0.13
30.85 0.891 / 0.134 30.48 0.882 / 0.149
Bias-Free
14.03 0.577 0.415
33.01 0.928 0.073
30.51 0.882/ 0.148
*Due to the unavailability code of Zo0 22 we cannot measure their exact computational metrics: however, their transformer based INR classification network inherently requires more computational resources than our MLP-based approach.

---

5 Related Work
5.1 Implicit Neural Representations
Implicit Neural Representation (INR) [48, 45 30) introduces an advanced approach that employs coordinate-based multilayer perceptrons (MLPs) for multimedia data representation and storage. The spectral bias phenomenon [3], wherein high-fr high-frequency signal components are more challenging to fit during the INR encoding process, has prompted numerous enhancement strategies. These improve- ments span various aspects, including novel activation function designs 45
41
[9 27], accelerated
sampling techniques (59] 22 57 61], meta-learning frameworks
44
7 12 data transformation
methods [43 60), and alternative approaches
42
55
32
These advancements have enhanced
the capacity of INRs to exploit their inherent memory efficiency and natural suitability for inverse
problems, enabling various downstream applications including image enhancement 126 9 G view
synthesis [30 32 54| 53 56). PDE solving [38 8 20). data compression [13] 46, 17 Despite
extensive research in this field, the role of bias terms in INRs remains unexplored. Our work bridges
this gap through comprehensive analysis and leverages these findings to enhance INRs' expressive
capability and downstream performance.
5.2 Neural Networks for Neural Representations
Unlike discrete representations such as pixels, neural representations contain less interpretable parameters, making them challenging for designing network for processing them. Existing methods in this domain fall into two distinct categories: (1) Symmetry-equivariann architectures [33]
62 23 19]. This line of research considers that neural parameters exhibit inherent symmetries, such as permutation symmetries, where neurons can be arbitrarily permuted without affecting their behavior. The goal is to design architectures that are equivariant to these symmetries. NFN [62
proposes Equivariant NF-Layers, which separately address the assumed symmetries of hidden neurons (HNP) and all neurons (NP), demonstrating effectiveness in downstream tasks. NG-GNN (23 represents the input of INRs as a neural graph structure and leverages well-established Graph Neural Networks (GNNs) for processing. ScaleGMN [99 extends the scope of such meta-architecttres from permutation symmetries to scaling symmetries and introduces a corresponding equivariant meessage-psssngg framework, achieving superior performance against other meta-network. (2) End- to-end methods [15]. Recently, MWT [I5] was introduced to address this problem without explicitly considering the equivariant symmetries of INRs. Instead, integrates the encoding process of INRs directly into the task-specific network's ' loop in an end-to-end manner, achieving state-of-the-art performance on the INR classification task. Differences: Unlike these approaches, Feat-Bias infuses established features into input-layer bias space. While offering limited insights into permutation- symmetric processing, achieves significant performance improvements through lightweight approach, leveraging novel findings about bias terms in INRs.
6 Conclusion and Limitations
This paper reveals the previously overlooked role of bias terms in INRs through analytical and empirical studies. Our findings show that, unlike in conventional MLPs, bias terms in INRs contribute minimally to nonlinear representation, with their gradient propagation having negligible impact on performance. Through comprehensive analysis, spatial aliasing caused by coordinate and activation symmetries, with input layer bias terms providing the most significant benefits. Based these insights, propose applying bias terms exclusively to input layers, challenging the conventional full-bias architecture and demonstrating consistent performance improvements. Moerover, we introduce Featuring INRs with Bias Terms (Feat-Bias) which initializes input-layer bias with pre-trained encoder features, bypassing the challenging process of parsing intractable neural parameters. This approach achieves substantial performance gains while reducing computational costs compared to existing INR post-processing methods.
demonstrate that bias terms primarily mitigate
The main limitation of our work lies in the lack of rigorous mathematical proof to theoretically explain the surprising behavior of bias terms in INRs, despite our intuitive analysis and empirical evidence. Future work will focus developing formal theoretical framework for these findings.

---

Acknowledgments and Disclosure of Funding
This work supported in part by National Key Research and Development Project of China (Grant No.
National Natural Science Foundation of China(Grant No. 92467204 and 62472249), and Shenzhen Science and Technology Program (Grant No. (0X1202208181010110000 and KIZD22022493310000000)
References
[1] Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution. Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 126-135 135, 2017.
[2] Hmrishav Bandyopadhyay, Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Tao Xiang. Timothy M. Hospedales, and Yi-Zhe Song. Sketchinr: A first look into sketches as implicit neural representations. CoRR, abs/2403.00344, 2024.
[3] Nuri Benbarka, Timon Höfer, Hamd ul Moqeet Riaz, and Andreas Zell. Seeing implicit neural representations as fourier series. WACV, pages 2283-2292. IEEE, 2022.
[4] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European conference on computer vision, pages 333-350. Springer, 2022.
[5] Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser-Nam Lim, and Abhinav Shrivastava. Nerv: Neural representations for videos. In NeurlPS, pages 21557-21568, 2021.
[6] Xiang Chen, Jinshan Pan, and Jiangxin Dong. Bidirectional multi-scale implicit neural repre- sentations for image deraining. In CVPR, pages 25627-2565 -2636.. IEEE, 2024.
[7] Yinbo Chen and Xiaolong Wang. Transformers meta-learners for implicit neural represen- tations. In ECCV volume 13677 of Lecture Notes in Computer Science, pages 170-187. Springer, 2022.
[8] Junwoo Cho, Seungtae Nam, Hyunmo Yang, Seok-Bae Yun, Youngjoon Hong, and Eunbyung Park. Separable meyysic--nffrree neural networks. In NeurlPS, 2023.
[9] Tomás Chobola, Yu Liu, Hanyi Zhang, Julia A. Schnabel, and Tingying Peng. Fast context- based low-light image enhancement via neural implicit representations. In ECCV(86), volume 15144 of Lecture Notes in Computer Science, pages 413-430. Springer, 2024.
[10] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR. OpenReview.net, 2021.
[11] Emilien Dupont, Adam Goliñski, Milad Alizadeh, Yee Whye Teh, and Arnaud Doucet. Coin: Compression with implicit neural representations. arXiv preprint 103.0:2003.,313,,, 2021.
[12] Emilien Dupont, Hyunjik Kim, S. M. Ali Eslami, Danilo Jimenez Rezende, and Dan Rosenbaum. From data to functa: Your data point is function and you can treat it like one. In ICML, volume 162 of Proceedings of Machine Learning Research, pages 5694-5725. PMLR, 2022.
[13] Emilien Dupont, Hrushikesh Loya, Milad Alizadeh, Adam Golinski, Yee Whye Teh, and Arnaud Doucet. COIN++: neural compression across modalities. Trans. Mach. Learn Res., 2022, 2022.
[14] Kodak.. Kodak dataset, 1999.
[15] Alexander Gielisse and Jan van Gemert. End-to-end implicit neural representations for classifi- cation. CVPR, 2025.
[16] Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT press Cambridge, 2016.
10

---

[17] Jiajun He, Gergely Flamich, Zongyu Guo, and José Miguel Hernández-Lobtto. RECOMBINER: robust and enhanced compression with bayesian implicit neural representations. ICLR. OpenReview.net, 2024.
[18] Langwen Huang and Torsten Hoefler. Compressing multidimensional weather and climate data into neural networks. In ICLR. OpenReview.net, 2023.
[19] Ioannis Kalogeropoulos, Giorgos Bouritsas, and Yannis Panagakis. Scale equivariant graph metanetworks, In NeurIPS, 2024.
[20] Namgyu Kang, Jaemin Oh, Youngjoon Hong, and Eunbyung Park. PIG: physics-informed gaussians as adaptive parametric mesh representations. CoRR, abs/2412.05994, 2024.
[21] Amirhossein Kazerouni, Reza Azad, Alireza Hosseini, Dorit Merhof, and Ulas Bagci. INCODE: implicit neural conditioning with prior knowledge embeddings. In WACV, pages 1287-1296. IEEE, 2024.
[22] Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. Accelerating neural field training via soft mining. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 20071-20080, 2024.
[23] Miltiadis Kofinas, Boris Knyazev, Yan Zhang, Yunlu Chen, Gertjan J. Burghouts, Efstratios Gavves, Cees G. M. Snoek, and David W. Zhang. Graph neural networks for learning equivariant representations of neural networks. In ICLR. OpenReview.net, 2024.
[24] Alex Krizhevsky, Geoffrry Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[25] Zhemin Li, Hongxia Wang. and Deyu Meng. Regularize implicit neural representation by itself. In CVPR, pages 10280-10288. IEÉE, 2023.
[26] Zhiheng Li, Muheng I., Jixuan Fan, Lei Chen, Yansong Tang. Jiwen Lu, and Jie Zhou. Learning dual-level deformable implicit representation for ECCV (69), volume 15127 of Lecture Notes in Computer Science, pages 352-368. Springer, 2024.
scale arbitrary super-resolution. In
[27] Zhen Liu, Hao Zhu, Qi Zhang, Jingde Fu, Weibing Deng, Zhan Ma, Yanwen Guo, and Xun Cao. FINER: flexible spectral-bias tuning in implicit neural representation by variable-periodic activation functions. CoRR, abs/2312.02434, 2023.
[28] Luca De Luigi. Adriano Cardace, Riccardo Spezialetti, Pierluigi Zama Ramirez, Samuele Salti, and Luigi Di Stefano. Deep learning on implicit neural representations of shapes. In ICLR. OpenReview.net, 2023.
[29] Qi Ma, Danda Pani Paudel, Ender Konukoglu, and Luc Van Gool, Implicit zoo: A large-scale dataset of neural implicit functions for 2d images and 3d scenes. In NewrlPS, 2024.
[30] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV (I), volume 12346 of Lecture Notes in Computer Science, pages 405-421 Springer, 2020.
[31] Amirali Molaci, Amirhossein Aminimehr, Armin Tavakoli, Amirhossein Kazerouni, Bobby Azad, Reza Azad, and Dorit Merhof. Implicit neural representation in medical imaging: A comparative survey. In ICCV (Workshops), pages 2373-2383, IEEE, 2023.
[32] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with multiresolution hash encoding. ACM Trans. Graph., 2022.
[33] Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, and Haggai Maron. Equivariant architectures for learning in deep weight spaces. In ICML, volume 202 of Proceed- ings of Machine Learning Research, pages 25790-25816. PMLR, 2023,

---

[34] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo. Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision. Trans. Mach. Learn. Res., 2024, 2024.
[35] Jeong Joon Park, Peter R. Florence, Julian Straub, Richard A. Newcombe, and Steven Lovegrove, Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, pages 165-174. Computer Vision Foundation / IEEE, 2019.
[36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zachary Vito Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, pages 8024-8035, 2019.
[37] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron C. Courville. On the spectral bias of neural networks. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 5301-5310. PMLR, 2019.
[38] Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. Physics-informe neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 378:686-7077 2011
[39] Sameera Ramasinghe and Simon Lucey. Beyond periodicity: Towards a unifying framework ECCV (33), volume 13693 of Lecture Notes in Computer
for activations coordinate-mlpp Science, pages 148. Springer, 2022.
[40] Simone Saitta, Marcello Carioni, Subhadip Mukherjee, Carola-Bibiane Schönlieb, and Alberto Redaelli. Implicit neural representations for unsupervised rupo-tresouttion and denoising of 4d flow MRI. Comput. Methods Programs Biomed., 246: 108057, 2024.
[41] Vishwanath Saragadam, Daniel LeJeune, Jasper Tan, Guha Balakrishnan, Ashok Veeraraghavan, and Richard G. Baraniuk. WIRE: wavelet implicit neural representations. In CVPR, pages 18507-18516. IEEE, 2023.
[42] Vishwanath Saragadam, Jasper Tan, Guha Balakrishnan, Richard G. Baraniuk, and Ashok Veeraraghavan. MINER: multiscale implicit neural representation. In ECCV (23), volume 1 13683 of Lecture Notes in Computer Science, pages 318-333. Springer, 2022.
[43] Junwon Seo, Sangyoon Lee, Kwang In Kim, and Jaeho Lee. In search of a data transformation that accelerates neural field training. In Proceedings ofthe the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[44] Kexuan Shi, Xingyu Zhou, and Shuhang Gu. Improved implicity neural representation with fourier bases reparameterized training. CoRR, abs/2401 .07402, 2024.
[45] Vincent Sitzmann. Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In NewrIPS, 2020.
[46] Yannick Strümpler, Janis Postels, Ren Yang, Lue Van Gool, and Federico Tombari. Implicit neural representations for image compression. In ECCV (26V(26), volume 13686 of Lecture Notes in Computer Science, pages 74-91. Springer, 2022.
[47] Matthew Tancik, Ben Mildenhall, Terrance Wang, Divi Schmidt, Pratul P. Srinivasan, Jonathan Barron, and Ren Ng. Learned initializations for optimizing coordinate- basee neural representa- tions. In CVPR, pages 2846-2851 Computer Vision Foundation / JEEE, 2021.
[48] Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In NewrIPS, 2020.
12

---

[49] Shengjie Wang, Tianyi Zhou, and Jeff A. Bilmes. Bias also matters: Bias attribution for deep neural network explanation. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 6659-6667. PMLR, R, 2019.
[50] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600- 612, 2004.
[51] David Wiesner, Julian Suk, Sven Dummer, David Svoboda, and Jelmer M. Wolterink. Implicit neural representations for generative modeling of living cell shapes. In MICCAI (4), volume 13434 of Lecture Notes in Computer Science, pages 58-67. Springer. 2022.
[52] Jelmer M. Wolterink, Jesse C. Zwienenberg, and Christoph Brune. Implicit neural represen- tations for deformable image registration. In MIDL, volume 172 of Proceedings of Machine Learning Research, pages 1349-1355 PMLR, 2022.
[53] Shuzhao Xie, Jiahang Liu, Weixiang Zhang, Shijia Ge, Sicheng Pan, Chen Tang, Yunpeng Bai, Cong Zhang, Xiaoyi Fan, and Zhi Wang. Sizegs: Size-aware compression of 3d gaussian splatting via mixed integer programming. ACM MM, 2025.
[54] Shuzhao Xie, Weixiang Zhang, Chen Tang, Yunpeng Bai, Rongwei Lu, Shijia Ge, and Zhi Wang. Mesongs: Post-training compression of 3d gaussians via efficient attribute transformation. In European Conference on Computer Vision. Springer, 2024.
[55] Runzhao Yang. TINC: tree-structured implicit neural compression. In CVPR, pages 18517-
18526. IEFF, 2023.
[56] Wei Yao, Shuzhao Xie, Letian Li, Weixiang Zhang, Zhixin Lai, Shiqi Dai, Ke Zhang, and Zhi Wang. Sd-gs: Structured deformable 3d gaussians for efficient dynamic scene reconstruction, 2025.
[57] Chen Zhang, Steven Tin Sui Luo, Jason Chun Lok Li, Yik-Chung Wu, and Ngai Wong. Non- parametric teaching of implicit neural representations. In Forty-first International Conference on Machine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024.
[58] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metric. In CVPR, pages 586-595. Computer Vision Foundation IEEE Computer Society, 2018
[59] Weixiang Zhang, Shuzhao Xie, Shijia Ge, Wei Yao, Chen Tang, and Zhi Wang. Expansive supervision for neural radiance field. arXiv preprint arXiv:2409.0800 2024.
[60] Weixiang Zhang, Shuzhao Xie, Chengwei Ren, Shijia Ge. Mingzi Wang, and Zhi Wang. Enhaneing implicit neural representations via symmetric power transformation. AAAI, pages 10157-10165. AAAI Press, 2025.
[61] Weixiang Zhang, Shuzhao Xie, Chengwei Ren. Siyi Xie, Chen Tang, Shijia Ge, Mingzi Wang, and Zhi Wang. Evos: Efficient implicit neural training via evolutionary selector. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2025.
[62] Allan Zhou, Kaien Yang, Kaylee Burns Adriano Cardace Yiding Jiang, Samuel Sokota, J. Zico Kolter, and Chelsea Finn. Permutation equivariant neural functionals. In NeurlPS, 2023.
13

---

A Additional Verification for Empirical Studies
(.2c and INR repre- sentation experiments (Secf4.I). We conduct experiments the Kodak dataset [14, maintaining all configurations identical to those described in Sec. 4.1 The quantitative results are presented in Tab. The configurations Standard, Bias Free, H, Bias, Output Bias, and Input Bias align with those in Tab. Additionally, we include H2 Bias and H3 Bias, representing networks with bias terms exclusively the second and third layers, respectively. Settings without underlines represent standard SIREN implementations, while underlined settings correspond to FINER. The symbol Ø indicates reconstructions exhibiting spatial aliasing. For each configuration block, we report averaged metries both with and without bias term backpropagation. Complementary results on the DIV2K dataset are presented in Tab. E supplementing the findings in Tab. [ The results in Tab. 5 and Tab. 4 further validate our observations from Sec. 2.2 and conclusions from Sec. 4.1] These findings substantiate both the magniudee of bias terms' impact in INRs and the effects of their gradient optimization, lending additional support to Proposition 1. This reinforces our claim that the primary role of bias terms in neural representations to eliminate spatial aliasing arising from the symmetry of coordinate space and activation functions.
In this section, we provide additional verification of our empirical studies (Sec 2.2
Table 4: Results of 2D image fitting tasks with varying bias configurations (Kodak dataset)
Ik Iterations
3k Iterations
5k Iterations
Settings
PSNR
SSIM
PSNR
SSIM
PSNR1
SSIM
Standard
30.05 / 30.05
0.832/ / 0.832
33.93 / 33,92
0.912 0.912
35.24/ 35.23
0.929 / 0.929
Bias Free
15.06 15.06
0.599 / 0.599
15.09 / 15.09
0.620 / 0.620
15.10/ 15.10
0.626 / 0.626
H, Bias
29.67 / 29.68
0.822 / 0.822
33.25 / 33.25
0.902 / 0,902
34,44
34.45
0.919
H: Bias
29.96 29.96
0.831 0.831
33.33 / 33.32
0.903 /
34.41 / 34.42
0.918 / 0,918
H; Bias
30.07 / 30.05
0.829 / 0,829
33.17 / 33.16
0.895 0.89
34.18/33.11
0.910 0,910
Output Bias
16.34/ 16.04
0.613 / 0.610
17.03 / 16.78
0.641 0.66
17.31/17.0
0.648 / 0.646
Input Bios
30.78 / 30.78
0.854 0.854
34.31 34.30
0.917
35.42/35.42
0.930 / 0.930
Standard
32,05 / 32.06
0.880/ / 0.881
36.18 / 36.18
0.940 / 0.940
37.52 / 37.53
0.952 / 0.952
Bias F
15.09 / 15.09
0.617 / 0.617
15.11 / 15.11
0.632 / 0.632
15.11 15.11
0.636 / 0.636
Hi Bias
30.83 / 30.82
0.853 / 0.853
34.46 / 34.45
0.921 / 0.921
35.82 / 35.81
0.935 / 0.935
H2 Bias
31.28 / 31.28
0.866 / 0.866
34.74 / 34.75
0.924 / 0.924
35.66/ 35.67
0.934 / 0.935
Hs Bias
31.37 / 31.36
0.865 / 0.864
34.60 / 34.60
0.919 / 0.919
35.60 / 35.59
0.931 / 0.931
Output Bias
16.19 / 16.07
0.628 0.627
16.82 / 16.79
0.652/0.65
17.10 17.04
0.655 / 0.657
Input Bias
/3.06/33.05
36.81 36.80
0.948 0.950
37.92/3.991
0
Table 5: Results of 2D image fitting tasks with varying bias configurations (DIV2K dataset)
1k Iterations
3k Iterations
Sk Iterations
Settings
PSNR
SSIM
PSNR1
SSIM
PSNR
SSIM
Standard
30.30/ / 30.30
0.882/ / 0.882
34,68 / 34.69
0.946 / 0.946
36.17 / 36.18
0.956 / 0,959
Bias Free
13.98 / 13.98
0.556/ / 0.556
14,00 / 14.00
0.574 / 0.574
14.03 / 14.03
0.577 / 0.577
H, Bias
29.88/ / 29.87
0.872 / 0.872
34.02 / 34.00
0.938 / 0.938
35.33 / 35.34
0.950/ / 0.950
H2 Bias
30.14 / 30.13
0.875 / 0.875
33.98 / 33.96
0.935 / 0.935
35.20/ 35.19
0.946 / 0.946
Hy Bias
30.18 / 30.14
0.871 / 0.869
33.65 / 33.63
0.927 / 0.926
34.77 / 34.76
0.934 / 0.933
Output Bias 2
14.67 14.49
0.567 0.562
15.11 14.96
0.593 0.583
15.11/ 15.13
0.586 0.588
Input Bias
31.20/31.18
0.903 0.902
35.33 35.32
/
36.61 36.63
0.962 0.962
Standard
32.26 / 32.25
0.916/ / 0.916
36,84 / 36.83
0.962 / 0.962
38.47 / 38.47
0.971 / 0,971
Bias Free
13,99 / 13.99
0.568 / 0.568
14.01 / 14.01
0.581 0.581
14.03 / 14.03
0.583 / 0.583
Hi Bias
31.06 31.08
/
35.21 / 35.23
0.949
36.63 / 36.62
0.958 / 0.958
Ha Bias
31.49 31.49
0.901 0.901
35.28 / 35.24
0.948 0.947
36.62/ 36.60
0.957 0.957
Hs Bias
0.901 0.899
35.21 / 35.18
/.946/0.944
36.32/36.34
0.952 0.951
Output Bias c
14.55 14.51
0.577 0.573
15.13 14.98
0.599 0.590
15.22 15.14
0.595 / 0.594
Input Bias
33.39/33.00
0.933/ 0.932
37.95 37.90
0.968 / 0.968
39.30 39.29
0.974
14

---

B Additional Experiments for Feat-Bias
In this section, we present additional experiments validate the effectiveness Feat-Bias (Sec
Following prior work 62
19 15 we evaluate our method on INR classification tasks using MNIST
and F-MNIST datasets. In addition the baselines discussed Sec[D] we include comparisons
with recent methods including Inr2Vec[28) DWS [33,, and NG-GNN [23] Beat-Biasywir and
Feat-BiasppnNova] represent our method using features extracted from ViT [10] and DINOv2 [34), respectively. We maintain identical configurations as described in Sec. D employing a 256 MI.P downstream network, training for 1000 iterations with cosine scheduler and learning rate of 1e-3. The quantitative results are presented in Tab. 6 Our Feat-Biase piatoov achieves the highest classification accuracy on both datasets, surpassing the recent state-of-the-ee MWT-L. Notably, on F-MNIST, method improves accuracy from 87.32% to 92.38%, demonstrating significant advancement in INR downstream tasks.
Table 6: INR classification on MNIST and Bushion-MNISS Datasets
Method
MNIST
F-MNIST
Method
MNIST
F-MNIST
MLP
17.5
19.91=0.477
Inr2Vec 28
23.69::.110
22.33:10.41
NFN 62
78.50=:.23
68.19:0.28
NFNNNP 662
79.11
68.94110.44
DWS 33
85.71
67.06=10.29
NG-GNN 23
91.40110.60
68.00110.20
ScalcGMN
96.57=0.10
80.46+0.32
ScaleGMN-B
19
96.59++0.4
00.16
WT [15
93.08+22.26
13.81=1.443
MWTxaa Teck
15
95.57++0.0
07.23
MWT [5]
96.58.10.32
83.86.10.91
MWT-L
98.33.10.13
87.32.10.16
Feat- Biasyar
95.79
90.13=0.49
Feat- Biasppnnoa
98.48:00.55
92.38+00066
C Additional Experiments for Ablation Studies
We present additional ablation studies to further verify the source of spatial aliasing. All experimental settings strictly align with those in Sec. 4..3 Tab preesentt ablation result for spatial aliasing on Kodak dataset, demonstrating consistency with the findings presented in Tab.
Table 7: Ablation study for spatial aliasing (Kodak Dataset)
Settings
standard
w/o coord. sym.
w/o act. sym. (+1)
w/o act. sym. (-1)
Full-Bias
35.23 0.922 0.089
32.23 0.880/ 0.191 32.17 0.877 0.200
30.38 0.838 0,257
30.44 0.840 / 0.254
Bias-Free
15.10 0.626 0.461
30.11 0.832 0.258
30.11 0.8332 0.259
15

---

D Additional Experiments for classification task
Experimental Settings. In this section, we further evaluate the effectiveness of our proposed Feat-Bias (Sec. (3 on INR downstream tasks. The downstream network of Feat-Bias consists of lightweight 256 MLP classifier. Since the CIFAR-I01 dataset is more complex than CIFAR-10, the number of iterations is changed to 5000, using cosine scheduler with learning rate of 1c-3. In addition, iteration count MWT has also been modified to 20 epochs. tab we list performance metrics at 10 epochs (i.e., the original default setting) and 20 epochs. All experiments are repeated three times, reporting both mean and standard deviation.
Quantitative Results. Tab. 8 shows the classification results for CIFAR-100 dataset [24]. where we also adopt Implicit-Zai -oo'ss 's default configuration with 1 64 SIREN network like the settings in the main text to ensure consistent experimental conditions across all comparisons. Since Implicitly Zoo does not provide the INR dataset CIFAR-100, only MWT compared with our method here. As evidenced by the results, our method achieves superior performance across accuracy, precision, and F1 score metrics meanwhile greatly reduces the training furthfr demonstrating effectiveness in INR post-processing tasks.
Table 8: INR classification CIFAR- 100 Datasets
Classification Task
INRs
Method
Accuracy
Precision ( f
F1
Time
Params
PSNR
SSIM
(kilo
(dB) 1
12.21 1nso
10.90_asz
10.09 2087
73.52
(min)
32.33
0.942
WTEeepocts
15.83
13.82_08
13.84
151.78 12m (min)
34.89
0.964
15
15.81 =0.44
14.94=0a6
13.86 +0.74
83.41 (min)
30.17
0.910
MWT Tisk Meppchhs
15
19.87 1021
17.88 10.1
18.06
169.42_2яı
(min)
33.06
0.948
pospocis
19.46:0as
18.09 +a.s
17.67
83.74 +0,97 (min)
24.45
0.752
MWT megonts
23.29 gust
21.56
21.78gn05
11.00 (min)
25.80
0.806
Feat- Bias-Bia
74.83
75.03
74.79.0m
144.70 :4 1 (sec)
151
Feat- Bita-Biisssnoova
28.51
78.71
78.46 2:015
145.38 1: 19 (sec)
weakk MWT
16

---

E Additional visualizations for spatial aliasing
Figure 5: Visualization for spatial aliasing: reconstructed signals demonstrate central symmetry, manifesting as aliasing artifacts between distinct regions of the input. Green and Pink denote the ground truth and reconstructed signals with spatial aliasing, respectively.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction have effectively reflected our contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed our limitations in the final part of the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

We have established sufficient assumptions for all theoretical result in the paper.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provide details to reproduce the main experimental results the section of Experimental Settings. We will also release the codes and checkpoints.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We will provide open access to the data and code the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccipps.c/ppulli//guu /ttoe//uuuddeeoooouuuubiiiiiii ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have specified all the training and test details in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We have reported error bar in the experiment sections.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
20

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: Yes]
Justification:
We have provided sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpp:////aaccr....ppoppp/////aa
Answer: [Yes]
Justification:
The research conformed with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The work performed demonstrates no significant societal impact. Currently,
there are evident examples suggesting that INR might lead to potential negative societal consequences.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
23

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
24


Reviews:
[{"review_id": "ZezLfIq23h", "reviewer": "Reviewer_x6DW", "summary": "- The authors propose Feat-Bias which initializes input-layer bias terms of an INR network with features extracted from pretrained models and keeps them frozen during training.\n- Feat-Bias results in improved INR classification performance.\n- The authors provide an explanation for why the bias terms can be replaced and how that can result in improved performance", "questions": "- Why do the authors think that the transferred bias terms result in improved performance? Could it be regularization?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The authors provide a clear explanation of how standard bias term training does not improve performance, that the role of bias terms is to prevent spatial aliasing, and that using bias terms in only the input layer is sufficient.\n- This is a simple, easy to implement idea that has a performance benefit.\nWeaknesses:\n- It is not clear to me that knowledge is really being distilled from the pretrained network into the INR bias weights rather than that the pretrained weight initialization has some kind of regularization effect. The fact that significant dimensionality reduction is needed to squeeze the pretrained weights into the INR weights suggests to me that it might be mostly regularization. I would like to have seen a baseline where a distribution was fit to the pretrained weights and then the INR bias weights were randomly sampled from that distribution.", "was_revised": true, "final_justification": "The authors have answered my questions and I have decided to maintain my score of accept.", "rebuttal": "We thank the reviewer for your positive feedback and insightful comments. We are delighted by the reviewer's appreciation of the originality, clarity, and effectiveness of our work. Herein, we address the raised concerns as follows:\n\n- **The Reason Why Transferred Bias Terms Result in Improved Performance**. As explained in lines 202 to 220 (Sec. 3), we believe that the Feat-bias can significantly improve INR classification performance because the fixed bias terms serve as ideal storage spaces for high-level features extracted from well-established pre-trained encoders (e.g., ViT or DINOv2). These bias terms act as a kind of distillation, transferring high-level features from large pre-trained networks to smaller input-layer bias spaces. In this manner, each INR can benefit from the representation capabilities of a large pre-trained network, leading to significant improvements when implementing additional networks for downstream INR tasks.\n- **Regularization Effect.** This is a very inspiring assumption. However, we do not think the performance boost comes from dimensionality reduction regularization. The dimension reduction in Feat-Bias can be seen as an operation to address the dimensional mismatch between features from pre-trained encoders and the flexible dimension settings of INRs. For instance, the decoded feature from ViT-base has 768 dimensions, but the number of first-layer biases is determined by the INR size. If the setting is 3x64, we only have 64 dimensions to represent the pre-trained features. In this case, dimension reduction is necessary. Such a dimensionality reduction operation results in some information loss from the pre-trained features, leading to a slight decrease in performance, which can be verified by the following experiment conducted on CIFAR-10:\n    \n    \n    |  | Accuracy | Precision | F1 |\n    | --- | --- | --- | --- |\n    | MWT (best) [2] | 46.94 | 46.48 | 46.51 |\n    | Scale-GMN-B (best) [1] | 56.14 | 57.67 | 56.12 |\n    | Zoo (best) [3] | 81.57 | 81.53 | 81.51 |\n    | Feat-Bias [768]  | 96.62 / 97.73 | 96.63 / 97.73 | 96.62 / 97.73 |\n    | Feat-Bias [768 → 256]  | 95.99 / 97.05 | 96.01 / 97.06 | 95.99 / 97.05 |\n    | Feat-Bias [768 → 64]  | 91.68 / 93.78 | 91.70 / 93.38 | 91.68 / 93.78 |\n    | Feat-Bias [768 → 32] | 85.98 / 89.44 | 85.94 / 89.44 | 85.94 / 89.44 |\n    \n    The experimental settings in this table are strictly aligned with those in Tab. 2 of our paper (lines 245 to 253). The reported results in the row of Feat-bias is with the format of (ViT/DinoV2). As shown, the 768-dimensional configuration demonstrates the best classification performance, indicating that dimensionality reduction negatively affects INR post-processing tasks due to the information loss from the pre-trained encoders. However, we found that by selecting a simple yet appropriate dimensionality reduction method (in our case, linear interpolation), the performance loss remains acceptable, far surpassing existing INR post-processing methods. This trade-off proves to be worthwhile, as it reduces the size of the INR architecture while preserving the memory efficiency advantages inherent to INR.\n    \n- **Additional Experiments for Feature Sampling.** We have conducted the sampling-based dimensionality reduction method you proposed and compared it with others. First, directly sampling features from the fittest distribution would break the inherent permutation of encoded features, resulting in suboptimal performance. This is quite similar to random sampling ( $x \\sim U(-\\frac{1}{\\sqrt{n}}, \\frac{1}{\\sqrt{n}})$), completely losing the prior of the pre-trained encoder. To address this problem, we maintain an additional map to save the permutation of the original features and permute the sampled bias terms according to this map. The experimental results are as follows：\n    \n    \n    |  | Random | ~ Fittest Dist. | Max pooling | Avg Pooling | Linear (Ours) | ~ Fittest Dist. + Re-permute |\n    | --- | --- | --- | --- | --- | --- | --- |\n    | ViT | 0.1066 | 0.1329 | 0.8288 | 0.8852 | 0.9168 | 0.9027 |\n    | DinoV2 | 0.1121 | 0.1297 | 0.7728 | 0.9185 | 0.9378 | 0.9193 |\n    \n    All experiments were conducted under the same settings, aiming to reduce the dimension from 768 to 64 (aligned with the INR classification benchmark in Table 2 of the paper). We tested both ViT and DinoV2 features respectively. The reported values represent classification accuracy on the CIFAR-10 dataset. Among the baselines, *Linear* represents the common linear interpolation method we implemented, and *~ Fittest Dist. + Re-permute* is our re-implementation for feature sampling, as described above. The results show that *~ Fittest Dist. + Re-permute* achieves comparable performance but slightly lags behind our method, while outperforming other baselines. We greatly appreciate the reviewer's academic acumen and will incorporate this baseline into the experimental section of the final manuscript.\n    \n\n[1] Scale equivariant graph metanetworks. In NeurIPS, 2024.\n\n[2] End-to-end implicit neural representations for classification. CVPR, 2025.\n\n[3] Implicit zoo: A large-scale dataset of neural implicit functions for 2d images and 3d scenes. In NeurIPS, 2024."}, {"review_id": "IoBcHDqGyY", "reviewer": "Reviewer_sknb", "summary": "This paper investigates the infulence of the bias terms in INRs through both theoretical analysis and empirical validation. The authors demonstrate that bias terms in INRs primarily serve to eliminate spatial aliasing caused by symmetry from both coordinates and activation functions. To further address this issue, the paper introduces Feat-Bias, which initialize input-layer bias with high-level features extracted from pre-trained models.\n\nExtensive experiments are performed on a representation task to validate the effectiveness of bias terms, and on a classification task using the CIFAR-10 dataset to demonstrate the performance of Feat-Bias.", "questions": "1. The classification experiment is conducted only on CIFAR-10, which may not sufficiently demonstrate the generalization ability of Feat-bias.\n\n2. Could the authors offer a more formal mathematical characterization of how bias terms influence coordinate symmetries and contribute to the suppression of spatial aliasing?\n\n3. Ablation experiment is limited.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. Analyzing the role of bias terms in INRs from the perspective of spatial aliasing caused by symmetry is both novel and well-motivated.\n\n2. Extensive experiments are carefully designed to validate the effect of bias terms.\n\n**Weaknesses**\n\n1. While the theoretical analysis is insightful, some of the claims would benefit from more formal proofs or clearer mathematical exposition.\n\n2. The classification results are promising, but currently limited to CIFAR-10. Additional datasets would help assess the generality and robustness of Feat-bias in broader settings.", "was_revised": true, "final_justification": "The author followed our advice and conducted experiments on more datasets and provided additional mathematical formulas to illustrate his theory. Therefore, I decided to improve the final score.", "rebuttal": "We express our gratitude to the reviewer for the insightful feedback and suggestions provided. We are delighted that our work has been acknowledged as innovative, well-motivated, and rigorously tested. In response to your valuable comments, we offer the following clarifications:\n\n- **Additional Experiments for INR Classification**\n    - **Supplementary Material Inclusion**. Due to space constraints, we have included the INR classification results for CIFAR-10 in the main text. Moreover, we have conducted INR classification tasks on MNIST, F-MNIST, and CIFAR-100, with detailed outcomes presented in Sections B and C of the supplementary materials. Notably, our method surpasses the state-of-the-art MWT-L [1] in terms of classification accuracy across all datasets. Particularly noteworthy is the substantial enhancement in accuracy observed on F-MNIST, increasing from 87.32% to 92.38%. The summarized results for MNIST and F-MNIST are as follows:\n        \n        \n        |  | MNIST | F-MNIST |  | MNIST | F-MNIST |\n        | --- | --- | --- | --- | --- | --- |\n        | NFN | 78.50±0.23 | 68.19±0.28 | ScaleGMN-B | 96.59±0.24 | 80.78±0.16 |\n        | DWS | 85.71±0.57 | 67.06±0.29 | MWT  | 96.58±0.32 | 83.86±0.91 |\n        | NG-GNN | 91.40±0.60 | 68.00±0.20 | MWT-L | 98.33±0.13  | 87.32±0.16 |\n        | ScaleGMN | 96.57±0.10  | 80.46±0.32 | **Feat-bias (Ours)** | **98.48±0.05**  | **92.38±0.06** |\n    - **INR Classification on CIFAR-100 (Supplementary Material)**.The experimental results for CIFAR-100 are summarized below. Given the absence of code implementation for CIFAR-100 in Implicit-Zoo [2], our method is compared solely with MWT, showcasing superior performance in terms of accuracy, precision, F1 score metrics, while also significantly reducing training time. These outcomes further validate the effectiveness of Feat-Bias in INR post-processing tasks.\n        \n        \n        |  | Accuracy ↑ | Precision ↑ | F1 ↑ | Time ↓ | Parmas ↓ |\n        | --- | --- | --- | --- | --- | --- |\n        | WT | 15.83±0.71  | 13.82±0.85  | 13.84±0.79 | 151.78 ±2.93 (min)  | 261 |\n        | MWT [mid-task] | 19.87±0.21 | 17.88±0.11 | 18.06±0.15  | 169.42±2.80 (min) | 261 |\n        | MWT | 23.29±0.57  | 21.56±0.67 | 21.78±0.65 | 168.0±1.27 (min)  | 261 |\n        | **Feat-bias (Ours)** | **78.51±0.16** | **78.71±0.13** | **78.46 ±0.15**  | **145.38 ±1.65 (sec)** | **151** |\n    - **Additional Experiment on STL10 Datasets**. We have previously conducted INR classification tasks on all datasets used in prior works [1][2]. To further underscore the generalization ability of Feat-Bias, we have extended our evaluation to include an INR classification task on the STL10 Datasets. Similar to the case of CIFAR-100, the absence of code implementation in Implicit-Zoo [2] has led us to compare our method solely with the recent state-of-the-art MTL [1]. The result of this experiment are presented below:\n        \n        \n        |  | Accuracy ↑ | Precision ↑ | F1 ↑ | Time ↓ | Parmas ↓ |\n        | --- | --- | --- | --- | --- | --- |\n        | MWT | 37.22±0.80 | 36.29±0.85 | 35.93±0.23 | 25.03±1.88 (min)  | 261 |\n        | **Feat-bias (Ours)** | **97.62±0.03** | **97.61±0.03** | **97.62 ±0.03**  | **3.84±0.11 (min)** | **151** |\n- **More Formal Mathematical Characterization for How Bias Terms Influence Symmetries of INRs**\n   - We appreciate the valuable suggestion provided by the reviewer, and we hereby present a more formal mathematical formulation. Consider a typical $L$-layer periodically activated representation network, where the output value of the $l$-th layer  is expressed as $\\mathbf{z} _ l= f_l(\\mathbf{z} _ {l-1}) =  \\sin(\\omega_0\\alpha^l(\\mathbf{W} _ l \\mathbf{z} _ {l-1}+\\mathbf{b} _ l)$. Here, $\\mathbf{W} _ l$ and $\\mathbf{b} _ l$ denote the weights and bias at layer $l$, $\\omega_0$ controls controls the network frequency, and $ \\alpha^l$ determines the activation function's periodicity. Specifically, when $l=1$, the activated value of the input layer can be defined as $\\mathbf{z} _ 1= f_1(\\mathbf{z} _ {0}) = f_1(\\tilde{\\mathbf{x}} ) = \\sin(\\omega_0\\alpha^1(\\sum _ {j=1} ^ {|\\mathbf{x}|}\\mathbf{W}_ {j} \\tilde{\\mathbf{x}} _ {j}) +\\mathbf{b} _ 1))$, with $\\mathbf{x}$ denoting the input coordinates and $\\tilde{\\mathbf{x}} \\in [-1,1] $ represents normalized coordinates( $\\tilde{\\mathbf{x}} = \\frac{2\\mathbf{x}}{\\max(\\mathbf{x}) - \\min(\\mathbf{x})} - \\frac{1}{2}$). The examination of centrally symmetric coordinate pairs $ \\langle \\tilde{\\mathbf{x}} ^ {+} , \\tilde{\\mathbf{x}} ^ {-} \\rangle$, where  $\\tilde{\\mathbf{x}} ^ {+} = \\{ x _ 1, … , x_k \\}$ and   $\\tilde{\\mathbf{x}} ^ {-} = \\{ - x _ 1, … , - x_k \\}$ (with $k$ denoting the coordinate dimension, e.g., $k=2$ for images), reveals that the presence of $\\mathbf{b} _ 1$ disrupts the symmetry effect of the input layer's activated value, resulting in $f_1(\\tilde{\\mathbf{x}} ^ +) = -f_1(\\tilde{\\mathbf{x}} ^ -)$. For Bias-free INRs, where $\\mathbf{b} _ l = \\mathbf{0}, \\forall l \\in L$, the absence of bias terms leads to $f(\\tilde{\\mathbf{x}} ^ +) = \\sin(\\omega_0\\alpha^1(\\sum _ {j=1} ^ {|\\mathbf{x}|}\\mathbf{W}_ {j} \\tilde{\\mathbf{x}} ^ + _ {j}) )) = - \\sin(\\omega_0\\alpha^1(\\sum _ {j=1} ^ {|\\mathbf{x}|}\\mathbf{W}_ {j} \\tilde{\\mathbf{x}} ^ - _ {j}) )) = -f(\\tilde{\\mathbf{x}} ^ -)\n$. This leads to a symmetric effect on the output value, where $| \\mathbf{y} ^ + |= |\\prod _ {l=1} ^ L f_l(\\tilde{\\mathbf{x}} ^ +)| = |\\prod _ {l=1} ^ L f_l(\\tilde{\\mathbf{x}} ^ -)| = | \\mathbf{y} ^ - |$ , which hinders the reconstruction of the signal with spatial aliasing, as natural signals should exhibit  $|\\mathbf{{y}}^ + _ {gt} | \\neq |\\mathbf{{y}}^ - _ {gt} |$.  The presence of bias terms solely in the first layer, i.e.,$ \\mathbf{b} _ 1 \\neq \\mathbf{0}$  ensures that  $|f_1(\\tilde{\\mathbf{x}} ^ +)| \\neq |f_1(\\tilde{\\mathbf{x}} ^ -)|$ thereby averting failed reconstruction with spatial aliasing. \n\n- **Concerns about Limited Ablation Study**\n    - **The Empirical Studies (Sec.2) as Essential Ablation Study.** The primary focus of the paper is to unveil the previously overlooked significance of bias terms in INRs via empirical ablation studies. Comprehensive ablation studies have been conducted and presented in Fig.2, Tab.2, Tab.4 (Appendix), and Tab.5 (Appendix) to address the question of the impact of bias terms from different layers on INRs. Given this perspective, our proposed approach of exclusively freezing bias terms in input layers does **NOT** necessitate additional ablation studies, as our method does not constitute a pipeline consisting of multiple function blocks.\n    - **The Role of Sec. 4.3 Ablation Study: Supplementary Verification for Spatial Aliasing.** In Sec. 4.3, another ablation study was carried out to confirm the source of spatial aliasing, rather than assessing the efficacy of our method. We believe that our experimental setup adequately confirms the source of spatial aliasing. Additional ablation studies on different datasets can be located in Sec. C of the appendix.\n    - **Ablation Study on Feature Downsampling Method**. Upon careful consideration, we have identified the only suitable area for an additional ablation study, which pertains to the feature downsampling method utilized in Feat-Bia. The results of this ablation experiment have been demonstrated in the Response to **Reviewer x6DW**, presented under the bolded title **Additional Experiments for Feature Sampling**. These results demonstrate the effectiveness of selecting linear interpolation as our method for dimension reduction.\n\n[1] End-to-end implicit neural representations for classification. CVPR, 2025.\n\n[2] Implicit zoo: A large-scale dataset of neural implicit functions for 2d images and 3d scenes. In NeurIPS, 2024."}, {"review_id": "iKa4glt9nM", "reviewer": "Reviewer_3n6q", "summary": "This paper investigates the role of bias terms in Implicit Neural Representations (INRs) and challenges the common assumption that biases primarily enhance the network’s nonlinearity. Through extensive analysis, the authors demonstrate that biases mainly serve to eliminate spatial aliasing artifacts rather than increase model expressiveness. Based on this insight, the paper proposes a novel bias modulation technique called Feat-Bias, which strategically leverages bias terms to reduce aliasing effects in INR reconstructions.\n\nUnfortunately, the paper is poorly written and difficult to follow. Although the authors emphasize reconstruction quality, the experimental setup for this task is weak. Conversely, the experiments primarily focus on the INR classification task. However, the section “3 Application: Featuring INRs with Bias Terms” is unclear and poorly organized. Furthermore, the key component, Feat-Bias, is insufficiently explained. In addition, Figure 4 is not referenced anywhere in the manuscript.", "questions": "1. The structure of the paper should be improved. The authors need to clearly specify which task is central to the paper—reconstruction or classification—and precisely describe the key architectural components.\n\n2. A theoretical justification for the proposed approach should be included.\n\n3. Additional experiments on diverse datasets and with different architectures are necessary.\n\n4. The Feat-Bias method is unclear and insufficiently described.\n\n5. Figures and visualizations should better emphasize the main advantages of the paper.", "limitations": "The authors have not adequately addressed their work's limitations and potential negative societal impact. To improve, they should explicitly discuss the scope and boundaries of their method, such as the types of data and tasks for which their approach may be less effective.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The main topic is very interesting.\n\n2. The hypothesis in the paper is well defined.\n\nWeaknesses:\n\n1. The paper lacks theoretical justification for the proposed hypothesis.\n\n2. The notation of (Observation #1), (Observation #2), and (Observation #3) is unclear. The authors should precisely formulate their theses and provide theoretical support.\n\n3. Reconstruction quality should be evaluated on different types of datasets, such as videos, 3D objects, and sounds—not only images.\n\n4. The baselines used for reconstruction tasks are weak. The authors should include comparisons with:\n\n- FreSh: Frequency Shifting for Accelerated Neural Representation Learning (ICLR 2025)\n- SPDER: Semiperiodic Damping-Enabled Object Representation (ICLR 2024)\n\n5. The authors should consider more architectures, such as Fourier Feature Networks and Positional Encoding.\n\n6. More activation functions should be tested.\n\n7. The Feat-Bias method is unclear and poorly described in the paper. Additionally, Figure 4 is not cited anywhere in the main text.\n\n8. Training curves in Figure 2 are incomplete. It is unclear what happens when the model is trained for 10,000 iterations.\n\n9. Figure 1 is not convincing since the differences in PSNR are small, and I do not observe any visible difference in the images.\n\n10. Perhaps dividing the reconstruction and original image for comparison could help illustrate the differences more clearly.", "was_revised": true, "final_justification": "Na podstawie wyjaśnień i rozszerzonych wyników eksperymentalnych przedstawionych w odpowiedzi, podniosłem swoją ocenę. Zdecydowanie zaleca się jednak, aby ostateczna wersja artykułu zawierała następujące elementy. Po pierwsze, powinna zawierać krótką dyskusję na temat wpływu różnych funkcji aktywacji. Po drugie, bardziej szczegółowe i jednoznaczne wyjaśnienie metody Feat-Bias, najlepiej poparte pseudokodem lub schematyczną ilustracją, znacznie poprawiłoby czytelność. Po trzecie, rozwinięcie teoretycznej motywacji stojącej za obserwacjami dotyczącymi terminów stronniczości pomogłoby lepiej ugruntować wkład w teorię. Wreszcie, wizualizacje powinny zostać ulepszone, aby wyraźniej podkreślić zalety proponowanego podejścia. Te uzupełnienia znacznie poprawiłyby czytelność, kompletność i ogólny wpływ pracy.", "rebuttal": "We sincerely thank the reviewer for your valuable comments and suggestions, and here is our response for your concerns:\n\n- **Lack of Formal Mathematical Proof**. As stated in Sec. 6 (line 338), we admit that our work lacks rigorous mathematical proof to explain our insights. However, we respectfully disagree that this undermines the contribution of our work. We challenge the conventional practice of implementing full-bias INR architecture in this field and explain the role of bias terms through intuitive analysis and empirical evidence, proposing that frozen first-layer bias terms should be the new paradigm with efficient parameters and improved reconstruction quality. Many foundational papers in the INR field are combinations of intuitive analysis, observation, and effective implementation without mathematical proof (e.g., SIREN, FINER, WIRE, Trans-INR [1]). Their theoretical frameworks might be later formulated in separate works published 2-3 years after (e.g., Sampling [2], Algebra [3]). At this stage, we aim to contribute our findings to the research community first, and then stimulate further studies in this direction in both theoretical frameworks and practical improvements.\n- **Additional Evaluation on Other Modalities**\n    - **1D Audio Fitting Task**. Fitting 1D audio data can be formulated as $F_\\theta(\\mathbf{x}): (t) \\mapsto (a)$, where $a$ represents the amplitude value at time step $t$.\n    For this task, we follow the settings of [4], utilizing LibriSpeech dataset and configuring both $\\omega$ and $\\omega_0$ to 100 in the SIREN architecture.\n        \n        \n        |  | SI-SNR ↑ | STOI ↑ | MSE (e-4) ↓ |\n        | --- | --- | --- | --- |\n        | Full-bias  | 11.21 | 0.896  | 2.835  |\n        | Input Bias (Ours) | 11.72 | 0.904 | 2.239 |\n    - **2D Text Fitting Task**. Fitting 2D synthesized text image data can be formulated as $F_\\theta(\\mathbf{x}): (x,y) \\mapsto (r,g,b)$. The dataset in this experiment was obtained from PE. We maintain all settings consistent with settings described in Sec.4.1 (line 223 -230).\n        \n        \n        |  | PSNR ↑ | SSIM ↑ | MSE (e-3) ↓ |\n        | --- | --- | --- | --- |\n        | Full-bias [T=1k] | 33.94 | 0.983 | 1.964 |\n        | Input Bias (Ours) [T=1k] | 34.38 | 0.985 | 1.775 |\n        | Full-bias  [T=3k] | 40.11 | 0.994 | 0.490 |\n        | Input Bias (Ours) [T=3k] | 40.49 | 0.996 | 0.443 |\n    - **3D Video Fitting Task**. Fitting 3D video data can be formulated as $F_{\\theta}: (x,y,t) \\mapsto (r,g,b)$.  Following the setting of DINER, the experiments were evaluated on the video from the UVG dataset.  Each scenario was trained for 100 epochs.\n        \n        \n        |  | PSNR ↑  | SSIM ↑  |\n        | --- | --- | --- |\n        | Full-bias  | 26.97 | 0.981 |\n        | Input Bias (Ours)  | 27.35 | 0.984 |\n    - **3D Shape Fitting Task**. We used Signed Distance Fields to represent 3D shapes. The fitting task can be formulated as $F_\\theta(\\mathbf{x}): (x,y,z) \\mapsto (s)$, where $(x,y,z)$ represents the coordinate of given points and $s$ denotes the signed distance to the surface. Following [5], we employed an $8\\times256$ MLP with SIREN architecture. We evaluated our method on the Stanford 3D Scanning Repository. The total iterations were set to $T=5,000$. The results are are as follows:\n        \n        \n        |  | IoU ↑ | CHD ↓ |\n        | --- | --- | --- |\n        | Full-bias  | 0.949  | 1.45e-6  |\n        | Input Bias (Ours)  | 0.954 | 1.41e-6 |\n    - **Conclusion**. The above quantitative results evaluated across different modalities demonstrate the enhanced reconstruction achieved by implementing frozen bias terms solely in the input layer (Input Bias), further supporting the main findings of our paper and challenging the conventional practice of implementing full-bias INR architecture in the community.\n- **Central Task: Reconstruction or Classification?**\n    - Our paper does **NOT** aim to design a task-specific method. Instead, it **uncovers the overlooked role of bias terms in INRs** through analytical and empirical studies. Surprisingly, we discovered that bias terms minimally contribute to nonlinear representation, with their gradient propagation having little impact on performance. Based on these findings, we propose applying bias terms exclusively to input layers, challenging the conventional full-bias architecture. This adjustment consistently improves reconstruction performance, setting new standards in the INR community for more efficient storage and enhanced reconstruction.\n    - Our proposed Feat-bias for INR classification serves as a practical application of our understanding of bias terms in INRs. Since backpropagation of bias terms does not affect reconstruction, these values remain stable throughout the encoding process. This stability makes bias terms ideal for storing high-level features extracted from pre-trained encoders. Leveraging the expressiveness of these features, we achieve significant performance enhancements in post-processing tasks, offering a straightforward and effective solution in this field.\n    - **In summary**, improved reconstruction performance stems from eliminating redundant bias terms, while enhanced classification performance results from our optimized utilization of fixed bias terms, both rooted in our understanding of the role of bias terms in INRs.\n- **Application for Other Activation Functions.** Considering that sine-based periodic activation functions are widely used in the field of Implicit Neural Representations (INR) (SIREN being the most popular and FINER achieving the highest performance), we focus our study on INR architectures utilizing SIREN and its variations. Our analysis is also centered around the symmetry of these activations, a characteristic lacking in other activation functions (such as WIRE). It is observed that introducing a bias implementation in the first layer can lead to marginal improvements in models like PE and WIRE. However, the effectiveness of this enhancement is unstable, indicating the need for additional regularization in these architectures, a direction we plan to explore in future research. Given that FINER stands as the state-of-the-art architecture in INR reconstruction, we assert that the alignment of our work with FINER underscores the significance of our research endeavors.\n- **Absence of Comparison in INR Reconstruction.** Our method stands orthogonal to existing INR optimization methods, enabling seamless integration with these established frameworks without the need for direct comparisons. Over the past years, there have been approximately 40 papers focusing on improving the expressive capacity or fitting speed of INRs from diverse perspectives such as partition, meta-learning, transformation, and sampling. While we acknowledge the substantial challenge in evaluating the compatibility of our findings with all these diverse methods, we respectfully contend that the absence of comparisons with specific acceleration methodologies (e.g., FreSh) does **NOT** undermine the significance of our approach for the following reasons:\n    - The prevailing trend in INR studies is to benchmark against baselines within the same category (e.g., EVOS [4] versus INT [5] as both employ sampling-based acceleration techniques). Given that our method represents the first work in enhancing INRs by adjusting specific bias terms, the lack of existing baselines necessitates comparison.\n    - Indeed, FreSh (ICLR 2025) did not conduct comparative analyses across any of other INR optimization methods.\n    - Enhancing INRs' reconstruction quality is just one part of our contribution. Our main focus is highlighting the overlooked role of bias terms in INRs and questioning the prevalent full-bias architecture in the field.\n    \n    We sincerely appreciate the suggestions provided and commit to incorporating experiments that align with various existing INR acceleration methods in the future work.\n    \n- **Presentation Issues**\n    - **Fig. 2 are incomplete.** In actuality, **Fig. 2 is comprehensive**. The training dynamics under various bias configurations are examined over 5k iterations, following the methodology of previous studies [2][3][8]. The training curves indicate the convergence of the reconstruction process. Additionally, we conduct a re-implementation of this experiment with 10k iterations, showing a stable trend and reinforcing the conclusions drawn from the results illustrated in Fig. 2.\n    - **Visible Difference in Fig. 1.** We appreciate your valuable suggestions. To highlight the visible differences under various bias configurations, we will enhance the comparison of local details in the image in the final version.\n    - **Notation of Observations**. Given that these observations are not formalized within a mathematical framework, we designated them as Observation #1, #2, #3 for clarity and to facilitate a quicker comprehension of our empirical findings. These notations will be revised in the final manuscript.\n    - **Description of Feat-Bias Method**. Regarding the Feat-Bias method as a practical application of our interpretation of bias terms, we consider it a straightforward engineering solution. This method involves transferring encoded features to the bias terms in the input layer, hence, we have provided a concise treatment in the current version. A more detailed description of this method will be included in the appendix.\n\n[1] Learning Transferable Features for Implicit Neural Representations. In NeurIPS, 2024.\n\n[2] A Sampling Theory Perspective on Activations for Implicit Neural Representations. ICML, 2024.\n\n[3] Implicit Neural Representations and the Algebra of Complex Wavelets. ICLR 2024.\n\n[4] EVOS: Efficient Implicit Neural Training via EVOlutionary Selector. CVPR, 2025.\n\n[5] Nonparametric Teaching of Implicit Neural Representations. ICML, 2024."}, {"review_id": "qfKGU7UGf2", "reviewer": "Reviewer_aC5b", "summary": "This paper studies the effect of bias term in commonly used INRs which utilize periodic activation functions. In contrast to piece-wise linear activation functions like ReLU, the bias term does not have the same influence on the network behavior and prediction performance as for commonly used ReLU-networks. The authors find that while the bias term has less influence on overall performance, completely omitting it leads to \"spatial aliasing\" an infolding of spatial content. As a solution, they propose using a frozen bias term in the first layer, which is distilled from a larger ViT. This approach aims to shift the INR's symmetry, thereby eliminating the infolding spatial content.", "questions": "1. Could the authors add a comparison between Feat-Bias and a randomly sampled bias vector which follows different distributions (e.g., uniform-like, Gaussian-like)? Is the use of a feature vector from a ViT necessary?\n2. While the authors provide several examples for \"spatial aliasing\" for bias free IRNs, it would be interesting to visualize the reconstruction results yielded by networks with frozen biases like done with Feat-Bias. For a fair comparison it would be even more interesting to qualitatively compare standards IRN with periodic activation function with learned biases, fixed biases and Feat-Bias.\n3. The visualization of the spatial aliasing suggested that the diagonal line is the center line, however, it seems as the artifacts are negatives of the original one flipped upside down, is there a reason why the authors used a diagonal line?\n4. In the context of INRs, [a,b] used INRs, or more precisely, Hypernetworks, to learn CNN kernel weights. Would the authors think that this kind of inherent symmetry might even be beneficial for kernel learning with INRs? \n\n[a] David W. Romero, Robert-Jan Bruintjes, Jakub Mikolaj Tomczak, Erik J Bekkers, Mark Hoogendoorn, and Jan van Gemert. Flexconv: Continuous kernel convolutions with differentiable kernel sizes. In International Conference on Learning Representations, 2022a. URL https://openreview.net/forum? id=3jooF27-0Wy.\n\n[b] Grabinski, Julia, Janis Keuper, and Margret Keuper. \"As large as it gets-studying infinitely large convolutions via neural implicit frequency filters.\" Transactions on Machine Learning Research 2024 (2024): 1-42.", "limitations": "As the authors state, a theoretical explanation and proof are missing. Therefore, the evidence relies solely on the empirical study and the intuition regarding the inherent symmetry induced by coordinates and the periodic activation function. However, the qualitative empirical analysis and evidence are incomplete and should be added to compensate for the missing theoretical evaluation.\n\nSince the authors don't present qualitative results for their approach, it's unclear whether the spatial aliasing problem is fully resolved or merely shifted. This is particularly relevant for the Feat-Bias configuration (a frozen bias term only in the first layer), which could potentially lead to shifted artifacts because the bias term isn't adapted during learning.\n\nIf the authors can thoroughly evaluate their configuration and demonstrate through qualitative empirical evaluation (going beyond the current performance evaluation) that adapting the bias term to a fixed Feat-Bias leads to fewer artifacts, and thus should become the new standard in INRs, I would be happy to raise my score.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Novelty: The study of the bias term in INRs utilizing periodic activation functions, as opposed to piece-wise linear ones, is both novel and compelling.\n- Efficiency: Employing a fixed bias term in the first layer and eliminating additional bias terms in subsequent layers can lead to savings in computational resources and memory consumption.\n- Clarity: The paper is well-written, with a clear and easy-to-follow line of argumentation. \n\nWeaknesses:\n1. While the intuition regarding spatial aliasing is compelling, the paper lacks a theoretical analysis to substantiate these insights. \n2. Although the authors provide good empirical evidence for spatial aliasing when a bias term is omitted, they do not offer qualitative empirical evidence demonstrating that their proposed approach (using a fixed bias term only in the first layer) effectively eliminates spatial aliasing. Thus, besides the empirical performance evaluation, qualitive results should be provided. \n3. The usage of a compressed feature vector from a larger ViT, while conceptually interesting, raises questions about its necessity and whether the feature vector genuinely remains useful given the potential for inherent aliasing introduced by the compression pipeline of this feature vector. \n4. The authors claim that the bias term in INRs with periodic sine activation results in a phase shift (line 111). However, this potentially interesting property is not further discussed, despite its possible role in the observed spatial aliasing.", "was_revised": true, "final_justification": "Based on the clarifications provided by the authors, I have decided to raise my score.", "rebuttal": "We sincerely thank the reviewer for your valuable comments and suggestions. In addressing your insightful comments, we provide the following clarifications:\n\n- **Lack of Formal Mathematical Proof.** We admit that our work lacks rigorous mathematical proof to explain our insights. In this paper, we challenge the conventional practice of implementing full-bias INR architecture in this field and explain the role of bias terms through intuitive analysis and empirical evidence, proposing that frozen first-layer bias terms should be the new paradigm with efficient parameters and improved reconstruction quality. Many foundational papers in the INR field are combinations of intuitive analysis, observation, and effective implementation without mathematical proof (e.g., SIREN [1], FINER [2], WIRE [3], Trans-INR [4]). Their theoretical frameworks might be later formulated in separate works published 2-3 years after (e.g., Sampling [5], Algebra [6] ). Our current objective is to share our discoveries with the research community, aiming to inspire further exploration in both theoretical frameworks and practical advancements in this domain. We intend to incorporate formal mathematical proof in our future work.\n- **Qualitative Results for the Elimination of Spatial Aliasing.** Due to the constraints of the rebuttal page, we are unable to provide additional visually represented qualitative results to illustrate the benefits of the frozen first-layer bias terms mechanism. However, we have included an illustrative visual representation of the impact of frozen bias terms in INRs in Fig. 1 (line 39) in the paper. From this, we can infer that, with the exception of the bias-free configuration (#6) and the sole implementation of bias in the output layer (#5), other bias configurations exhibit the elimination of spatial aliasing. Our proposed approach of solely implementing input bias (#2) achieves optimal reconstruction performance, outperforming the conventional full-bias architecture (#1). In the final version, we plan to incorporate additional visual comparisons and enhance the analysis of local image details.\n- **Concerns Regarding Phase Shift.** In line 111 of the manuscript, we highlighted that bias terms in sine activation primarily introduce phase shifts rather than enforcing strict gating states, suggesting that the influence of bias terms in periodic activation functions should manifest significantly smaller effects compared to traditional ReLU-based activation functions. While we recognize that such phase shifts may have unique implications on the final reconstruction, we have not identified a clear and explicable relationship between the shifted phase and the ultimate reconstructed outcome, even after applying recent frameworks designed to explain the properties of INRs [7]. We value this insight greatly and intend to delve deeper into this aspect in our future work.\n- **Comparing Feat-Bias with Other Sampling Methods.** The results of this experiment have been detailed in the response to **Reviewer x6DW**, presented under the bolded title **Additional Experiments for Feature Sampling.**\n- **Is the use of a feature vector from a ViT necessary?** For INR post-processing task, such as INR classification, the utilization of a feature vector is essential. In the specified lines 202 to 220 (Sec. 3), we posit that Feat-bias can significantly enhance INR classification performance by leveraging fixed bias terms as optimal repositories for high-level features derived from established pre-trained encoders. These bias terms function as a distillation mechanism, transferring high-level features from extensive pre-trained networks to more compact input-layer bias spaces. This process allows each INR to capitalize on the representation capabilities of a large pre-trained network, resulting in substantial advancements when integrating additional networks for downstream INR tasks. Regarding INR reconstruction, the distillation of the feature vector is deemed unnecessary. The pivotal factor lies in the configuration of bias term **presence** (bias terms exclusively in input layers v.s. bias terms across all layers), rather than considerations of adaptiveness or inclusion of high-level feature information, as these aspects do not significantly impact the reconstruction ability, demonstrating near-identical performance.\n- **Presentation Regarding Diagonal Line.** We employ diagonal lines to showcase the impact of spatial aliasing, emphasizing that this aliasing stems from center symmetry rather than horizontal (x-axis) or vertical (y-axis) symmetry. Additionally, we aim to illustrate that the artifacts are essentially inverted negatives of the original one flipped both vertically and horizontally.\n- **Potential Benefits for INR-based Kernel Learning.** The insights derived from our analysis of bias have the potential to enhance INR-based kernel learning under the following conditions: (#1) symmetry in input structure and (#2) symmetry in activation functions (such as SIREN and FINER). While the original configurations of FlexConv (ICLR 2022) and NIFF (TMLR 2024) met condition #1 but did not fulfill condition #2 due to FlexConv utilizing MFN as the INR backbone and NIFF employing a PE+ReLU network, neither of which are periodically activated INRs. Given that FINER [2] has exhibited superior expressive capabilities compared to MFN and PE+ReLU, integrating FINER or SIREN into FlexConv and NIFF stands to benefit from the insights presented in our study.\n- **Comprehensive Evaluation of the Proposed Bias Mechanism**. Here, we present additional experimental results demonstrating that exclusively applying frozen bias terms to input layers consistently outperforms fully biased networks. This approach is poised to set **new standards in the INR community**, promising enhanced efficiency in storage and reconstruction capabilities.\n    - **Supplementary Material Inclusion.** We offer further validation of our empirical investigations (Section 2.2) and the proposed bias mechanism through additional datasets and bias configurations, including more comprehensive analyses of bias terms and the impacts of gradient optimization. Detailed settings and results are accessible in Section A of the supplementary material.\n    - **Additional Evaluation on Other Modalities**\n        - **1D Audio Fitting Task**. Fitting 1D audio data can be formulated as $F_\\theta(\\mathbf{x}): (t) \\mapsto (a)$, where $a$ represents the amplitude value at time step $t$.\n        For this task, we follow the settings of [8], utilizing LibriSpeech dataset and configuring both $\\omega$ and $\\omega_0$ to 100 in the SIREN architecture.\n            \n            \n            |  | SI-SNR ↑ | STOI ↑ | MSE (e-4) ↓ |\n            | --- | --- | --- | --- |\n            | Full-bias  | 11.21 | 0.896  | 2.835  |\n            | Input Bias (Ours) | 11.72 | 0.904 | 2.239 |\n        - **2D Text Fitting Task**. Fitting 2D synthesized text image data can be formulated as $F_\\theta(\\mathbf{x}): (x,y) \\mapsto (r,g,b)$. The dataset in this experiment was obtained from PE. We maintain all settings consistent with settings described in Sec.4.1 (line 223 -230).\n            \n            \n            |  | PSNR ↑ | SSIM ↑ | MSE (e-3) ↓ |\n            | --- | --- | --- | --- |\n            | Full-bias [T=1k] | 33.94 | 0.983 | 1.964 |\n            | Input Bias (Ours) [T=1k] | 34.38 | 0.985 | 1.775 |\n            | Full-bias  [T=3k] | 40.11 | 0.994 | 0.490 |\n            | Input Bias (Ours) [T=3k] | 40.49 | 0.996 | 0.443 |\n        - **3D Video Fitting Task**. Fitting 3D video data can be formulated as $F_{\\theta}: (x,y,t) \\mapsto (r,g,b)$.  Following the setting of DINER [9], the experiments were evaluated on the video from the UVG dataset.  Each scenario was trained for 100 epochs.\n            \n            \n            |  | PSNR ↑  | SSIM ↑  |\n            | --- | --- | --- |\n            | Full-bias  | 26.97 | 0.981 |\n            | Input Bias (Ours)  | 27.35 | 0.984 |\n        - **3D Shape Fitting Task**. We used Signed Distance Fields to represent 3D shapes. The fitting task can be formulated as $F_\\theta(\\mathbf{x}): (x,y,z) \\mapsto (s)$, where $(x,y,z)$ represents the coordinate of given points and $s$ denotes the signed distance to the surface. Following [10], we employed an $8\\times256$ MLP with SIREN architecture. We evaluated our method on the Stanford 3D Scanning Repository. The total iterations were set to $T=5,000$. The results are are as follows:\n            \n            \n            |  | IoU ↑ | CHD ↓ |\n            | --- | --- | --- |\n            | Full-bias  | 0.949  | 1.45e-6  |\n            | Input Bias (Ours)  | 0.954 | 1.41e-6 |\n        - **Conclusion**. The above quantitative results evaluated across different modalities demonstrate the enhanced reconstruction achieved by implementing frozen bias terms solely in the input layer (Input Bias), further supporting the main findings of our paper and challenging the conventional practice of implementing full-bias INR architecture in the community.\n\n[1] Implicit neural representations with periodic activation functions. In NeurIPS, 2020.\n\n[2] FINER: flexible spectral-bias tuning in implicit neural representation by variable-periodic activation functions. CVPR, 2024.\n\n[3] WIRE: wavelet implicit neural representations. CVPR, 2023\n\n[4] Learning Transferable Features for Implicit Neural Representations. In NeurIPS, 2024.\n\n[5] A Sampling Theory Perspective on Activations for Implicit Neural Representations. ICML, 2024.\n\n[6] Implicit Neural Representations and the Algebra of Complex Wavelets. ICLR 2024.\n\n[7] Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributions. CVPR 2024\n\n[8] EVOS: Efficient Implicit Neural Training via EVOlutionary Selector. CVPR, 2025.\n\n[9] DINER: Disorder-Invariant Implicit Neural Representation. CVPR, 2024.\n\n[10] Nonparametric Teaching of Implicit Neural Representations. ICML, 2024."}]

Author rebuttal:

Meta-review: This paper challenges the conventional full-bias design in implicit neural representations, showing that bias terms mainly mitigate spatial aliasing rather than enhance expressivity. The proposed Feat-Bias method leverages pretrained features as input-layer biases, yielding strong improvements in INR classification and reconstruction efficiency. Reviewers found the idea simple, new, well-substantiated, and practical.

Concerns have been raised regarding limited theoretical understanding and experimental scope. The concerns were largely addressed during the response phase, which led reviewer to be unanimously supportive toward acceptance of the paper. Overall, this is a novel and impactful contribution, and I recommend acceptance.
Decision: Accept (poster)