Exploring State-Space Models for Data-Specific Neural Representations
Authors: Jinsung Lee
Suha Kwak
Venue: ICLR 2026 Poster (2026)
Keywords: state-space model
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: This paper first applies the compressive property of state-space models to data-specific neural representations, which aim to represent a datum by overfitting a neural network.

Abstract: This paper studies the problem of data-specific neural representations, aiming for compact, flexible, and modality-agnostic storage of individual visual data using neural networks. Our approach considers a visual datum as a set of discrete observations of an underlying continuous signal, thus requiring models capable of capturing the inherent structure of the signal. For this purpose, we investigate state-space models (SSMs), which are well-suited for modeling latent signal dynamics. We first explore the appealing properties of SSMs for data-specific neural representation and then present a novel framework that integrates SSMs into the representation pipeline. The proposed framework achieved compact representations and strong reconstruction performance across a range of visual data formats, suggesting the potential of SSMs for data-specific neural representations.

Published as a conference paper at ICLR 2026
EXPLORING STATE-SPACE MODELS FOR DATA-SPECIFIC NEURAL REPRESENTATIONS
Jinsung Lee & Suha Kwak Graduate School of Al Pohang University of Science and Technology (j (jinsunglee, suha. .wak @postecc..ee... ac.kr
ABSTRACT
This paper studies the problem of data- satai-pecciiic neural representations, aiming for compact, flexible, and modality-agnostic storage of individual visual data using neural networks. Our approach considers visual datum as a set of discrete ob- servations an underlying continuous signal, thus requiring models capable capturing the inherent structure of the signal. For this purpose, we investigate state-space models (SSMs), which are well-suited for modeling latent signal dy- namics. We first explore the appealing properties of SSMs for data-specific neural representation and then present novel framework that integrates SSMs into the representation pipeline. The proposed framework achieved compact representa- tions and strong reconstruction performance across a range of visual data formats, suggesting the potential of SSMs for data data-specific neural representations.
INTRODUCTION
Recent years have witnessed growing interest in overfitting neural network single visual datum such as image (Dupont et al. 2021 Strümpler et al. 2022), video (Chen et al. 2021 Mentzer et al. 2022), or 3D instance (Martin- Brualla et al. 2022 Zhang et al. 2020). This data-specific neural representation paradigm. prevalent in implicit neural representations (INRs) SSizzmann et al. 2020 and neural compression (Ballé al. 2016) Cheng et al. 2020), aims to directly encode datum into an embedding or the weights of a compact neural model. Such paradigm not only serves as an effective data compression method but also offers standardized data format that can accommodate various modalities for future neural network training (Dupont et al.) 2022), with some approaches further enabling downstream applications such as spatial/temporal super-resolution (Chen et al. 2022b), denoising (Xu et al. )022), and in/outpainting (Skorokhodov et al. 2021 Chen et al. 2023).
The central objective of the data-specific neural representations is to represent single datum with minimal parameter complexity and maximal reconstruction quality. One of the effective strategies to achieve this comes from the recognition that visual data are essentially arrays of pixels sampled at discrete intervals from continuous signals (Sitzmann et al. 2020; Xu al. 2022 Tancik et al. 2020 Saragadam et al.) 2023). The core idea behind this approach is to project input data onto set of established basis functions and only save their coefficients, so that the coefficients reconstruct not only the input but also the continuous signal from which the input is sampled. Although this concept has served fundamental principle for effective compression and reconstruction of visual data (Cooley et al., 1969: Richardson 2011), modern approaches to data-specific neural representation do not take into account due the lack of well-established neural network architectures that incorporate the concept; they have instead focused merely on coordinate-to- RGB mapping (Martin- Brualla et al. 202T Strümpler et al. [2022), bit-level quantization (Xu et al. [0188] Gordon et al. 2023) or improving the capacity of conventional neural networks to implicitly manage input redundancies (Zhou et al.)
2018 Li et al. 2018).
Recently, the rise of state-space models (SSMs) has opened a new pathway to this challenge, as SSMs provide a framework for modeling continuous signals in a way that aligns with the objectives of compact neural representations. To be specific, the hidden state of SSM was initially designed to represent the coefficients that reconstruct observed data using a set of orthogonal polynomial bases (Gu et al. 2020) 2022b), which generalizes to the traditional compression algorithms. Although

---

Published as a conference paper at ICLR 2026
the design of SSMs has become more implicit (Gu et al. 2021b Smith et al. 2022 Gu et al. 202 la
such that their hidden states no longer explicitly represent coefficients of such continuous bases, they still retain the desirable properties necessary for effective signal modeling (Gu et al., 2021b Guo
et al.
2025
Rao 1987
Rao & Arun 1992), so it is worth exploring their applications.
Driven by this motivation, explore the potential for incorporating SSMs within data-specific neural representations. We investigate the effectiveness of SSMs in compressing input data and capturing underlying signal structures, and empirically demonstrate their benefits in enhancing the reconstruction quality. However, a naive application of SSMs presents key challenges: primarily operate on 1D sequence inputs, necessitating unnatural scanning for multi-dimensionn data, and (2) they inherently preserve input sequence length, which makes them unsuitable for an effective compression method. To address these limitations, we propose structured state-space kernel (S3K), which distills the expressive power of SSMs into convolutional kernels. We design the kernel parameters in a way that the convolution output matches the hidden state representation of SSMs, effectively preserving their reconstruction capability. Through seamless integration with convolution, it naturally processes multi-dimensional inputs while inherently enabling expressive downsampling. In summary, contribution is fhree-fold as follows:
(1) they
We for the first time investigate the integration of state-space models (SSMs) into data-specific neural representation frameworks, providing theoretical background that explains their potential benefits in improving both expressive power and efficiency.
We introduce S3K, an SSM-derived convolutional kernel that inherits the expressive power of SSMs while mitigating their limitations for multi-dimensional processing and downsampling.
Our framework shows promising results across diverse visual data reconstruction tasks images (Kodak (Kodak] 1993), CLIC2020 (Toderici et al. 2020)), videos (Bunny (Roosendaal 2008).
UVG (Mercat et al. 2020), DAVIS (Perazzi et al., 2016)), and 3D objects (Objaverse (Deitke
et al. 2023) highlighting its potential for advancing data-specific neural representations.
2 RELATED WORK
State-space models. SSMs are a family of sequence- pequencc-to-sequeee tos-uueeeeee models that embeds historical data in state-space representation, using differential equations that involve hidden states and sequential inputs. HiPPO (Gu et al. (020). an early state-space model, treats an input sequence as samples taken from a continuous function. This function is then approximated using a predefined set of orthogonal polynomials, with their coefficients being dynamically updated by the incoming sequential inputs. LSSL (Gu al. 2021b) generalizes HiPPO by replacing HiPPO parameters into learnable ones, while still retaining its ability to continuously remember and store the history of observed tokens. has developed into S4 (Gu al. 2021a| 2022a| Nguyen et al. 2022) and S5 (Smith al.. 2022).
which addresses technical inefficiencies of the previous work. Recently, Mamba (Gu & Dao 20233
has been introduced as a state-space model that adapts its parameters based on the input sequence.
Although originally designed for sequential inputs, this model has inspired various adaptations across
different visual perception tasks, including images (Zhu et al., 2024a Hu et al. 2024, Ruan &
Xiang, 2024 Zhu al. 2024b; Nguyen et al., (022), videos (Li et al. 2025 Yang et al. 2024
Chen et al. 2024 Li et al. 2024). and 3D scenes (Liang et al.] 2024 Xing et al. al. 2024). Although
these efforts extend SSMs multi-dimensional inputs, they primarily target sequence modeling
tasks such as classification or siquence-to-sequenee translation. In contrast, our work investigates
SSMs as compact representations through compression of input data, perspective that has received comparatively less attention.
Implicit neural representations. INRs aim at constructing a model that effectively captures continuous signals, including 3D scenes (Park et al. 2019 Mildenhall et al. (222)) images (Strümpler et al. 2022] Guo et al., 2023) and videos (Zhang et al. 2021] Chen et al. 202T). INR typically represents continuous signal by parameterizing field, which involves mapping between the coordinate space and the signal space. The emergence of INRs has rapidly advanced the field of data-specific neural representations, offering promising avenues for efficient compression (Strümpler
Chen
et al. 2022). continuous signal modeling (Martin-Brualla et al., 2021 Chen et al., (022a), and lask-specific adaptation (Pumarola et al.) | Chen al. [022] Given this shared objective of data-specific modeling, we evaluate our method on a standard INR benchmark, highlighting its potential as a new architectural direction within the INR paradigm.
2

---

Published as a conference paper at ICLR 2026
Neural compression. This line of work enables neural networks to learn compact representations of images and videos by incorporating advanced techniques such as entropy modeling (Ballé et al. 20188: Cheng et al. 2020) quantization (Yang et al. 20200a b). Early methods introduce autoencoder- style architecture (Ballé et al., 2016, Mentzer et al. 2018). where the input is encoded into a compressed latent vector and subsequently reconstructed by the decoder. Our method adopts similar encoder-decoder formulation while introducing state-space models (SSMs) as a new architectural component for learning compact representations, highlighting the unexplored potential of SSMs in neural compression.
PRELIMINARY: STATE-SPACE MODEL
SSM is function that maps a 1D input signal d(x) to a 1D output signal y(z) of the same length through the latent state h(x) E CN based on the following linear differential equation:
(x(x) Ah(x) + Bo(x), y(z) = Ch(x),
where A N is the state transition matrix, and B CN and C CN projection parameters. Solving the linear differential equation
I
to explicitly express h yields]
h(x)= % ((-T)) Bo(T)dT ECY,
(2)
where each h& (a:) C corresponds to:
hx(z) / e(e-r)):
Bolr)dr cll-t))
( [0.z]
:= (d,
(3)
Here, (1))) is complex function space inner product in the given domain [21, Intuitively, Eq. tells that the k-tth element of the hidden state (xr is projection of the input é(T)rz²
onto the function &x- Gu et al. 20200 2021b 2022b a) have established that through appropriate parameterization of the matrix A. {Ex} can serve as a set of basis functions, which enable the model to effectively capture and retain key information from the entire sequence history up to the current position z. This ability to project input signals onto learned basis makes SSMs natural fit for compression, since such a capability to model signal representation becomes beneficial. Various ways to parametrize
have been explored: HiPPO (Gu al., 2020)), diagonal plus low rank (Gu et al.) 2021a), and diagonal (Gupta et al. [2022] Gu et al. (022a).. Among these, using diagonal parameterization of A has gained popularity for easier formulation while maintaining expressivity (Smith et al. 2022 Gu & Dao 2023
EXPLORING SSMS FOR APEACIPECIIIC NEURAL REPRESENTATIONS
This section delves into advantages and proper architecture designs of data- satai-ppeciiic neural representa- tions using SSMs. First, we examine how SSMs encode input data and highlight their effectiveness data-specific neural representation. Then, through an extensive experimental analysis, we identify the key characteristies of network architectures incorporating SSMs for this purpose.
4.1 WHAT SSMs ENCODE?
To understand the operational principles of SSMs and interpret their features, bring up classical signal processing task, sinusoid problem, which aims at estimating parameters of sinusoidal signals that make up the input signal. Given input function @(t) that takes 1D coordinate E [0, L.J, are interested in finding (t) and ea such that
((1)
Σ
Gne in(())
(4)
See Appendix 1.1ml for details.
3

---

Published as a conference paper at ICLR 2026
Enceder
Decodar
Table Image reconstruction quality PSNR of differ- ent architectures incorporating various SSMs. (a), (b), and (c) indicate the encoder variants shown in Fig.
SSM stook
(o
ConsTransppen (AAAA
SSM Block
Baseline
(b)
(c)
ombedding
image
Transfomer Viewani et 2017
24.75
24.87
23.96
24:67
$4 (Ga al. 20218
25.79
24.65
24.68
25.82
Figure I: Baseline architecture incorporating SSMs for image reconstruction
S4D Fou
25.49
24.99
25.48
26.06
SAND [Smil or at
25.75
25.76
24.84
24.90
26.44
Mainii (Gu Dan 2023
24.90
24.82
24.78
26.58
SSM block black
Conv
Tet core
Fx3 corls
Fxd OOTM
3xl 00/N
Ix3 coWV
SSM
16)
331
16,81
044.44
(a) Stacked
(b) Image pyramid
(c) Laplacian pyramid
Figure 2: Encoder variants incorporating SSMs for image reconstruction
where (() the n-th sinusoidal basis and 6n its coefficient. Note that this form generalizes to various sinusoidal transformation methods, e.g., setting (t) - 2mrdd/ T. leads to the choice of bases used in discrete Fourier transform (Cooley et al., 1969). Under a proper choice of 0, (t), estimating the parameter c, that approximates the input signal @(t) offers effective method for compression.
Interestingly, the SSM formulation allows the input signal to be decomposed into sinusoidal form: Theorem 4.1. Let A be diagonalizable over C with non-zero distinct eigenvalues {A+}- Given A, B, and the hidden state computed by Eq. (2), there exists function f (A, + E CNNN with which one can decompose the imput function ((I) as a linear combination of complex exponentials:
(11)= Σ (n
where on the n-th element of f(A, ByT.
The conclusion of the theorem implies that the SSM parameters A and B inherently capture signal characteristics the input, as well as the hidden state h. This highlights the unique capability of SSMs: they are particularly favorable for reconstructing the input since they encode the input function (tt) itself, unlike traditional data-specific neural representation frameworks that were originally designed to capture specific patterns semantics.
To verify the effectiveness SSMs in the context of compact data-specific neural representation, conduct input reconstruction experiments where the input is compressed into embedding and then reconstructed by lightweight decoder, which allows us to directly assess how well the encoder captures and retains the information of the input in a compact form. To this end, we first design a simple encoder-decoder architecture that can naturally incorporate various SSMs (Fig. I The encoder consists of an SSM block computing the output signal by Eq. (T) and a single convolutional layer for downsampling the signal, while the decoder is composed of a deconvolutional layer (Noh al., 2015) for upsampling. Note that the SSM block is attached directly onto the raw input and that the downsampling operation of the encoder is essential for compression due to the length-preserving (Gu al.] (021a)). S4D Gu et al. 2022a), S4ND (Nguyen al. 2022), S5 (Smith et al. 2022) and Mamba (Gu & Dao 2023). as candidates for the SSM block.
nature of SSMs (Sec. 3). We consider the established SSM architectures,
For evaluation, we train and evaluate this baseline architecture coupled with the diverse SSMs on 1K randomly sampled images from CIFAR- 100 (Krizhevsky et al., 2009). Each model is trained for 300 epochs each image, adhering to the prevailing practice in data- specifc neural representations that focuses on optimizing lightweight model to facilitate overfitting on a single sample (Dupont
al. 2021 Chen et al. 2021). Since the SSMs are designed to operate ID sequences (except for
4

---

Published as a conference paper at ICLR 2026
S4ND Nguyen et al. 2022) ), preprocess the imput images by flattening the 32 32 pixel grid before passing through the SSMs. We report experimental results using the baseline architecture in adopt a Transformer (Vaswani et al. (017)) in place of the SSM block, as it is widely used architecture for data-specific representation (Yan et al. Mentzer et al. 2022 Liu al. 2023 Lu et al. (02T) and has demonstrated strong performance. "Tranffrmmer indicates a single multi-heat attention layer used in this context. The results show that, under the baseline architecture, every SSM consistently outperforms the transformer in reconstruction quality. This finding is nor trivial, particularly considering that transformers are widely recognized for their superior performance when operating on short token sequences (Gu Dao 2023). This suggests that for input reconstruction, the transformer's s ability to compute semantic relationships between tokens less beneficial than in other tasks, while the input function modeling property of SSMs proves to be more advantageous, as discussed in Sec.
the 'Baseline' column of Table D For comparison,
4.2 EXPLORING ARCHITECTURES INCORPORATING SSMs
To explore encoder architectures that better leverage SSMs, we experiment with several design variants. Given focus evaluating how well the encoder compresses the input into embedding, we maintain a fixed decoder architecture across all configurations. We first evaluate a stacked architecture where SSM and convolutional layers alternate to form deep network (Fig. 2(a)). and observe consistent performance drop for SSM models (Table a)). We hypothesize that this decline stems from the way SSMs encode input features (Sec. 4.11 since SSMs project the input onto implicitly parameterized basis functions and stacking them results in multiple layers of such projections, repeated projection amplifies artifacts and limits the achievable reconstruction rate, analogous to generation loss from information theory (Cover 1999).
To address this, we introduce an 'Image pyramid' variant (Fig. 2(b)). where SSMs are applied at it enables the
multiple resolutions of the input. This approach improves performance (Table
use of representations across different scales and increases model capacity without having to stack SSM blocks. While Mamba (Gu & Dao 2023) shows slight drop in this setting, the overall trend confirms the benefit of incorporating SSMs across multiple resolutions.
We further explore 'Laplacian pyramid' variant (Fig. 2(c).. widely used decomposition method in traditional compression techniques (Burt & Adelson 1987 Richardson 201T). Since a Laplacian pyramid introduces less redundancy across scales than the 'Image pyramid variant, the separate SSM blocks can be more effectively utilized. Results Table ()) show consistent gains, with SSM-based models benefitting the most.
From these image reconstruction experiments, we outline key insights on incorporating SSMs: (1) Stacking SSMs in the encoding process does not yield effective results, (2) attaching SSMs to downsampled images to provide intermediate multi-scale features proves ad vantaggo,,,, (3) employing Laplacian pyramid decomposition further enhances performance.
PROPOSED METHOD
This section presents our method for data-specific neural representations using SSMs. We introduce our novel module, structured state-space kernel, which addresses the two major limitations applying SSMs to neural representations of visual data: (1) their design for one-dimensional signals, which does not align directly with visual data, and (2) their inability to compress input sequences due their nature, which typically necessitates additional components for downsampling. Our module overcomes these challenges by leveraging structured kernels derived from SSMs, enabling efficient encoding and reconstruction of visual data.
5.1 STRUCTURED STATE-SPACE KERNEL
To implement the continuous- time dynamics of SSMs on a discrete sequence, the state update of Eq. (T) is often approximated discretized intervals using step size parameter A. For instance,
5

---

Published as a conference paper at ICLR 2026
can apply the hidden state update between h (zz,) and h(z;-1) using the Euler method:
h(x;)) h(0;-1) (11-111)
= h((--1) + A(Ah(ri-1) + Bo(x;))
== Ah(r;-1) + Ba(r;))
(6)
Depending on the choice of the discretization method and A, the way A and B are constructed may vary. Eq. (6) can be expressed as a convolution, where the hidden states evolve according to:
h-1 = 0, ho- Ah + B Bøo,
h1 Aho + Bai
ABoo + BΦi,
h A - Bioo A B + AB B
= [AA' TAALB L BB ... ABB) (d0 01 1...1...-111 $
where we denote (xx;)) and of (.,)) h, and di, respectively, for brevity. Given that the last hidden state is the projection of the entire sequence onto the basis functions defined by A and B (Sec.3). some previous work (Gu al. 2020 2022b) have demonstrated that the input signal can be reconstructed solely from the last hidden state. Inspired by this idea, we employ h for downsampling, as effectively compresses the input into a compact representation while allowing for an efficient operation by skipping the computation of intermediate hidden states, i.e., /ho....,hh..) Once we construct the convolutional kernel K so that
K = AAL A
AB C
we can convolve this kernel to obtain the compressed input. We refer to this kernel as structured state-space kernel (S3K), as is factorized by leveraging the state transition matrix A and the projection matrix B. In practice, we adopt the diagonal parameterization Gupta al. (0222) to ease the computation of the power terms and adopt multi-input multi-output (MIMO) framework (Smith et al. 2022) by letting B E C to handle C channels of input sequence simultaneously. The convolution using this structured kernel acts as lossy compression mechanism, theoretically allowing the reconstruction of the original input signal using the learned parameters.
A (Gu et al. 2022a
Theorem 5.1. A diagonalizable over C with non-zero distinct eigenvalues {De and A be the step size used for the discretization of A. Given the final hidden state h E CN after applying S3K, there exists a function R: (A, B, h) + H CIXN with which one can reconstruct the input sequence as:
R(A, (cA((AARA))
1.1.....,N N
k=12....
This finding further supports the interpretation of S3K as a lossy compression mechanism, where the transformed representation retains sufficient information reconstruct the original input signal through the learned state-space parameters.
5.2 EXTENSION TO MULTI-DIMENSIONAL S3K
We extend S3K to multiple dimensions via outer products of multiple independent ID S3Ks. This follows naturally from the definition of nD basis functions as outer products of ID basis functions in continuous space (Cheney 1986; Nguyen et al. (022). The resulting nD kernel K now has L N, C), where (T, [_
are spatial dimensions of the kernel, and N and C' represent output and input channel dimensions, respectively. This formulation enables nD convolution operations on multi-dimensional inputs using structured kernels, akin to the traditional convolution.
dimensions (I,(1, (2)
5.3 ENIIANCING EXPRESSIVITY
While the structured kernel theoretically finds effective basis functions and their corresponding coefficients that represent the input data, practical implementation reveals limited expressivity due to the small number learnable parameters. To address this, we introduce several modifications to enhance the power of our model as follows.

---

Published as a conference paper at ICLR 2026
Encoded output
Input
Dnverted Rastlenee
pixel
voxel
S3K Conv
S3K Conv
S3K Conv
S3K Conv
xxl convolution
Laplacian Pyramid
Figure 3: Structure and operation of the proposed LPNet-S3K architecture
Input-adaptive B: Instead of using fixed kernel, adopt adaptive mechanism (Chen et al., 2020; Gu & Dao 2023) where the kernel parameters depend on the input, allowing dynamic adjustments diverse signals.
Real-valued SSM parameters: To improve numerical stability and expressivity, we follow the real parametcrization of A and B, which has shown strong empirical performance Mamba (Gu &
Dao 2023
Subsequent convolution layer: We integrate 1 x convolution layer to further enhance the representation capacity while allowing the state size N to differ from the output channel size, providing greater flexibility model architecture design.
These modifications refine S3K into a flexible neural network module with stronger representation capacity, allowing S3K to be effectively integrated into data-specific neural representation frameworks. Additional implementation details can be found in Appendix A.8
5.4 MODEL ARCHITECTURE
We illustrate the architecture of our final model Fig. 3 We follow the design choices of the last encoder variant in Sec. 4.2 multi-scale signals decomposed by Laplacian pyramid (Fig. 2(c)..
and attach an S3K convolution layer, instead of an SSM block and the following convolutional layer, to each level of the Laplacian pyramid. We make two key modifications to complete
model: (1) replacing the intermediate MLP blocks with inverted bottleneck layers (Liu et al., 2022)).
more advanced module that has shown superior performance across various domains (Woo et al.)
2023;
Chen et al. [023] IZhao et al. (002)), and (2) using SiL.U (Elfwing et al.) 2018) activation and RMSNorm (Zhang Sennrich 2019) that have been frequently used in SSMs (Gu & Dao 2023
Nguyen et al.) |022] Smith et al. 2022). We call this network Laplacian Pyramid Network with S3K
(LPNet-S3K) for ease of reference.
EXPERIMENTS
To evaluate the effectiveness of the proposed LPNet-S3K architecture, we overfit the network to individual visual inputs across diverse data formats, including image, video, and 3D object. We first validate efficacy of the LPNet and S3K architectures in the context of data-specific neural representations by evaluating their performance on images and 3D objects. Then, we evaluate our model on video INR benchmarks (NeRV), demonstrating efficacy on the standard literature benchmarks.
Datasets. We evaluate our models on standard image and video reconstruction benchmarks. For im- ages, we use Kodak [Kdakk 1993) and CLIC2020 (Toderici et al. (020), 20220) consisting of high-resolution photographs. For videos, we follow common NeRV benchmarks, including Bunny (Roosendaal 2008). UVG (Mercat et al. 2020). and DAVIS (Pont-Tuset et al. 2017). For 3D data, we randomly sample 1K furniture objects from Objaverse (Deitke et al. 2023). ). To enable compatibility with framework, we voxelize each point cloud into a binary voxel grid, forming a cube shaped tensor that encodes the object's geometry. We describe details of each dataset in Appendix A.9.
7

---

Published as a conference paper at ICLR 2026
Table Comparison between different architectures on Kodak, CLIC2020, and Objaverse. Results are reported in *SNR/MS-SSIN,
Table 3: Comparison with the existing NeRV methods on Bunny with different model sizes. Results are reported in PSNR.
Methoo
2D images
3D points
Model size
0.35M
0.75M
1.5M
3.0M
Kodak
CLIC2200
Objaverse
HNeeY
Chen et
2123
35.57
37.43
ComoNeXt
17.17007536
DNeRY
7ain or m.
$1.15
13.30
35.22
38.09
LPNes Cost
27.44X0.9132
25.41/0.8505
17,67/0.2815
OS-NeRV
31,20
33.82
36.44
38. 38.65
Mamba
27.51/0.9227
26. 1699.694
12.25/0.7322
36.76
39.64
LPNer-S3K (Our)
28.09/0.9331
26.33/0.8692
18.340.8492
Oure-S
32.93
35.74
37.83
39.99
Table 4: Comparison with the existing NeRV methods on UVG. indicates results reproduced by official codebases.
Table 5: Decoding speed
comparison on UVG
Beauty
Bexp-
Honry
Jockey
Ready:
Shuke
Avg:
Decnding Speed
HNoRV
(Che
N013
sechid (1)
(1)
HNNeRV 32.70
1.24
344.83
"PNeRtV
2002
34.18
35.55
31.51
25.54
35.30
30.27
33.22
Ous-H 32.79
1:74
544.83
DS-NeRV
30M
33.97
35:22
32.86
27.10
35.04
29:40
33:31
PNext 33.22
1.99
301.10
"SNeRV
23240
3.08
33.76
35.66
38.44
33.78
26.57
35.11
29.83
33.28
Ohmap 34.35
1.99
301 10
Ours
3.084
34.04
36.32
39.50
31.30
27.92
35:54
30:47
33.66
SNekV 33.28
10.08
59.52
Oua-PP
3.3M
34.22
36.54
38.71
35.40
29.31
35.85
30.50
34.36
Our 33,66
10.08
59.52
Table 6: Comparison with the existing NeRV methods DAVIS. indicates results reproduced by the official codebase.
Method
Bike Blackswan
BMX
Breakeiance
Camel
endaht
Dunce
RNORY
Chen
3.M
30.55
93.35
20.56
10:45
36.71
25.61
31:11
24.60
25.60
$1.04
26.30
DNGRV
3.4M
90.24
90.92
29.63
10.38
17.38
29:35
31.65
24.85
20:13
$1.32
20.57
*yNeRV
29.67
28.76
31.02
24.39
28.15
30.95
29.74
0S Nele
3.0
52,55
29.36
32.21
27.26
29.48
35.88
25.05
28.79
53.29
SNekV
30M
31.20
53.81
31.65
31 40
28.68
31.27
35.79
25.14
30.11
34.11
31.56
Dum-S
10M
34,33
34,58
11.22
33386
30.04
13:35
36.69
26.47
51255
$3.21
33,86
Dun-P
3.MM
35
34
32,48
13.15
20.04
32.38
12:15
25.04
30.94
33.97
33.25
Bunny
UVG
DWiefiv
33
- HNtfV
PS4H (88)
22
-0
23
28
27
Figure 4: Rate-distortion plot on Bunny (left) and UVG (right).
Evaluation metries. We adhere to the standard evaluation protocols, reporting Peak Signal-to-Non -o-issse Ratio (PSNR, in dB) and/or Multi-Scale Structural Similarity Index Measure (MS-SSIM) (Wang et al. 2003) as fidelity metrics across all reconstruction tasks.
Implementation details. For images and 3D objects, we adopt simple setup by attaching multiple 2D or 3D deconvolutional layers (Noh et al. 2015) on top of LPNet-S3K to reconstruet the input. For videos, we replace the convolutional encoders of HNeRV (Chen et al.) 2023), SNeRV (Kim) et al. 2024b), and PNeRV-L (Zhao et al. 2024) with LPNet-S3K, denoted as 'Ours-H', "Ours-S" and OOrrsPP, respectivlly.. Additional implementation details are provided in Appendix A.9.2
6.1 QUANTITATIVE COMPARISONS
Images and 3D objects. Quantitative results reported in Table To validate the effectiveness of LPNet (Sec. (S4c. the proposed bascline architecture, we construct a ConvNeXt (Liu et al. (022) variant that follows the same configuration of I. I.Net,, ConvNeXt is widely adopted encoder in
8

---

Published as a conference paper at ICLR 2026
Ground Truth
LPNet-S3K (Ours)
LPNet-Mamba
LPNet-Conv
ConvNeXt
PSNR 34 SSIM, 0.89
PSNR-23.39 MS SSIM 0.63
PSNR:24.70 MS SSIM 0.87
PSNR-24.75 ME-55 M: 0.87
POPPIR
POPPLE
2A.7.
PSNR.24.1
MS SSIM
PSNR.23.51
MS 5SIM:
ME-SSIM
25 SSI:: 0.99
PSNR:19.03 MS-SSIM:
PSNR.20.00 S3IM:
PSNR:21.80
Figure 5: Reconstruction results images (Kodak, CLIC2020) and voxelized points (Objaverse)
Ground Truth
Ours-S (3.0M)
SNeRV (3.0M)
PSNR:33.86
PSNR:31.40
DNeRV (3.4M)
PNeRV (3.4M)
HNeRV (3.0M)
PSNR:30.88
PSN -29.7
PSNR:30.4
Figure 6: Reconstruction results on DAVIS video
the literature on data-specific neural representations (Chen et al. 2023 Zhao et al. 2023 2024 Kim
cet al. 2024b). For fair comparison with ConvNeXt, integrate standard convolutional layers into PNet, denoted as LPNet-Conv. The performance gap between ConvNeXt and LPNet-Conv the superiority of the LPNet architecture. We then compare LPNet-Conm and LPNet-S3K to assess the contribution of S3K convolution; LPNet-S3K outperformed LPNer-Conv in
across all benchmarks as shown in the table, underscoring the benefits of SSMs for data-specific neural representations. To assess the distinct advantage of S3K, we also experiment with Mamba (Gu Dao 2023) as an alternative SSM (LPNet-Mambu), Specifically, we switch the S3K convolutions with Mamba followed by a standard convolutional layer. LPNet-S3K outperforms LPNet-Mami showing its effectiveness as compression- compression-speecalizzee specialized SSM. Refer to Appendix A.9.3 for further ablation studies. It is worth noting that the approach introduced by LPNet and S3K is orthogonal to prevalent techniques data-specific neural representations, such as quantization (Kim et al.)
2024a| Ladune et al. 2023 Damodaran al., 2023) or learning image priors from large-scale
datasets (Ballé et al. 2016 2018 Cheng et al. 2020 Strümpler et al. 2022 Catania & Allegra
2023). This distinction highlights the potential complementarity of our framework: believe that
integrating LPNet-S3K with these existing techniques could further enhance performance and offer new insights into the design of compact and effective data-specific neural representations.

---

Published as a conference paper at ICLR 2026
NeRV benchmarks. We evaluate our method standard NeRV benchmarks, with results shown in Table (Bunny). Table44and 5 (UVG), and Table 6 (DAVIS). We include convolution- based NeRV models as our baselines, following Yan et al. (2024) and Kim et al. (2024b). Additional comparisons to MI.P-hased methods are provided in Appendix A.9.4. On Bunny, our method ranks the best across all model sizes. On UVG, our method outperforms previous arts and even surpasses models of larger sizes: Ours-S achieves 33.66 PSNR with only 3.0M parameters, exceeding the performance of bigger models like DNeRV and PNeRV. We provide an additional rale-distortion plot in Fig. 4 where we vary the bits-per bits-eer-pixel (bpp) and compare the corresponding model performance. Our model consistently outperforms the baselines across different bpp levels, demonstrating its effective compression capability across a wide range of compression ratios. It is noteworthy that our method enhances the performance while leaving the decoder part unchanged, ensuring the inference cost remains the same (T This aspect offers a meaningful advantage in NeRV, since video decoding speed is critical for its real-time streaming applications. On DAVIS, our model ranks either first or second across various vidcos, surpassing most prior methods. All results are obtained by employing existing decoders, indicating that the performance gains are entirely from our SSM-based encoder; this also implies even greater potential of our method with dedicated decoder design.
6.2 QUALITATIVE ANALYSIS
We present qualitative comparisons on images and 3D objects in Fig. 5 showing reconstruction results on Kodak (Kodak, 1993). CLIC2020 (Toderici et al.] (020), and Objaverse (Deitke et al. 2023). Across all datasets, our model consistently preserves finer details, such as high-frrquenny textures in the background (first row) or legible text on signage (second row), and underlying structure of 3D geometry (third row). We provide video reconstruction results in Fig. 6 Our model shows superior performance despite its smaller model size compared to PNeRV and DNeRV, and effectively preserves fine details, such as a person's hand or typographies on a t-shirt. Additional qualitative results are provided in Appendix A.9.5
CONCLUSION AND FUTURE DIRECTIONS
In this paper, we present the first attempt to link SSMs to data-specific neural representations. To this end, we explore network architectures for effective SSM integration and analyze how different architectures suit the characteristics of SSMs. As a result, we propose S3K, which harnesses the expressiveness of SSMs while enabling natural multi-dimensional data processing and downsampling. These results are theoretically supported and together lead to a novel and powerful data-specific neural representation framework. Our framework achieves strong performance across diverse visual data formats, including images, videos and 3D objects, and remains superior on challenging NeRV benchmarks despite not being designed for NeRV.
In the following, we outline potential directions to improve and extend our framework.
Designing a dedicated decoder: This work focuses primarily on designing the SSM-based encoder, while employing simple upsampling decoder or decoders from other methods. While results demonstrate that the encoder alone significantly contributes to the performance improvements, decoder tailored to the characteristies of the SSM encoded features may further improve the performance of our model.
2. Reducing encoding complexity: Constructing input-sized kernel using state-space model parameters leads to substantial computational overhead: about 20x more memory and 4x more FLOPs than a plain convolution. While effective, this approach may limit scalability. More efficient alternatives- such as avoiding explicit kernel construction through mathematically equivalent formulations (Nguyen al. 2022 Gu et 2021a), or employing -pptiiiee implementations (Gu & Dao (023) could alleviate this burden.
3. Application to autoencoders: Although our method is proposed for data-specific neural net- works, its compressive property can be exploited to produce compressive representation beyond individual inputs. Note that modern autoencoders used for generative modeling often rely on convolution- baasdd architectures (Black Forest Labs, 2023 Rombach et al. 2022) or signal pro- cessing methods (Agarwal et al. 2025). S3K, which aligns closely with both convolution and signal processing principles, could enable compressive autoencoders that encode inputs using fewer tokens.
10

---

Published as a conference paper at ICLR 2026
8 ACKNOWLEDGEMENTS
This research was supported by the High-Performance Computing Support Project (RQT-25- 070271) and the IITP grants (RS-2024-00509258, RS-2024-00469482, RS-2024-0045788., RS- 2019-11191906), funded by the Government of the Republic of Korea (Ministry of Science and ICT).
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, al. Cosmos world foundation model platform for physical ai. arXiv preprint XXiv:25010033755 2025. T0
Johannes Ballé, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compression. arXiv preprint arw.v: 2016.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with scale hyperprior. arXiv preprint arXiv:/802.01436. 2018.
Black Forest Labs. Flux.
cowp:/th......mlalaaaa..a....b forest latsstt-aab/fllus 2023. 10
Peter J Burt and Edward H Adelson. The laplacian pyramid as compact image code. In Readings in computer vision, pp. 671-679. Elsevier, 1987.
Lorenzo Catania and Dario Allegra. Nif: A fast implicit image compression with bottleneck layers and modulated sinusoidal activations. In Proceedings of the 31 ACM International Conference on Multimedia, pp. 9022-9031, 2023. 9
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European conference on computer vision, 333-350, Springer, 2022a. 2
Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, and Limin Wang. Video mamba suite: State space model as a versatile alternative for video understanding. arXiv preprint arxiv:2403.09662
2024.
Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser Nam Lim, and Abhinav Shrivastava. Nerv: Neural representations for videos. Advances in Neural Information Processing Systems, 34:21557-21168,, 2021.
Hao Chen, Matthew Gwilliam, Ser-Nam Lim, and Abhinav Shrivastava. Hnerv: A hybrid neural representation for videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10270-10279, 2023.
9
10
Yinpeng Chen, Xiyang Dai, Mengehen Liu, Dongdong Chen, Lu Yuan, and Zicheng Liu. Dynamic convolution: Attention over convolution kernels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11030-11039, 2020.
Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, and Xiaolong Wang. Videoinr: Learning video implicit neural representation for continuous space-time super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2047-2057, 2022b. II
Ward Cheney. Multivariate approximation theory: Selected topics. SIAM, 1986.
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. Proceedings of the IEFF/VVF conference on computer vision and pattern recognition, pp. 7939-7948, 2020.
J Cooley, Lewis, and P Welch. The finite fourier transform. IEEE Transactions on audio and electroacousties, 17(2):77-85, 1969.
Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999.
11

---

Published as a conference paper at ICLR 2026
Bharath Bhushan Damodaran, Muhammet Balcilar, Franck Galpin, and Pierre Hellier. Rqat-inr: Improved implicit neural image compression. In 2023 Data Compression Conference (DCC), pp. 208-217. IEEE, 2023. [
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13142-13153, 2023. 2 TO
Emilien Dupont, Adam Goliñski, Milad Alizadeh, Yee Whye Teh, and Arnaud Doucet. Coin: Compression with implicit neural representations. arXiv preprint arXiv:2103.0312. 2021.
Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From data to functa: Your data point a function and you can treat it like one. arXiv preprint arXiv:2201.12200 2022. 0
Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid- linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3-11, 11, 2018.
Cameron Gordon, Shin-Fang Chng, Lachlan MacDonald, and Simon Lucey. On quantizing implicit neural representations. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 341-350, 2023.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, arXiv preprint arXiv:2312.00752 2023.
10
Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. Hippo: Recurrent memory with optimal polynomial projections. Advances in neural information processing systems, 33: 1474-1487, 2020.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces, arXiv preprint arXiv:2111.00330 2021a.
10
Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré, Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in neural information processing systems, 34:572-585, 2021b.
Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems, 35:35971- 35983, 2022a.
2
5 7
Albert Gu, Isys Johnson, Aman Timalsina, Atri Rudra, and Christopher Ré. How train your hippo: State space models with generalized orthogonal basis projections. arXiv preprint arXiv:2206. 12037,
2022b.
3 6
Hang Guo, Jinmin Li. Tao Dai, Zhihao Ouyang, Xudong Ren, and Shu-Tao Xia. Mambair: A simple baseline for image restoration with state-space model. In European Conference Computer Vision, pp. 022-241. Springer, 2025. 2
Zongyu Guo, Gergely Flamich, Jiajun He, Zhibo Chen, and José Miguel Hernández- Lonato. Com- pression with bayesian implicit neural representations. Advances in Neural Information Processing Systems, 36:1938-11999 -9966 2023. 2
Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are effective as structured state spaces. Advances in Neural Information Processing Systems, 55522982-22994, 2022. 3
Vincent Tao Hu, Stefan Andreas Bauma, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Fis- cher. and Bjor Ommer. Zigma: Zigzag mamba diffusion model. arXiv preprint 13802 2024. 2
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017. 8
12

---

Published as a conference paper at ICLR 2026
Hyunjik Kim, Matthias Bauer, Lucas Theis, Jonathan Richard Schwarz, and Emilien Dupont. C3: High-performange and low-complexity neural compression from a single image or video. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9347-9358, 2024a
10
Jina Kim, Jihoo Lee, and Je-Won Kang. Snerv: Spectra-preserving neural representation for video. In European Conference on Computer Vision, pp. 332-348. Springer, 2024b.
TO
Subin Kim, Sihyun Yu, Jaeho Lee, and Jinwoo Shin. Scalable neural video representations with learnable positional features. Advances in Neural Information Processing Systems, 35:12718- 12731, 2022.
I0
Eastman Kodak. Kodak lossless true color image suite (photocd pcd0992).
URL http://rOk.
ua/graphies/kodak, 6:2, 1993. 10
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ho Man Kwan, Ge Gao, Fan Zhang, Andrew Gower, and David Bull. Hinerv: Video compression with hierarchical encoding- based neural representation. Advances in Neural Information Processing Systems, 36, 2024. 10
Théo Ladune, Pierrick Philippe, Félix Henry, Gordon Clare, and Thomas Leguay. Cool-chic: Coordinate-based low complexity hierarchical image codec. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 13515-13522, 2023.
Serge Lang. Algebra, volume 211. Springer Science & Business Media, 2012.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta-learning for domain generalization. In AAAI, 2018.
Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Yali Wang, Limin Wang, and Yu Qiao. Videomamba: State space model for efficient video understanding. In European Conference on Computer Vision, pp. 237-255. Springer, 2025. 2
Shufan Li, Harkanwar Singh, and Aditya Grover. Mamba-nd: Selective state space modeling for multi-dimensional data. In European Conference on Computer Vision, pp. 75-92. Springer, 2024.
Dingkang Liang, Xin Zhou, Wei Xu, Xingkui Zhu, Zhikang Zou, Xiaoqing Ye, Xiao Tan, and Xiang Bai. Pointmamba: simple state space model for point cloud analysis. arXiv preprint
arXiv:2402.10739, 2024. 2
Jinming Liu, Heming Sun, and Jiro Katto. Learned image compression with mixed transformer- enn architectures. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14388-14397, 2023.
Zhen Liu, Hao Zhu, Qi Zhang. Jingde Fu, Weibing Deng, Zhan Ma, Yanwen Guo, and Xun Cao. Finer: Flexible spectral-bias tuning in implicit neural representation by variable-periodee activa- tion functions. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern
Recognition, pp. 2713-27227 2722, 2024. 10
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976-11986, 2022.
Ming Lu, Peiyao Guo, Huiqing Shi, Chuntong Cao, and Zhan Ma. Transformer-based image compression. arXiv preprint arXhv:2111.06707, 2021.
Ricardo Martin- Brralla,, Noha Radwan, Mehdi SM Sajjadi, Jonathan Barron, Alexey Dasovitskiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pp. 7210- 7219, 2021.
13

---

Published as a conference paper at ICLR 2026
Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Condi- tional probability models for deep image compression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2018. B
Fabian Mentzer, George Toderici, David Minnen, Sung-Jin Hwang, Sergi Caelles, Mario Lucic, and Eirikur Agustsson. Vct: A video compression transformer. arXiv preprint (tt.i::22066070000 2022.
Alexandre Mercat, Marko Viitane,, and Jarno Vanne. Uvg dataset: 50/120fps 4k sequences for video codec analysis and development. In Proceedings of the 1 11th ACM Multimedia Systems Conference, pp. 297-302, 2020.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021. 2
Eric Nguyen, Karan Goel, Albert Gu, Gordon Downs, Preey Shah, Tri Dao, Stephen Baccus, and Christopher Ré. S4nd: Modeling images and videos as multidimensional signals with state spaces. Advances in neural information processing systems, 35:2846-2861, 2022.
Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmentation. In ICCV, 2015.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 165-174, 2019.
Federico Perazzi, Jordi Pont-Tuset, Brian Mc McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Homunng A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 724 732,
2016. 2
Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine Hormung, and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.0000 2017. 7
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10318-10327, 2021.
Rao. Sensitivity analysis of state space methods in spectrum estimation. ICASSP 87. IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 12, pp. 1517-1520. IEEE, 1987.
Bhaskar D Rao and KS Arun. Model based processing of signals: A state space approach. Proceedings of the IEEE, 80(2):283-300 1992. 2
lain E Richardson. The H. 264 advanced video compression standard. John Wiley & Sons, 2011.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022. I0
Ton Roosendaal. Big buck bunny. In ACM SIGGRAPH ASIA 2008 computer animation festival,, pp. 62-62. 2008. 2
Jiacheng Ruan and Suncheng Xiang. Vm-unet: Vision mamba unet for medical image segmentation. arXiv preprint arXiv:2402.0249) 2024. 2
Vishwanath Saragadam, Daniel LeJeune. Jasper Tan, Guha Balakrishnan, Ashok Veeraraghavan, and Richard G Baraniuk. Wire: Wavelet implicit neural representations. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 18507-18516, 2023.
14

---

Published as a conference paper at ICLR 2026
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings the IEEE conference computer vision and pattern recognition, pp. 1874-1883 1883, 2016.
Seungjun Shin, Suji Kim, and Dokwan Oh. Efficient neural video representation with temporally coherent modulation. In European Conference on Computer Vision. Springer, 2024.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances neural information processing systems, 33.7462-7473, 2020.
10
Ivan Skorokhodov. Savva Ignatyev, and Mohamed Elhoseiny. Adversarial generation of continuous images. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10753-10764, 2021. T
Jimmy TH Smith, Andrew Warrington, and Scott W Linderman. Simplified state space layers for sequence modeling. arXiv preprint arXiv: 04933, 2022. 2
5
Yannick Strümpler, Janis Postels, Ren Yang. Lue Van Gool, and Federico Tombari. Implicit neural representations for image compression. In European Conference on Computer Vision, pp. 74-91.
Springer, 2022.
9
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich- Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
George Toderici, Wenzhe Shi, Radu Timofte, Lucas Theis, Johannes Balle, Eirikur Agustsson, Nick Johnston, and Fabian Mentzer. Workshop and challenge on learned image compression (clic2020). In CVPR, 2020. 7
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurlPS, 2017. 5
Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for image quality assessment. In The Thrity-Seventt Asilomar Conference on Signals, Systems & Computers, 2003, volume 2, pp. 1339-14422 Ieee, 2003. 8
Sanghyun Woo, Shoubhik Debnath, Ronghang Hu,, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16133-
16142, 2023. 7
Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, and Lei Zhu. Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assiste Intervention, pp. 578-588. Springer, 2024. 2
Dejia Xu, Peihao Wang, Yifan Jiang, Zhiwen Fan, and Zhangyang Wang. Signal processing for implicit neural representations. Advances in Neural Information Processing Systems, 35:13404-
13418, 2022. 1
Yuhui Xu, Yongzhuang Wang, Aojun Zhou, Weiyao Lin, and Hongkai Xiong. Deep neural network compression with single and multiple level quantization. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Hao Yan, Zhihui Ke, Xiaobo Zhou, Tie Qiu, Xidong Shi, and Dadong Jiang. Ds-nerv: Implicit neural video representation with decomposed static and dynamic codes. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 23019-23029, 2024. 5
Yibo Yang, Robert Bamler, and Stephan Mandt. Improving inference for neural image compression. Advances in Neural Information Processing Systems, 33:573-584, 2020a. 3
15

---

Published as a conference paper at ICLR 2026
Yibo Yang, Robert Bamler, and Stephan Mandt. Variational bayesian quantization. In International Conference on Machine Learning, pp. 10670-10680. PMLR, 2020b. B
Yijun Yang, Zhaohu Xing, and Lei Zhu. Vivim: video vision mamba for medical video object
segmentation. arXiv preprint arXiv:2401.4466. 2024. 2
Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances Neural Information Processing Systems, 32, 2019.
Kai Zhang, Germot Riegler, Noah Snavely, and Vladlen Koltun. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492 2020.
Yunfan Zhang, Ties Van Rozendaal, Johann Brehmer, Markus Nagel, and Taco Cohen. Implicit neural video compression. arXiv preprint arXiv:21 2021. 2
Qi Zhao, M Salman Asif, and Zhan Ma. Dnerv: Modeling inherent dynamics via difference neural representation for videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and
Puttern Recognition, pp. 2031-2040, 2023. g 10
Qi Zhao, M Salman Asif, and Zhan Ma. Pnerv: Enhancing spatial consistency via pyramidal neural representation for videos. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 19102-19112, 2024.
10
Lei Zhou, Chunlei Cai, Yue Gao, Sanbao Su, and Junmin Wu. Variational autoencoder for low bit-rate image compression. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 2617-2620, 2018. |
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417, 2024a. 2
Qinfeng Zhu, Yuanzhi Cai, Yuan Fang, Yihan Yang, Cheng Chen, Lei Fan, and Anh Nguyen. Samba: Semantic segmentation of remotely sensed images with state space model. Heliyon, 10(19), 2024b.
16

---

Published as a conference paper at ICLR 2026
A APPENDIX
This material provides proofs for the theorems in the main paper (i.e. Theorem 4.1 and Theorem 5.1), and additional details omitted in the manuscript due to the space constraint.
A.1 SOLVING THE LINEAR DIFFERENTIAL EQUATION
We solve the linear differential equation of the state-space model (Eq. (T). and derive its solution in the form of Eq. 22) As the derivation in this section depends the original state-space formulation, we first restate Eq. T from the main paper for reference:
N' (x()) Ah(x) + Bx(x)).
(10)
We start by solving homogeneous first-order matrix ordinary differential equation (ODE), M/ (x) A (2)), which is standard matrix ODE. Its solution is h4 (x: e A=C, where C C is constant vector determined by initial conditions. Allowing the constant C to vary with Le. C := u(z),
derives the particular solution of the form h(x) (Ar)()). Plugging this Eq. (10 yields:
d u(x) Ah(z) + Bo(②)
(11)
dx
Ar u(x) u Ac + BO(x),
(12)
and canceling the terms gives:
Ar u (x) Bó(x))
(13)
+ ul (z) = E --ArB((r)
(14)
+ u(x) = /-² TF
E - c-ATBo(T)dT + C".
(15)
For practical implementation, since we set h(0) u(0) 0, we can set C' 0. Thus, h(x)
(1)) becomes:
hxx) Ar u(x)
(16)
Av
Ar BobT)dT
(17)
/ Bo(()dd
(18)
which matches the target expression (Eq.
A.2 PROOF FOR THEOREM 3.1
Theorem 3.1. Let A be diagonalizable over C with non-zero distinct eigenvalues { Given A, B, and the hidden state computed by Eq. (2). there exists a function f (A, + E NNN with which one decompose the input function (t)) linear combination of complex exponentials:
((1) ())
(19)
where is the n-th element of fAA, B)..
Proof. Since is diagonalizable, we can write
E(T, L) c(L-r)AB
(20)
= -Vell-jjV--B 'B,
(21)
for some matrix V E CNN and diagonal matrix A E CNNN
Let V- so that
b1
V =
(22)
bN

---

Published as a conference paper at ICLR 2026
Since e(!-77)A is diagonal,
elb-r)deb b1 ell elb-r)dabz
ell-T)Ab=
(23)
e ell ell-r)Aor bNN
and thus, multiplying V yields:
clI--r)asb b1 ell. T2222
VelL-TIAD=
(24)
Vine T))n (lne((-r))mb bn -==1 U (I. bn
(25)
En= uNnelL T)) bn
N
where Usj is the (i,j)-th element of the matrix V. Hence, the (T, x)) from Eq. (0) can be expressed as:
ET L)
eknelL- (agelLL-))nn -))ab
(26)
dkn dnneeAn (b-F)
(27)
for some constant den. Hence, it becomes natural to choose sinusoidal bases e00, (1- and express the input function ((t) as:
(L-))
(28)
Note that (t-i(t-t)) consists of complex exponentials with N distinct frequencies, which ensures their linear independence (Lang) 2012). Since this property allows them serve as valid basis functions, can obtain orthonormal basis functions (t)) that span the same functional space as (e2n(1-0)). Let the change-of-basic matrix from (t)) to (cAn(L-1)) defined P:
w vi((1)
(t)
P
(29)
(()
(L-(L-
Then, EN from Eq. 27 can be rephrased to:
ed1(L-7)
e>a(L-7)
(x L)
-dkk
dez
dEN]
(30)
(AN(L--T)
(1(T)
W2(7)
[dka
dez
deN) P
(31)
(T)(T))
N
Ikn (T)()))
(32)
2

---

Published as a conference paper at ICLR 2026
where 9kn is the inner product between [dxt da2 danl and n-th column of P. Sim-
ilarly, we can express (t) from Eq. (28) with different bases, i.e., _n mn Un (t), where
mi mg mN
[c1 C2 CN P. Then, plugging sb(1) En=1 mn (n) into
Eq. (3) gives:
lk
mn
(T), D)
(33)
Myn (T),
Σ
ghi (T))00001111111
(34)
==1
7711
m2
mnykn
19ka 9k2
9kN
(35)
M
P'c.
9hl 9k2
9kN
(36)
where с == |er
C2
cx]²
Let
[h1
h2
hN]
and G
Stacking Eq. (36) yields:
GP C
c= (GPT)-'E,
(37)
which enables us rewrite <(t) by plugging to Eq. (28
A.3 PROOF FOR THEOREM 4.1
Theorem 4.1. Let A be diagonalizable over C with non-zero distinct eigenvalues { and A be
the step size used for the discretization of A. Given the final hidden state h E C after applying S3K, there exists a function (A, B, h) + H E C with which one can reconstruct the imput sequence as:
R h) [cM(LLA-A)]] 1=12....
(38)
1.12.....
Proof. Let LA ZL-1. Note that
T h = 1" (T (
(39)
The diagonalizability of A gives:
(x-r)A --
(40)
Let V B = B have elements, then
h: V-1h
(41)
A(T A(T-r)
Bo(r)dr
(42)
h
f'
T
Ck
B ((()-r
(43)
Note that our goal is to recover ((T) from h. This can be accomplished by finding the dual basis function (f((T)} of the basis function (T)u(T-T)) for E {1,2, ...,,,..,,,.. N). The dual basis function (T) satisfics satisfies
{ ( = dij-
(44)
3

---

Published as a conference paper at ICLR 2026
where day is the Kronecker delta. It is worth to note that having fi((T) leads to the expression of )(T)
as:
(Tr)
caff((T),
(45)
which can easily shown when plugging Eq. finding the dual basis {fx] that corresponds to {e^x T) linear combination of (Ax(T-T);
to Eq. (43 Thus, our problem is now converted
We start from expressing fy(T) as
fT(T) Σ ()xea((T-T)
(46)
for some zjk- Plugging Eq. (46)
Eq.
gives:
(( T)
M
A -5) d1
(47)
k=1
Σ Zjk
e(Ac+An)(T -T dej
(48)
we let the matrix Z = {zik] and G = [gis] = ( (
dithe }. directly solving the integral yields:
eAActde)T diz = ZG =1 I.
(49)
A + d
Hence, we obtain Z = G According to Eq. (46).
EA1
-72((T-7)
f(r) = G-1
(50)
(dN(T-T))
and plugging this to Eq. 44 gives:
(A1(T-T)
(72(T--7)
((T) = c' c² cd
(51)
(AN(TTT)
C1
where :
If we put everything together,
CN
A1(T-T))
TT)
= = G-
(52)
(x((T-
(T-Tr)
V-1h - B-IB
(T
G--1
(53)
(xx(T-r)
We abused element-wise division operation here for simplicity. (0, A, 2A, (T. })A} completes the proof.
Evaluating ((T) at 7
4

---

Published as a conference paper at ICLR 2026
Incermedlate feoture maps (from the last two layens
output image
Input image
60
050
Figure A1: Feature map visualizations of three encoder variants explored in Sec 4.2
Figure A2: SSM block output of Image pyramid (top) and Laplacian pyramid (bottom) variants. The rightmost column shows correlation matrix between these maps.
(a), (b) and (c) correspond to each variant Fig.
Table A1: Performance (PSNR) of different architec- tures incorporating various SSMs for the ID signal task. (a), (b), and (c) indicate the encoder variants illustrated in Fig. A4
Encode
Decoder
Input agnal
Recomdracted giaall 1004)
Conv m
8 1024
SSM Block
PSNR
Baseline
(a)
(c)
Signal embedding
Transoomer Veewani
2017
28:09
27.9%
27.27
27.47
Cu 2011
29.81
28.43
28.66
31.20
$407 Toul cuse6
28.99
26.24
27.73
29.94
Figure A3: Baseline architecture for ID signal reconstruction
S4ND Ngowa
2023
29.98
28.17
31.05
Manins (Gu & Eal
2023
30.62
26.78
27.30
30.73
A.4 VISUAL MATERIALS TO SUPPORT EMPIRICAL FINDINGS IN SEC.
Figure A1 supports our hypothesis on why stacking SSM layers deteriorates the reconstruction: the feature maps from stacked SSM layers show clear block-like artifacts, and these distortions persist into the reconstructed output. The consistency between feature-level and output-level artifacts aligns with hypothesis that stacking SSM projections accumulates errors, ultimately limiting the achievable reconstruction quality.
Fig. A2 illustrates that residual maps provide meaningfully distinct inputs the model: in the 'Laplacian pyramid' variant, the outputs of the SSM blocks vary substantially across scales, whereas the 'Image pyramid' variant, the outputs exhibit noticeable redundancy.
A.5 SSM FOR ID SIGNAL RECONSTRUCTION
As natural extension of the reconstruction experiment in Sec. on ID signal reconstruction to further validate the effectiveness of SSM-based architectures in capturing signals. We follow the similar experimental setup described in Sec. 4.2 with the primary modifications being the use of 1D convolutional layers in place of 2D ones and adjustments to the input dimensionality. Specifically, we convert input images into signals by flattening them in zig-zag manner, transforming the input shape from (3, 32, 32) to (3, 1024). The kernel size of the convolution is maintained to 8, resulting in an encoded embedding that is one-cighth the length of the input. For clarity, illustrate the modified baseline architecture and its variants in Fig. A3 and Fig. A4 respectively.
conduct additional studies
The results of the experiment demonstrated in Table Al observed in Sec. S.2. which suggests that SSMs, when placed in an appropriate reconstruction setting, hold strong potential in signal reconstruction implied in Sec. 4.1
We find the similar tendency have

---

Published as a conference paper at ICLR 2026
SSMi block
Comu
AUD
C100
SSM block
SSM block
Conv
Conv
Downsample
Ixn.conv
COTM
pyremi
Lallaesn
3x1 conv
1xl conv
3Xl cDnV
55M block
5SM block
5SM block
55M block
55M block
5SM block
55M block
[8,
512)
(8,256)
18. 320)
[9.1024]
(8,522
18, 256)
18.128)
(a) Stacked
(b) Signal pyramid
(c) Laplacian pyramid
Figure A4: Encoder variants incorporating SSMs for ID signal reconstruction
Table A2: Parameter count comparison different variants of S3K 2D convolution.
subsequent
Adaptive
Adaptive
Real
16.
Method
x 1 conv.
B
A,B
param
K N, 16
Co.,))(..D.K)
CDK -
3088
X
6N ANC
288
X
X
X
6N + 4NC + (N 1)D
560
S3KConv2d (C.N.D.K)
/
X
x
8N + 2NK + (N + I)D
656
/
V
x
8N | 2 K (N 1)D
664
+ 2NK + (N I)D 6N + 2NK K (N I)D
624
V
632
Table A3: Computation cost comparison between SSM-convolttion variants, measured on Kodak.
Method
FLOPs (G)
Memory (MB) J.
Time (ms) +
PSNR
S4D Conv
2.00
453.3
100.4
26.73
Mamba Cony
2.36
667.3
94.4
27.51
S3KConv
3.34
3445.8
62.4
28.09
A.6 PARAMETER COUNT ANALYSIS
As discussed in Sec. 5.3 the direct application of S3K convolution suffers from limited expressivity, primarily due to insufficient parameter budget that constrains the network's representational capacity. Table A2 presents how each modification contributes to the parameter count of the 2D convolution using S3K. Following the standard convolutional network notation, Conv2d(C. D, K) denotes a 2D convolutional layer that transforms an input with C' channels into D output channels using kernel of size K. Similarly, S3KConv2d(C: N, D, K)) performs the transformation but introduces an intermediate state of size N to model the structured state-space dynamics. The primary contributor the high parameter count in standard convolutional networks the CDK term, which involves the multiplication of four factors and grows rapidly with channel and kernel size. In contrast, S3K layers are designed with more compact parameterization, where the largest terms involve only two multiplicative factors, resulting in significantly fewer parameters. Among all the architectural modifications, the most significant increase in parameter count arises from the subsequent 1 x convolution, which projects the N-dimensional latent state into the desired size of output feature. On the other hand, the adaptivity spate parameters also introduces a relatively modest parameter increase, though the exact impact depends on the choice of state size N.
A.7 COMPUTATIONAL COST ANALYSIS
As discussed in Sec. 7 our applicit-kerne formulation leads to additional computational burden compared to other SSM-based convolutions that avoid kernel materialization. For explicit comparison, we benchmark these approaches under the same setting by replacing the convolution in our architecture with an S4D or Mamba block followed by a standard convolution layer in Table A3 The results highlight the trade-offs: SSM-convolutions that do not materialize the kernel (i.e., S4D, Mamba) consume less memory relative to our explicit kernel, but require additional runtime due to the sequential structure of their implicit state updates. Conversely, S3KConv incurs higher memory due to explicit kernel construction, but benefits from more parallel, convolution like execution and achieves better reconstruction quality.

---

Published as a conference paper at ICLR 2026
A.8 MODEL IMPLEMENTATION DETAILS
This section provides additional implementation details that were omitted from the main paper due to space constraints.
A.8.1 INITIALIZATION OF A AND B
We mainly follow the initialization scheme introduced in S5 (Smith et al. 2022) and Mamba (Gu & Dao 2023). For complex initialization A and B for ablation, we follow HiPPO initialization A and use cigenvectors V from diagonalization of A for initialization of B, as done in S5 (Smith et al. 2022). For real implementaionn, we set A, the diagonal elements of A, be A, - 1), and employ normal initialization for B.
A.8.2 OPERATIONAL DETAILS OF S3KCONV2D LAYER
We elaborate on operational details of S3KConv2d layer, which naturally extends to S3K convolutions for N-dimensional inputs.
S3KConv2d layer takes the same arguments of the ordinary convolutional layer: input channel dimension Cin- output channel dimension Cour. and kernel size (K,, 1, K2). For simplicity, we assume trivial settings for stride, padding, and dilation. Let the input Xo R B Wo, then we first project input to have C' channel dimension by 1 x convolution: construct a kernel for each spatial dimension. For clearer explanation, RxCx K2 K2, which represents the local window extracted during convolution. The following operations are applied in parallel across all such windows as the kernel slides over the input. Note that in Mamba (Gu & Dao 2023), a linear layer is applied to the length- - input sequence to obtain the input-adaptive B of length T.. Since the kernel of i-th dimension needs to be length-K, 1D
[BBxCxHoXWo Now we focus the input X
proj (dm: doun) be the linear projection layer that transforms the channel dimension from din dous produce B B used to construct the i-th dimension S3K ID kernel. Then, we set (din, doot) (K2, N) for i = 1 and (K1, N) for i = 2, so that we obtain (+) E BBBC'x X.N With A (4) E R&N and step size
kernel, we apply linear layer to the other spatial dimension. Specifically, let B(i) (
()) E RN,, we discretize A (6) and ((i) using zero-order hold (ZOH) method:
A
A (940)
B (A
(6)
I)B()
(54)
and compute the kernel in Eq. 7 Since have kernel K BXC*K; we take outer product of these kernels to construct 2D kernel K E R BxCxXXx K2 x K2 and apply this kernel to the input X
(ou) E BBXXXIX1
where (X((ou)) Jin.
ΣΣΣ
((X,). (p. K A.am
(55)
A2
An
We also incorporate gating mechanisms (Gu et al. and residual connections for complete implementation, as they have proven effective and are widely adopted standard components in SSM block designs. Specifically,, we project the initial input Xo to have N channels: X E R Bx X and 2D average pool with the same kernel size (K1, K2) and the stride produce the tensor X matching the output size of the convolution. SiLU (Elfwing et al. 2018) activation is applied to this residual tensor, which is then used to gate the convolution output via element- wise multiplication. Finally, add x back the gated output complete the residual connection.
2022a Nguyen et al.
2022 Gu & Dao 2023
A.9 EXPERIMENT DETAILS
This section provides additional experiment details that were omitted from the main paper due to space constraints.
A.9.1 DATASETS
Kodak, CLIC2020. The Kodak dataset (Kodak) [993) is a set of 24 natural photographs of
resolution 512 x 768. The CLIC2020 (Toderici et al. 2020) dataset includes 41 images of varying
7

---

Published as a conference paper at ICLR 2026
Table A4: Implementation details of the experiments from Sec.
Learning
Dataset
Encoder
Enc. strides
Decoder
Dec, strides
Feature dims.
rate
Con<NeXt
[16, 4,
Conv Transpose
[4, 44,4,4,2, 2]
[64, 64, 64, 16]
le-2
Kodak
LPNet-Conv
[16. 4,
Conv ConyTranspos:2D
[4, 4,
[64, 64, 64, 16]
3e-2
1. PNet-Mamhu
[32, 2, 21
[4,4,4,
[64, 64, 64. 16]
1e-2
LPNet-S3k
[32, 2, 2]
Conv Transpose
[4,4,
[64, 64, 64, 16]
2e-2
Con<NeXt
[4, 2]
Conv' Conn/Transpose2D
[4,2,
[64, 64, 64, 16]
le-2
LPNet- Conv
[4,
Transpos:2D
2,,
[64, 64, 16]
5e-3
CLIC2020
LPNet-Mamba
2..
Conv CoarYraaspos:2D
2,,
[64, 64, 64, 16]
le-2
LPNet-S3K
18,
Conv TaaTrraaspop2D
[4.2.
[64, 64, 64, 16]
1.6e-2
ConvNeXt
2,
Conv 3r
2,,
[64, 64, 64, 16]
30:3
LPNet-Conv
2,
Cony TransposeSD
2,
[64, 64, 16]
3e-3
Objaverse
Mamhha
2, [4, 2]
Conv Comy/ransspoe331
2, 2|
164, 64, 64, 16]
4e-3
LPNet-S3K
Cony SanYTranspos:3D
2,
[64, 64, 64, 16]
3e:3
Bunny
LPNet-S3K
4. 2]
SNeRV KKim et
2024b
[5.4.4.2.2) 2]
[64, 64, 64, 16]
3e-4
LPNet-S3K
[10, 2]
HNeRV
Chen et
2023
3,,
[64, 64, 16]
2e-4
UVG
LPNet-S3K
[10, 3.
SNeRV Kim et
20246
[5.4.433.2
[64, 64, 64, 16]
2e-4
LPNet-S3K
[10, 3.
PNeRV-L
Zhao ot m
2024
2c-4
LPNet-S3K
[10, 3,
SNeRV Kim et ml 2074b
[5,4,4,3,]
[64, 64, 64, 16]
2e-4
DAVIS
LPNet-S3K
[10, 2]
PNeRV
2e-4
resolutions, allowing side lengths up to 2048 pixels. Both datasets are commonly used for image compression tasks, they contain rich high-frequency details and complex scenes.
Bunny, UVG, DAVIS,
Bunny (Roosendaal 2008 is a 132-frame, animated short film, while
UVG (Mercat et al. 2020) is long 1080p video dataset comprising sequences of 300 or 600 frames. Both are widely used benchmarks for video compression. DAVIS (Huang et al.) 2017) is a densely annotated featuring short 1080p video clips, commonly used for video segmentation. Since NeRV benchmarks often include a subset of DAVIS, we follow this convention and select the following video clips; 'bike-packing', 'blackswan', 'bmx-trees', 'breakdance', 'camel', "car-roundabout" 'car-shadow', 'cows', 'dance-twirl', and 'dog'.
Objaverse. Objaverse is large-scale dataset containing over 800K web-crawled 3D objects, which covers a wide range of functional categories and geometric variations. As one of the largest publicly available collections of 3D assets, it often used for 3D understanding, neural rendering, shape reconstruction, and vision-language grounding. In our work, we take the IK 3D objects from 'Furnitures' subset, which consists of everyday household items such as sofas, chairs, and tables. We voxelize each furniture object to have binary grid of size 323 (approximately 33K voxels), where voxel occupancy encodes the object's structure.
A.9.2 IMPLEMENTATION DETAILS
We provide detailed implementation configurations in Table A4 Since the decoder architecture remains consistent across all experiments, we list the different experimental variants under the 'Encoder' column. The 'Enc. strides' column specifies the strides applied at each encoder stage, indicating the spatial downsampling factor between successive layers. Analogously, 'Dec. strides' indicates the upsampling factor at each decoder stage. PNeRV (Zhao et al. (024) does not employ conventional upsampling methods such as deconvolution (Noh et 2015) or pixelshuffle (Shi et al.) 2016), thus we do not specify its decoder strides. We observe that SSM-hased encoders perform better when the first encoder layer uses a larger kernel size, while standard convolution- based encoders tend to perform well with comparably more uniform stride settings across layers. Accordingly, we use larger stride in the first encoder layer for SSM-based models to match this behavior. The 'Feature dims.' column specifies the channel width at each encoder stage, which is kept consistent across all experiments. Since each model has different characteristics, we observe that they require distinct learning rates achieve optimal performance. Therefore, we carefully search across range of learning rates for each dataset and report the best results.
8

---

Published as a conference paper at ICLR 2026
Table A5: Ablation study of L.PNet-S3K components on the Bunny dataset. We employ HNeRV (Chen et al
2023
decoder for the experiment.
subsequent conv.
Adaptive
Adaptive
Inverted Bottlneck
Real
SiLU
RMS norm
PSNR
A, B
act.
X
X
36.37
X
36.67
X
36.92
V
X
X
X
X
36.26
/
X
V
X
X
X
36.94
V
X
J
V
37.03
J
X
36.92
36.99
37.04
Table A6: Ablation of state size on the Bunny dataset.
Table A7: size on the Kodak dataset.
Ablation of state
Table A8: Ablation of state size on the Objaverse dataset.
state size
PSNR
state size
PSNR
state size
PSNR
16
36.74
27.65
17.56
32
36.88
27.98
17.98
64
37.03
16
28.09
32
18.34
128
36.92
32
28.04
64
18.29
A.9.3 ABLATION STUDIES
We present an ablation study in Table A5 to evaluate the impact of the architectural modifications introduced in Sec. 5.3 and Sec. 5.4 In addition, we evaluate an alternative design inspired by Mamba (Gu & Dao 2023), where the step size A is made adautaadapiive. The results show that each proposed component contributes positively to performance, except for the adaptive A: performance drops significantly from 36.92 to 36.26 when A is made input-dependent. We hypothesize that this is due to a mismatch between the objective input-selectivity. a primary reason for adopting adaptive A, and input reconstruction. Since input reconstruction demands uniform attention across all regions of the input for accurate reconstruction, the input selectivity introduced by adaptive A may be less effective than in other tasks. We also find that the complex initialization of A and B (Smith) et al. 2022) is equally helpful, aligning with observations Mamba (Gu & Dao 2023) that such initialization of A is aids in processing continuous inputs. However, for performance optimization in terms of speed, we stick to the real parametrization.
We state size of 8 (images), 32 (3D data), 64 (vidcos), and apply the same state size across all S3K layers for each setting. Empirically, increasing the state size in the first S3K layer meaningfully enhances the performance, likely because this layer is the primary opportunity to capture high-frequency spatial details and a larger state size helps preserve this. In contrast, increasing the state size in later layers yields only marginal gains. For simplicity and consistency, we adopt the first layer's 's optimal state size for all layers. In our experiments, performance improves as the state size increases but begins to saturate once the state size is sufficiently large relative to the signal complexity. In practice, this occurs around 8 for images, around 32 for 3D data, and around 64 for videos-values we therefore use throughout the model.
We also provide thorough ablations state size of S3K layers in Table A6
A8
A.9.4 COMPARISON TO MLP- MLP-BASED METHODS
LPNet-S3K is yet inapplicable to coordinate-tor mapping methods (i.e., MLP-based methods) such as Liu et al. 2024); Sitzmann al. 2020) (images) or Kwan et al. (2024); Kim et al. (2022) (videos), categorizing itself as convolution based neural representation, in contrast to other mainstream neural representation MLP-bau MLP--ased neural representation (coordinate based. INR). Hence, we have compared methods only with cnavolution-basee methods in Sec. 6. it has been standard practice to compare models within the same family (Kim et al. [2024b] Yan et al. 20244] Kim et al. 2002 Shm
et al. 2024), due their fundamentally different characteristics which are illustrated in Table A9

---

Published as a conference paper at ICLR 2026
MIP- MLP--aaee NeRV
Convolution based NeRV
36
better
HiNeRV NeuriPS 2023)
Table A9: Comparison between MLP-based meth-
Ours-P
ods and convolution-based methods
PSNR (dB)
C3 (CVPR 2024)
DNeRV (CVPR 2023)
PNeRY
Type
recon quality
decoding speed
Examples
(ECCV 2024)
SNeRV
(CVPR 2024)
SIREN Sizzmann
2020
HNeRV
FINER Lau all 4024
32
[CVPR 2023)
MLP based (INR)
high
HiNeRV
2024
NeRV
hen el al.
31
NVP
C3 Km Wi
A0443
30
HNeRV
Chen
(NeuriPs 2021)
DNcRV
7ahn er
2124
29
Convolation
high
PNeRV
0
50
100
150
200
250
300
350
-based
DS-NeRV
Yan et al
2124
Decoding speed (FPS)
SNeRV
Kim
Ours
Figure A5: Reconstruction quality and decod- ing speed trade-off in modern NeRV models. collect models of 0.02-0.03bpp on a 600- frame UVG video, which occupy 300 epochs to train. FPS measured with NVIDIA A6000ada GPU.
Table A10: Comparison between LPNet- S3K FINER.
Method LPNet-S3K
Epochs
Num params
Training time (s/epoch)
Memory (MB)
Kodak
CLIC
300
150K
0.05
35210
28.09
26.33
FINER (Liu et al. 2024
300
199K
0.4
4106
28.68
27.12
Fig. A5 also exhibits their differences clearly: convolution-base models offer very fast decoding speed while sacrificing their reconstruction quality. On the other hand, MI.P-based methods in general show higher fidelity, but have slower decoding speed, which limits their real-time application. Unlike the prevailing NeRV trend of trading decoding speed for higher reconstruction quality, LPNet-S3K serves fidelity enhancer for convolution-basee models without adding inference cost.
We also provide a comparison with MLP-based image neural representation, FINER (Liu et al. 2024) in Table A10 FINER shows better fidelity compared to LPNet-S334 but requires much more time and memory for training. On the other hand, LPNet-S3K shows greater efficiency, while underperforming in terms of reconstruction quality.
A.9.5 ADDITIONAL QUALITATIVE RESULTS
We provide additional qualitative results in Fig. A6 (Kodak), Fig. A7 (CLIC2020) Fig. A8 (Obja- verse), and Fig. A9 A14 (DAVIS).
10

---

Published as a conference paper at ICLR 2026
Ground Truth
LPNet-S3K (Ours)
UPNet-Mamba
LPNet-Com
ConvNeXt
PSNR:26.80 26.80 MS-SSIMM 0.33
PSNR:26.45 MS SSIM: 0.93
PSNR:25.79 IMS NNS-SSIM: 0.91
PSNR-26.32 MS SSIM: 0.92
P5NR:30.15 MS-S5IM: PSNR:29.45 M5-S5IM: 0.92
MS-SSIM:
PSNR-29.03 M5-55IM:
Figure A6: Reconstruction results on the Kodak dataset
Ground Truth
LPNet-S3K (Ours)
LPNet- Mamba
LPNet- Conv
ConvNeXt
PS MS-SSIM-D.. 0.91
PSNR226.4 MS-SSIMM 0.90
PSNR 56
MS-SSIM: 0.88
PSNR.25.34 34
MS-SSIM 0.88
PSNR:29.04 SS:M::SM::: 0.50
PSNR:228.0 MMS-SSIM: 0.89
PSNRR:2755
MSS-SSIM:
PSNR:27.81 R/S-SSIM:: 0.89
PSNR 43 MS S5 MLO.95
PSNR22 16
MS
SSIML 0.35
PSNR MS..5IMM.0
PENR MS SSIMLO.5E
25.7028.70 MM SS Mt
0.95
PSNR228.50 MS-SSIM: 0.94
PSNR 27.32
MS 115-SSIM:03 0.53
P5NR MS SSIM: 0.87
Figure A7: Reconstruction results on the CLIC2020 dataset
11

---

Published as a conference paper at ICLR 2026
Ground Truth
LPNet-53K (Ours)
LPNet-Mamba
LPNet-Conv
ConvNeXt
PSNR:24.04 MS-SSIM: 1.00
PSNR-22.83 MSSSSIM 1.00
PSNR:21.26 MS-SSIM: 0.97
PSNR:19.330 MS-SSIM: 0.96
PSNR:22.91 MS-SSIM:
PSNR-22.99 MS-SSIMA 0.99
PSNB:21.02 MS-SSIME
P5NR:20.19
PSNR:23.22 MS S3IM: 0.99
PSNR-23.00 MS SSIM 0.95
PSNR:23.39 MS-SSIM: 0.99 PSNR:20.991 MS M: 0.97
Figure A8: Reconstruction results on the Objaverse dataset. For clearer comparison, we present the difference visualization under each reconstruction result.
12

---

Published as a conference paper at ICLR 2026
Ground Truth
Ours-S (3.0M)
SNeRV (3.0M)
PSNR:34.58
PSNR:33.83
DNeRV (3.4M)
PNeRV (3.4M)
HNeRV (3.0M)
PSNR:30.92
PSNR:29.11
PSNR:30.35
Figure A9: Reconstruction results 'blackswan' from the DAVIS dataset
Ground Truth
Ours-S (3.0M)
SNeRV (3.0M)
PSNR:30.04
PSNR:28.68
DNeRV (3.4M)
PNeRV (3.4M)
HNeRV (3.0M)
PSNR:27.38
PSNR:27.89
PSNR:26.71
Figure A10: Reconstruction results on 'camel' from the DAVIS dataset
Ground Truth
Ours-S (3.0M)
SNeRV (3.0M)
PSNR:32.35
PSNR:31.27
DNeRV (3.4M)
PNeRV (3.4M)
HNeRV (3.0M)
PSNR:29.35
PSNR:28.76
PSNR:28.61
Figure All: Reconstruction results on 'car-roundabout" from the DAVIS dataset
13

---

Published as a conference paper at ICLR 2026
Ground Truth
Ours-S (3.0M)
SNeRV (3.0M)
PSNR:36.69
PSNR:35.79
DNeRY (3.4M)
PNeRV (3.4M)
HNeRV (3.0M)
PSNR:31.95
PSNR:31.02
PSNR:31.11
Figure A12: Reconstruction results on 'car-shadow' from the DAVIS dataset
Ground Truth
(3.0M)
SNeRV (3.0M)
PSNR:25.12
PSNR:25.14
DNeRV (3.4M)
PNeRV (3.4M)
HNeRV (3.0M)
PSNR:24.88
PSNR:24.39
PSNR:24.60
Figure A13: Reconstruction results on 'cow" from the DAVIS dataset
Ground Truth
Ours-S (3.0M)
SNeRV (3.0M)
P5NR:30.225
PSNR:30.41
(3.4M)
(3.4M)
(3.0M)
PSNR:29.13
PSNR:28.16
PSNR:28.60 28.60
Figure A14: Reconstruction results on 'dance-twirl' from the DAVIS dataset
14


Reviews:
[{"review_id": "B8rOKZhB0H", "reviewer": "Reviewer_DAda", "summary": "This well-written paper proposes using modern state-space models (SSMs) for data-specific implicit neural representations (intentionally overfitting SSMs to a single datum) for image, object, and video data. They introduce a structured state-space kernel (S3K) that turns an SSM into a convolutional kernel, allowing for natural n-D preprcessing and learnable downsampling. The final encoder uses a Laplacian-pyramid design with S3K layers and plugs into existing decoders, and they compare against a rich set of baselines.", "questions": "Are fidelity metrics (PSNR/SSIM) enough to support compression claims? It would be nice to see, e.g., bitrate comparisons / rate-fidelity curves?\n\nAlthough acknowledged, the compute/memory overhead is significant. Could the authors implement one of the suggested efficient formulations, at least in one case/example?\n\nCould you compare against SSM-convs that don’t build explicit kernels?\n\nHow sensitive are results to state size? Does this work for very high res images/videos on realistic training budgets?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Clear link between SSMs and continuous INRs through basis expansions. \n\nS3K has a concise definition and extends clearly to nD using 1D outer products, allowing it to potentially drop into many encoder models.\n\nGood ablations studies and architectural insights: they evaluated (a) stacked, (b) image pyramid, and (c) Laplacian pyramid variants (showing the last to be preferable across models). This shows that stacking in SSMs may harm reconstruction (they speculate through repeated projection amplifying artifacts). \n\nThey provide a number of ablations, including on input adaptivity, real vs. complex parameters, inverted bottlenecks, and SiLU/RMSNorm. \n\nOn video benchmarks, replacing only the encoder improves PSNR. \n\nTheory shows relationship between diagonalizable A and spectrum and supports compression framing.", "weaknesses": "Fidelity metrics (PSNR/SSIM) alone don’t substantiate “compact representation.'\n\nThe memory/compute overhead of explicit kernels is substantial (the authors acknowledge this).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "28SE5Wd5W6", "reviewer": "Reviewer_ADR1", "summary": "This paper explores the integration of state-space models (SSMs) into data-specific neural representations (e.g., implicit neural representations for images, videos, and 3D objects). The authors first analyze the theoretical foundation of SSMs, showing that their latent states effectively capture the coefficients of continuous basis functions and thus suit compact signal modeling. Based on this insight, they propose a new Structured State-Space Kernel (S3K), which translates SSM dynamics into convolutional kernels, enabling multi-dimensional processing and inherent downsampling.\n\nThe final architecture, LPNet-S3K, integrates S3K layers into a Laplacian pyramid structure and demonstrates strong performance across diverse modalities—images (Kodak, CLIC2020), videos (Bunny, UVG, DAVIS), and 3D objects (Objaverse). The results show consistent gains over ConvNeXt- and Mamba-based baselines and over established NeRV-family models, validating both theoretical and empirical advantages.", "questions": "See in weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Instead of simply integrating the off-the-shelf state-space model, this paper provides the first systematic theoretical and empirical exploration of applying SSMs to data-specific neural representations. The analysis linking SSM parameters to signal reconstruction coefficients offers a new, signal-processing–grounded perspective on neural compression. In particular, it found a non-trivial conclusion that SSMs consistently outperform the transformer in data representation, which is also explained from the perspective of a classical sinusoid signal processing task.\n\n2. The work revisits how SSMs should be incorporated and identifies some design findings: stacking SSMs can harm reconstruction; applying them to downsampled multi-scale features improves quality; and Laplacian pyramid decomposition provides the best balance between redundancy and expressivity.\n\n3. The experiments are broad and convincing—covering 2D, video, and 3D data. LPNet-S3K consistently improves reconstruction quality with comparable model size and decoding speed. Note that performance gains are achieved via encoder modification, keeping decoders identical to baselines.", "weaknesses": "While empirical findings (e.g., stacking degradation, Laplacian advantage) are useful, their underlying reasons are only intuitively discussed. A deeper analytical or visualization-based explanation (e.g., spectral bias, basis overlap) would strengthen the claims.\n\nWhile this paper shows the advantages of state space model in data representations, I encourage the author to build a SOTA implicit neural representation-based neural codec based on it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B7ek39xOzH", "reviewer": "Reviewer_kdKF", "summary": "The authors explored the application of state space model (SSM) in visual representation. They analyzed the feasiblity to use an SSM to extract the frequency domain information and designed a pyramidal network structure (S3K). They combined this design with different convolution encoders and verified the performance on multiple input modalities, incouding DVS, 2d image and 3d points. The result shows that their method have some advantages in visual representation quality and extendability to different modalities.", "questions": "1. In Section 3.1, the authors claimed that SSM can encode a signal into frequency domain (coefficients of a sine series). For this purpose, DFT is the common approach, with mature and efficient algorithmic and hardward implementations. For digital image, there are also DCT and many other methods to yield the frequency domain representation of an image. What are the advantages to use SSM compared with these transformations?\n2. In Section 4.2, the authors extend the dimansion of the kernel by making outer product. What is the meaning of this operation in the aspect of State Space Models?\n3. What is the relationship between an input (2d image for instance) and the input function $\\phi(t)$?\n4. What are the advantages of the proposed S3K method compared with the broadly used VAEs?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Explored the novel application of SSM in visual representation encoding\n- Clearly organized architecture and consistent logic\n- Flexibile input forms, covering multi modalities", "weaknesses": "- Insufficient analysis on the motivation of method design\n\n  The authors didn't explain the meaning of taking outer product of the 1d convolution kernel into multi dimansional\n\n- Some methods not clearly described\n\n  The article didn't define how to convert an image into the input of the proposed S3K Conv module\n\n- No comparison with populat visual encoding methods like VAE and diffusion\n\n  VAE method is commonly used in visual encoding in various visual and multimodal models. New methods should compare with important baselines and clarify the improvements and advantages\n\n- Format issue in figure caption and referneces\n\n  Figure numbering skipped 2, and the later figure references are mis-marked", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ShZE1zS9aX", "reviewer": "Reviewer_My7V", "summary": "This paper proposes to use state-space models (SSMs) to build neural representations. The main motivation for the use of SSMs is that many signals are discrete observations of an underlying continuous signal, permitting the use of classical signal processing techniques. In this paper, the authors extend an older approach of compressing signals by storing them as their coefficients in a fixed basis, which can be done using SSMs. This paper proposes a new SSM-based architecture which they call S3K. Based on their analysis of different architectures, this paper combines S3K with multi-scale features and a Laplacian pyramid to create a new SSM-based architecture for neural representations, which they call LPNet-S3K. The proposed method is evaluated on 2D image reconstruction, video reconstruction, and 3D shape reconstruction.", "questions": "**(Q1)**: How does the proposed method compare to recent methods such as Boost [1] and NVRC [2]?\n\n**(Q2)**: How does the decoding speed of convolutional methods compare with that of efficient MLP methods such as InstantNGP [3]?\n\n**(Q3)**: Can SSM-based methods be extended to create coordinate neural networks?\n\n[1] Zhang, Xinjie, et al. \"Boosting neural representations for videos with a conditional decoder.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[2] Kwan, Ho Man, et al. \"NVRC: Neural video representation compression.\" Advances in Neural Information Processing Systems 37 (2024): 132440-132462.\n\n[3] Müller, Thomas, et al. \"Instant neural graphics primitives with a multiresolution hash encoding.\" ACM transactions on graphics (TOG) 41.4 (2022): 1-15.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "**(S1) Novelty**: this paper introduces the new idea of using SSMs to create neural representations and proposes a novel SSM-based architecture. \n\n**(S2) Architectural analysis**: This paper analyzes different architectures to create the best architecture for SSM-based neural representations.\n\n**(S3) Theoretical justification**: This paper also provides some theory to back up their claims.", "weaknesses": "**(W1) Motivation/Limited applicability**: the S3K/LPNet-S3K method is mainly evaluated on toy tasks of the following form: given a signal, reconstruct that signal. The proposed method is not evaluated for more complicated tasks involving neural representations. Potential more complex tasks that are signal-to-signal include video interpolation, video superresolution, and 3D reconstruction from 2D partial views. Additionally, it is not possible to build S3K-based neural representations that map coordinates to function values, which represent a significant class of neural representations used today. \n\n**(W2) Computational trade-off of convolutional methods vs MLP methods**: The authors mention that, compared to convolutional methods like the one proposed in the paper, MLP methods offer higher fidelity but slower decoding speed. It seems to me that this may not matter in practice when there exist fast and efficient MLP neural representations such as InstantNGP [3]. \n\n**(W3) Experimental setup**: For the 2D image and 3D object reconstruction experiments, the only baselines are ablations of the proposed model and ConvNeXt, which was not designed as neural representation. I think the performance would be better contextualized by adding previous neural representation methods, such as FINER, which was mentioned in the Appendix A.8.4. Additionally, recent baselines for video reconstruction are either not mentioned in the main paper or not included in the paper. These include HiNeRV (only mentioned in the Appendix), *-Boost [1], NVRC [2]. \n\n**(W4) Empirical results**: The empirical justification of the method is mixed. For 2D and 3D reconstruction, the proposed method is worse than MLP methods but better than the use of a vanilla convolutional network (ConvNeXt). For video reconstruction, the method is generally better than all but HiNeRV, with some recent methods possibly missing. The increased performance comes at the cost of much more expensive training due to the increased expense of the LPNet-S3K encoder (20x more memory, 4x more FLOPS). \n\n[1] Zhang, Xinjie, et al. \"Boosting neural representations for videos with a conditional decoder.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[2] Kwan, Ho Man, et al. \"NVRC: Neural video representation compression.\" Advances in Neural Information Processing Systems 37 (2024): 132440-132462.\n\n[3] Müller, Thomas, et al. \"Instant neural graphics primitives with a multiresolution hash encoding.\" ACM transactions on graphics (TOG) 41.4 (2022): 1-15.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)