Continuous Space-Time Video Super-Resolution with 3D Fourier Fields
Authors: Alexander Becker
Julius Erbach
Dominik Narnhofer
Konrad Schindler
Venue: ICLR 2026 Poster (2026)
Keywords: continuous space-time video super-resolution
arbitrary-scale super-resolution
low-level vision
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce a novel formulation for continuous space-time video super-resolution. Instead of decoupling the representation of a video sequence into separate spatial and temporal components and relying on brittle, explicit frame warping for motion compensation, we encode video as a continuous, spatio-temporally coherent 3D Video Fourier Field (VFF). That representation offers three key advantages: (1) it enables cheap, flexible sampling at arbitrary locations in space and time; (2) it is able to simultaneously capture fine spatial detail and smooth temporal dynamics; and (3) it offers the possibility to include an analytical, Gaussian point spread function in the sampling to ensure aliasing-free reconstruction at arbitrary scale. The coefficients of the proposed, Fourier-like sinusoidal basis are predicted with a neural encoder with a large spatio-temporal receptive field, conditioned on the low-resolution input video. Through extensive experiments, we show that our joint modeling substantially improves both spatial and temporal super-resolution and sets a new state of the art for multiple benchmarks: across a wide range of upscaling factors, it delivers sharper and temporally more consistent reconstructions than existing baselines, while being computationally more efficient. Project page: https://v3vsr.github.io.

Published as a conference paper at ICLR 2026
CONTINUOUS SPACE-TIME VIDEO SUPER- RESOLUTION WITH 3D FOURIER FIELDS
Alexander Becker
Julius Erbach
Dominik Narnhofer
Konrad Schindler
ETH Zurich
ABSTRACT
We introduce a novel formulation for continuous space-time video super- resolution. Instead of decoupling the representation of video sequence into separate spatial and temporal components and relying on brittle, explicit frame warping for motion compensation, we encode video as continuous, spatio- temporally coherent 3D Video Fourier Field (VFF). That representation offers three key advantages: (1) it enables cheap, flexible sampling arbitrary locations in space and time; (2) it is able to simultaneously capture fine spatial detail and smooth temporal dynamics; and (3) offers the possibility to include an analytical, Gaussian point spread function in the sampling to ensure aliasing-free reconstruction at arbitrary scale. The coefficients of the proposed, Fourier-like sinusoidal basis predicted with neural encoder with large nempool receptive field, conditioned on the low-resolution input video. Through extensive experiments, show that our joint modeling substantially improves both spatial and temporal super-resolution and sets a new state of the art for multiple benchmarks: across wide range of upscaling factors, it delivers sharper and temporally more consistent reconstructions than existing baselines, while being computationally more efficient. Project page: https: //vpsss//vvsx github. io
INTRODUCTION
Video super-resolution (VSR) seeks improve the perceptual quality by reconstructing high- resolution (HR) videos from low-resolution (LR) inputs. VSR is an elementary capability for video editing and analysis, because the spatial resolution and frame rate of video recordings are limited by hardware and power constraints. For instance, mobile devices and action cameras typically lack a continuous optical zoom and rely VSR for digital enlargement.
10
32
v (ours)
GiB
PSNR [dB]
31
BF-STVSR
Vi
MoTIF
29
1.5
2.5
3
3.5
Forward time [s]
To be practical, super-resolution scheme should not be tied to a specific magnification, but support the reconstruction of videos with arbitrary upsampling factors, in both space and time. Several recent works have addressed this by representing video as a continuous function often an implicit neural representation (INR).
Figure Performance vs. computation time. v outperforms other VSR models by about 2 dB PSNR, while being significantly faster. PSNR is measured on the Adobe240 test set, for x4 spatial and x8 temporal SR. For compute see Sec. 4.3
Once that function has been inferred from the available (low-resolution) observations, it can be sampled at any desired locations, respectively raster spacing. Existing implementations of that principle, however, share an important limitation: they separate spatial and temporal modeling, i.e., each frame pair is spatially represented by a 2D function (typically a 2D INR), and the motion in between frames is represented by another function (typically optical flow field). This factorization conceptually unsatisfactory, since it risks losing spatio-tempore correlations. Perhaps more importantly, it is also practically undesirable: exchange information between different frames one

---

Published as a conference paper at ICLR 2026
must rely on explicit warping, making super-resolution vulnerable to errors of the underlying flow estimation - which tend to be worst in critical regions, c.g., near object boundaries. To make matters worse, optical flow is estimated from two consecutive images. "Temporal" modeling often does not go beyond pairs of adjacent frames, because chaining or fusing flow vectors over longer, more meaningful temporal contexts is hard due to error build-up, over-smoothing and (dis-)occlusions.
An important capability for an arbitrary-scale super-resolution method is correct anti-aliasing: at the time when the representation learned, is not known at which scales it will later be sampled. Consequently, the representation must contain high-frequency detail up to the highest possible upsampling factor, which means that, when super-resolving by lower factors, those details will lie beyond the Nyquist limit. To avoid artifacts, the sampling mechanism must be designed in way that suppresses unrepresentable frequencies. With INRs, this is complicated: their representation resides in an abstract latent feature space that is hard to manipulate. Sampling them with an integrative observation model (a "point spread function"" requires workarounds that add complexity and computational cost.
To tackle these challenges, we resort to drastically simpler representation: a combination of 3D sine waves in (I, Lee, we sidestep the above-mentioned problems by jointly embedding the space and time dimensions in a single, unified representation. Our VFF representation to knowledge, the first such model for video super-resolution is conceptually simple and interpretable, and at the same time a natural choice if one aims to decode the video at arbitrary spatial and temporal scales: Like previous methods, VFF can be queried at any desired, continuous spatio-temporal coordinate but at much lower computational cost. However, VFF avoids the potentially error-prone, warping-based decoupling of spatial and temporal modeling, and is, by construction, amenable to longer-term temporal modeling over multiple frames. What is more, translational motions correspond phase shifts in VFF, making it easy to capture (and lear) a dominant component of video motion. Moreover, the functional form of VFF permits closed- form sampling with Gaussian PSFs, important for correct anti-aliasing when moving across scales, especially in out-of-distributton settings.
Intuitively, VFF can be thought of as continuous (ar, y, t)-cuboid that can be queried at arbitrary spatio-temporal locations to obtain video with the desired resolution and frame rate. To map a (low resolution, low frame rate) input video to that cuboid, we predict the corresponding basis coefficients, using neural video backbone. Inference of the super-resolved video amounts to sampling the representation at the corresponding spatio-tempores grid points.
The resulting, end-to-end trainable continuous space-time video super-resolution (C-STVSR) method, which we name V3, outperforms existing baselines by a substantial margin across different tasks and datasets, while being computationally more efficient (Fig. I. In summary, contributions are:
VFF, a radically simple, yet highly effective comtinuous-domain video representation, consisting of a single trigonometric expansion of the joint (ar, y, () space.
V², an end-to-end framework to predict the parameters of VFF directly from a low-quality input video, using backbone encoder with large spatio-temporal receptive field.
An extensive experimental evaluation across different super-resolution tasks, in which y outperforms prior C-STVSR approaches by up to x2 dB in PSNR, while at the same time reducing runtime and memory footprint.
2 RELATED WORK
Space-time video super-resolution (STVSR) aims to enhance both spatial resolution and frame rate in unified framework. A straightforward baseline for STVSR is two-stage pipeline: first apply a video frame interpolation method such as SuperSloMo (Jiang et al. 2018) or DAIN (Bao et al. 2019), then follow up with a single-image or video SR model such as RCAN (Zhang et al.) 2018) or EDVR (Wang et al., 2019). While intuitive, this separation ignores correlations across space and time and often introduces temporal flicker. To avoid it. one-stage STVSR methods such Zooming Slow-Mo (Xiang et al.) 2020) and TMNet (Xu et al., (021) perform spatial and temporal upsampling jointly. These methods generally assume fixed, integer upscaling factors, limiting their usability.
2

---

Published as a conference paper at ICLR 2026
To go beyond fixed scales requires arbitrary-seale super-resolution. which has predominantly been studied for single images. Early work relied on meta-learning (Hu et al.) 2019) or implicit neural representations (INRs. e.g. Chen et al. 202I) to enable sampling arbitrary resolutions without retraining. Lee & Jin (022) employ a Fourier basis within such an INR framework, while Becker et al. (2025) introduce learned, non-orthogonal sinusoidal basis for fast and theoretically guaranteed anti-aliasing. Naive frame-by-trame application of image super-resolution to videos ignores temporal dependencies and leads to flickering. Hence, dedicated methods have been proposed for arbitrary-scaa video SR (AVSR), which incorporate temporal guidance features (Li al. 2024, Shang al., (2044). These methods, however, do not support temporal upsampling.
So-called continuous STVSR (C-STVSR) represents both space and time as continuous domains and therefore able to upsample in space and in time. It includes AVSR as well as fixed-scale STVSR (and also single-image SR) as special cases and, arguably, constitutes the most general and practical formulation. C-STVSR was pioneered by VideoINR (Chen et al., 2022), where videos are parametrized as two separate, decoupled INRs in image space and time. The temporal INR acts as an optical flow predictor, estimating the motion field from the intermediate frame at time function of t, followed by backward warping to reconstruct intermediate features. However, modeling continuous backward flow fields is challenging: a location's 's content and thus its motion varies with time. Even under linear motion, the backward flow field changes structurally over time, producing discontinuities at motion boundaries that are challenging learn. In contrast, forward flow (key-to-intermediate) tracks each pixel's s trajectory through time as a continuous curve. For linear motion, the displacement vector field preserves its structure and simply scales with t, making easier to model. Building on this idea, MoTIF (Chen al. 2023b) learns forward motion trajectories and applies softmax splatting (Niklaus & Liu (0200 for forward warping, leaving occlusion handling and conflicting motions to the decoder. BF-STVSR (Kim al., 2025) incorporates Fourier basis in the latent space and B-splines to parametrize the motion field, but still depends on explicit warping to map the two keyframes to the intermediate. Furthermore, none of the mentioned methods has principled mechanism for anti-aliasing. Instead, they count on implicitly learning the necessary. adaptive high pass filtering from data, which makes the learning problem significantly more complicated and comes without any guarantees.
t to the keyframe
In summary, the existing literature shows a growing interest in C-STVSR, but exposes a clear conceptual gap: existing method provides unified, spatio-temporally consistent representation that would (i) keep the system design simple: (ii) circumvent explicit, error-prone warping; (iii) allow for multi-frame motion context; and (iv) include efficient mechanism for the elementary anti-aliasing operation.
3 METHOD
To describe our formulation for continuous space-time video super-resolution first define the problem sctup, then introduce our VFF video representation and its conditional parameterization, and finally show how to embed it in an end-to-end learning framework.
3.1 PROBLEM STATEMENT
Let V'r E RT denote a low-resolution RGB video. The objective of C-STVSR is to recover a continuous spatio-temporal signal,
Va y. :):
0 3
(1)
The observed video satisfies V = D(V), where D describes the degradation - in the case of C- STVSR the discrete sampling of the signal in space and time. Once the signal 1' has been recovered, can be sampled at an arbitrary, smaller grid spacing (sx smaller space, rx smaller in time) to obtain a high-resolution, high-frame rate video Vhr E RrrxxaH x&W x3 To reconstruct V one must invert D. which is highly ill posed and only possible if one has access to a strong prior for V. In modern super-resolution schemes, including V3, that prior i.e., generic expectations about the spatio-temporal structures and patterns in natural videos takes the form of a neural network and is learned from data.
3

---

Published as a conference paper at ICLR 2026
Fourier basis functions
Spatio-temporal voxel grid
Video Fourier Field (continuous)
Sampled output video
Input video
jen-og)
Local region
Bi
Sumple (Fq. 4)
T
Bg
SH
sW
= phase shift & scale
8um
B
Vix, v.t)
Figure 2: Overview of val A backbone encoder predicts a voxel grid of local phase shifts and weighting coefficients for set of 3D Fourier basis functions. Their sum describes, within local interval, the continuous function Vxx, y, t. that we call the Video Fourier Field. The function can be sampled at different spatio-temporal resolutions (Eq. to obtain an output video.
Below, we introduce a compact, but expressive parametrization of V (Sec. practical algorithm to estimate its free parameters from training data (Sec. 3.3
3.2 followed by
3.2 VIDEOS AS 3D FOURIER SERIES
The core of our representation is collection of sinusoidal 3D basis functions (B:). in (I, space, parameterized by frequencies wi E R and phase shifts Φi E R::
B
3 R
(I, + a; sin (w, (z, y, t) + ox)
(2)
With their help, we parametrize the continuous video signal as a finite trigonometric expansion,
V(x,y,1) = Σ Be w.),
which we refer Video Fourier Field (VFF). We call our representation Fourier Field because expresses the video as a finite sum of sinusoidal functions, inspired by the classical Fourier transform. Unlike true Fourier series, consisting of infinite set of orthogonal functions with integer frequencies to guarantee completeness, our formulation employs a finite set of sinusoids with continuous frequencies and phase shifts. This relaxation sacrifices strict orthogonality but retains the key advantage of Fourier features: continuous, band-limited representation that can be queried at arbitrary spatio-temporal resolutions.
Note that V is defined on continuous domain, and can thus be sampled at arbitrary rates both spatially and temporally, as needed for C-STVSR. In practice, to keep the number of basis functions small, we split the (x, y, 1)-space into local, axis-aligned regions ("voxels"") and fit individual VFFs to them. In this way, the coefficients (a, Φi) can be adjusted to the local video content, but the overall representations still covers the whole, continuous domain.
One advantage of the VFF is that it can easily be queried with linear point spread function (PSF) to achieve correct anti-aliasing for any desired scale factor. Following Fourier theory,
Vx = Σ B.(x,V.) ((wi,
(4)
exactly describes V under Gaussian PSF with variance a. Aliasing-free sampling thus amounts to simply rescaling the individual basis functions by the frequency-depender exp(_1le.l1f/sopp3) where a is inversely proportional to the effective sampling rate, as dictated by the Nyquist limit (see, .p..enhpe et al. 1997). Note that Eq, 4 can be implemented as a matrix multiplication followed by element- wise addition (for the shift) and scaling, which is much more efficient than explicit filtering or oversampling. Compared to existing C-STVSR methods (Chen) et al. 2022 2023b Kim et al. 2025), where scale-appropriate PSF has to be learned from data as
factor ((w,,)) =
4

---

Published as a conference paper at ICLR 2026
part of the neural model, efficient, but also ensures good generalization. unaffected by training biases, cf. [ecker et al. (2025). Note also, can be set to = 0 for individual dimensions, for instance to express Gaussian blur in space but point sampling in time, or to a small constant value representing the narrow temporal PSF induced by finite exposure times in many consumer cameras. Contrary to other C-STVSR methods (Chen et al. 2022 2023b Kim et al. 2225), our trigonometric expansion over (æ, y, t) can be sampled at arbitrary spatial and temporal resolutions with a single function call, without explicit frame warping. The formulation can naturally encode translational motion as simple phase shifts in the frequency domain (Kuglin 1975). and the sinusoidal basis compactly captures frequency patterns across scales.
the tigmaalthhoreticalll correct PSF is not only more parameter-
3.3 CONDITIONAL FOURIER PARAMETERIZATION
Building on the VFF formulation, we now have to infer the parameters of the representation V for an input video, so that higher-resolution output videos can be sampled from it. We employ domain-specific neural video encoder, E, that aggregates semantic features of every input voxel over large apatio-temport receptive field. Unlike approaches that are restricted to pairwise optical flow, our model by design leverages a substantially larger temporal context and can reason jointly over multiple frames. Consequently, the representation can capture long-range dependencies, handle occlusions and disocclusions more robustly, and capture non-linear and periodic motion patterns that a simple frame-to-framn interpolation cannot adequately handle (see Fig. 55)
As mentioned above, in practice we use a voxel grid of local basis expansions, in line with other SR methods that do the same with local INRs (Chen al. 2 2023b) Kim et al., 2025). We emphasize that each grid cell still contains only single, unified and continuous function f... Global consistency across cell boundaries is maintained through the backbone encoder, which aggregates information over large spatio-temporal receptive fields to predict the local field parameters. Concretely, after obtaining a grid of semantic features Exx) E T H x from the encoder, matching the size of the input video, we apply small convolutional network to map those features to a 3D grid of VFF parameters, {(aj. Φÿ)};. where aj = (a1....,ay) and Φj (øI...., ΦN ) define the VFF at each voxel index j E N3. We find that it is not necessary to locally vary the frequencies [w] between different grid cells. In other words, the base frequencies are learned once during training. At inference time they are kept fixed for all input videos and all cells, and only their amplitudes and phases are modulated to fit the input. Beyond saving compute, sharing a common frequency basis turns out to slightly improve stability and coherence of the reconstructed video.
The complete super-resolution system (Fig. 2 consists of the backbone encoder, the VFF basis, and the PSF-aware sampler (Eq. It differentiable, and trained end-to-end.
3.4 IMPLEMENTATION AND TRAINING
y is implemented in JAX (Bradbury et al. 2018). We use N = 512 basis functions. As backbone encoder we employ RVRT (Liang et al.) 2022). with the embedding dimension set to 90 and 12 attention heads. During training, the spatial upsampling factor is sampled randomly from the interval 2(122,44 4), and input data is generated with bicubic downsampling in space and subsampling of (high- speed) video frames in time, in prior work. Training patches have 80 80 pixels and 14 frames. We use standard data augmentations (random flipping, rotation, and resizing) and train for 2.5 x 106 iterations with an L1 reconstruction loss using the AdamW (Loshchilov & Hutter 2017) optimizer (Ir= 10-4 31 = 0.9, 32 = 0.999, e = 10->) and a Cosine Annealing scheduler (Loshchilov & Hutter 2016). We furthermore employ gradient clipping at global L2 norm of 1. As in prior work, all parameters are trained from scratch, except for the flow component in RVRT (RAFT, Teed & Deng 2020), which we finetune only for the last 3 x 10S iterations. We use a batch size 16 and train on 16x Nvidia GH200 chips. Inference only requires single consumer GPU (RTX 3090 Ti).
4 EXPERIMENTS
We evaluate 3 on several benchmarks. Besides C-STVSR performance, we also test the special cases of spatial-only and temporal-only SR, well as temporal consistency. Following the
5

---

Published as a conference paper at ICLR 2026
Quantitative results for spatio-temporal super resolution (PSNRt SSIM1). The spatial scaling factor is set to x4, the temporal to x8 (Adobe and GoPro) and x2 (Vid4). Columns ending in Center are evaluated on key- and center frames only, and Average denotes all output frames. We show two-stage method for comparison (t).
Table 1:
GoPro Center
GoPro Average
Adobe Center
Adobe Average
Method
Vid4
Par.
DAIN EDVR
23.48 / 0.654
28.58 0.842
26.64 0.799
27.45 / 0.809
25.64/ 0.759
44.7 M
ZoomingSloMo
25.72 0..77
30.69 0.885
30.26 0.882
11.1 M
TMNet
25.96 0,780
30.14/0.877
28.83 0,851
29.41 0.855
28.30 0.835
12.3 M
VideoINR
25.61 /0.771
00.26/0.877
29.41 / 0.867
29.92 / 0.875
29.27 / 0.865
11.3 M
MOTIF
25.79 0.775
31.04 0.888
3..04/08877
30.63 / 0.884
29.82 / 0.875
12.6 M
BE-STVSR
25.85 0.777
31,17 0.890
30.22/0.880
30.83 0.888
30.12 / 0.880
13.5 M
26.76 0.818
32.96 0.923
32.26 0.916
32.91 / 0.922
32.29 0.917
13.7 M
V Large
26.82 / 0,821
33.09 / 0.925
32.36 0.921
33.08 0.924
32.45 0.919
20.6 M
literature (Chen et al.) 2022; 2023b Kim et al. 2025), we train on the 240 fps Adobe240 (Su et al. 2017) dataset, which consists of 133 videos at resolution x280x pixels. Input videos are obtained by spatial downsampling with random scaling factors and fixed temporal subsampling by a factor of x8 to obtain input videos with 30 fps; all ground truth frames, randomly sampled in space and time serve ground truth for supervision.
4.1 C-STVSR PERFORMANCE
We begin by evaluating V on the full, spatio-temporally continuous C-STVSR task. Table 1 shows quantitative results for various methods the test of Vid4 (Liu & Sun 201T) (spatial x4, temporal x2) GoPro (Nah et al., (017) and Adobe240 (both spatial x4, temporal x8). We follow the evaluation protocol of previous works and compute PSNR and SSIM in the luminance channel, one for the input and center frames ("Center")) and once for all output frames ("Average"), We find that V sets new state of the art on all three datasets and outperforms the baselines by a substantial margin - in most cases > 1.5 dB in PSNR. It appears that our unified Fourier representation is able to recover more spatial detail and reconstruct motion more coherently than methods that factor the representation into a spatial and temporal component.
For a fair comparison, we have matched the capacity of V to recent competitors, yet RV x14M parameters is small model by modern standards. To check the scalability of our model, therefore also train a version with a higher parameter count (V*-Large, 20.6M parameters), obtained by increase the size of the attention embedding in the backbone encoder 144. This further boosts performance, albeit with diminishing returns, and often outperforms prior art by a full 2 dB in PSNR. The experiment suggests that current models have not yet reached saturation point, and super- resolution quality could still be increased by further scaling up the model.
Figure shows visual comparison (on non-keyframes) for sample from the Adobe240 dataset. y faithfully reconstructs high-frequee content over time. In the example, it is the only method that recovers both legible text and the characteristic "accordion" structure of the articulated bus joint.
4.2 DECOUPLING SPATIAL AND TEMPORAL SR
Our core contribution is scale-agnostic, spatio-temporally unified video representation. V3 is not in any way specialized to purely spatial arbitrary scale video super-resolution (AVSR, no temporal upsampling) or video frame interpolation (VFI, spatial upsampling). Therefore, these "edge cases" are of particular interest, and they can be easily realised by simply setting the temporal, respectively spatial upsampling factor x1.

---

Published as a conference paper at ICLR 2026
Frame 4
Frame 12
Frame 20
Frame 28
Frame 36
XGMGED TOP CABE PRIN
MIZED TOP HEA CAME PR2VE
1B TOP KEALT PROVIDE
TOP HEALTH
TOP HEALTH EE PROWDER.
ARE PROVIDER.
VideoINR
und it.
nd it.
d it.
it.
OGMAZE TOP CABE PREN
HAIZED POP HEA CABE PROVI
NED TOP HEALT CABIE PROVIDE
TOP HEALTH ARE PROVIDER.
T HEALTH
DE PROVIDER
MoTIF
und it.
ud it.
d it.
it.
OGMZED TOP CABE PRON
MIZED TOP HEA CABE CABE PROVI
VED TOP HEALT PROVIDE
TOP HEALTH ARE PROVIDER.
TOP HEALTH DE PROVIDER
BF-STVSR
und
nd it.
d it.
it.
OGNIZED TOP CARE PROM
NIZED HEA CARE PROVIO
IZED TOP HEALT CARE PROVIDE
TOP HEALTH ARE PROVIDER.
TOP HEALTH DE PROVVDDE.
(Sann)
und it.
nd it.
d it.
it.
OGNIZED TOP CARE PRON
NIZED TOP HEA CARE PROVID
ZED TOP CARE PROVIDE
ED TOP HEALTH ARE PROVIDER.
TOP HEALTH IE PROVIDER.
und it.
nd it.
d it.
it.
Figure 3:
Qualitative comparison of C-STVSR methods (x4 spatial, x8 temporal). V recovers legible text as well as thin stripe patterns.
4.2.1 ARBITRARY-SCALE VIDEO SR
To measure AVSR performance, we follow the experimental settings of the dedicated AVSR literature (Li et al. 2024 Shang et al. 2024) and evaluate on the REDS (Nah et 2019) validation set, consisting of 240 videos with x280x pixels and 100 frames, captured with a GoPro camera at 24 fps. Table 2 compares V to other C-STVSR methods (all trained on Adobe240) in this setting. It has been reported (Shang et al. 2024 that, when operating at 1 temporal upsampling, C-STVSR methods barely outperform arbitrary-see scale single-image super-resolution (AISR). We therefore include three methods representative of three generations of AISR: CNN- (LTE, Lee & Jin 2022), modern transformer- (CLIT, Chen al. 2023a), and the the time of writing

---

Published as a conference paper at ICLR 2026
Table 2: Spatial video super resolution (AVSR) on the REDS validation set (PSNR* SSIM1).
Methods marked with f indicate arbitrary-scale image SR applied frame-by-trame for comparison.
Method
x2
x3
x4
x6
X8
# Par.
Bicubic
31.51 0.911
26.82 0.788
24.92 / 0.7113
22.89 / 0.622
21.69 0.574
RDN-LTE i
34.73 0.994
30.73 / 0,866
28.75 0.804
26.56 / 0.718
25.24 0.669
22.5 M
RDN-CLITY
34.63 0.942
30.63 0.865
28.63 0.801
26.43 / 0.714
25.14 / 0.661
37.7 M
RDN-HIIF
$4.57/0.99
30.59 0.864
28.60 0.799
26.44 / 0.712
25.15 0.659
23.2
VidcoINR
31.59 /0.900
30.04 0.852
28.13/0.791
25.27 / 0.687
23.46/0.619
11.3 M
MoTIF
31.03
30,44 0.862
28.77/0.80
25.63 / 0.698
/5.12/0.664
12.6
BF-STVSR
3..92
31.17 0.881
00.11
26.78 0.728
25.40 / 0.668
13.5
V
36.53 / 0.963
32.31/ 0.908
29.92 / 0.849
27.39 0.754
25.96 0.690
13.7 M
-a-ge
36.70 0.964
32.53 0.911
30.13 0.853
27.54 0.760
26.10 0.696
20.6
Table Decoupling of spatial-only (Sx4, Tx1) and temporal-only (Sx1, Tx8) VSR on Adobe240. Input sequences have 30 fps.
Setting
VideoINR
MoTIF
BE-STVSR
V
4x4, Tx1
31.84 00900
32.95 / 0.916
33.03 / 0.917
34.25 / 0.938
Sx1, Tx8
24.45 0.712
28.09 0.843
29.37 / 0.867
33.43 / 0.936
strongest method based on hierarchical positional encoding, HIIF (Jiang et al, 2025 Indeed, v appears to be the first C-STVSR method to substantially surpass per-frame image AISR across all scaling factors within and outside the training distribution. We attribute the gain to our model's better ability to transfer information between frames. The unified spatio-temporal basis gives V access to larger temporal context window and lets exploit redundancy between frames for spatial super-resolution, rather than merely avoid flickering.
For completeness, Tab. (top) shows spatial-only and temporal-only upsampling for the strictly in- distribution Adobe240 test set. Other C-STVSR methods fare better in this setting, apparently they are quite sensitive to the (small) domain shift to REDS. Still, v works best by a healthy margin.
4.2.2 VIDEO FRAME INTERPOLATION
We next evaluate our approach for video frame interpolation (VFI), the complementary special case where the spatial scaling is fixed to x1. Table 3 (bottom) compares the pure frame interpolation capabilities of C-STVSR methods on Adobe240. For this analysis, we evaluate on the centerframes, for x8 temporal super resolution.
V once more brings substantial performance gain, demonstrating the temporal modeling power of the underlying VFF. The improvement is also evident in the qualitative results in Fig. Warping- based competitors suffer from artifacts caused by inaccurate optical flow that leads to misaligned features. Specifically, the baselines often fail to handle abrupt motion boundaries with occlusions (handrail, marked with red box) and have tendency to incorrectly combine the two nearest input frames, producing duplicate textures (text, marked with blue box).
In contrast, VFF obviates feature warping and thereby achieves more coherent and visibly sharper frame interpolation, supporting our claim that a native motion representation (z, y, t)-space is more robust than one that depends on an external optical flow estimator.
4.3 TEMPORAL CONSISTENCY
The larger temporal context of VFF allows us capture non-linear dynamics. We also hypothesize that representing a video in a 3D frequency space simplifies motion modeling (Kuglin 1975) and avoids crror-prone warping. All these factors should aid temporal consistency. To illustrate this, Fig. [5 depicts the temporal profile of a vertical image column as predicted by several video super- resolution methods.
8

---

Published as a conference paper at ICLR 2026
VideoINR
MoTIF
BF-STVSR
(ours)
GT
Figure 4: Frame interpolation (x8, center frames). v faithfully recovers high frequency content.
VidcoINR
MoTIF
BE-STVSR
(i(uns)
Figure 5: Temporal consistency. The red rectangle corresponds vertical image column time. V faithfully reconstructs complex, non-linear image motion and reduces block artifacts.
While v propagates structures smoothly across time, preserving both curvature and fine details, the baselines exhibit bias towards linear, non-periodic motion (center, wave pattern caused by the red facade), as well as blocky temporal discontinuities (top left, white window frame).
quantiaaua evaluates temporal consistency. Following Chu al. (2018) Cao et al. (202 )); Kim et al. (2025) we compute 10F to measure per-pixel differences in the optical flow (i.e. end point error) between adjacent ground truth frames and the corresponding target frames. Intuitively, tOF quantifies how much the motion trajectories in the super-resolved video deviate from the ground truth motion, so large values imply temporal inconsistencies such flicker or unnatural motion. The numbers confirm the superior temporal consistency of V
4.4 ANALYSIS OF LEARNED BASIS FUNCTIONS
Figure 6 analyzes the spatiotemporal behavior basis functions B, learned by V The figure shows that V is able to pick up structure in the data, with higher concentration of components along the coordinate axes of the polar plot (i.e., horizontal and vertical wave patterns representing axis-aligned structures). In terms of the marginal distribution of spatial frequencies, non-uniform distribution with more higher frequencies than lower ones, increasing the capacity for reconstruction of fine details for super-resolution. We further visualize the average predicted magnitude ("coefficient") of each basis function for an example sequence (Vid4, calendar), using a color map in the left subplot of Fig. Magnitudes decrease with increasing frequency, consistent with results from classical Fourier analysis.
4.5 COMPUTATIONAL COMPLEXITY
Table lists computational requirements in terms of inference time and VRAM, for various methods. Measurements have been performed a canonical input patch size of shape 14 x 80 x 80 with upsampling factors of x8 temporally and x spatially, on an RTX 3090 Ti GPU,
Table Temporal consistency of recent C-STVSR methods in terms of tOF (evaluated on Vid4,
x 4 spatial and x2 temporal super-resolution).
Bicubie
VideoINR
MoTIF
BF-STVSR
V3
V -Laree
0.595
0.344
0.354
0.323
0.254
0.250

---

Published as a conference paper at ICLR 2026
90"
Count
135.
45"
10
Spetial direction [rads]
180
Count
10
Spatial frequency
Count
270
0.01
0.02
0.03
0.04
0.05
Temporal Frequency
Figure 6: Analysis of globally learned basis functions. Left: Polar plot visualizing spatial direction spatial frequency (encoded radius) of the learned basis functions. The color map visualizes average predicted magnitudes for each component for an example video. Right: Marginal distributions of spatial direction, spatial frequency, and temporal frequency of basis functions.
Table Computational complexity of various C-STVSR methods.
VideoINR
MoTIF
BF-STVSR
V3
Inference time
3.03 2.6 GiB
1.88
1.90 10.4 GiB
1.27 s
VRAM
8.4 GiB
6.1 GiB
LIMITATIONS AND FUTURE WORK
At very high scaling factors, the outputs of y will tend be overly smooth, limitation shared by all regression- based SR methods, as a consequence of the discriminative training objective, which favors low distortion over perceptual realism. Generative models (e.g., based denoising diffusion) would likely deliver results that are more visually pleasing, despite spurious details. In other words, the absence of hallucinated details a price to pay for low reconstruction error.
finite 3D Fourier sum is rather simple and could theoretically present representational bottleneck the presence of extensive high-frequency content. So far did not observe problems for the vidcos and scaling factors we tested. We suppose that the chosen number basis functions (N 512), combination with the local fitting to small (z. y. -oxelss,, is sufficient for practical purposes. Note also, it is straightforward to increase N if needed.
The VFF parameterization
Finally, it may be useful extend v more advanced video degradations beyond spatio-tempore downsampling. E.g., the degradation operator could be used to accomodate sensor noise, motion blur or compression artifacts. Most obviously, VFF would be natural match for any local, linear degradation for instance, motion blur achieved by simply setting Ouime > 0.
CONCLUSION
We have presented V novel scheme for continuous space-time video super-resolution. Its core component VFF, a principled, compact video representation continuous (a, y, .)-space based on Fourier-like 3D frequency decomposition. By combining VFF with a contemporary neural video encoder to predict its free parameters, we construct a clean, unified spatio-temporal super-resolution method that upsample videos by arbitrary scaling factors in both space and time. V exhibits excellent practical performance and outperforms competing methods by > dB of PSNR, while at the same time offering faster runtime.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Wenbo Bao, Wei-Sheng Lai, Chao Ma, Xiaoyun Zhang, Zhiyong Gao, and Ming-Hsuar Yang. Depth-aware video frame interpolation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3703-3712, 2019.
Alexander Becker, Rodrigo Caye Daudt, Dominik Narhhofer, Torben Peters, Nando Metzger, Jan Dirk Wegner, and Konrad Schindler. Thera: Aliasing-free arbitrary-scale super-resolution with neural heat fields. Transactions on Machine Learning Research, 2025. ISSN 2835-8856.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman- and Qiao Zhang. JAX: composable transformations of Python+NumPy programs. htttps://ggiii
com/ jax ml/1ma ma/jax 2018.
Yanpeng Cao, Chengcheng Wang, Changjun Song, Yongming Tang, and He Li. Real-time super- resolution system of 4k-video based on deep learning. In 2021 IEEE 32nd International Conference on Application-specife Systems, Architectures and Processors (ASAP), pp. 69-76.
IEEE, 2021.
Kelvin CK Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Basicvsr: The search for essential components in video super-resolution and beyond. In Proceedings the IEEF/CVF conference on computer vision and pattern recognition, pp. 49.44947-4555 2021.
Hao-Wei Chen, Yu-Syuan Xu, Min-Fong Hong, Yi-Min Tsai, Hsien-Kai Kuo, and Chun- Yi Lee. Cascaded local implicit transformer for arbitrary-scale super-resolution. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, 2023a.
Yi-Hsin Chen, Si-Cun Chen, Yen-Yu Lin, and Wen-Hsiao Peng. MoTIF: Learning motion trajectories with local implicit neural functions for continuous space-time video super-resolution. In JEEE/CVF International Conference on Computer Vision, pp. 23131-23141, 2023b.
Yinbo Chen, Sifei Liu, and Xiaolong Wang. Learning continuous image representation with local implicit image function. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8628-8638, 2021.
Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, and Xiaolong Wang. VideoINR: Learning video implicit neural representation for continuous space-time super-resolution. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2047-2057,
Mengyu Chu, You Xie, Laura Leal-TTaxx, and Nils Thuerey. Temporally coherent gans for video super-resolution (tecogan). arXiv preprint 18 1(2)33 2018
Xuecai Hu, Haoyuan Mu, Xiangyu Zhang, Zilei Wang. Tieniu Tan, and Jian Sun. Meta-SR: A magnification-arbitaayy network for super-resolution. In IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1575-1584, 2019.
Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsus Yang. Erik Learned-Miller, and Jan Kautz.
Super SloMo: High quality estimation of multiple intermediate frames for video interpolation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9000-9008, 2018.
Yuxuan Jiang, Ho Man Kwan, Tianhao Peng, Ge Gao, Fan Zhang, Xiaoqing Zhu, Joel Sole, and David Bull. Hiif: Hierarchical encoding based implicit image function for continuous super- resolution, Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2289- 2299, 2025.
Eunjin Kim, Hyeonjin Kim, Kyong Hwan Jin, and Jaejun Yoo. BF-STVSR: B-splines and Fourier- best friends for high fidelity spatial-temporal video super-resolution. In IKEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 28009-28018 2025.
Charles D Kuglin. The phase correlation image alignment method. In IEEE Int. Conf. on Cybernetics and Society, 1975, pp. 163-165, 1975.
11

---

Published as a conference paper at ICLR 2026
Jaewon Lee and Kyong Hwan Jin. Local texture estimator for implicit representation function. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1929-1938, 1938, 2022.
Zekun Li, Hongying Liu, Fanhua Shang, Yuanyuan Liu, Liang Wan, and Wei Feng. SAVSR: Arbitrary-scale video super-resolution via a learned scale-adaptive network. In AAAI Conference on Artificial Intelligence, volume 38, pp. 3288-3296, 2024.
Jingyun Liang, Yuchen Fan, Xiaoyu Xiang, Rakesh Ranjan, Eddy llg. Simon Green, Jiezhang Cao, Kai Zhang, Radu Timofte, and Luc V Gool. Recurrent video restoration transformer with guided deformable attention. Advances in Neural Information Processing Systems, 35:378-393, 2022.
Ce Liu and Deqing Sun. A bayesian approach adaptive video super resolution. In IEEE Conference on Computer Vision and Puttern Recognition, pp. 209-216, 2011.
Chengxu Liu, Huan Yang, Jianlong Fu, and Xueming Qian. Learning trajectory-awar transformer for video super-resolution, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5687-5696, 2022.
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint 160iv:1608. 039983, 2016.
Iyya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 2017.
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In IEEE Conference Computer Vision and Pattern Recognition, pp. 3883-3891, 2017.
Seungjun Nah, Sungyong Baik, Scokil Hong. Gyeongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee. Ntire 2019 challenge video deblurring and super-resolution: Dataset and study. In JEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, June 2019.
Simon Niklaus and Feng Liu. Softmax splatting for video frame interpolation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5437-5446, 2020.
Alan Oppenheim, Alan S Willsky, and Syed Hamid Nawab. Signals & systems. Pearson Educación, 1997.
Zhongwei Qiu, Huan Yang, Jianlong Fu, and Dongmei Fu. Learning spatiotemporal frequency- transformer for compressed video super-resolution. European conference on computer vision, pp. 257-273. Springer, 2022.
Wei Shang, Dongwei Ren, Wanying Zhang, Yuming Fang, Wangmeng Zuo, and Kede Ma. Arbitrary-scale video super-resolution with structural and textural priors. European Conference on Computer Vision, pp. 73-90, 2024.
Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, and Oliver Wang. Deep video deblurring for hand-held cameras. In IEEE conference on Computer Vision and Pattern Recognition, pp. 1279-1288, 2017.
Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow. In European Conference on Computer Vision, pp. 402 402-419, 2020.
Xintao Wang, Kelvin CK Chan, Ke Yu, Chao Dong, and Chen Change Loy. EDVR: Video restoration with enhanced deformable convolutional networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P Allebach, and Chenliang Xu. Zooming Slow-Mo: Fast and accurate one-stage space-time video super-resolution. In /EEF/CVF Conference Computer Vision and Pattern Recognition, pp. 3370-3379, 2020.
Gang Xu, Jun Xu, Zhen Li, Liang Wang, Xing Sun, and Ming- Cheng. Temporal modulation network for controllable space-time time video resoutiun.. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6388-63397 2021.
12

---

Published as a conference paper at ICLR 2026
Yulun Zhang, Kumpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image super- resolution using very deep residual channel attention networks. European Conference on Computer Vision, pp. 286-301, 2018.
13

---

Published as a conference paper at ICLR 2026
A
ADDITIONAL QUALITATIVE RESULTS
Figures
10 and 13 show additional qualitative C-STVSR results for various datasets.
Figures
and 12 show additional spatial- and temporal-only SR results, respectively.
Frame
Frame 4
Frame 5
Frame 6
Frame 7
DAGZO
PAGGO
PALGO
PAGA
PAGO
VideolNR
XXtOO
PNGTO
OAGAO
XOGAO
XAGO
MOTIP
OOGZO
OoGGO
PAGOO
OOGZO DMI B01
PAGGO
BF ST-STSSR
FUmEE our
For BEE
OAGAO
O000O
000TO
ON0TO
OnGTO
(ours)
IRAE IM
14I 863
SPOR ANI
DOLOO
ONLGO
DALYO
DNL0O
DOLOO
ARIIES
BALL
ARSIEES
AERIIES
ARRIHES
Figure 7: Qualitative comparison of various C-STVSR methods at different time steps on Adobe240.
The spatial scaling factor and the temporal factor to x8. Best viewed zoomed
14

---

Published as a conference paper at ICLR 2026
Frame 4
Frame 6
Frame 8
Frame 10
Frame 12
VideoINR
MOTIF
BF-STVSR
(ours)
Figure 8: Qualitative comparison of various C-STVSR methods different time steps on Adobe240. The spatial scaling factor is set to x4 and the temporal factor to x8. Best viewed zoomed in.
B ADDITIONAL QUANTITATIVE RESULTS
B.1 COMPARISON WITH AVSR METHODS
As a complement to Sec. 4.2.1 we also train our method specifically for the AVSR task, enabling direct comparison with prior work (Li al. 2024 Shang et al.) 2024). We denote this variant as 22.5 which is architecturally identical to Large but trained on the REDS training set without temporal upsampling and supervised solely the temporal coordinates of the input frames.
With the numher of basis functions reduced to N 384.
15

---

Published as a conference paper at ICLR 2026
Frame 4
Frame 6
Frame 8
Frame 10
Frame 12
2111
VideoINR
75
5011
2011
5611.
5011.
5
MOTIF
501
50.
501.
201.
VIL
SA11
5611
5611.
5611.
5
BF-STVSR
501
501.
501.
501. 75
551
5511
BA 5511
5511.
5511.
5
501
501
501.
501.1
501. 75
(ours)
0000
VIE 0000
VIL
1000
551 BA RA
8848
5511
EA 5511
5511 501. 7
5511, 5
-
501
501
W
501
501. 75
Figure 9: Qualitative comparison of various C-STVSR methods at different time steps GoPro. The spatial scaling factor set to x4 and the temporal factor to x8. Best viewed zoomed in.
Table reports PSNR and SSIM metrics for multiple scaling factors on the REDS validation set.
Atthuuhh not explicitly designed for AVSR our model remains a more expressive C- e
STVSR method - v2.6 clearly surpasses dedicated AVSR approaches in PSNR. For example, at x2 scaling it exceeds the next best method by nearly 1 dB. In SSIM, only outperformed by ST-AVSR (Shang et al.) 2024), which leverages auxiliary pre-trained VGG16 encoder extract multi-scale structural and textural features. is worth noting that ST-AVSR also provides a baseline variant (B-AVSR) without this auxiliary encoder. Since 2255 likewise does not rely external feature extractors, B-AVSR arguably offers the most direct comparison in terms of training data.
Related qualitative samples provided in Fig. 14
16

---

Published as a conference paper at ICLR 2026
Frame
Frame 12
Frame
Frame
Frame
Frame 10
Frame
Frame 30
AREE
ARRE
MARET
MARE
VideoINR
28
28
28
AREE
AREE
MARE
MARE
28
MoTIF
28
28
28
MARE
BF-STVSR
AREE
MARE
MARE
Sobbe
28
28
28
28
AREE
IAN
HARE
Sabbr
MARED
Saies
MARE
28
(ours)
28
28
28
AREE
HARE
Saddy
MARED
MARÉ
Sundey
28
28
28
28
Figure 10: Qualitative comparison of various C-STVSR methods different time steps Vid4. The spatial scaling factor is set to x4 and the temporal factor to x2. Best viewed zoomed in.
B.2 COMPARISON WITH VSR METHODS
We go step further and compare 2 to standard video super-resolution (VSR) methods. In contrast to AVSR, VSR methods do not support arbitrary spatial scaling factors and are specifically trained for single upsampling rate (usually x4). In Tab. 7 compare V2. with BasicVSR (Chan et al. 2021), TTVSR (Liu et al. 2022), and FTVSR (Qiu et al. 2022). We find that our method achieves the highest PSNR among all compared models and matches the perf-rerro approach in SSIM, despite being the only method supporting arbitrary-scale upsampling.
B.3 ABLATION STUDY
In Tab. 8 we analyze several design choices of our method. To isolate the effect of the VFF capacity, we vary the size of the basis while keeping the rest of the model fixed. Lee. we chose different numbers N of basis functions and retrain the mapping from the 256-dimensional encoder output as well as the N-dimensional VFF basis.
On both Vid4 and Adobe240, smaller basis (N 256) leads noticeable performance drop, which is further exacerbated when reducing to N = 128. On the other hand, increasing the dimension of the basis to N 768 only marginally improves performance. We consider N 512
17

---

Published as a conference paper at ICLR 2026
VI
MoTIF
BF-STVSR
(ours)
GT
TOASTE
TEASTO
Parnet
purne
5-5 QuiNe
5- OURN1
34
10
BANN
CARARIR
CLEARIN
CLEARES
CLEADIN
CONE
Figure 11: Spatial Super Resolution x4 on Adobe240 dataset.
Table 6: Comparison of methods trained the AVSR task (PSNRt / SSIM1) the REDS validation set, with various spatial scaling factors. We re-trained SAVSR on REDS for fair comparison. Also note that ST-AVSR (*) leverages auxiliary pre-trained VGG-16 model for their structural and textural prior.
Method
x2
x3
x4
x6
x8
# Par.
SAVSR
36.20/ 0.959
32.18 / 0.901
29.89 0.843
27.13 0.744
25.53 0.672
18.9 M
B-AVSR
35.94 0.960
31.86 0.910
29.67
0.861
26.83
0.771
/
14 M
ST-AVSR*
/ 0.9969
33.41 / 0.937
31.03 / 0.897
27.89 / 0.812
0.746 26.04/
27.9 M
V2.
37.89 / 0.970
33.70 0.922
31.20 / 0.876
28.39 / 0.788
26.80 0.725
20.6
good compromise, noting that larger basis may in some applications be warranted, and would still be supported by the current encoder capacity.
We further examine the importance of learning the VFF frequencies by pre-setting with canonical 3D plane-wave Fourier basis. Frequency vectors are sampled uniformly from the 3D Fourier frequency space (k-space) and fixed during training. This variant performs significantly worse, which is expected: sampling 512 vectors uniformly in 3D frequency space results in a very sparse grid - only 5/51 = 8 unique frequencies per axis - limiting representation power. In contrast, learning the basis functions globally allows to flexibly distribute constrained number of basis functions so as to maximize their representational power, exploiting structure in the dataset. This is visible in Fig, where learned frequencies concentrate along coordinate axes and distribute non-uniformly over space-time. Notably, learning the base incurs only a negligible overhead: just 3.V additional trainable scalars and extra inference cost.
18

---

Published as a conference paper at ICLR 2026
VideoINR
MoTIF
BE-STVSR
(ours)
GT
Figure 12: Video Frame Interpolation x8 Adobe240 dataset.
VR
MoTIF
BF-STVSR
V3
GT
Qualitative comparison of various C-STVSR methods on Adobe240. The spatial scaling factor is set to x4 and the temporal factor to x8. We selected an example with large motion blur due to camera motion.
Figure 13:
Finally, we ablate the number of training iterations: Reducing to 50% of iterations leads to almost no performance loss (e.g., 26.76 vs. 26.71 dB PSNR on Vid4 x4). At 25% the drop is more noticeable, but 3 remains the best method by a substantial margin (c.g., V at 25%: 31.84 vs. second-best BF-STVSR 100%: 30.15 12 dB).
19

---

Published as a conference paper at ICLR 2026
S-AVSR
ST-AVSR
V2.5
GT
CALVIN
JEP
Figure 14: Qualitative comparison of various AVSR methods to our method on x4 spatial super resolution. Note that checkpoint for B-AVSR was available the time of writing.
Table 7: Comparison of our method with various recent video super-resolution (VSR) methods. Note that ours the only one capable of arbitrary-scale upsampling, while still outperforming others.
PSNR
SSIM
BSs
27.24
0.825
TTVSR
27.64
0.840
FTVSR
27.57
0.839
V25
27.77
0.840
B.4 DEGRADATION STUDY
We analyze two types of image degradation during test time, as shown in Table All experiments are conducted on the Vid4 dataset with spatial upscaling factor of x and a temporal upscaling factor of x2. (1) Noise only: We add zero-mean Gaussian noise with a specified standard deviation
to images normalized to the range [-1, 1]. The corrupted images are then rescaled to 8-bit [0, 255] and rounded. (2) Noise compression: We apply the same noise corruption, followed by H.264 video compression of the resulting frames. both cases, we compare the model's predictions against the non-degraded high-resolution ground-truth images.
20

---

Published as a conference paper at ICLR 2026
Table 8: Ablation experiments Vid4 and Adobe240 (average).
Vid4
Adobe240
PSNR
SSIM
PSNR
SSIM
N 128
26.19
0.797
30.55
0.884
256
26.50
0.809
31.55
0.903
N 512
26.72
0.817
32.16
0.914
= 768
26.74
0.818
32.29
0.917
Fixed wi
22.85
0.651
24.80
0.738
50% train iters.
26.71
0.817
32.12
0.914
25% train iters.
26.60
0.814
31.84
0.909
Table We ablate how each method's performance degrades under input corruptions on the Vid4 dataset with x4 spatial and x2 temporal super resolution. We evaluate two settings: 1) Gaussian noise and 2) Gaussian noise followed by H.264 video compression.
Degradation: N₂ (Gaussian noise)
Method
10-3 3
7.5 10-3
10-2
2.5 10-2
5 10
VideoINR
25.55 / 0.766
25.50/ /..66/0.76 0.763
25.45 0.759
25.04 / /0.729
24.35 24.35/0.677 0.677
MoTIF
25.73 / 0.771
25.68 0.729
25.62 / 0.763
25.16 0.730
24.46 0.675
BF-STVSR
25.82 / 0.774
25.77 / 00771
25.71 / 0.767
25.28 0.735
24.53 0.679
V3
26.68 / 0.814
26.50 0.806
26.28 0.796
25.58 / 0.757
25.18 0.721
Degradation: C o N₂ (H.264 of noised input)
X 10-3
7.5 x 10-3
1 x 10-2
2.5 X 10-2
5 X 10-2
VidcoINR
24.38 / 0.688
24.38 0.688
24.38 0.687
24.32 / 0.684
24.06 0.665
MoTIF
24.54 / 0.690
24.53 / 0.690
24.53 / 0.690
24.47 / 0.686
24.21 0.666
BF-STVSR
24.64 / 0.694
24.63 0.694
24.63 0.694
24.57 00.99
24.29 0.670
V3
25.35 / 0.726
25.35 0.726
25.34/ 0.726
25.29 / 0.722
25.02 0.706
C
USE OF LARGE LANGUAGE MODELS
LLMs were used exclusively for text polishing and grammar refinement.
21


Reviews:
[{"review_id": "RwGj2jbzFb", "reviewer": "Reviewer_wVPd", "summary": "The paper proposes a continuous space–time video super-resolution (C-STVSR) framework that represents an entire video as a spatio-temporally coherent 3D “Video Fourier Field” (VFF)—a finite sum of sinusoidal basis functions. This avoids decoupling space and time and eliminates brittle frame warping. A neural encoder with a large spatio-temporal receptive field predicts local frequencies, phases, and weights for these basis functions from the low-resolution input, enabling sampling at arbitrary spatial and temporal scales.  The VFF can be queried with a closed-form Gaussian point-spread function, yielding aliasing-free reconstruction by frequency-dependent rescaling rather than expensive filtering, which also generalizes well across scales.  Compared to prior C-STVSR methods and local INR approaches, the unified 3D trigonometric expansion captures long-range temporal dependencies, handles occlusions more robustly, and naturally models translation as phase shifts. Experiments show sharper, more temporally consistent results and about a 2 dB PSNR gain on Adobe240 test set, while being faster.", "questions": "- Given VFF’s continuous spatio-temporal sampling, can the representation be extended to video frame interpolation, prediction and or video generation, while preserving temporal coherence and anti-aliasing guarantees?\n- What motivates introducing a Gaussian PSF with tunable variance?\n- Why was JAX chosen over PyTorch/TensorFlow?\n- Could you suggest an ablation or variant that reduces training cost without degrading final quality?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A continuous-domain video representation that models videos as a single finite trigonometric expansion over (x, y, t) (“Video Fourier Field”), enabling arbitrary spatio-temporal sampling, encoding translation as phase shifts, and supporting analytic, aliasing-free queries.\n\n- The proposed end-to-end C-STVSR framework (V3) whose backbone aggregates over a large spatio-temporal receptive field to predict VFF parameters, leveraging a shared frequency basis for efficiency and stability, and a PSF-aware sampler; the whole system is differentiable and trained end-to-end.\n\n- Strong empirical results: consistent PSNR/SSIM gains across scales on REDS and markedly better video frame interpolation on Adobe240, yielding sharper, more temporally coherent outputs and avoiding warping artifacts typical of optical-flow-based baselines.", "weaknesses": "- Since the entire (x,y,t)-space is partitioned into local, axis-aligned regions, does this induce any discontinuities across region boundaries?\n\n- The description of Sec. 3.3 Conditional Fourier Parameterization is unclear. Is the voxel grid for the local basis expansion sample-specific or shared across all samples? How should inputs with different spatial resolutions and aspect ratios be handled? What exactly is the relationship between the voxel grid and the Video Fourier Field? Please provide a more detailed explanation.\n\n- After training, is the common frequency basis identical for all samples? Have you considered presetting a bank of frequency bases as anchors of frequency basis?\n\n- Why is training conducted on 16× GH200? If you used the same compute cost in training stage as prior methods, how would the performance be?\n\n- How does the method handle severe motion blur arising from fast or large motions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uevY7fOVaM", "reviewer": "Reviewer_MxXA", "summary": "This paper introduces a novel formulation for continuous space-time video super-resolution. Unlike previous approaches that decouple video representation into separate spatial and temporal components and rely on frame warping for motion compensation, the proposed method encodes video as a continuous, spatiotemporally coherent 3D Video Fourier Field (VFF). The resulting model, dubbed V³, achieves substantial improvements in both spatial and temporal super-resolution and establishes a new state of the art across multiple benchmarks. However, certain issues regarding the experimental verification remain to be addressed.", "questions": "(1) The training details, specifically the loss function and the hyperparameter σ, need to be clearly stated.\n\n(2) The experimental comparisons should be revised to include benchmarks against AVSR methods.\n\n(3) Ablation studies must be added to systematically evaluate the impact of core model parameters, such as the number of basis functions N and the hyperparameter σ.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "(1) This work represents video as a continuous, spatiotemporally coherent 3D Video Fourier Field, departing from conventional approaches that decouple video representation into separate spatial and temporal components.\n\n(2) The proposed model, V³, substantially improves both spatial and temporal super-resolution performance, setting a new state-of-the-art on multiple benchmarks.", "weaknesses": "(1) The training procedure of V³ lacks clarity. Key implementation details are missing, such as the specific form of the loss function and the value of the hyperparameter σ.\n\n(2) The work would benefit from comprehensive ablation studies. At a minimum, a systematic evaluation of the impact of two key design choices is necessary: the number of basis functions (N) and the value of the hyperparameter σ on final model performance.\n\n(3) The experimental comparisons in Section 4.2.1 have several shortcomings. First, comparisons with Arbitrary-Scale Video Super-Resolution (AVSR) methods are absent, and no visual results from AVSR models are provided. Second, the selection of the two image super-resolution models (LTE and CLIT) in Table 2 is not justified in the text, and the models themselves are not properly cited. Furthermore, as LTE and CLIT are from 2023, including comparisons with more recent methods—such as HIIF [1] would strengthen the evaluation.\n\n[1] \tYuxuan Jiang, et. al. HIIF: Hierarchical Encoding based Implicit Image Function for Continuous Super-resolution. CVPR 2025.\n\n(4) There are minor typographical errors: a comma is missing in Equation (4), and an unintended line break appears on line 396.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xaRbDJDVpV", "reviewer": "Reviewer_H6t6", "summary": "The paper proposes V3, a novel method for continuous space-time video super-resolution (C-STVSR), centered around a new representation called the Video Fourier Field (VFF). VFF models a video as a continuous 3D trigonometric expansion over joint spatial and temporal coordinates (x,y,t), avoiding the common practice of decoupling space and time via optical flow and explicit frame warping. \nKey contributions include:\nVFF: A unified, interpretable, continuous video representation using a finite sum of 3D sinusoidal basis functions.\nEnd-to-end framework (V3): A neural encoder predicts VFF coefficients from low-resolution input, enabling arbitrary spatial and temporal upsampling.\nAnalytical anti-aliasing: Incorporates a closed-form Gaussian point spread function (PSF) for aliasing-free reconstruction at any scale.\nState-of-the-art performance: V3 outperforms prior C-STVSR methods by >1.5 dB PSNR across multiple benchmarks while being faster and more memory-efficient.", "questions": "I'm a bit confused about the method in this paper. The paper proposes modeling a video as a 3D Fourier series, called a Video Fourier Field (VFF). Does this mean that each individual video is represented by its own, independently trained VFF?  \n\nThis seems different from conventional data-driven video super-resolution (VSR) algorithms. In traditional VSR, once the model is trained, it can theoretically take any input video and produce a high-resolution output.  \n\nIf, in this paper, every video requires its own dedicated VFF representation, how can this approach be applied in real-world scenarios?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper introduces a completely novel approach by modeling video as a joint 3D Fourier field, establishing a new paradigm in the field of video super-resolution.\n\nThis paper leverages the classical principle that \"translational motion manifests as phase shifts in the frequency domain\" and builds a model that does not rely on traditional optical flow algorithms.\n\nThe experimental validation is comprehensive: extensive tests on multiple tasks (such as continuous spatiotemporal super-resolution, spatial super-resolution, and video frame interpolation) and various public datasets consistently demonstrate that its performance significantly surpasses existing state-of-the-art methods, with a notable improvement (PSNR gain exceeding 1.5–2 dB).", "weaknesses": "The paper employs a globally fixed set of frequencies during training, with only independent modulation of amplitude and phase for each voxel. While this enhances stability, it may limit the model's adaptability to diverse motion spectra (e.g., the differences between high-frequency vibrations and slow translations). The paper does not analyze what specific frequencies are learned or explore how these frequencies correlate with different types of motion. Providing ablation studies or visualizations of the learned frequency distributions would more convincingly demonstrate the expressive power of these basis functions.\n\nThere is a lack of comparison and review of classical data-driven methods, such as BasicVSR, TTVSR, and FTVSR.\n\nThe degradation model is overly simplified: training assumes ideal bicubic downsampling and clean inter-frame subsampling. However, real-world videos often contain noise, compression artifacts, or motion blur. Although Section 5 mentions this as future work, even a brief experiment (e.g., validating the model's robustness under mild noise or JPEG compression) would more strongly support its claimed practical applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0HxigVw4sL", "reviewer": "Reviewer_hNzY", "summary": "This paper introduced INR framework  for arbitrary video super resolution. In particular, the authors utilize 3D Fourier feature embedding to resolve aliasing components during reconstruction. By using 3D representation, they are able to achieve better reconstruction results compared with 2D Fourier embedding or 2D Fourier embedding + B-spline embedding.", "questions": "- How does this algorithm behaves for large motion gap such as big fluctuation in optical flow? or abrupt frame changes(key-frame)? In my guess, V^3 depends on 3D Fourier representation which includes time domain resulting in further aliasing components along with temporal axis when they encounter abrupt or big motion changes along with temporal axis\n\n- Isn't there any aliasing results in slomo reconstruction? A task related with slomo is focusing on temporal super resolution, so VFF only provides temporal enhancements which may contradict with spatial super resolution.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- 3D video Fourier Field is proposed to reconstruct 3D arbitary super resolution in video. This is easy to handle and simply explained.\n- The proposed VFF shows high jump in quantitative comparisons in Tables 1 & 2. It's meaningful gap.\n- Temporal consistency looks very coherent in Fig. 5. The core deliver for vsr is usually accurate prediction of temporal coherency. In that terms, V^3 algorithm shows optimal solution for video super resolution.", "weaknesses": "- I would like to see computational complexity including memory consumption and inference time as table in main text.\n- This is regression model, so when we see the results related with texts in Fig.10, we couldn't guess the text information. Nevertheless, the proposed method is highly impactful in quality perspective.\n- no detailed analysis on Fourier coefficients. It will be helpful to show Fourier features aligned with video contents such as DFT spectra of the original contents and estimated coefficients.\n- No controlling method for weighting spatial or temporal axis, independently. For example, temperature methods sometimes boiled down the effect of reconstruction. Here in VFF, there is no induced effect by user side. VFF automatically estimated Fourier coefficients along temporal and spatial axis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)