Trajectory-aware Shifted State Space Models for Online Video Super-Resolution
Authors: Qiang Zhu
Xiandong MENG
Yuxuan Jiang
Fan Zhang
David Bull
Shuyuan Zhu
Bing Zeng
Ronggang Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Video Super-resolution
Online
Mamba
Trajectory
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Online video super-resolution (VSR) is an important technique for many real-world video processing applications, which aims to restore the current high-resolution video frame based on temporally previous frames. Most of the existing online VSR methods solely employ one neighboring previous frame to achieve temporal alignment, which limits long-range temporal modeling of videos. Recently, state space models (SSMs) have been proposed with linear computational complexity and a global receptive field, which significantly improve computational efficiency and performance. In this context, this paper presents a novel online VSR method based on Trajectory-aware Shifted SSMs (TS-Mamba), leveraging both long-term trajectory modeling and low-complexity Mamba to achieve efficient spatio-temporal information aggregation. Specifically, TS-Mamba first constructs the trajectories within a video to select the most similar tokens from the previous frames. Then, a Trajectory-aware Shifted Mamba Aggregation (TSMA) module consisting of proposed shifted SSMs blocks is employed to aggregate the selected tokens. The shifted SSMs blocks are designed based on Hilbert scannings and corresponding shift operations to compensate for scanning losses and strengthen the spatial continuity of Mamba. Additionally, we propose a trajectory-aware loss function to supervise the trajectory generation, ensuring the accuracy of token selection when training our model. Extensive experiments on three widely used VSR test datasets demonstrate that compared with six online VSR benchmark models, our TS-Mamba achieves state-of-the-art performance in most cases and over 22.7% complexity reduction (in MACs). The source code for TS-Mamba is available at https://github.com/QZ1-boy/TS-Mamba.

Published as a conference paper at ICLR 2026
TRAJECTORY-AWARE SHIFTED STATE SPACE MODELS FOR ONLINE VIDEO SUPER-RESOLUTION
Qiang Zhu', Xiandong Meng', Yuxuan Jiang', Fan Zhang', David Bull? Shuyuan Zhu", Bing Zeng'. Ronggang Wang!
Pengghheng Laboratory, *University of Electronic Science and Technology of China Universstty of Bristol, "Shenzhen Graduate School, Peking University {zhuqiang, mengxd) @pcl.pcc...m on eezsyQuestc. edu. en, rgwanglpkusz. edu. cn
ABSTRACT
Online video super-resolution (VSR) is important technique for many real-world video processing applications, which aims to restore the current high-resolution video frame based on temporally previous frames. Most of the existing online VSR methods solely employ one neighboring previous frame to achieve temporal alignment, which limits long-range temporal modeling of videos. Recently, state space models (SSMs) have been proposed with linear computational complexity and a global receptive field, which significantly improve computational efficiency and performance. In this context, this paper presents novel online VSR method based on Trajectory-aware Shifted SSMs (TS-Mamba), leveraging both long- term trajectory modeling and low-complexity Mamba to achieve efficient spatio- temporal information aggregation. Specifically, TS-Mamba first constructs the trajectories within video select the most similar tokens from the previous frames. Then, a Trajectory-awarc Shifted Mamba Aggregation (TSMA) module consisting of proposed shifted SSMs blocks is employed aggregate the selected tokens. The shifted SSMs blocks are designed based on Hilbert scannings and corresponding shift operations to compensate for scanning losses and strengthen the spatial continuity of Mamba. Additionally, we propose trajectory- function to supervise the trajectory generation, ensuring the accuracy of token selection when training our model. Extensive experiments on three widely used VSR test datasets demonstrate that compared with online VSR benchmark models, our TS-Mamba achieves state-of-the-a performance in most cases and over 22.7% complexity reduction (in MACs). The source code for TS-Mamba is available https: /ttpttgttt//////// com. Q21-boy TS-Mamba.
loss
INTRODUCTION
Among various video super-resolution (VSR) application scenarios, online VSR has recently attracted significant interest due to the growing popularity live video conferencing and live broadcasting applications (Fuoli et al., 2023; Xiao et al., 2023). In online VSR, the current high-resolution (HR) video frame typically restored using only its low-resolution (LR) counterpart and previous frames. This is constrained by the requirements for low latency and low computational complexity inherent to these online real-time applications.
In VSR model, temporal alignment or aggregation is module employed compensate for the information from neighboring frames before generating the current HR frame. Advanced temporal alignment aggregation modules have been recently developed, which are based on deformable convolution networks (DCN) (Wang et al., 2019; Tian et al., 2020), flow-guided deformable alignment models (Chan al., 2022a; Zhu al., 2024c), non-local attention mechanisms (Li al., 2020; Yi et al., 2019), Vision Transformer based spatio-tempon information aggregation (Liu et al., 2022a; Tang et al., 2023) or Diffusion models (Wang et al., 2025; Liu al., 2025; Zhuang et al., 2025).
"Corresponding Authors.

---

Published as a conference paper at ICLR 2026
30.75
TS-Mamba
(3.0M)
9DAN(3.9M)
30.70
Reduce 22.7% MACS
TMP(3.1M)
KSNet (3.0M)
30.65
PSNR (dB)
DAP-128(-)
30.60
30.55
Params.
1M
30.50
2M
3M
B (3.0M)
30.45
4M
110
120
130
140
150
160
170
MACs (G)
Figure I: Comparison existing online VSR methods with our TS- TS-Mamba in terms of PSNR and MACs the REDS4 dataset. Our TS-Mamba outperforms these SOTA methods and significantly reduces complexity in terms of MACs.
Although they offer superior VSR performance, these methods are typically associated with high complexity and, therefore, are not ideal for online VSR.
To mitigate these limitations, recent online VSR methods have adopted more efficient temporal alignment modules, such as lightweight optical flow networks (Sajjadi et al., 2018; Xiao et al., 2023), deformable attention mechanisms (Fuoli et al., 2023; Yang et al., 2023), and temporal motion propagation modules (Zhang et al., 2024b). For example, CKBG (Xiao et al., 2023) utilized lightweight optical flow network to estimate motion between frames and perform motion compensa- tion. DAP (Fuoli et al., 2023) designed deformable attention pyramid module to dynamically focus on the most salient locations between frames and progressive refine the offsets to improve temporal alignment performance. FDAN (Yang et al., 2023) proposed flow-guided deformable attention propagation module that introduces the optical flow to guide the offset generation to efficiently exploit the temporal information between frames. Despite their efficiency, these methods predominantly use short-term temporal information based on convolutional neural networks (CNN) typically from a single previous frame, which restricts their ability to further enhance reconstruction quality. While incorporating long-term temporal alignment can improve performance, it often introduces significant applications.
computational overhead, resulting in challenges for real-time or Therefore, it is valuable to develop the efficient long-range models for online VSR applications.
Recently, low-complexity state space models (SSMs) (Gu et al., 2021; Gu & Dao, 2023) have been proposed with linear computational complexity and with relatively large receptive fields, which can potentially improve performance with limited complexity. Inspired by this, we propose a Trajectory- aware Shifted Mamba for online VSR, denoted TS-Mamba, leveraging long-temn trajectory modeling and lowccompyyxtty Mamba for achieving the token-level spatio-temporal aggregation. In TS-Mamba, trajectories within video are first constructed for selecting the most similar tokens from the previous frames. A trajectory-awant shifted Mamba aggregation (TSMA) module is then employed that consists of shifted SSMs blocks to aggregate the selected tokens. The shifted SSMs blocks are designed with specific procedures based on Hilbert scannings and four shift operations compensate for scanning losses and strengthen the spatial continuity of Mamba. Moreover, we propose a trajectory- accuracy of token selection when training our model. The proposed TS-Mamba model enables efficient long-term video modeling with significantly reduced computational complexity. The primary contributions are summarized as follows:
loss function to supervise the trajectory generation, optimizing the
TSmMamb is the first SSMs-based online VSR model, which aggregates long-term spatio- temporal information from previous frames at the token level for restoring current HR frame. This is different from existing online VSR methods which typically use CNN-based temporal alignment to exploit temporal information from a single previous frame.
This is also the first time to introduce video trajectories into Mamba to select the most similar tokens from previous frames and construct the new trajectory- aware shifted Mamba model for efficient token-level spatio-temporal information aggregation.
2

---

Published as a conference paper at ICLR 2026
The novel shifted SSMs blocks are designed based on four different shift operations and Hilbert scannings to effectively compensate for the intra-window and inter- inter-windde window losses of Hilbert scannings and strengthen the local spatial continuity of Mamba.
The proposed method has been benchmarked on three widely used test datasets and shows superior VSR performance with more than 22.7% computational complexity reduction in terms of MACs over five (SOTA) online VSR methods (as shown in Figure 1).
RELATED WORK
2.1 VIDRO RESOLURIOOO
Video super-resolution (VSR) is fundamental low-Ievel vision task that aims to restore an HR video from its LR counterpart. Existing VSR methods are typically learning- based, utilizing various deep neural networks (Teed & Deng, 2020; Zhu et al., 2019; Arnab et al., 2021; Ho et al., 2022). For example, optical flow-based methods (Chan et al., 2021; Liu et al., 2022b) explore the temporal motion between frames to align them; deformable convolution networks (DCN)-based methods (Tian et al., 2020; Wang et al., 2019; Dong et al., 2023) learn the motion offsets between frames for feature alignment. Moreover, Now-guided deformable-buse 2024c) combine optical flow and DCN to achieve better feature alignment. Non-local attentionnbusel methods (Li et al., 2020; Yi et al., 2019) aggregate global information for feature aggregation. Vision Transformer-basee based methods (Liu et al., 2022a; Tang et al., 2023; Lin et al., 2022) aggregate long- term spatio-temporal information in video to restore SR frames. Diffusion-baase al., 2025; Liu et al., 2025; Zhuang al., 2025) build the long-range modeling using designed diffusion models for information aggregation. However, these methods are still associated with high complexity and are therefore not best suited for real-time online VSR applications. Recently, some Mamba-base based works (Xiao & Wang, 2025; Tran et al., 2025) were proposed that use low- low-complexity Mamba with global receptive filed to improve VSR performance. Although Mamba-baser based methods achieve some reduction in model complexity, their high-overhead repeated scannings hinders efficient implementation of real-time online VSR.
methods (Chan et al., 2022a: Zhu et al.,
methods (Wang
2.2 ONLINE VIDEO IOPER-RESOLUTION
Due to the specific requirements of online applications, online VSR methods are expected to be lightweight and have low latency. Therefore, most existing online VSR methods have been proposed (Saijadi et al., 2018; Cao et al., 2021; Fuoli et al., 2023; Xiao et al., 2023; Jiang et al., 2025a) with efficient feature alignment modules. For example, EGVSR (Cao et al., 2021) and CKBG (Xiao et al., 2023) utilized lightweight optical flow networks to estimate motion between frames and perform motion compensation. KSNet (Jin et al., 2023) proposed a kernel-split manner to reparameterize convolutional kernels the high-value channel, enabling representation of dynamic information and reducing complexity along the channel dimension. TMP (Zhang et al., 2024b) employs an efficient temporal motion propagation method that leverages motion field continuity to achieve fast feature alignment. DAP (Fuoli et al., 2023) designed a deformable attention pyramid module to dynamically focus on the most salient locations between frames and progressive refine the offsets to achieve temporal alignment improvement. FDAN (Yang et al., 2023) proposed a flow-guides deformable attention propagation module that introduces the optical flow to guide the offset generation effi- ciently exploit the temporal information between frames. It is noted that, however, these online VSR methods are only based on one previous frame in feature alignment due to the complexity limitation, which hinders further improvement of VSR performance. Different from existing deformable-basd methods (Fuoli et al., 2023; Yang et al., 2023) that based on short-term spatio-tempor aggregation, in this work, our MS-Mamba introduces of trajectories and designs shifted SSMs blocks, enabling it improve the ability for long-range spatio-temporal information aggregation.
2.3 STATE SPACE MODELS
State space models (Gu al., 2021; Gu & Dao, 2023), e.g., Mamba, have been widely employed in vision tasks (Liu et al., 2024; Zhu et al., 2024a) due to their linear computational complexity and ability model global dependencies. Recently, some Mamba-based methods are proposed for
3

---

Published as a conference paper at ICLR 2026
Trajectory-aware Slaffeed SSMs Network (TS-Mamba)
lix
ER
Trajectory Generation
Toben Edectio
V
TSMA
Pix R()
2
(1
(dr)
ia U(+
158
TSMA
SS3D
lntea Window Competuation Beazch
5:000
SSMs Block (8cuz-1)
Adiliio
Millyls
Compasatiio
2:
Figure 2: The architecture of the TS-Mamba network. Trajectories of videos are first generated and the similar tokens from previous frames are selected along trajectories. Then, the selected tokens alongside the current frame token are fed into the trajectory-awar shifted Mamba aggregation (TSMA) module to achieve the long-term
information aggregation.
image/video super-resolution. For example, MambalR (Guo et al., 2024), and MambalRv2 (Guo et al., 2025) used Mamba to achieve the global receptive field. TAMambalR (Peng et al., 2025) introduced texture-aware state space model to focus on textures regions for improving SR quality. VSRM (Tran et al., 2025) proposed the of apatio-temporal aggregation. MamEVSR (Xiao & Wang, 2025) proposed interleaved Mamba and a cross modality Mamba to interleave tokens and further leverage spatio-temporal information to capture finer details. Typically, Mamba converts 2D images into 1D tokens through scanning (Qiao et al., 2024; Shi et al., 2025), resulting in spatial continuity loss inherent to images. To enhance the ability of Mamba, some advanced scanning techniques have emerged address this issue, such as bidirectional scanning (Hu et al., 2024; Shi et al., 2025), cross scanning (Liu et al., 2024), continuous 2D scanning (Yang et al., 2024a), and local scanning (Huang et al., 2024). To the best of our knowledge, the use of Mamba has not yet been investigated for the online video super-resolution task. Unlike existing Mamba-based works (Xiao & Wang, 2025; Tran al., 2025) that neglect the local spatial continuity of Mamba, we introduce sophisticated shift operations for Hilbert scannings to enhance the ability of Mamba to maintain local spatial continuity, improving the online VSR performance with high efficiency.
Mamba and the camporal-to-spatiin Mamba to ability
METHODOLOGY
In online video super-resolution, when reconstructing the 8h frame in low-resolution video, denote the current LR frame as Iř,R and temporally previous LR frames as { E [t - T, []}- The proposed trajectory-awart shifted state space models, TS-Mamba, are illustrated in Figure 2. Here, all these LR video frames {IÉ k E [ - T.E} are first fed into the token and trajectory generation G(-) module to extract the current frame token Q and the tokens of previous LR frames V:
== (Fi.) {qf} i [1,
(1)
G= = i [
(2)
where G(-) consists of convolution layer and N, residual blocks to generate features and tokens from video frames, the token number, and T is the temporal window size. Based on the generated tokens {g.), the trajectories T of Lin frame can be formulated as set of trajectories,
T { (2 v
N1,N [t- T,4),
4

---

Published as a conference paper at ICLR 2026
where 2² [1, WT, and H and W represent the height and width of the feature (for LR frame), respectively. Each trajectory Tf contains sequence of coordinates (af ut) [1, N]}, and the end point of trajectory of is associated with the coordinate (x), v() of token g--
then select s the most similar tokens V. along the trajectories and feed them into the proposed trajectory-aware shifted Mamba aggregation (TSMA) module alongside token Q to achieve spatio- temporal information aggregation:
F TSMA V.)-
Finally, the aggregated feature FLn and current LR frame In are fed into the reconstruction network R(-) and the upsampling U(-) network, respectively, to produce the super-resolved frame l's
I'n R(Fin) + U(Ln)):
(5)
in which R(-) consists of two convolution layers, N2 residual blocks, and pixelshuffle layer. U(-) here represents the bicubic upsampling operation.
3.1 TOKEN SELECTION
In order to select the most similar tokens along trajectories, we reformultee tokens Q, V assoociated with trajectories T'. Based on the formulation of the trajectories in Equation 3, tokens Q, and V can be formulated as:
@ = (gr;), [I, V [u++) i E [1,N], k -
We compute cosine similarity between the token Q, and tokens V to select the most similar tokens along trajectories. The indices of the selected tokens and the selected tokens can be formulated as:
4r
[hi] Top-k
hy [1,7 1],
12
gr;
V. au 1 i i NJ.
Thus, the process of the TS- Mamba network is described as:
IS TS-Mamba 2, V,T*) fo TfET!
R(TSMA( (17, to ))= + U(ER).
(8)
3.2 TRAJECTORY-AWARE SHIFTED MAMBA AGGREGATION
Mamba networks typically used to convert 2D images into ID tokens via scanning, resulting in spatial continuity losses inherent to the images. Existing works (Zhang et al., 2024a; Xiao & Wang, 2025) do not analyze the degree of discontinuous regions but instead repeatedly multiple scannings, making these methods hard to maintain the spatial continuity of the image and instead lead to greater complexity.
To address this issue, in this work, we first analyzed the spatial discontinuity in Hilbert scannings and then proposed trajectory-aware shifted Mamba aggregation (TSMA) module that combines a standard SSMs block and the proposed shifted SSMs (S- (S-SMMs) blocks in the "Scan-Shift-Scar" manner to compensate for the mnnd-wwindww and mindow losses of Hilbert scannings. As illustrated in Figure 2, in the TSMA module, token Q and selected tokens V. are first concatenated along the channel dimension and fed into two paths in a specific "Scan-Shift-Scan' manner, i.e., ! or 2, each of which consists of a standard SSMs block and two parallel S-SSMs blocks to compensate for the losses according to the scanning of the standard SSMs block. The output of each path is concatenated and then aggregated by convolution layer and deformable attention block (DAB) (Xia et al., 2022) to obtain the output feature. Each SSMs/S-SS SSSMs block and DAB is preceded by layer normalization (LN) and is followed by residual connection. In each SSMs/S-SSM. block. the trajectory- aware tokens are scanned based on spatial Hilbert selective scannings along the temporal dimension (SS3D) capture long-term temporl characteristics.
5

---

Published as a conference paper at ICLR 2026
Four Types Iilbert Scannings
DTSCE) f- 16
01A f-
BRIL f-
U.009,5
(b) Procedure (3))))
(c) Skifled Windows 74/.///////2..2222///////01/
Figure 3: Illustration of Hilbert scannings and shifted windows generated by seven procedures. (a) Four types of Hilbert scannings. (b) The procedure (1), 3), and eliminatioon value d. (c) Shifted windows and climination values for procedures P(L, PILUUL(112/3UURR(2238
respectively.
3.3 DISCONTINUITY FOR HILBERT SCANNING
To evaluate the spatial discontinuity of Hilbert scannings in local windows, define the discontinuity degree D& as follows. If the four adjacent areas are successively scanned, the region consisting of these four scanned areas is considered as continuous (Da 0); otherwise, the discontinuity degree Da equals the number of areas that are not successively scanned. For a region consisting of four adjacent areas, the range of discontinuity degree Da {0, 1,2,3). This is illustrated in Figure (a), where four typical Hilbert scannings, i.e., Sconn = [Scan-1, Scan-2, Scan-3, 3, Scan-4] are shown on 4 grid. Here the region with Da = marked by green circle and the region with D2 2 is marked by a red circle.
Moreover, we extend the general case that based on the 8x8 grid further discuss the discontinuity degrees. An x region is partitioned into four 4x4 local regions and illustrate the discontinuity degree Da within and between local windows under Scan in Figure 3 (b). can be observed that both intra- wirraww discontinuity and inter-winder discontinuity exist. In particular, due the nature of Hilbert scanning, the central region between windows is widely spaced (the inter-window discontinuity). resulting in inter-level gaps. Here, the discontinuity degree Da equals we mark this region with a gray circle in Figure 3 (b).
3.4 SHIFTED SSMS BLOCK
To eliminate the discontinuity of Hilbert scannings, we propose the "Scan-Shift-Scan" manner that combines window shifting with specific Hilbert scannings to strengthen the continuity of SSMs. The shifting can be defined based on the shift direction and shift position, e.g., Up 1 position (U(I)) Up Left I position (UT(1)) and Down Right position (DR(2)). Our "Scan-Shift-Scaal designed based on the four Hilbert scannings (shown in Figure (a)) and these window shifting processes. As shown in Figure 3 (b), illustrate the procedure of Scan 1- (1) (1)-Sccan-3 as an example. The local windows are first partitioned by (1) shift operation and then cyclic fed as the shifted windows. It can be inferred that the second scanning (Scan-3) 3) the shifted window can eliminate the discontinuity first scanning (Scan-1). 1). To simplify the description of procedure, we define the procedure
manner is
P(L,))(p).)) Sc1(1) Sf(p) Sc2(i),
where the first and the second scannings are denoted as Sc, Sc2()) E I 1,j E (1,2,3, 3,4,. Shift operations are denoted Sy(p) E (U()),L(p)) U UR(P), D(p), (DL(p,,R(pp) can be formulated as P (1),
{1,2,3)}. Therefore, procedure Scan 1 (L)-scnnn ()
To evaluate the discontinuity elimination, we set three symbols and define elimination value 5 to mark and calculate the elimination. Specifically, we use "\". "(", and "\" on the circle for representation that eliminates 1, 2, and discontinuity degrees, respectively, in Figure 3 (b)- (c). The elimination value is calculated by summing the eliminated discontinuity degrees that consist of intra- windoww discontinuity elimination and inter-window discontinuity elimination, i.e.,

---

Published as a conference paper at ICLR 2026
5= Oitra + Ointer- We have investigated possible procedures and illustrated the representative shifted windows generated by six shift operations, i.e., U'L(I), UL(2), (L),),, and U (R),, U UR2,), UR3,), under the first scanning (Sean-1) and second scanning (Scan-3) in Figure 3 (c). It can be inferred from Figure 3 (b)-(c) that procedure P(1, U()),)) 3) achieves the best elimination (5=18), and the best intra-window discontinuity elimination (8mma=18) but doesn' 't eliminate inter-window discontinuity (0), We infer that the other three procedures also achieve the best climination: P(2, L(I), 4), P(3, D(1), 1), (4, R(1), 2). Moreover, procedures P(1, U L3),, 3), and P(I, (R3), have the best inter-wing wierww discontinuity elimination (6) but worse than procedure P(1, (1), 3) for intra- wira-window discontinuity elimination (8).==8) The more procedures and details are provided in the supplementary.
Based on the formula of procedure, we can calculate the elimination value å of the procedure Sf)) We summary all the procedures with our supplementary and we can infer its range value of elimination value 8 E [4, 18), the range value of antra-window discontinuity elimination dintra- [0, 18] and the range value of interw discontinuity elimination dinter [0, 6j. There- fore, we elaborately find out the combinations of shift operations and Hilbert scannings to construct two S-SSMs blocks in parallel branches, i.e., intra- window compensation branch (IntraWCB) and interww compensation branch (InterWCB), to optimally eliminate corresponding discontinuities. procedures for the parallel SSMs blocks construct
As illustrated in Figure 2, TSMA module: ①: ((1,U(1),) P(1, (L, @: P(2, L(1), 4) P(2, (3). 4) to achieve sufficient climination of discontinuity.
3.5 SELECTIVE SCANNING ALONG TEMPORAL DIMENSION
To achieve temporal token aggregation, we implement spatial Hilbert-based selective scanning along the temporal dimension, i.e., SS3D. As shown in Figure 2, showcase the SS3D processing with Scan-1. The current token {9+;} and selected tokens {u ny }ー=1 are scanned to convert spatio- temporal neighboring pixels into a ID token sequence. Each token sequence undergoes selective scanning based the local windows. This process interweaves selected tokens with current tokens, enabling information to interact across spatial and temporal dimensions to capture long-term spatio- temporal characteristics. By scanning apatio-temporall] adjacent pixels, SS3D preserves local spatial information and progressively captures global temporal patterns.
3.6
LOSs FUNCTION
We adopt Charbonnier loss (Lai et al., 2018) the spatial loss function supervise the SR frame generation:
Capa /
M ss 1l + e2,
(10)
in which l is the HR frame and the is set to 1 x 10-4
To supervise the trajectory generation for ensuring the accuracy of token selection, we first employ the formulation of trajectories of LR video in Equation to generate trajectories of HR video:
ThR= = { { (((H)) = (a vt } [1, & E [t T.t..
(11)
Based on this, we propose our trajectory-aware loss function:
Ciry | ((TAR) /
(12)
where Is is the downsampling with scale factor & that subsamples every coordinate to LR size.
Overall, the total loss is:
Liotal = Cspa + XLtrj>
(13)
in which the hyperparameter À is set to 0.I.
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Following previous online VSR research (Jin et al., 2023; Zhang et al., 2024b), we use REDS (Nah al., 2019), and Vimeo-90K (Xue et al., 2019) as training datasets. REDS4 is used for evaluating the
7

---

Published as a conference paper at ICLR 2026
Table I: Comparison with state-of-the-art online VSR methods. The runtime, FPS, parameters, and PSNR(dBySSSM are reported on three benchmarks with BI and BD degradations.
Category
BI degradation
BD degradation
Supp. HRun. FPSt MACs|,Params. (M)
Methods
Frame 2e (ms) (1/s)
(G)
REDS4(RGB)
Vid4(Y))
Simeo-90K-T(YY) (PSNR/SSIM)
Vid4(Y)t
(PSNR/SSIM)
(PSNR/SSIM)
(PSNR/SSIM)
BasicVSR
P+F
x
63
15.9
397
6.3
31.42/0.8909
27.24/0.8251
37.53/0.9498
27. 27.96/0.8555
IconVSR
P+F x
70
14.3
452
8.7
31,67/088948
27.39/0.8279
37,8 37,84/0.9524
28. 28.04/0.8570
Bidirectional
+
P+F X
13.0
418
7.3
32.39/0.9066
27.79/0.8400
38.21/0.9550
29. 19,04/0,875
SSI.-bi
P+F x
41.7
31.06/0.8931
17.15/0.8208
37.06/0.9458
27. 84/56608.431
DFVSR
P+F x
7.1
32.76/0.9081
27. 27.92/0.8427
38.51/0.9571
29.56/0.8983
MIA-VSR
P+F x 318
3.1
3220
16.5
32.78/0.9220
28.20/0.8507
IART
P+F X
180
5.6
5020
13.4
32.90/0.9133
28 26/0.85177
38.62/0.9557
29. 29.68/0.8884
VSRM
P+F X
223
4.5
2174
17.1
33.11/0.9162
28.44/0.8552
Bicubic
N
V
16.14/007297
23, 23.78/0.6344
31.30/0.8687
21.80/0.5246
RRN
V
34
29.4
193
3.4
28.82/0.8234
25 15.85500.666 7660
36.69/0.9432
27 27.69/0.8488
VSsicVSR++*
V
25.0
146
50.44/0.8686
27.06/0.8173
37.11/0.9464
27.49/0.8426
Online
DAP-128
V
26.3
165
30.59/0.8700 8703
37729/0.9476
FDAN
P
V
34
29.4
146
3,9
30.71/0.8722
27. 14/0.8206
37,36/0 94831
27.76/0.8477 8471
KSNet
P
V
31
32.3
145
3.0
30.69/0.8724
27.14/0.8208
37.34/0.9490
27.63/0.8444
TMP
V
25
40.1
176
3.1
30.67/0.8710
27. 10/.0008810 167
37.33/0.9481
27.61/0.8426
VSRM
V
32.7
30.64/0.8701
27.10/0.8163
37.28/0.9477
27.57/0.8425
TS-Mamba
V
29
33.5
112
3.0
30.73/0.8277
27.17/0.8009
37.36/0.9482
27.70/0.8473
models trained on the REDS dataset, while Vimeo-90K-T and Vid4 (Liu & Sun, 2013) are utilized for benchmarking the models trained on the Vimeo-90K dataset. Two degradations, BI (bicubic) and blur degradation (BD), are used to perform downsampling and the downsampling factor is set to å 4. For BI downsampling, the HR frame is downsampled by bieubic filter. For BD downsampling. the HR frame is first blurred by a Gaussian filter with standard deviation a = 1.6, and then the blurred frame is subsampled for every pixels to generate the LR frame. PSNR and SSIM are adopted as performance evaluation metrics. Runtime (Run.), FPS (frames per second), MACs, and parameters (Params.) are computed on LR frame of size X80x320 evaluate model complexity and speed.
In the experiments, the numbers of residual blocks N1 and N₂ are set to 2 and 13, respectively. The token size is 4x4 and the window size 8x8. The selected token number is set as 3. Random flips, rotations, and temporal inversion operations are performed for data augmentation. Adam optimizer (Kingma, 2014), and Cosine Annealing scheme (Loshchilov during network training. The HR patch size is 256x 256 and the batch size is 8. The total number of iterations is 600K. The proposed method is implemented on the PyTorch platform with two NVIDIA GeForce RTX 3090 GPUs Following (Liu et al., 2022a), lightweight optical flow network (Kong et al., 2021) is adopted to update trajectories. The temporal window size T set as 15 based on (Zhang et al., 2024b) when training REDS (Nah al., 2019). For the Vimeo-90K (Xue et al., 2019) dataset, the original sequence is temporally flipped to obtain a 14- 14-frame sequence.
Hutter, 2016) are used
We compare our approach with five SOTA online VSR methods, including RRN (Isobe et al., 2020), DAP-128 128 (Fuoli et al., 2023), FDAN ( (Yang et al., 2023), KSNet (Jin et al., 2023), and TMP (Zhang et al., 2024b), and eight bidirectional propagation VSR methods, BSs (Chan et al., 2021)
ISO (Chan et al., 2021). BasicVSR++ (Chan et al., 2022a), SSL (Xia et al., 2023), DFVSR (Dong al., 2023), MIA-VSR (Zhou al., 2024b), IART (Xu et al., 2024) and VSRM (Tran et al., 2025)). Additionally, we implemented another methods, i.e., "Basic the backward propagation branch of VSR models, i.e., Basic VSR++ and VSRM, and reducing its model size for online VSR application. We use "P","F" and "N" to represent those with the previous support (supp.) frames, future support frames and no support frames.
and "VSRM** by removing
4.2 OVERALL PERFORMANCE
As shown in Table 1, the quantitative results demonstrate the superior performance of the proposed method over other online VSR models in terms of PSNR and SSIM. We also supplement the results of FDAN and KSNet models on Vid4 and Vimeo-90K-T datasets based on their released pre-traue trained models and source codes for comprehensive comparison. These results are reported in Table I with
8

---

Published as a conference paper at ICLR 2026
T
REDS4 011 011 (BI)
GT
FDAN
KSNNtt
TMP
TS Mamba
Vid4_City.0n _ity__y999 (BI)
GT
+S+++
FDAN
KSNet
TS-Mamba
Vimeo90K-T.00 BB)
GT
BSsicVSR++
FDAN
KSNet
TMP
TS-Mamba
Vid4 Foliage 025 (BD)
GT
BSs
FDAN
KSNet
TMP
TS-Mamba
Figure 4: Visual results on BI degradation (REDS4, Vid4) and BD degradation (Vimeo-90K-T, Vid4).
Table 2: Results of the ablation study REDS4 dataset.
Models
PSNR(dBy) SSIM Params.
Rus,(ms)
MACs(G))
(v1.1) w/o Trajectory
30.45 0,8678 30.70 / 0.8721
1.7
20
84
(v1.2) w/o Loyj
3.0
29
112
(y1.3) w/o IntraWCB
30.58 / 0.8702
2.8
25
97
(v1.4) w/o InterWCB
30.61 / 0.8706
2.8
25
97
(v1.5) w/o InraWCB-InterWC
30.52 / 0.8689
2.4
21
85
(v1.6) w/o ((1)
30.65 0.8710
3.0
27
112
(v1.7) w/o UL(3)/D(C
30.67 0.8714
3.0
27
112
(v1.8) w/o (v1.6) + (v1.7)
30.61 0.8702
3.0
25
111
TS-Mamba (ours)
30.73 / 0.8727
3.0
29
112
Figure 4 presents qualitative comparisons, from which better visual quality than other online VSR methods for both BI and BD degradations.
observe that method shows
Following (Fuoli et al., 2023; Zhang al., 2024b), VSR methods that process 720p (1280 720) videos in at least 24 in terms of FPS are recognized as real-time (R-T.) methods (Fuoli et al., 2023), and have labelled all the tested methods in Table according their runtime. It noted that TS-Mamba model achieves the second fastest inference speed among all online VSR methods. TMP the one with the fastest runtime as it was implemented with the CUDA accelerator (high MACs but low runtime) while M'mmaamba is not. Moreover, TS-Mamba also offers significant reduction in terms of MACs (about 36.3%) and marginal reduction in parameter numbers compared to TMP, as shown in Figure 1.
4.3 ABLATION STUDY
To further verify the effectiveness of our contributions, we have conducted ablation studies the REDS4 dataset with BI degradation.
We first confirmed the contribution of two trajectory-aware designs, i.e.. trajectory generation and trajectory-awart loss, by creating the following variants. (v1.1) w/o Trajectory G(-) and Token Selection module were removed from TS-Mamba; (v1.2) w/o Ltrj the trajectory-aware loss function

---

Published as a conference paper at ICLR 2026
3666
GT
w/o Trajectory
w/o Lirs
w/e IntraWCB w/o InterWCB
w/o IntraWCB
w/o (11)/((1)
w/o UL(3)/D((3) (v1.6)+(v1.7)
w/o
TS-Mamba (ours)
REDS4 400.045 (BI)
+ InterWCB
Figure 5: Visual results of the ablation study on REDS4 dataset.
Params.4 (M)
Run.. (ms)
MACs (G)
PSNR(dB)1/SSIMF
30.64/0.8712
2.8
25
96
REDS4_015
GT
FDAN
2
30.68/0.8720
2.9
27
104
3
30.73/0.8727
3.0
29
30.74/0.8727
3.1
TMP
VSRM*
TS-Mamba
Table 3: Ablation study of selected token number Figure 6: failure case on REDS4 dataset.
was removed when training the TS-M IS-Mamba model. We further verified our proposed TSMA module in terms of compensation branches and shift operations, by obtaining (v1.3) w/o IntraWCB and (v1.4) w/o Inter IntraWCB and InterWCB were removed from the TSMA module, respectively; (v1.5) w/o Intra WCB InterWCB both Int and InterWCB
removed from the TSMA module. We also tested the adopted shift operations in compensation branches, and implemented (v1.6) w/o (11) the U (L) shift operations were removed IntraWCB, (v1.7) w/o U (L) the (TL(3)/UUG) (3) shift operations were removed in InterWCB and (v1.8) w/o (v1.6)+(v1.7) all the shift operations were removed in TSMA module. As shown in Table 2, the performance of all these variants is evidently lower than that of the full TS- which fully confirms the effectiveness of each key component in our design. Moreover, we further provide the visual results of these variants on REDS4 dataset in Figure 5. It is found that the visual results demonstrates the contributions of our designs, particularly in realistic textures and fine details of the car.
To confirm the value of the token number in TS Mamba, TS-Mamba, and presented the results Table It noted that as s increases, the VSR performance improves, but with higher model complexity. When = it is diffieult to obviously improve VSR performance. To trade off between complexity and performance, we set s = 3 in this work.
tested different values with
LIMITATIONS
We investigate our results and find out the failure cases. A failure case when highly dynamic rotation occurs is visualized in Figure 6. The generated trajectories and compensated manner of our TS- Mamba are inaccurate enough when dynamic rotation occurs car tire and rotation information cannot be reconstructed, thus limiting the performance of our method. Due to the high difficulty of modeling rotation, other online VSR methods also fail obtain complete rotation information.
CONCLUSION
In this paper, we proposed Trajectory-aware Shifted SSMs (TS-Mamba) for online VSR, leveraging long-term trajectory modeling and low-complexity Mamba to achieve efficient spatio-temporal information aggregation. In TS-Mamba, trajectories a video are first constructed to select the most similar tokens from the previous frames. A trajectory-aware shifted Mamba aggregation module is then employed, which consists of shifted SSMs blocks to aggregate the selected tokens. The shifted SSMs blocks designed based on Hilbert scannings and shift operations to compensate for the scanning losses and strengthen the spatial continuity of Mamba. Extensive experiments on three widely used VSR benchmarks have demonstrated the effectiveness and efficiency of method.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the Key Research and Development Program of Peng Cheng Laboratory under Grant PCL2024A02, the China Scholarship Council, the University of Bristol, and the UKRI MyWorld Strength in Places Programme (SIPF00006/1).
REFERENCES
Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučié, and Cordelia Schmid. Vivit: A video vision transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6836-6846, 2021.
Yanpeng Cao, Chengcheng Wang, Changjun Song, Yongming Tang, and He Li. Real-time super- resolution system of 4kdv based on deep learning. In 2021 IEEE 32nd International Conference on Application-spetti specific Systems, Architectures and Processors, pp. 69-76. IEEE, 2021.
Kelvin CK Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Basievsr: The search for essential components in video super-resolution and beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4947-4956, 2021.
Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Basicvsr++: Improving video super-resolution with enhanced propagation and alignment. Proceedings of the IEEE/VV Conference on Computer Vision and Pattern Recognition, pp. 5972-599289 5921, 2022a.
Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Investigating tradeoffs in real-world video super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5962-59711 2022b.
Shuting Dong, Feng Lu, Zhe Wu, and Chun Yuan. Dfvsr: Directional frequency video super- resolution via asymmetric and enhancement alignment network. In International Joint Conference on Artificial Intelligence, pp. 681-689, 2023.
Dario Fuoli, Martin Danelljan, Radu Timofte, and Luc Van Gool. Fast online video super-resolution with deformable attention pyramid. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 1735-1744, 2023.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, arXiv preprint ar arXiv:2312.00072,, 2023.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396 2021.
Hang Guo, Jinmin Li, Tao Dai, Zhihao Ouyang, Xudong Ren, and Shu-Tao Xia. Mambair: A simple baseline for image restoration with state-space model. In Proceedings of the European Conference on Computer Vision, pp. 222-241. Springer, 2024.
Hang Guo, Yong Guo, Yaohua Zha, Yulun Zhang, Wenbo Li, Tao Dai, Shu-Tao Xia, and Yawei Li. Mambairv2: Attentive state space restoration. In Proceedingy of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 28124 28133,283,, 2025.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 2022.
Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova. Pingchuan Ma, Johannes Fischer, and Björ Ommer. Zigma: A dit-style zigzag mamba diffusion model. In Proceedings of the European Conference on Computer Vision, pp. 148-166. Springer, 2024.
Tao Huang, Xiaohuan Pei, Shan You, Fei Wang, Chen Qian, and Chang Xu. Localmamba: Visual state space model with windowed selective sean. In Proceedings of the European Conference on Computer Vision, pp. 12-22. Springer, 2024.
11

---

Published as a conference paper at ICLR 2026
Takashi Isobe, Fang Zhu, Xu Jia, and Shengjin Wang. Revisiting temporal modeling for video super-resolution. arXiv preprint arXiv:2008. 05765, 2020.
Yuxuan Jiang, Jakub Nawała, Chen Feng, Fan Zhang, Xiaoqing Zhu, Joel Sole, and David Bull. Rtsr: A real-time super-resolution model for avl compressed content. In IEEE International Symposium on Circuits and Systems, pp. 1-5. IEEE, 2025a.
Yuxuan Jiang, Siyue Teng, Qiang Zhu, Chen Feng, Chengxi Zeng, Fan Zhang, Shuyuan Zhu, Bing Zeng, and David Bull. Compressed video super-resolution based on hierarchical encoding, arXiv preprint arXiv:2506. 14381, 2025b.
Shuo Jin, Meiqin Liu, Chao Yao, Chunyu Lin, and Yao Zhao. Kernel dimension matters: To activate available kernels for real-time video super-resolution. In Proceedings of the 31 ACM International Conference on Multimedia, pp. 8617-8625, 2023.
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
Lingtong Kong, Chunhua Shen, and Jie Yang. Fastflownet: A lightweight network for fast optical flow estimation. In IEEE International Conference on Robotics and Automation, pp. 10310-10316. IEEE, 2021.
Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Transactions Pattern Analysis and Machine Intelligence, 41 41 1):599-22613, 2018.
Wenbo Li, Xin Tao, Taian Guo, Lu Qi, Jiangbo Lu, and Jiaya Jia. Mucan: Multi-corresondeence aggregation network for video super-resolution. In Proceedings of the European Conference on Computer Vision, pp. 335-351. Springer, 2020.
Xin Li, Kun Yuan, Bingchen Li, Fengbin Guan, Yizhen Shao, Zihao Yu, Xijun Wang, Yiting Lu, Wei Luo, Suhang Yao, al. Ntire 2025 challenge on short-form uge video quality assessment and enhancement: Methods and results. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1092-1103 2025.
Jing Lin, Xiaowan Hu, Yuanhao Cai, Haoqian Wang, Youliang Yan, Xueyi Zou, Yulun Zhang, and Luc Van Gool. Unsupervised flow-aligned sequence-to-sequeme learning for video restoration. In International Conference on Machine Learning, pp. 13394-13404. PMLR, 2022.
Ce Liu and Deqing Sun. On bayesian adaptive video super resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2):346 2013.
Chengxu Liu, Huan Yang, Jianlong Fu, and Xueming Qian. Learning trajectory-aweet transformer for video super-resolution. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 5687-5696. 2022a.
Meiqin Liu, Shuo Jin, Chao Yao, Chunyu Lin, and Yao Zhao. Temporal consistency learning of inter-frames for video super-resolution. IEEE Transactions on Circuits and Systems for Video Technology, 33(4):1507-1522 2022b.
Yong Liu, Jinshan Pan, Yinchuan Li, Qingji Dong, Chao Zhu, Yu Guo, and Fei Wang. Ultra- vsr: Achieving ultra-realistic video super-resolution with efficient one-step diffusion space. In Proceedings of the 33rd ACM International Conference on Multimedia, pp. 7785-7794, 2025.
Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, Jianbin Jiao, and Yunfan Liu. Vmamba: Visual state space model. Advances in Neural Information Processing Systems, 17:103331-0006 2024.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.0399 2016.
Xiandong Meng, Xuan Deng, Shuyuan Zhu, and Bing Zeng. Enhancing quality for compressed videos by jointly exploiting spatial details and temporal structure. In 2019 IEEE International Conference Image Processing (ICIP), pp. 1193-1197. IEEF, 2019.
12

---

Published as a conference paper at ICLR 2026
Xiandong Meng, Xuan Deng, Shuyuan Zhu, Xinfeng Zhang, and Bing Zeng. A robust quality enhancement method based on joint spatial-temporal priors for video coding. IEEE Transactions Circuits and Systems for Video Technology, 3166):400122414, 2020.
Seungjun Nah, Sungyong Baik, Seokil Hong. Gyeongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee. Ntie 2019 challenge on video deblurring and super-resolution Dataset and study. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1996-2005, 2019.
Long Peng, Xin Di, Zhanfeng Feng, Wenbo Li, Renjing Pei, Yang Wang. Xueyang Fu, Yang Cao, and Zheng-Jun Zha. Directing mamba to complex textures: An efficient texture-aware state space model for image restoration. arXiv preprint arXiv:2501. 16583, 2025.
Yanyuan Qiao, Zheng Yu, Longteng Guo, Sihan Chen, Zijia Zhao, Mingzhen Sun, Qi Wu, and Jing Liu. Vl-mamba: Exploring state space models for multimodal learning. arXiv preprint arXiv:2403. 13600, 2024.
Yajun Qiu, Qiang Zhu, Shuyuan Zhu, and Bing Zeng. Dual circle contrastive learning-baste based blind image super-resolution. IEEE Transactions on Circuits and Systems for Video Technology. 34(3): 1757-1771, 2023.
Bin Ren, Hang Guo, Lei Sun, Zongwei Wu, Radu Timofte, and Yawei Li. The tenth ntire 2025 efficient super-resolution challenge report. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 917-966, 2025.
Mehdi SM Sajjadi, Raviteja Vemulapalli, and Matthew Brown. Frame-recurrent video super- resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 6626-6666 6634, 2018
Yuan Shi, Bin Xia, Xiaoyu Jin, Xing Wang, Tianyu Zhao, Xin Xia, Xuefeng Xiao, and Wenming Yang. Vmambair: Visual state space model for image restoration. IEEE Transactions Circuits and Systems for Video Technology, 2025.
Jun Tang, Chenyan Lu, Zhengxue Liu, Jiale Li, Hang Dai, and Yong Ding. Ctvsr: Collaborative spatial-temporal transformer for video super-resolution. IEEE Transactions on Circuits and Systems for Video Technology, 34(6):5018 5032, 2023.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Proceedings of the European Conference on Computer Vision, pp. 402-419. Springer, 2020.
Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu. Tdan: Tenporally-deformabll alignment network for video super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3360-3369, 2020.
Dinh Phu Tran, Dao Duy Hung, and Dacyoung Kim. Vsrm: A robust mamba-based framework for video super-resolution, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1471 114221, 2025.
Xijun Wang, Xin Li, Bingehen Li, and Zhibo Chen. Liftvsr: Lifting image diffusion to video super- resolution via hybrid temporal modeling with only 4x rtx 4090s. arXiv preprint arXiv:2506.08559, 2025.
Xintao Wang, Kelvin CK Chan, Ke Yu, Chao Dong, and Chen Change Loy. Edvr: Video restoration with enhanced deformable convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1954-1963, 2019.
Bin Xia. Jingwen He, Yulun Zhang, Yitong Wang, Yapeng Tian, Wenming Yang, and Luc Van Gool. Structured sparsity learning for efficient video super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22638-22641 2023.
Zhuofan Xia, Xuran Pan, Shiji Song, Li Erran Li, and Gao Huang. Vision transformer with de- formable attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4794 4803, 2022.
13

---

Published as a conference paper at ICLR 2026
Jun Xiao, Qian Ye, Rui Zhao, Kin-Man Lam, and Kao Wan. Self-feature learning: An efficient deep lightweight network for image super-resolution. In Proceedings of the 29th ACM International Conference Multimedia, pp. 4408-44 4416, 2021.
Jun Xiao, Xinyang Jiang, Ningxin Zheng, Huan Yang, Yifan Yang, Yuqing Yang, Dongsheng Li, and Kin-Man Lam. Online video super-resolution with convolutional kernel bypass grafts. IEEE Transactions Multimedia, 25:8972-8999 8987, 2023.
Zeyu Xiao and Xinchao Wang. Event-based video super-resolution via state space models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12564-12574, 2025.
Kai Xu, Ziwei Yu, Xin Wang, Michael Bi Mi, and Angela Yao. Enhancing video super-resolution via implicit resampling-base alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1346-2555 2024.
Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with task-oriented flow. International Journal of Computer Vision, 127:1106-1125, 2019.
Chenhongyi Yang. Zehui Chen, Miguel Espinosa, Linus Ericsson, Zhenyu Wang, Jiaming Liu, and Elliot J Crowley. Plainmamba: Improving non-hierarchical mamba in visual recognition. arXiv preprint arXiv:2403.1769) 2024a.
Cuixin Yang, Jun Xiao, and Kin-Man Lam. Spatial-frequency fusion for arbitrary-scale ultra-high- definition image super-resolution. In International Workshop on Advanced Imaging Technology, volume 13164, pp. 443-448, SPIE, 2024b.
Xi Yang, Wangmeng Xiang, Hui Zeng, and Lei Zhang. Real-world video super-resolution: A benchmark dataset and decomposition based learning scheme. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4781-4790, 2021.
Xi Yang, Xindong Zhang, Lei Zhang, and Lei Zhang. Flow-guided deformable attention network for fast online video super-resolution. In IEEE International Conference on Image Processing, pp. 390-394. IEEE, 2023.
Xi Yang, Chenhang He, Jianqi Ma, and Lei Zhang. Motion-guided latent diffusion for temporally consistent real-world video super-resolution. In Proceedings of the European Conference on Computer Vision, pp. 224-242. Springer, 2024c.
Peng Yi, Zhongyuan Wang, Kui Jiang, Junjun Jiang, and Jiayi Ma. Progressive fusion video super- resolution network via exploiting non-local spatio-temporal correlations. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. -110--115,,, 2019.
Guozhen Zhang, Chuxnu Liu, Yutao Cui, Xiaotong Zhao, Kai Ma, and Limin Wang. Vfimamba: Video frame interpolation with state space models. Advances in Neural Information Processing Systems, 1111002225-072255 2024a.
Yuehan Zhang and Angela Yao. Realviformer: Investigating attention for real-world video super- resolution. In Proceedings of the European Conference on Computer Vision, pp. 412-41244 Springer, 2024.
Zhengqiang Zhang, Ruihuang Li, Shi Guo, Yang Cao, and Lei Zhang. Tmp: Temporal motion propagation for online video super-resolution. IEEE Transactions on Image Processing, 2024b.
Shangchen Zhou, Peiqing Yang, Jianyi Wang. Yihang Luo, and Chen Change Loy. Upscale-a-videe videoo: Cemporal-consisterr diffusion model for real-world video super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2535-2545, 2024a.
Xingyu Zhou, Leheng Zhang. Xiaorui Zhao, Keze Wang, Leida Li, and Shuhang Gu. Video super- resolution transformer with masked inter ceatrtinntrranann frame attention. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 25399-25408, 2024b.
14

---

Published as a conference paper at ICLR 2026
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. In International Conference Machine Learning, pp. 22429-62442 PMLR, 2024a.
Qiang Zhu, Haoyu Zhang, Shuyuan Zhu, Guanghui Liu, Bing Zeng, and Xiaozhen Zheng. Deep video super-resolution with flow-guided deformable alignment and sparsity-based temporal-spatial enhancement. In 2022 IEEE 24th International Workshop on Multimedia Signal Processing, pp- 1-6. IEEE, 2022.
Qiang Zhu, Pengfei Li, and Qianhui Li. Attention retractable frequency fusion transformer for image super resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshop, pp. 1756-1763, 2023.
Qiang Zhu, Feiyu Chen, Yu Liu, Shuyuan Zhu, and Bing Zeng. Deep compressed video super- resolution with guidance of coding priors. IEEE Transactions on Broadcasting, 70(2):505-515, 2024b.
Qiang Zhu, Feiyu Chen, Shuyuan Zhu, Yu Liu, Xue Zhou, Ruiqin Xiong, and Bing Zeng. Dvsrnet: Deep video super-resolution based on progressive deformable alignment and temporal-sparse enhancement. IEEE Transactions on Neural Networks and Learning Systems, 2024c.
Qiang Zhu, Jinhua Hao, Yukang Ding, Yu Liu, Qiao Mo, Ming Sun, Chao Zhou, and Shuyuan Zhu. Cpga: Coding priors-guided aggregation network for compressed video quality enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2964-2974, 2024d.
Qiang Zhu, Yajun Qiu, Yu Liu, Shuyuan Zhu, and Bing Zeng. Compressed video quality enhancement with temporal group alignment and fusion. JEEE Signal Processing Letters, 31:1565-1555 2024e.
Qiang Zhu, Yuxuan Jiang, Shuyuan Zhu, Fan Zhang, David Bull, and Bing Zeng. Blind video super- resolution based on implicit kernels. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2025.
Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. Deformable convnets v2: More deformable, better results. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 9308-9316, 2019.
Junhao Zhuang, Shi Guo, Xin Cai, Xiaohui Li, Yihao Liu, Chun Yuan, and Tianfan Xue. Flashvsr: To- wards real-time diffusion- streaming video super-resolution. arXiv preprint arXiv:2510. 12777777 2025.
15

---

Published as a conference paper at ICLR 2026
A MORE ANALYSIS FOR TSMA MODULE
In this section, we provide full analysis the evidence of our procedures in our TSMA module. Four Hilbert scannings (as shown in Figure 7) and ten shift operations (U(1), D(1), L(1), R(),, (11(1//.)). ((122333) are adopted to determine the suitable procedures for the best climination performance.
Scan-1
Scan 2
Scan-3
Scan-4
Figure 7: Four types of Hilbert scannings.
We first provide the procedures of previous methods (Zhang et al., 2024a; Xiao & Wang, 2025),
ie., multiple scannings with no shift operations, to prove the necessity of shift operations. Four procedures of previous methods under Scan-1 and their corresponding elimination values 8 are illustrated Figure
Scan-1-2 Scan-1; Scan-1-> Scan-2;
Scan-1-+ Scan-3: Scan-1-> Scan-4.
It can be found from Figure 8 that the discontinuity of Scan-1 cannot be eliminated when using Scan-1 as the second scanning. few of the intra-window discontinuity can be eliminated and the wnterwwindow discontinuity cannot be eliminated when using Scan-2/Scan-3/See second scanning. These results imply that using multiple scannings on local windows makes it hard eliminate the discontinuity of Hilbert scans.
the
Based on this problem, we introduce the shift operations under Hilbert scannings to enhance the discontinuity climination. To find suitable shift operations for specific scanning to achieve the best climination performance. We attempt some procedures under first scanning is Scan-1 with four different shift operations (U(1), D(1), L(1), R(1)) to determine the second scanning. These procedures are illustrated in Figure 9, We can find from Figure that:
Scan-1-t((1 -Scann-33) has the best elimination performance (5=18).
O0ar:11-D00-Gaan--1/caan-/// have the same elimination value (8=14).
have the same elimination value (6=14),
Scan
have the same elimination value (6=14).
4 have the same elimination value (6=14), From these results, can be inferred that the procedure (1): -itt--1-t(()))aaa
the most
suitable procedure.
Based the observation, extend the procedure (1) other first scannings construct the other three procedures:
2
3-Scaan---D(1)-Sccaa (4):caan- (uSSaa---RR(]-Scan--
These four procedures and their climination values 8 are illustrated in Figure 10. It can be inferred that these four procedures can obtain the best elimination performance. Besides, we also found that these procedures have obvious symmetry. Furthermore, the UI((/2/3) and U R(1 11/2/3) shift operations also have the same symmetry. The procedures the UL((1/2/3) and U(1/2/33) shift operations are illustrated in the Figure. (c) in our paper (also shown in Figure 11 in this file). Therefore, in our paper, we have determined to use the procedures (1) and (2) to construct our shifted SSMs blocks in the TSMA module. It is noted that although the second Hilbert scanning brings new discontinuous areas, these areas have already been overcome in the first scanning because the discontinuous areas do not overlap when using the Hilbert scannings.
16

---

Published as a conference paper at ICLR 2026
Windows(Scar (Scan-1)
Local Windows(Scan-1)
Local Windows(Scarm)
Local Vindows(Soan--2
Scan-1 8can-1(6-
Scan-1 - Scan-2(6
Local Windows(Scan
Local Windows(Scan-3)
Local Windows(Scan
Local Windows(Scan-4)
Scan-1 - 8can-3(8 = 4)
Scan-1 Scan-4(6
Figure 8: Four procedures of previous methods and the corresponding elimination values.
Local Windomm/taa
Window (111)
D(A).
Local Windmm/Bour 1)
Scan-1
Scan-1 D(1) Boan-1 /Boa/////caaseeeeee Scaan-
Local [lndown/Bonn-1)
Window
Perthiniigg(())
213),5
8can-1 + [(1) + Scan- 1iaae11//cee 1/Scan 2/ /Sean - -4
R(1) ///m/// /BBaa /Scan-6
Figure 9: Determining suitable procedures under first scanning Scan-1 and four shift operations.
B ADDITIONAL EXPERIMENTS
In this section, we conduct the additional experiments of ablation study and compared methods for comprehensive comparison.
B.1 ADDITIONAL ABLATION STUDIES
(I) Ablation Study of Different Deformable-bused Modules for Our TS-Mambu. We implement these alignments, i.e., deformable convolution network (DCN), flow-guided deformable alignment (FGDA), deformable attention (DA), and flow-guided deformable attention (FDA) to replace the TSMA module in our TS TS-Mamba for comparison. Their corresponding results are reported in Table It can be see that TSMA module achieves the best PSNR/SSIM performance and the lowest complexity, which demonstrates the effectiveness of our TSMA module.
(2) Ablation Study of Temporal Window Size. We set different temporal window size T from 3 to 23 to determine the optimal size temporal window and the corresponding results are provided in Table 5. Besides, previous works also the 7=15 as the fixed window size, in experiment,
17

---

Published as a conference paper at ICLR 2026
010,5-
Window Pari
D13
Beau-1 > D(1) Boaa-8
Boaz-3 D(L) Bcan-1
Wlodow (etltbolsgg((11
Lond Winlown(rear1 1)
Window RR)))
803,5 18
[(1)
Scan-4 - R(1) - Scan-2
Figure 10: Four procedures and corresponding elimination 8.
U 15
UL(2), =
UL(3). 14
Scan-1 + UL(1/2/3) - Scan-3
UR(I), 6- 15
UR(2),
UR(3),
Scan-1 UR(112/3)
Scan-3
Figure 11: Six procedures and elimination value 5 for Scan-1 UL012/33/URR/2/23 Scan-3.
for fair comparison, we also set our temporal window size as 15. From Table 5 see that when 7 is increase, the PSNR is increase, which indicates that longer temporal information help optimize the designed models and achieve the better VSR performance.
B.2 ADDITIONAL COMPARISONS
(I) Comparisons of Manba-based Methods. We first discuss the designs of MambalR (Shi al., 2025), MambalRv2 (Guo et al., 2025), TAMambalR (Peng et al., 2025), and VSRM (Tran et al., 2025) with our TS-Mamba. MambalR proposes residue state space block with local enhancement and channel redundancy reduction based on four directions scannings boost the restoration ability of Mamba. MambalRy2 designs an attentive state-space model with semantic-guided neighboring mechanism to encourage strong interaction between pixels under single direction scanning, which effectively eliminating the high complexity and redundancy of nulti-dirretional scans in existing methods. TAManbalR introduces texture-aware state space model that modulates the transition
18

---

Published as a conference paper at ICLR 2026
Table 4: Comparison with different alignments with our TS-Mamba.
Models
PSNR(dB)//SSMM
Params.(M)
(ms)
MACs(G))
DCN
30,59/0.8696
2,8
30
132
FGDA
30.64/0.8701
3.1
42
148
DA
30,67/0,87
3.0
33
145
FDA
30.69/0.8714
3.0
142
TSMA (ours)
30.73/0.8727
3.0
29
112
Table 5: Comparison of different temporal window sizes with our TS-Mamba.
Models
PSNR(dB)1/SSIMT
Params.(M)
(mn)(
MACs(G))
T
30.57/0.8698
2.4
23
96
T==
30.65/0.8713
2.7
25
103
T'
30.70/0.8722
2.8
107
T 15 (ours)
30.73/0.8727
3.0
29
112
7'= 19
30.73/0.8730
3.3
118
T 23
30.74/0.8731
3.7
33
123
matrix of Mamba with multi-directional perception blocks and focus on regions with complex textures to enhance texture awareness. In the VSR task, VSRM introduces spatial-to-tempraal Mamba and a temporal-to-spatial Mamba blocks based bidirectional scannings to extract long-range spatio-temporal features and enhance receptive fields efficiently. These Mamba-based methods usually use the multiply scannings or extra interaction module to enhance the ability of Mamba but they neglect the local spatial continuity of Mamba, which cause the limited SR performance. Our TS-Mamba analyzes the local spatial discontinuity and definites the degree of discontinuity and combines the Hilbert scannings and shift operations to obtaín strength the ability of Mamba. Notes that Hilbert scannings have the better spatial continuity than bidirectional scannings and cross scannings, and introduce of shift operations is a simple yet efficient way that strength the local spatial continuity of Mamba and without increasing the complexity, which is effectively help our TS-Mamba information aggregation.
achieves the high efficiency
Table 6: Comparison of Hamba-basede methods with our TS-Mamba on REDS4 dataset.
Category
Methods
PSNR(dBY//SSIMF
Params.(MJ)
MACs(G)
MambalR
32.25/0.9019
20.42
779.7
MambalRv2
32.48/0.9054
23.10
1567.2
Image SR
TAMambaIR
HambalR-light
26.89/0.8195
0.92
84.6
2ambalRv2-light
27.36/0.8389
0.79
75.6
VSRM
33.11/0.9162
17.1
2174
Video SR
VSRM
30.64/0.8700
3.1
136
TS-Mamba (ours)
30.73/0.8727
3.0
112
To evaluate these methods with our TS-Mamba, we conduct experiments of MambalR, Mam- baIRv2, and VSRM on REDS4 dataset with BI degradation and implement a transformed method, i.e., "VS""""*** by removing the backward propagation branch of VSRM and reducing its model size for online VSR application. Their results are provided in Table 6. It is found that Mamba-based VSR methods has better SR performance than Mamba-based ISR methods, which due to the temporal information lose impacts the restoration quality. Besides, VSRM has the best performance than other methods but its high complexity leads to suitable for online real-time applications. Compared with two Mamba- based online methods, i.e., VRSM* and TS-Mamba, our TS-Mamba achieves the high PSNR and low complexity, which due to the long-range spatio-temporal information aggregation
19

---

Published as a conference paper at ICLR 2026
Table Comparison of Mamba-based image SR methods.
Set5 PSNR/SSIM
BSDS100
Manga109 PSNR/SSIM
Params.
MACs
Methods
PSNR/SSIM
(M)
(G)L
MambaIR
38.57/0.9627
32.58/0.9048
40.28/0.9806
20.42
221.0
TAMambalR
38.58/0.9627
32.58/0.9048
40.35/0.9810
16.07
180.0
MambalRv2
38.65/0.9631
32.62/0.9053
40,42/0.9810
22.90
445.8
Table Comparison different VSR methods with our TS-Mamba on RealVSR dataset.
Methods
ILNIQE//NRMM1
Params.
Run.(ms)
RealVSR
34.39/3.795
2.7
772
BasicRealVSR
30.37/6.582
6.3
73
Real/iformer
28.61/6.588
5.3
49
VSRM
30.29/6.613
17.1
223
VSRM*
33.29/4.368
3.1
31
TS -S-Mamba (ours)
32.54/5.161
3.0
and shifted Mamba compensation, while VRSM* only use two previous frames to aggregate tem- poral information, limiting the exploration of long-range spatio-temporal information. Due to the TAMambalR method without releasing its source code, we cannot compared with other methods. To compare it with other methods much as possible, we collect the image SR results of three image SR methods from their papers as a reference and are provided in Table 7. It can be found that TAMambalR reduces the complexity compared with MambalR and MambalRy2, but did not exceed Ma in SR performance.
(2) Comparisons of Real-world VSR Methods. We conduct latest Mamba-based VSR model, i.e., VSRM, its variant VSRM* and our TS-Mamba model on RealVSR dataset for evaluation real- world scenarios. For further making a comprehensive comparison, we adopted three representative real-world VSR methods, i.e., RealVSR (Yang et al., 2021), BasicReal VSR (Chan et al., 2022b), RealViformer (Zhang Yao, 2024) on RealVSR dataset, the corresponding experimental results of compared methods and our method are provided in Table 8. It found that since no noise or unknown complex degradations were introduced during the training process, the general VSR results are not as effective the RealVSR methods. Compared to the RealVSR, our TS-Mamba achieves better VSR results while maintaining lower complexity, making it potential replacement for RealVSR.
(3) Comparisons of Recent VSR Methods. Recently, some methods advanced structures such as CNN (Meng et al., 2019; 2020; Zhu et al., 2022; Qiu et al., 2023; Zhu et al., 2024e; Yang et al., 2024b; Zhu et al., 2024d;b; Xiao et al., 2021; Zhu et al., 2025; Jiang et al., 2025b), Transformer (Liu et al., 2022a; Zhu et al., 2023; Zhou et al., 2024b; Ren et al., 2025; Li et al., 2025) and diffusion models (Zhou al., 2024a: Yang et al., 2024c), achieve the promising VSR performance. Here, we discuss some representative methods and compared them with our TS-Mamba.
Some diffusion-based VSR methods use diffusion for long-range information modeling achieve the superior generation performance. For example, LiftVSR (Wang et al., 2025) introduces hybrid temporal modeling mechanism that decomposes temporal learning into dynamic temporal attention (DTA) and attention memory cache (AMC). DTA for fine- grained temporal modeling within short frame segment and AMC for long-term temporal modeling across segments. UltrVSL (Liu et al., 2025) proposes a lightweight recurrent temporal shift module that by partially shifting feature components along the temporal dimension to enable effective propagation, fusion, and alignment across frames without explicit temporal layers. Additionally, it introduces a temporally asynchronous inference strategy to capture long-range temporal dependencies under limited memory constraints. Flash (Zhuang et al., 2025) proposes a diffusion-based one-step real-time VSR that consists train-frient frienff three stage distillation pipeline, a locality constrained sparse attention that adopted a KV-cache maintain the spatio-temporal consistency and preserve the high fidelity of videos.
20

---

Published as a conference paper at ICLR 2026
Table 9: Comparison of long-range modeling duffuusion-basee methods with TS-Mamba.
Methods
PSNRT
SSIM
PM))
[ms.(
LiftVSR
24.34
UItraVSR
24.50
0.6962
10.5
89
FSashVSR
24.11
0.6511
1780.14
15500
TS-Mamba (ours)
30.73
0.8727
3.1
29
Table 10: Comparison with different general VSR methods with our TS-Mamba on REDS4 dataset.
Category
Methods
PSNR(dB)//SSMM)
Params.(M)) (un,(ms) MACs(G))
CNN-based
DFVSR
32.76/0.9081
7.1
S2SVR
31.96/0.8988
13.4
194
3462
Transformer- based
MIA VSR
22.78/0.92220
16.5
318
3220
IA-RT
32,90/0.9 9138
13.4
180
5020
VSRM
33. 1/0.9162
223
2174
Mamba-based
VSRM*
30.64/0.8701
31
136
TS- Mamba (ours)
30.73/0.8727
3.0
29
112
We provide the results of these methods and our TS-Mamba REDS4 dataset in Table 9. It is found that our TS-Mamba model achieve the best PSNR and SSIM performance with a large margin while significant efficiency. These recent works the all long-range information in videos for current frame reconstruction which brings high complexity while our TS-Mamba selects the most similar token based on the trajectories to aggregate the long-range spatio-temporal information, which avoiding to all information processing so as to reduce model complexity for online VSR.
Besides of diffusion-based methods, some CNN-based and Transformer-baset methods also proposed in the VSR task. For example, A CONN-base method, i.e., DFVSR (Dong et al., 2023), proposed directional frequency representation and a directional rrqquency-enhancee alignment to represent the property of frequency of detail and direction information, and use double enhancements of task-related information to generate the high-quality feature. S2SVR (I.in al., 2022) proposes an sequence-to- sequence model with an unsupervised optical flow estimator to maximize its potential in capturing long-range dependencies among frames. With reliable optical flow, the accurate correspondence is established among multiple frames for improving the restoration performance. We added their results and two latest Transformer based VSR methods, i.e., MIA-VSR (Zhou et al., 2024b) and LA-RT (Xu et al., 2024) REDS4 dataset with BI degradation in Table 10. It is found that although S2SVR, DFVSR, MIA-VSR and IA-RT achieves the better VSR performance than out TS-Mamba model, but they have the high complexity and low inference speed, which cannot apply into the real-time online VSR processing.
These two VSR methods uses the consecutive one two frames achieve the temporal alignment or aggregation, while our TS-Mamba model utilizes the long-range spatio-temporal information based on trajectories in video from all the previous frames for spatio-temporal aggregation. Duo the trajectory- awware aggregation, our TS- Mamba can more easy to obtain the better VSR performance.
(4) Comparisons of Real-time SR Methods. Recently, some real-time SR methods were proposed. For comprehensive comparison with real-time SR methods, we compare these methods in quantitative or empirical comparison. EGVSR (Cao al., 2021) designed a lightweight CNN network structure based on spatio-temporal adversarial learning and efficient upsampling method to reduce the computation and guarantee the high visual quality. RTSR (Jiang et al., 2025a) utilizes a dual teacher knowledge distillation network for optimization of compressed content various quantization levels to achieve the www.oomllextty SR. Different from these two time methods, our TS-Mamba introduces the low-complexity and global receptive field Mamba with long-range temporal trajectories achieve the long- rangrrane apatio-tempora aggregation while these two models only use CNN network with local receptive field and neglect the long-rang temporal information, which limits they further improve SR
21

---

Published as a conference paper at ICLR 2026
Table 11: Comparison of real-time methods with TS-Mamba on Vid4 dataset.
Methods
PSNR(dB)//SSINN1 Para (mn.(m))
MACs(G)4
EGSVR
25.88 0.80
2.68
70
57.1
RTSR
25.59 0.75
0.06
1.07
TS- Mamba (ours)
27.17 / 0.82
3.00
29
112
2010540000003--1 (BI)
GT
FDAN
KSNet
TMP
Basic V TS-Mamba
REE FIN
REE FIN
REE FIN
RREE FIN
RREE FIN
REE FIN
Vid4 Calendal 011 (BI)
GT
VSR+
FDAN
KSNet
TMP
TS-Mamba
Vimeo90K-T 001 001 0812(BD)
GT
BS+
FDAN
KSNct
TMP
TS-Mamba
Yd4_Walk_026 (BD)
GT
BasicVSR++
FDAN
KSNet
TMP
TS-Mamba
Visual comparison results on BI degradation (REDS4, Vid4) and BD degradation (Vimeo- 90K-T, Vid4).
Figure 12:
GT
Basic- VSR++*
FDAN
KSNet
TMP
TS- Mamba
Rdd_Calenndrr (BI)
Figure 13: Temporal consistency comparison results BI degradation for Calendar video in Vid4 dataset.
performance. We provide the results of these two real-time methods with our TS-Mamba model on Vid4 dataset with BI degradation in Table II. Noted that RTSR achieves the lowest complexity and fastest inference speed but has the unsatisfactory SR performance.
22

---

Published as a conference paper at ICLR 2026
(5) Additional Visual Results. To provide comprehensive comparison, compare our TS- Mamba with four state-of-the online VSR methods, i.c., BasicVSR++* FDAN (Yang et al., 2023), KSNet (Jin al., 2023), and TMP (Zhang et al., 2024b) in visual results and temporal consistency of restored high-resolution videos.
We first provide Vimeo-90ks -OK-- (Xue et al., 2019) on BI and BD degradations. These results are illustrated Figure 12. be found from Figure 12 that our TS-Mamba model achieves better visual results than other online VSR methods.
visual results three test datasets, i.e., REDS (Nah al., 2019), Vid4, and
Moreover, further provide comparison of temporal consistency. The temporal profiles of five online VSR methods and our TS-Mamba model on Vid4 calendar video on BI degradation are illustrated in Figure 13. It can be found in Figure 13 that our TS- TS-Mamba model can also achieve better temporal consistency in restored videos.
23


Reviews:
[{"review_id": "YoywqqaWkw", "reviewer": "Reviewer_2xrU", "summary": "This paper proposes TS-Mamba, a state-of-the-art method for online video super-resolution (VSR), leveraging trajectory-aware shifted state space models (SSMs) for efficient spatio-temporal information aggregation. The model addresses challenges in long-term temporal modeling for real-time applications while keeping computational complexity low. The approach combines token-level spatio-temporal aggregation with a novel trajectory-aware shifted Mamba aggregation (TSMA) module. TS-Mamba constructs trajectories within video frames to select similar tokens from previous frames and aggregates them using shifted SSM blocks. This enables improved video frame restoration with reduced computational overhead. The proposed model is evaluated on several benchmark datasets (REDS, Vimeo-90K-T, Vid4) and outperforms five state-of-the-art online VSR methods in terms of PSNR/SSIM while reducing complexity by over 22.7%.", "questions": "See the weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Performance: The model achieves superior performance in terms of PSNR/SSIM and visual quality across multiple benchmark datasets (REDS, Vid4, Vimeo-90K-T) and degradation types (BI and BD), demonstrating its robustness in real-world video restoration scenarios.\n\n2. Computational Efficiency: TS-Mamba successfully reduces complexity by 22.7% in terms of MACs compared to existing methods, making it a strong candidate for real-time online VSR applications. The model is also one of the fastest among the tested methods.\n\n3. Comprehensive Ablation Study: The authors provide an extensive ablation study validating the importance of trajectory-aware components and the shifted SSM blocks, as well as the impact of different design choices (e.g., token number, shift operations). This strengthens the validity of their claims and helps illustrate the contributions of each part of the model.", "weaknesses": "1. Lack of Comparison. The proposed scheme fails to discuss or compare with several recent restoration schemes that leverage state-space models (SSMs, e.g. Mamba) for superior performance. For instance, MambaIR and MambaIRv2 introduced a residual Mamba-based backbone (with convolution and channel attention) to capture global dependencies in image super-resolution and denoising, outperforming a SwinIR Transformer baseline. The more recent TAMambaIR improves efficiency by modulating the state-space transition for complex textures and using multi-directional scanning, achieving state-of-the-art results across image restoration tasks (e.g. super-resolution, deraining, low-light enhancement). In the video domain, VSRM proposes dual Spatial-to-Temporal and Temporal-to-Spatial Mamba blocks for long-range spatio-temporal feature extraction and a deformable cross-Mamba alignment module for flexible frame alignment, yielding new state-of-the-art performance on VSR benchmarks. Likewise, The authors should incorporate and evaluate these methods to ensure a comprehensive comparison with the current state-of-the-art in video super-resolution. （I believe that the VSR method, when transformed into the online VSR setting, can be compared.）\n\n2. Insufficient Theoretical Justification: While the paper presents an innovative approach, the theoretical justification for the trajectory-aware shifted state space model is somewhat lacking. Specifically, a formal analysis of the long-term spatio-temporal aggregation process and a comparison with existing models using similar principles (e.g., in flow-guided deformable attention models) would provide more clarity. Additionally, the authors mention the use of shifted operations to enhance spatial continuity but could further explore the theoretical implications of this operation on model behavior.\n\n3. Insufficient Validation. A notable weakness is the absence of experiments on real-world VSR datasets. The method’s results are only reported on standard synthetic benchmarks (REDS4, Vid4, Vimeo-90K) with bicubic or simulated blur degradation, but no evaluations on real degraded videos were provided . As real applications involve unknown and complex degradations (compression artifacts, sensor noise, motion blur, etc.), the paper should have validated the approach on established real-world video SR benchmarks. For example, testing on datasets like RealVSR (ICCV 2021), which is built with a dual-camera system on the iPhone 11 Pro Max, would demonstrate the model’s robustness to in-the-wild conditions\n\n4. Potential on Certain Datasets: The model performs exceptionally well on the benchmarks but lacks a more nuanced discussion of failure cases or scenarios where the method might struggle. A brief mention of potential limitations (e.g., when frames are highly dynamic or occlusions are prevalent) would enhance the robustness of the paper's claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DgR6Yrd8UJ", "reviewer": "Reviewer_BFtp", "summary": "This paper focuses on online video super-resolution which aims to reconstruct a high-resolution video from its low-resolution counterpart using only past frames, i.e., without access to information from future frames. This research has high industrial values, especially for real-time and streaming applications. In this paper, authors propose a variant of Mamba models, called TS-Mamba, to model the long-term temporal information bewtween frames. Specifically, to effectively leverage similar features from previous frames, the proposed methods aims to select similar features along the trajectory. This token selection strategy keeps spatial consistency and avoids artifacts in reconstruction. The original Mamba models efficiently process 1D sequential data, but the Raster scan mechanism cannot handle image data, especially in boundary areas. To this end, TS-Mamba introduces a variant of Hilbert scan in the spatial domain, which is restricted within and between local windows. By incoorporating the shifting operator, the proposed model can effectively process image content from adjacent windows for restoration. To better supervise model during training, it additionally propsoes trajectory loss. The experiment results have demonstrated promising results in REDS4 and Vimeo-90K datasets with BI degradation and BD degradation. Overall, this is a good paper. Unlike previous methods, it introduce Mamba structure for online video super-resolution and has demonstrated a better trade-off between restoration quality and speed. However, I still have several suggestions, questions and confusions about this paper listed as below.", "questions": "I include the questions and concerns on this paper in Weakness section. Please authors prepare their rebuttal reference to the questions listed in Section 6.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper comprehensively discusses the related works in online video super-resolution and points out the existing limitations of the existing methods.\n2. TS-Mamba solely utilizes past frames for online video super-resolution and aggregates long-range information through trajectory-aware token selection, which motion paths across multiple previous frames.\n3. The proposed method is efficient without sacrificing quality, leading to a better trade-off between reconstruction quality and processing time.", "weaknesses": "1. what is $$v_{\\tau_{i}^{h_{j}}}? Are they math typos in Eq.(7) and Eq.(8)?\n2. The method proposes a trajectory-aware method and define a temporal trajectory among video frames in Eq.(3). However, I am confused on this definition. It is better to elaborate more on what positions among video frames belong to the same trajectory.\n3. The method introduces a dual-path block, i.e., intra-window compensation branch and inter-window compensation branch. What is the key difference between two blocks? Why one block can handle intra-window content and another one can hanld inter-window content?\n3. The method proposes variant scanning strategies. What scanning strategy it adopts in experiment? And why using this scanning strategy instead of other strategies?\n4. It is better elaborate on the descriptions of shifted SSMs block, especially for math notations. It seems that elimintation value is a hyper-parameter. Does it matter for the restoration performance?\n5. In the paper, it proposes trajectory-aware loss. How to compute this loss? Does it compute L2 loss between the trajectories in LR and the corresponding counter in HR images?\n6. It is better to demonstrate visual results associated with Table 2. However, due to page limited, it is still acceptable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OcqYdZk5P3", "reviewer": "Reviewer_zoqq", "summary": "This paper focuses on online video super-resolution (VSR) and proposes a new model called TS-Mamba, which is built upon State Space Models (SSMs). The method introduces long-term trajectory modeling combined with the lightweight Mamba framework to improve spatio-temporal information aggregation efficiency in video sequences.", "questions": "N/A", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The integration of Mamba-based spatial modeling with trajectory-aware design is conceptually clear and implemented in a well-structured manner.\n\nThe proposed Trajectory-aware Shifted Mamba Aggregation (TSMA) module is technically sound and engineering-wise elegant.\n\nThe model achieves reasonable complexity-efficiency trade-offs — it incorporates long-term temporal modeling while maintaining relatively low MACs and parameter counts compared to prior online VSR baselines.", "weaknesses": "1. Biased motivation.\n\nThe paper claims that “most existing VSR methods solely employ one neighboring previous frame”, yet this statement overlooks numerous recent works that already explore long-range temporal modeling. For instance:\n\n[1] Wang, Xijun, et al. LiftVSR: Lifting Image Diffusion to Video Super-Resolution via Hybrid Temporal Modeling with Only 4×RTX 4090s. arXiv:2506.08529 (2025).\n\n[2] Liu, Yong, et al. UltraVSR: Achieving Ultra-Realistic Video Super-Resolution with Efficient One-Step Diffusion Space. arXiv:2505.19958 (2025).\n\n[3] Zhuang, Junhao, et al. FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolution. arXiv:2510.12747 (2025).\nThese studies demonstrate substantial progress in long-term temporal modeling, undermining the paper’s claim that such research is scarce. This weakens the motivation and positioning of the work.\n\n2. Lack of analysis on real-time VSR methods.\n\nIn lines 109–117, the authors argue that optical-flow or deformable-convolution-based methods are computationally expensive. However, multiple studies have achieved real-time performance with efficient architectures, such as:\n\n[1] Cao, Yanpeng, et al. Real-Time Super-Resolution System of 4K-Video Based on Deep Learning. IEEE ASAP 2021.\n\n[2] Jiang, Y., Nawała, J., Feng, C., et al. RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content. IEEE ISCAS 2025.\nThe paper fails to include any quantitative or empirical comparison with these real-time approaches and instead relies on generic qualitative claims. This lack of evidence weakens its argument regarding computational superiority.\n\n3. Outdated comparison on standard benchmarks.\n\nThe experiments are conducted on REDS4, Vid4, and Vimeo-90K-T datasets, but do not include recent state-of-the-art models that have achieved higher PSNR and SSIM. For example:\n\nLin, Jing, et al. Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration. ICML 2022 — achieving 31.96 dB and 37.63 PSNR on REDS4 and Vimeo-90K-T, outperforming the proposed method.\n\nDong, Shuting, et al. DFVSR: Directional Frequency Video Super-Resolution via Asymmetric and Enhancement Alignment Network. IJCAI 2023.\nWithout comparisons to these recent methods, the experimental evaluation is incomplete and fails to convincingly establish the method’s competitiveness.\n\nLimited novelty in applying Mamba to VSR.\nThe use of Mamba-based State Space Models for video super-resolution is not new.\n\nEvent-based Video Super-Resolution via State Space Models (CVPR 2025) already introduced a similar Mamba framework for VSR.\nThe authors do not clarify how TS-Mamba fundamentally differs from or advances beyond these prior Mamba-based designs. Thus, the methodological contribution appears incremental rather than novel.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PfEY5vlUXr", "reviewer": "Reviewer_7jKz", "summary": "This paper proposes a trajectory-aware shifted state space model for online video super-resolution, named as TS-Mamba. TS-Mamba leverages trajectory to select relevant tokens from previous frames, shifted Hilbert scanning to compensate the spatial discontinuities, and a trajectory-aware loss to supervise trajectory generation. Experimental results show the competitive or superior of TS-Mamba compared with SOTA, with reduced complexity.", "questions": "(1) It is suggested to add a new set of experiments ablate the temporal window size. This is a low-effort, high-impact experiment that directly validates a core contribution of the paper. \n(2) The authors should revise the introduction and related work sections to include a concise yet specific \"Limitations of Prior Work\" paragraph that directly motivates the design choices of their own model.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The integration of temporal trajectory into Mamba for online VSR is original. \nThe trajectory mechanism for video is a well-motivated and reasonable way to capture the longer temporal dependencies while maintaining low complexity. That is suitable for online video-related tasks. \nNovel use of state space models (SSMs) with elaborately designed shifted operations to compensate for Hilbert scanning discontinuities，and sufficient illustrations enhance understanding and readability. \nThis paper is a well-organized and well-presented, with ablations and detailed training settings. The illustration of the SSMs scan is impressive and clear.", "weaknesses": "Author proposes a trajectory-aware shifted Mamba aggregation module to achieve the long-term spatio-temporal information aggregation. However, the existing trajectory/temporal alignment methods (e.g., deformable alignment, flow-guided deformable alignment) also can achieve this goal, a more detailed comparison with these methods would strengthen this part. \n\nThe experiments only use standard window sizes, without specific analysis on long-term dependency (e.g., varying temporal window T). A comparison between using the fixed temporal window (i.e., T=15) and other temporal window sizes would help determine the optimal size of temporal window of trajectory modeling for the proposed method.\n\nThe author did not thoroughly analyze the shortcomings of existing methods, and the description is also quite vague. It is suggested that the author conduct a more in-depth analysis of the shortcomings of existing methods to further highlight the advantages of the method proposed in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)