WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool
Authors: Zizun Li
Jianjun Zhou
Yifan Wang
Haoyu Guo
Wenzheng Chang
Yang Zhou
Haoyi Zhu
Junyi Chen
Chunhua Shen
Tong He
Venue: ICLR 2026 Poster (2026)
Keywords: computer vision
3D reconstruction
machine learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: WinT3R: a feed-forward reconstruction model capable of online prediction of precise camera poses and high-quality point maps.

Abstract: We present WinT3R, a feed-forward reconstruction model capable of online prediction of precise camera poses and high-quality point maps.
Previous methods suffer from a trade-off between reconstruction quality and real-time performance.
To address this, we first introduce a sliding window mechanism that ensures sufficient information exchange among frames within the window, thereby improving the quality of geometric predictions without introducing a large amount of extra computation.
In addition, we leverage a compact representation of cameras and maintain a global camera token pool, which enhances the reliability of camera pose estimation without sacrificing efficiency.
These designs enable WinT3R to achieve state-of-the-art performance in terms of online reconstruction quality, camera pose estimation, and reconstruction speed, as validated by extensive experiments on diverse datasets.

Published as a conference paper at ICLR 2026
WINT3R: WINDOW-BASED STREAMING RECON- STRUCTION WITH CAMERA TOKEN POOL
Zizun Li!.2 Jianjun Zhou 2.34 Yifan Wang2.
Haoyu Guo
Wenzheng Chang266
Yang 2.o Haoyi Zhu Junyi Chen 2.0
Chunhua Shen'
Tong He2.3.
Uniiversity Science and Technology of China
2Shanghai Al Lab
Shanghai Innovation Institute
*Zhejiang University FFudan University
Shanghai Jiao Tong University *Corresponding Author
Global
Point Map
Camera Token Pool
Camera Token Pool
Camera Tuken Pool
Window
Point Map
T3R
W3R
WinT3R
Streaming (17 FPS)
Figure I: Overview. Given image stream, our method Win processes input images in sliding-window manner, where adjacent windows overlap by half of the window size. Unlike pre- vious online reconstruction methods, our model generates extremely compact camera tokens during online reconstruction to serve as global information for historical frames. This enables the recon- structions of subsequent windows leverage these global cues for more accurate results. Our model achieves high-quality geometry reconstruction while maintaining real-time performance at 17 FPS.
ABSTRACT
We present WinT3R, feed-forward reconstruction model capable of online pre- diction of precise camera poses and high-quality point maps. Previous methods suffer from trade-off between reconstruction quality and real-time performance. To address this, we first introduce sliding window mechanism that ensures suf- ficient information exchange among frames within the window, thereby improv- ing the quality of geometric predictions without introducing a large amount of extra computation. In addition, we leverage a compact representation of cam- eras and maintain a global camera token pool, which enhances the reliability camera pose estimation without sacrificing efficiency. These designs enable WinT3R to achieve state-of-the-art performance in terms of online reconstruction quality, camera pose estimation, and reconstruction speed, as validated by exten- sive experiments diverse datasets. Code and models are publicly available at https://github.comm/ LiZizun/Wini 33R

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Real-time reconstruction of 3D) geometry from image streams is a fundamental problem with nu- merous practical applications. This task requires incrementally integrating newly arrived frames into existing reconstructions within a unified coordinate system at high speed. A typical approach
involves traditional SLAM methods (Mur-Artal al.) 2015 Davison al. 2007 Engel al. 2014 Forster et al., 2016 Teed & Deng, 2021). which first extract features for tracking, then perform Bundle Adjustment (BA) to jointly refine camera poses and sparse 3D structures, and finally employ
loop-closure detection to mitigate accumulated drift. While they achieve real-time localization and sparse mapping, they are not suitable for online dense reconstruction.
With the rapid advances in deep learning, some recent approaches demonstrate promising recon- struction capabilities, yet they face a trade-off between reconstruction quality and real-time perfor- mance. Specifically, offline methods (Wang et al. 2025a d: Zhang al. 2025 Yang et al., 2025) achieve high-quality reconstruction by performing full attention across image tokens all frames. They fail to achieve real-time time performance and cannot flexibly incorporate
frames into existing
reconstruction results. In contrast, online methods (Liu et al. 2025 Wang & Agapito [024] Chen al. 2025b Wu et al. 2025 Zhuo al. 2025 Team et al. 2025) like CUT3R (Wang al. (025h) achieve real-time reconstruction in a streaming manner by enabling image tokens from each new frame to interact with the state tokens. However, due the lack of direct and sufficient interaction between image tokens of adjacent frames, the reconstruction quality remains suboptimal compared with offline methods.
To overcome these challenges, we propose WinT3R, real-time and high-quality 3D reconstruction method based on a sliding-windde window strategy and camera-token pool mechanism. Our design is motivated by two key observations. First, adjacent frames typically exhibit strong correlations, thus, the quality of geometric predictions can be improved if the image tokens can directly interact with those from neighboring frames. Second, camera tokens can be represented much more compactly than image tokens, which enables direct interaction with all historical frames without compromis- ing real-time performance, thereby yielding more reliable camera pose estimation with global perspective.
Based on these observations, first propose online sliding- winddww mechanism that processes input image streams in real time. Within this design, image tokens interact not only with the state tokens but also directly with other image tokens within the same window. Moreover, compact camera token for each frame and store them in an expandable pool. When estimating the camera parameters for newly arrived frames, the model leverages all historical camera tokens in the pool, thus achieving more accurate estimates within real-time computational constraints.
maintain
We train our model using variety of public datasets (Baruch et al. 202 I Dai al. 2017 Li & Snavely 2018 Li et al. 2023) Reizenstein et al., [021]] Roberts et al.] 2021 Wang et al., 2020 Yeshwanth et al., [023] Xia et al. [2224] Yao et al. (0200) and our private synthetic datasets. Ex- periments demonstrate that our model effectively mitigates the aforementioned issues and processes input image streams in real time at over 17 FPS while accurately predicting camera poses and point maps, thereby achieving state-of-the-an performance in online reconstruction tasks.
Our main contributions are summarized as follows:
1. We propose online window mechanism, enabling sufficient interaction of image tokens within the same window and across adjacent windows.
2. We maintain a camera token pool, which functions as a lightweight "global memory" and improves the quality of camera pose prediction with global perspective.
3. Experiments demonstrate that WinT3R achieves state-of-the oftte-----h--rr performance online 3D reconstruction and camera pose estimation, with the fastest reconstruction speed date.
RELATED WORK
Structure from Motion (SfM) aims to jointly reconstruct 3D scene structures and camera poses from multi-view images (He et al. 2024 Zhang [997] Wang et al. 2024a| Agarwal et al.) 201T). This task poses severe challenges due to the scale and complexity of real-world scenes. Traditional
2

---

Published as a conference paper at ICLR 2026
approaches are categorized as incremental methods (Snavely) 2008 Schonberger & Frahm 2016
Snavely et al.] 2006; Wu et al. 2011), which progressively align images via iterative bundle ad-
justment (Hartley 2003) but sufter from emror accumulation: global methods (Govindu 2004: Arie-
Nachimson et al. 2012 Crandall et al. 2012), which directly optimizes global camera poses but remains sensitive to erroneous pairwise constraints; and hybrid methods (Cui et al. 2017 Moulon cct al. 2013) that combine both paradigms to improve scalability. Recent advancements integrate
deep learning to enhance robustness: Learned features (DeTone et al. 2018 Sun et al. 2021 and
matchers (Sarlin et al. 2020, Lindenberger et al. 2023] Li et al. 2025) improve correspondence
reliability, while differentiable optimization frameworks (Tang & Tan| [008]] Brachmann & Rother
2021 enable end-to-end trainable pipelines. Despite progress, challenges remain in dynamic scenes, textureless regions, and the generalizability of learning- methods beyond synthetic data.
Multi-view Stereo (MVS) methods (Furukawa & Ponce, 2009
Camphell et al. 2008) predomi-
nantly adopt a depth-map fusion paradigm, where depth maps are estimated per view and merged into unified 3D reconstruction. Early approaches (Liu et al. 2009; Wang et al. 2021 iteratively propagate depth hypotheses via randomized initialization and cost aggregation. While efficient, these methods struggle with textureless regions and occlusions due to reliance handerafted simi- larity metrics. The advent of deep learning catalyzed significant advancements: MVSNet (Yao et al.] 2018) pioneered cost-volume construction via differentiable homography warping and 3D CNN reg- ularization, establishing an end-to-end trainable framework. Recently, direct RGB-to-3D methods like DUSt3R (Wang et al., (024b) and MASt3R (Leroy et al. 2024) estimate point clouds from pair of views, but they require additional global alignment process to handle multi-view tasks. Of- fline methods like VGGT (Wang al. 2025a FLARE (Zhang et al. 2025) and 3 (Wang et al., 2025d) move step forward DUSt3R (Wang et al., (024bb) to operate on multi-view images, but they cannot dynamically add new estimations to previous results.
Online Reconstruction Methods encompass simultancous localization and mapping (SLAM)
(Zhang & Singh| 2015 Shan et al. 202T| Engel et 2014 //hu et al. (022)) and dynamic scene re- construction Yu et al. 2018: Bescos et al. 2018). Monocular SLAMM systems estimate ego-motion
and 3D structure in real time from video, but they generally assume known camera intrinsics. Re-
cent learning-basse methods (Civera et al. 2008 Tateno et al. 2017 Yang & Scherer 2019) Team jet al. 2025: Chen et al.) 2025a) have bridged scalability and flexibility. MASBR-SLAN (Murai
ett al. 2025) exploits dense diel-view 3D reconstruction prior (building DUS&3R (Wang al.,
2024byMASt3R (Leroy et (024)) for real-time monocular SLAM. It models scenes with generic camera geometry, unifying pose estimation, dynamic point-cloud fusion, and loop closure. Innova- tions like CUT3R (Wang et al. (0255) and Spann3R (Wang & Agapito (024) enabled feed-forward reconstruction from video sequences. Fully depending on memory or state tokens, these methods suffer from severe geometric distortions. In contrast, our compact representation of camera tokens and local point maps alleviates this problem, yielding superior reconstruction quality.
METHOD
Given stream of input images, WinT3R predicts local point map and camera pose for each frame in real-time, as illustrated in Figure 2 We first propose an online window mechanism to process images in a sliding window manner, facilitating information exchange within the window and enriching image tokens with state tokens (Section 3.1). Next, we predict the local point map for each frame through a lightweight convolutional head and estimate the camera pose for each frame based on a camera token pool (Section 3.2) b.2). Finally, we describe our training objectives (Section 3.3
3.1 ONLINE WINDOW MECHANISM
The input is a stream of (T) of RGB images I, E R3x observing the 3D scene. For each coming image I, we first send it to a ViT encoder to obtain the image token F, E RNCC.
(I) Inspired by CUT3R (Wang et al. 2025b). we maintain a set of state tokens S for the scene, which allow image tokens to read contextual information and simultancously update these state tokens. However, in CUT3R, information between frames can only be shared indirectly through these state tokens. To leverage the strong correlation among adjacent frames, we introduce sliding window
En.oder(I.).
3

---

Published as a conference paper at ICLR 2026
Carrent Canen Tokes
Histeried Camara Tiken
Image Token
Canacia Takais
Previous State
Hoal
Window
Cimza kokmn Pood
VIT
Enceder
Branch Decoder
Point Local Point Maps
Head
New
Global Point Map
WinT3R pipeline. We detail the reconstruction process within single window. All images are first passed through frame-wise VIT encoder, which outputs image tokens. Camera tokens are then appended to these tokens. Then the tokens within this window are collectively fed into decoder to interact with state tokens. Finally, the image tokens output by the decoder sent to lightweight convolutional head to predict local point maps. Meanwhile, the camera tokens, along with those in the camera token pool, are jointly fed into camera head predict camera parameters, while these camera tokens are simultaneously added to the camera token pool.
Figure 2:
mechanism to facilitate more direct cross-frame communication between image tokens and state to- kens, thereby enhancing prediction quality. Specifically, for the input image stream, we set a sliding window of size During each interaction step, enable comprchensive information exchange across frames, all image tokens in the current window are used as input.
lg F.".eeww Ig'. Flkew,, S = Decoders([g: F
(2)
where Wi denotes the current window, and 9: denotes the learnable camera token prepended to the image tokens F., which is used for camera pose prediction. The decoder is equipped with two branches interconnected with each other. One branch inputs image tokens and camera tokens, which is designed perform Alternating-Attenne Attenntoon VGGT (Wang et al., 2025a) and outputs both global (gf and Fi)) and local (g) and F enriched tokens for these frames. The other branch inputs state tokens S and outputs updated tokens S, which have exchanged information with the image tokens within the window W, Specifically, we initialize the state tokens as set of learnable tokens the beginning of the reconstruction process.
With this design, the image tokens can not only read contextual information from the state tokens, also interact directly with other tokens the current window. Furthermore, to enhance continuity between adjacent windows, we set the sliding window stride to w w/2, ensuring neighboring windows share half their frames. This design allows predictions for the overlapping region to be updated based subsequent w/2 frames.
To balance the real-time requirements online processing and the reconstruction performance the model, we select window size of 4 and a stride of 2 in our implementation. During the inference process, we check if the window is full. images arrive until the window reaches the full size. For the last image, we duplicate it to fill the remaining window slots. Regarding the overlapping region between the initial prediction and the updated prediction, select the camera pose from the updated prediction and the point map with the higher confidence score as the final output.
not, current image tokens will wait for subsequent
3.2 POINT MAP AND CAMERA PREDICTION
Based on the enriched image and camera tokens, we predict the point map P; and camera pose è; for each frame. The point map of each frame is defined in its own local camera coordinate system, which
4

---

Published as a conference paper at ICLR 2026
Key
Key
visible
Quars
maksed
(a) Fully visible
(b) Causal
(c) Sliding Window
Figure 3: Attention mask. (a) Full attention, all imput tokens are covisible. (b) Causal attention,
each token can only see itself and the tokens before it in the sequence. (c) Sliding window attention, each token can only see tokens in current window and the tokens in history windows.
mainly contains local geometric information, so we consider the prediction relies primarily on local cues. Since the image tokens F: have already captured sufficient contextual information through interactions with the state tokens S and other image tokens within the window, we directly feed them into the point map head to predict the local point map P; and its corresponding confidence C, To optimize efficiency and quality, we avoid the computationally expensive DPT head and the linear head which introduces grid-like artifacts, opting instead for lightweight convolutional head:
P,, C; = ConvHead F.).
(3)
In contrast, the camera pose represents the position and orientation of each frame within the entire 3D scene. Therefore, predicting the camera pose requires a more comprehensive utilization of global information to achieve reliable results. To this end, store all historical camera tokens in a pool and leverage all of them when predicting the camera pose for each incoming frame. Furthermore, make camera tokens more expressive, concatenate the local camera token g and the global camera token oe along the channel dimension to form the final camera token g:-
gi = ChannelCat(g/. g."),
(4)
Pool cam Pool L [g]]ee,,
(5)
(alsew, = CameraFFcad((g/leews Pool
(6)
Here the camera parameters é R is the concatenation of rotation quaternion q R4 and trans- lation t E R33 indicates adding new calculated camera tokens to the pool.
For each frame, vector in our implementation. The number of such camera tokens is significantly fewer than the number of image tokens, ensuring the real-time performance of our system. Considering that the output of the camera parameter é; is only a 7-dimensional vector, which is of significantly lower- dimensional than the point map P. 3x3 this compact token design does not compromise prediction accuracy. Compared with other methods like caching memory tokens that require storing all keys and values for every attention layer, our approach drastically reduces storage overhead and computational cost.
model outputs only single camera token g which is 1536-dimensional
To better leverage these compact camera tokens, we design camera head with sliding window masked attention that matches the decoder's architecture. Our attention mask illustrated in Fig- ure (c). This attention mask enables the model to predict camera tokens of current window condi- tion on all previous windows, without being affected by subsequent windows at training stage.
3.3 TRAINING OBJECTIVE
We train our model end-to-end using camera pose loss and point map loss:
Liotal Leamera Lpmap-
5

---

Published as a conference paper at ICLR 2026
We normalize the prediction and ground truth respectively. Specifically, we first calculate the norm factors as the averaged point map scale weighted by confidence:
Σi= EjEM, P.
norm([Pil (C
Ei=I logCi,
Then we normalize both the predicted and the ground-trute camera translations and point maps using the norm factors. The local point map loss includes confidence-awee -wwrre regression term MAS13R (Murai et al. 2025):
T ppmpp Σ ;=1 JEM,
C
C Pagan (j. alogCi,j
where M, denotes the valid pixel mask. We apply l loss for the point map regression term Tegr Following x² WWang et al., 2025d), we supervise the relative camera pose, avoiding manually defin- ing coordinate system. The network adaptively predicts camera poses in learned coordinate frame. Consequently, we employ relative camera pose loss, supervising the pairwise relative poses for all frames rather than the absolute pose of each frame. The pairwise relative camera parameters Cjj from view i to j for the predicted and the ground truth are the concatenation of relative rotation quaternion q: E I+ and relative translation tij-
R
4i = 93 2 q:.
(10)
Li rotate(t, tj 4j),
(11)
where 9; is the conjugate of qj and & denotes quaternion multiplication, rotate(t, q) applies the rotation represented by quaternion q to translation Our camera pose loss compares the predicted relative camera parameters éi with the ground truth Cij using & Loss:
1
Leamera N(N 1) ins
Ea(a cty)-
(12)
In our implementation, we found that the supervision from both the é based camera loss and point map loss is equally critical, so we simply add them to form the final loss.
4
EXPERIMENTS
4.1
TRAINING DATASETS
We train model using large collection of datasets, including: GTASfm (Wang & Shen 2020
WiLdRGBD (Xia et al. 2024). CO3Dv2 (Reizenstein et al. (021), ARKitScenes (Baruch et al.
2021
TartanAir (Wang et 2020 Scannet (Dai al. 2017). Scannet++ (Yeshwanth et al,
2023
BlendedMVG Yao et al. 2020). MatrixCity (Li et al.] 2023). Taskonomy (Zamir et al.
2018
MegaDepth (Li Snavely 2018), Hypersim (Roberts al. (202I), and a synthetic dataset of
video games. Our datasets cover wide range of scenarios, such as object level and scene level. real- world data and synthetic data, video sequences and multiview images. We employ three sampling strategies: random sampling, interval sampling, and overlap view sampling.
4.2 IMPLEMENTATION DETAILS
Our model is initialized with pretrained weights of DUSt3R (Wang et al. 2024b) and trained using AdamW (Loshchilov & Hutter 2019) optimizer. The full model has 750 million parameters. We train our model in two stages. In the first stage, we train the model with 12-frame data for 100 epochs, setting the maximum learning rate to le-4 and using a batch size of 4 per GPU. This stage is conducted on 64 NVIDIA A800 GPUs and takes 7 days. In the second stage, we fine-tune the model using 60-frame data for 12 epochs, with maximum learning rate of 2e-6, completing in 4 days 32 A800 GPUs. All imput images during training have variable aspect ratios, with the longest edge fixed at 512 pixels.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative 3D reconstruction results on DTU and ETH3D datasets.
DTU
ETH3D
Method
Type
Acc.
Compl
Overall
Acc.
Comp.
Overall
Fast3R Yang al.
2025
Offinee
3.083
2.329
2.706
0.638
0.738
0.688
FLARE Zhang al.
21755
Offiine
2.077
1.982
2.030
0.522
0.542
0.530
VGGT
Wang
Offiine
1.140
1,439
1,289
0.186
0.144
0.165
Spann3R Wang Aganito
Online
3.554
1,546
1.139
SLAM3K Cill
201
Online
6.672
5.256
5,964
0.626
0.888
0,757
CUT3R (Wang el al
21725
Online
4,454
1.944
3.199
0.513
0.503
0.518
Point3
Online
4.887
1.688
3.288
0.662
0.579
0.621
Stream Zhoo & a.
2025
Offinnc
3,997
1.651
2,823
0.581
0.359
0.470
Online
1.838
0.272
Table Quantitative 3D reconstruction results on 7-Scenes and NRGBD datasets.
7-Scenes
NRGED
Method
Type
Acc|
Compl
Overall
Acc
Comp.
Overall
Fast3R Yang al
2025
Offiine
0.040
0.059
0.049
0.074
0.052
0.063
FLARE
Ziann 0.
2U25
Otflnne
0.019
0.026
0.022
0.022
0.018
0.020
Wang
Ofline
0.026
Spenn3R Wann & Agapito
2024
Online
0.054
0,044
0.049
0.134
0.078
0.106
SLAM3 LLu
1125
Online
0.069
0,060
0.064
0.130
0,082
0.106
CUT3R
WWang ef al:
202.15
Online
0.023
0.027
0.025
0.086
0.048
0.067
PoimBk
Online
0,026
StreaMGGr
Zhuu al.
2025
Online
0.047
0,030
0.038
0.096
0.049
0.074
Online
Table 3: Camera Pose Estimation on Tanks and Temples, CO3Dv2 and 7-Scenes datasets.
Tanks and Temples
CO3Dv2
7-Scenes
Method
Type: RRA@300
RTA
301C8:00
RRA@:00
RTA
AUCD
RRAP30T
RTA#
AUCO
Far33
Yong
71.69
50.18
82.18
FLARR
Ziang df
MBS
Offine 85.17
87.62
70.97
96.35
93.52
73.79
100.0
95.68
75.90
VOGT
Wing al
N225
Offline 93.83
95.72
91.17
98.98
89.89
100.0
97.36
Sponn RR
Wine & Apapin 2024
Online 65.52
68.54
40.78
93.81
89.95
70.41
99.98
95.10
72.60
Online 92.35
91.86
76.22
96.33
92.67
75.94
100.0
95.36
74.49
Point3
Wa 2025
Online 74.64
79.27
42.63
95.51
91.21
67.99
100.0
94.13
66.81
StramVGGT
Online
92.81
Online 94.53
94.35
81.34
98.66
95.90
84.61
100,0
97.40
78.59
4.3 3D RECONSTRUCTION
Following the evaluation protocol of VGGT Wang et al. 2025a we evaluate 3D reconstruction quality on object-centric DTU (Jensen et al.] 2014) and scene level ETH3D (Schops et al.) (017)) datasets, reporting Accuracy, Completeness, and Overall (Chamfer distance) for point map estima- tion as VGGT. We sample keyframes every 2 images and align the predicted point maps and the ground truth using the Umeyama (Umeyama) 2002) algorithm. further evaluate our method on scene-level 7-Scenes (Shotton et al. 2013) and NRGBD (Azinović et al. 2022) datasets, with stride of 40 (7-Scenes) or 100 (NRGBD). We compare our method with other online reconstruction methods and offline reconstruction methods, shown in Table 1 2 and Figure 4
method demonstrates state-of-the-an performance among online methods across broad spectrum of 3D re- construction tasks, encompassing both real-world and synthetic data, both object-level and scene- level.
4.4 CAMERA POSE ESTIMATION
For the camera pose estimation task, to ensure fair comparisons, we selected Tanks and Temples (Knapitsch et al. 2017), CO3Dv2 (Reizenstein et al. 2021), and 7-Scenes (Shotton al. 2013) datasets for evaluation. All evaluated models have either been trained on these datasets or not at all. These datasets encompass both object-level and scene-level contexts, and synthetic data. For Tanks and Temples, we select 30 frames per scene with a stride of 10; for CO3Dv2, randomly sample 10 frames per scene; for 7-Scenss we sample frames with
well real-world

---

Published as a conference paper at ICLR 2026
Input images
Spann3R
CUT3R
Stream VGGT
Ours
Ground Truth
Figure 4: Qualitative comparison of 3D reconstruction. Compared with other online methods, WinT3R achieves higher reconstruction accuracy while also enabling faster reconstruction speed.
Input Images
Spann3R
CUT3R
Stream VGGT
Ours
Figure 5: Qualitative comparison of wn-the-wild multi-view 3D reconstruction. We demonstrate reconstruction results on wild sequences across indoor, outdoor, and object-level scenes. Our method consistently achieves the most photorealistic reconstruction results.
stride of 40. We evaluate them using Relative Rotation Accuracy (RRA) and Relative Translation Accuracy (RTA) at a given threshold (e.g., RRA @ 30 for 30 degrees), and AUC@ 30 which serves as unified evaluation metric, defined as the area under the accuracy-thresholdd curve for the minimum of RRA and RTA across varying thresholds. The results in Table 3 show that our model delivers state-of-the-ar performance among online methods.
threshold
8

---

better perfor- Published as a conference paper at ICLR 2026
Table 4: Video Depth Estimation on Sintel, BONN and KITTI datasets.
Sintel
BONN
KITTI
Method
Type
Abs Rell
6 <11.25
Abs Rell
8 <1.25
Abs Rell
8 <1.255
FPS1
Fast3R Yang al.
2025
Offlnee
0.641
0.423
0.193
0.774
0.136
0.834
30.6
FLARE Zhang al.
2125
Offline
0.729
0.336
0.152
0.790
0.356
0.570
VGGT
Wang
Offline
0.292
0.682
0.055
0.971
0.073
0.963
46.6
Spann3R Wang Apanito
Online
0.597
0,953
0.251
CUT3K Whing m
20240
Online
0.417
0.507
0.078
0.937
0.122
0.876
12.9
Point3 Wi al AFS
Online
0.461
0.455
0.060
0.962
0.137
0.839
StreamVGGT 72hoo
2025
Online
0.343
0.604
0.057
0,974
0.185
0,700
13.7
Ours
Online
0.374
0,506
0.070
0.912
0.081
0.949
17.2
Table 5: Ablation Study on 7-Scenes and NRGBD datasets.
7-Scenes
NRGBD
Method
Acc|.
Compl
Overall
Acel
Compl
Overall
pool
0.126
0,200
0,163
0.220
0.480
0.350
w// window
0.123
0.300
0.212
0.253
0.556
0.404
w/a overlap
0.126
0,265
0.195
0.220
0.349
0.285
Full model
0.118
0.205
0.161
0.217
0.298
0.258
Table 6: Camera Pose Ablation Tanks and Temples, CO3Dv2 and 7-Scenes datasets.
Tanks and Temples
CO3Dv2
7-Scenes
Method
RRA@301
RTA@ 301
AUCO#307
RRA@301
KIA@301
AUC 301
RRA 30"
RTA@ 301
AUC@331 301
pool
28.24
40.93
8.87
76.01
78.23
38.10
65.38
41.22
11.54
w/c window
30.69
43.77
12.05
74.54
75.63
37.83
47.76
32.69
7.39
w overlap
30.13
44.83
11.83
81,23
80.44
44.31
56.34
40.98
11.54
Full model
35.88
51.32
15.73
83.54
81.98
47.17
67.92
43.32
15.01
4.5 VIDEO DEPTH ESTIMATION
We evaluate video depth estimation by aligning the predicted depth maps to the ground truth with per-sequence scale. This alignment enables the assessment of both per-fraun framf depth accuracy and inter-frame depth consistency. We report the Absolute Relative Error (Abs Rel) and the prediction accuracy in Table 4 the results show that method demonstrates comparable mance than other online approaches. Furthermore, we also evaluate inference efficiency of KITTI (Geiger al., 2013) dataset single NVIDIA A800 GPU, the result shows that our model runs at the highest speed among online reconstruction methods, running at 17.2 FPS.
4.6 ABLATION STUDIES
To quantify the contribution of each individual component, we conduct series of ablation studies on our proposed method. Specifically, we remove each element in our model to validate the ef- fectiveness of designs. "w/o pool" indicates that the camera head only within the current window for prediction, rather than conditions camera tokens of all historical windows. "w/o window" indicates the model inputs images frame by frame. "w "w/o overlap" indi- cates that there is no overlapping between the frames of adjacent windows, the stride is set equal the window size. our ablation studies, models were trained on 224 224 resolution from scratch without using any pretrained weights. For "w/o pool", "w/o overlap" and our full model, we set a window size of 4.
the camera token
We first validate the effectiveness of our design in reconstruction quality 7-Scenes and NRGBD datasets. To further verify the efficacy our camera pose prediction design, we compare the pose estimation accuracy across all ablated models. As demonstrated in Table 5 and Table 6, the use of a camera token pool leads to a significant improvement in camera pose prediction accuracy. Our online window and online mechanism also significantly enhance the quality of 3D reconstruction.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this paper, we propose WinT3R, an online model for continuous prediction of camera poses and point maps from streaming images. Our framework only employs state tokens align new re- constructions with existing scene geometry, but also utilizes camera tokens to compactly represent global information for each frame. This representation enables the model to capture global infor- mation of historical frames, drastically reducing storage overhead and computational costs. Further- more, our overlapping sliding window strategy enhances continuity across consecutive windows, facilitating comprehensive information exchange. Experimental results demonstrate improvements in reconstruction accuracy and efficiency, validating the efficacy of our design for online 3D recon- struction tasks.
ACKNOWLEDGMENTS
This work is supported by Shanghai Artificial Intelligence Laboratory.
REFERENCES
Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M Seitz, and Richard Szeliski. Building rome in day. Communications of the ACM, 54(10))105-112 201 1.
Mica Nachimacn, Shahar Z Kovalsky, Ira Samnllmacher-Shlzzemmnn Amit Singer, and Ronen Basri. Global motion estimation from point matches. In 2012 Second international conference on 3D imaging, modeling. processing, visualization & transmission, pp. 81-88. IEEE, 2012.
Dejan Azinović, Ricardo Martin-Brualla, Dan B Goldman, Matthias Nießner, and Justus Thies. Neural rgb-d surface reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6290-6301, 2022.
Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data, arXiv preprint arXiv:2111.08887 2021,
Berta Bescos, José M Fácil, Javier Civera, and José Neira. Dynaslam: Tracking, mapping, and inpainting in dynamic scenes. IEEE robotics and automation letters, 34):4076-4088, 2018.
Eric Brachmann and Carsten Rother. Visual camera re-localization from rgb and rgb-d images using dsac. IEEE transactions pattern analysis and machine intelligence, 44(9):5847-5865, 2021.
Neill DF Campbell, George Vogiatzis, Carlos Hernández, and Roberto Cipolla. Using multiple hypotheses to improve depth-maps for multi-view stereo. In Computer Vision-ECCT 2008: 10th European Conference on Computer Vision, Marseille, France, October 12-18, 2008, Proceedings, Part 10, pp. 766-779. Springer, 2008
Junyi Chen, Haoyi Zhu, Xianglong He, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Zhoujie Fu, Jiangmiao Pang, et al. Deepverse: 4d autoregressive video generation as world model. arXiv preprint arXiv:2506.0/ 103, 2025a.
Zhuoguang Chen, Minghui Qin, Tianyuan Yuan, Zhe Liu, and Hang Zhao. Long3r: Long sequence streaming 3d reconstruction. arXiv preprint arXiv:2507. 18255, 2025b.
Javier Civera, Andrew J Davison, and JM Martinez Montiel. Inverse depth parametrization for monocular slam. IEEE transactions on robotics, 24(5):932-944 2008.
David J Crandall, Andrew Owens, Noah Snavely, and Daniel Huttenlocher. Sfm with mrfs: Discrete-continuosd optimization for large-scale structure from motion. IEEE transactions on pattern analysis and machine intelligence,
2012.
Hainan Cui, Xiang Gao, Shuhan Shen, and Zhanyi Hu. Hsfm: Hybrid
In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1212-1221, 2017.
10

---

Published as a conference paper at ICLR 2026
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 0825-5828883 2017.
Andrew J Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. IEEE transactions on pattern analysis and machine intelligence, 29(6):1052-100 1052-1067,
2007.
Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference on computer vision and paltern recognition workshops, pp. 224-236, 2018.
Jakob Engel, Thomas Schöps, and Daniel Cremers. Lsd-slam: Large-scale direct monocular slam. In European conference on computer vision, pp. 834 849. Springer, 2014.
Christian Forster, Zichao Zhang, Michael Gassner, Manuel Werlberger, and Davide Scaramuzza. Svo: Semidirect visual odometry for monocular and multicamera systems. IEEE Transactions on Robotics, 33(2):249-265, 2016.
Yasutaka Furukawa and Jean Ponce. Accurate, dense, and robust multiview stereopsis. IEEE trans- actions pattern analysis and machine intelligence, 12(8):1362-133 2009.
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The international journaall of robotics research, 32(11):121--1277, 2013.
Venu Madhav Govindu. Lie-algebraic averaging for globally consistent motion estimation. Pro- ceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recog- nilion, 2004. CVPR 2004., volume 1, pp. H-I IEEE, 2004.
Richard Hartley. Multiple view geometry in computer vision, 2003.
Xingyi He, Jiaming Sun, Yifan Wang, Sida Peng, Qixing Huang. Hujun Bao, and Xiaowei Zhou. Detector-free structure from motion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Rasmus Jensen, Anders Dahl, George Vogiatzis, Engin Tola, and Henrik Aanæs. Large scale multi- view stereopsis evaluation. In Proceedings the recognition, pp. 406-413, 2014.
the IEEE conference on computer vision and pattern
Arno Knapitsch, Jaesik Park, Qian-Yt Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4):1-13, 2017.
Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision, pp. 71-91. Springer, 2024.
Yixuan Li, Lihan Jiang, Linning Xu, Yuanbo Xiangli, Zhenzhi Wang, Dahua Lin, and Bo Dai. Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3205-3215, 2023.
Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 2041-20500 2018
Zizhuo Li, Yifan Lu, Linfeng Tang, Shihua Zhang, and Jiayi Ma. Comatch: Dynamic covisibility- aware transformer for bilateral subpixel-level semi-dense image matching. arXiv preprint arXiv:2503.23925 2025.
Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Pollefeys. Lightglue: Local feature matching at light speed. In Proceedings of the IEFE/CVF International Conference on Computer Vision, pp. 17627-17638, 2023.
Yebin Liu, Qionghai Dai, and Wenli Xu. A point-cloud-basee multiview stereo algorithm for free- viewpoint video. IEEE transactions on visualization and computer graphics, 16(3):407-41,, 2009.
11

---

Published as a conference paper at ICLR 2026
Yuzheng Liu, Siyan Dong, Shuzhe Wang, Yingda Yin, Yanchao Yang, Qingnan Fan, and Baoquan Chen. Slam3r: Real-time dense scene reconstruction from monocular rgb videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 16651-16662 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https:
//aaxii..rr/aa//1110.... 05101
Pierre Moulon, Pascal Monasse, and Renaud Marlet. Global fusion of relative motions for robust, accurate and scalable structure from motion. In Proceedings of the IEEE international conference computer vision, pp. 3248-3255. 2013.
Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: A versatile and accurate monocular slam system. IEEE transactions on robotics, 3166):147-1113,, 2015.
Riku Murai, Eric Dexheimer, and Andrew J Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors. In Proceedings of the Computer Vision and Pattern Recognition Confer- ence, pp. 16695-16705, 2025.
Jeremy Reizenstein, Roman Shapovalov. Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d cat- egory reconstruction. In Proceedingg of the IEEESCVE international conference on computer vision, pp. 10901-10911. 2021.
Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: photorealistic synthetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10912-10922, 2021.
Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 4938 4947, 2020.
Johannes L Schonberger and Jan-Michael Frahm. Stracture-from-mmtion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104-4113, 2016.
Thomas Schops, Johannes L Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3260-3269, 2017.
Tixiao Shan, Brendan Englot, Carlo Ratti, and Daniela Rus. Lvi-sam: Tightly-coupled lidar-visual- inertial odometry via smoothing and mapping. In 2021 IEEE international conference roboties and automation (ICRA), pp. 5692-5698. IEEE, 2021.
Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb-d images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2930-2937, 2013.
Noah Snavely. Bundler: Structure from motion (sfm) for unordered image collections.
http://phototour. cs. washington. edu/bundler/, 2008.
Noah Snavely, Steven M Seitz, and Richard Szeliski. Photo tourism: exploring photo collections in 3d. In ACM siggraph 2006 papers, pp. 835-846. 2006.
Jiaming Sun, Zehong Shen, Yuang Wang. Hujun Bao, and Xiaowei Zhou. Loftr: Detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8922-8931, 2021.
Chengzhou Tang and Ping Tan. arXiv: 1806 04807, 2018.
Ba-net: Dense bundle adjustment network.
arXiv preprint
12

---

Published as a conference paper at ICLR 2026
Keisuke Tateno, Federico Tombari, Iro Laina, and Nassir Navab. Con-slam: Real-time dense monoc- ular slam with learned depth prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6243-6252. 2017.
Aether Team, Haoyi Zhu, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Chunhua Shen, Jiangmiao Pang, et al. Aether: Geometric-aware unified world modeling. arXiv preprint arXiv:2503.1894 2025.
Zachary Teed and Jia Deng. Droid-slam:
Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems, 34: 161566-166666 2021.
Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns.
IEEE Transactions pattern analysis and machine intelligence, 13(4)337--880 2002.
Fangjinhua Wang, Silvano Galliani, Christoph Vogel, Pablo Speciale, and Marc Pollefeys. Patch-
matchnet: Learned multi-view patchmatch stereo. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2021.
Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv preprint arXiv:2408. 16061, 2024.
Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 21686-21697, 2024a.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5294-5306, 2025a.
Kaixuan Wang and Shaojie Shen. Flow-motion and depth network for monocular stereo and beyond. IEEE Robotics and Automation Letters, 5(2):3307-3314 2020.
Qianqian Wang, Yifci Zhang, Aleksander Holynski, Alexci A Efros, and Angjoo Kanazawa. Con- tinuous 3d perception model with persistent state. arXiv preprint arXiv:2501. 123877 2025b.
Ruicheng Wang. Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5261-5271, 2025c.
Shuzhe Wang. Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 20697-20709, 2024b.
Wenshan Wang, Delong Zhu, Xiangwei Wang. Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4909- 4916. IEEE, 2020.
Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiang- miao Pang, Chunhua Shen, and Tong He. \pi"3: Scalable permutation -qquivaria visual geome- try learning. arXiv preprint arXiv:2507. 13347, 2025d.
Changchang Wu et al. Visualsfm: A visual structure from motion system. 2011.
Yuqi Wu, Wenzhao Zheng, Jie Zhou, and Jiwen Lu. Point3r: Streaming 3d reconstruction with explicit spatial pointer memory, arXiv preprint arXiv:2507.02885 2025.
Hongchi Xia, Yang Fu, Sifei Liu, and Xiaolong Wang. Rghd objects in the wild: scaling real-world 3d object learning from rgh-d videos. In Proceedings of the IEEE/VV Conference on Computer Vision and Pattern Recognition, pp. 22378-22333 2024.
Jianing Yang, Alexander Sax, Kevin J Liang. Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. arXiv preprint arXiv:2501.1392 2025.
13

---

Published as a conference paper at ICLR 2026
Shichao Yang and Sebastian Scherer. Cubeslam: Monocular 3-d object slam. IEEE Transactions Robotics, 35(4):925 938, 2019.
Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mvsnet: Depth inference for unstruc- tured view stereo. In Proceedings of the European conference computer vision (ECCV), pp. 767-783, 2018.
Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1790-1799, 2020.
Chandan Yeshwanth, Yuch-Cheng Liu, Matthias Nießnnr, and Angela Dai. Scannet++: A high- fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 12-22, 2023.
Chao Yu, Zuxin Liu, Xin-Jun Liu, Fugui Xie, Yi Yang, Qi Wei, and Qiao Fei. Ds-slam: A seman- tic visual slam towards dynamic environments. 2018 IEFE/RSJ international conference on intelligent robots and systems (IROS), pp. 1168-1174. IEEE, 2018.
Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE con- ference computer vision and pattern recognition, pp. 3712-3722, 2018.
Ji Zhang and Sanjiv Singh. Visual-lidar odometry and mapping: Low-drift, robust, and fast. In 2015 IEEE international conference on robotics and automation (ICRA), pp. 2174-2181, IEEE, 2015.
Shangzhan Zhang, Jianyuan Wang, Yinghao Xu, Nan Xue, Christian Rupprecht, Xiaowei Zhou, Yujun Shen, and Gordon Wetzstein. Flare: Feed-forward geometry, appearance and camera es- timation from uncalibrated sparse views. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 21936-21947, 2025.
Zhengyou Zhang. Motion and structure from two perspective views: from essential parameters to euclidean motion through the fundamental matrix. Journal of the Optical Society of America A, 14(11):2938- 2950, 1997.
Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin Os- wald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11796,, 2022.
Dong Zhuo, Wenzhao Zheng, Jiahe Guo, Yuqi Wu, Jie Zhou, and Jiwen Lu. Streaming 4d visual geometry transformer. arXiv preprint arXiv:2507.11539, 2025.
A APPENDIX
A.1
ARCHITECTURE DETAILS
The VIT encoder and state decoder maintain the same architecture as CUT3R (Wang et al. 20256).
The encoder has 1024 dimensions and 24 blocks, while the state decoder has 768 dimensions and 12 blocks. The image decoder employs alternating attention mechanism with 768 dimensions and 12 blocks. The learnable state tokens are configured as 1024 tokens, each with 768 dimensions. For camera pose prediction, our prediction head adapted from VGGT (Wang al.) (2225)), consisting of a dis-dimmns,,nnl,, 4-layer transformer block, followed by an MLP layer to output the final camera parameters. Our point map head is lightweight convolutional head adapted from MoGe Wang et al. 2025c). modified to accept 768 di-mensioonl input.
A.2 LONG SEQUENCE COMPARISONS
In online settings, long sequence prediction is also highly important, and we have conducted addi- tional comparisons for long sequences. We selected two models with relatively strong performance, CUT3R (Wang et al. 2025b) and StreamVGG* (Zhuo et al. 2025). for comparison. commpres the efficiency of the models when processing different numbers of frames at resolution of 5x2288.
14

---

Published as a conference paper at ICLR 2026
Table Long Sequence Comparison on 7-Scenes and NRGBD datasets.
7-Scenes
NRGBD
Method
Acc|p
Compl
Overall
Acc.
Comp4
Overall
CUT3R
Wang el
2025b
0.083
0,042
0.062
0.194
0.089
0.142
Stream VGGT Zhin et 2125
0.041
0.020
0.031
10
0.027
0.068
Ours
0.037
0.031
0.034
0.095
0.075
0.085
7 compares the quality of different models when processing 200 frames data. The results show that model maintains real-time performance while significantly outperforming CUT3R in prediction quality. Moreover, it achieves reconstruc- tion quality close that of Stream while being 14 times faster. The experimental results further demonstrate the compactness and effectiveness of our model's camera token pool.
when predicting over hundreds of frames,
Inference Time
Pork GPU Memory Usage (GB)
FPS
2014
1877
125
1
113
2
34
150
300
Figure 6: Inference efficiency of 3D reconstruction. We demonstrate inference efficiency of dif- ferent frame numbers. Our method achieves almost the fastest and the most GPU memory efficient.
A.3 LIMITATIONS
Our model, which utilizes compact camera tokens to assist in predicting camera parameters for subsequent frames, demonstrates promising results. However, still struggles avoid the issue of accumulated errors when processing very long videos or large number of images. Secondly, during the training process, temporal data must be passed through the model sequentially, which requires longer training times compared to offline models. Designing streaming model that conserves training resources remains a challenge to addressed.
A.4 LLM USAGE STATEMENT
In the writing of this paper, the LLM serves writing assistant, used for language translation and to provide concise, accurate, and academic language expression, as well as to correct grammatical errors. All the core ideas, experiments, formulas, methodologies, and figures in this paper originate from the authors and are independent of the LLM.
15

---

Published as a conference paper at ICLR 2026
Input Images (200Frames)
CUT3R
VOGT
Figure 7: Long sequence visualization.
16

---

Published as a conference paper at ICLR 2026
Figure 8: More visualization results.
17


Reviews:
[{"review_id": "mPx0gm8dlC", "reviewer": "Reviewer_Qrbo", "summary": "This paper (“WinT3R”) presents a feed-forward architecture for predicting point maps from streaming RGB video. At a high level, it combines sliding-window attention with a global persistent state, conceptually related to CUT3R and the streaming variant of VGGT (StreamVGGT). The method uses overlapping windows to encourage direct interaction among neighboring frames while maintaining memory efficiency. In addition, a global camera token pool stores compact historical camera tokens to support future camera pose prediction. The overall idea is simple and well-motivated.\n\nOverall, the design choices are reasonable and the empirical results are strong. My concerns are primarily about figure presentation and some missing ablations; details follow.", "questions": "- What are the training time and GPU resources used in the ablation study?\n\n- Which model size (and window size) is used for those ablations?\n\n- There is no ablation that removes the global persistent state entirely. How would performance change without it (e.g., sliding-window only, no global state)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The approach is simple, straightforward, and reasonable: combining sliding-window attention with a global state is a natural way to balance local interaction and long-range consistency.\n\n- Experimental results indicate state-of-the-art reconstruction quality among online/streaming methods.\n\n- The global camera token pool, while somewhat pragmatic, is an effective design specifically for camera pose estimation in a streaming setting.", "weaknesses": "- Result visualization could be improved. For example, in Figure 4 it is difficult to tell which columns are better without clearer legends or more distinct visual cues.\n\n- Consider adding videos or point-cloud visualizations in the supplementary material to better convey qualitative improvements.\n\n- The performance improvement over StreamVGGT is relatively modest, which could lessen the perceived contribution of this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9SexaV45Wy", "reviewer": "Reviewer_i3GJ", "summary": "This paper proposes a new online streaming 3D reconstruction method called WinT3R that combines a sliding-window attention mechanism with a camera token pool memory to achieve real-time performance. It is an improvement based on the prior work CUT3R. The novelty mainly comes from the attention between image tokens within windows, and representing prior frames via compact tokens in a global camera token pool. The experimental results on standard benchmarks show strong improvements over existing online methods.", "questions": "It seems like the selection of the window's length and stride is important here. Different video sequences will typically have non-uniform motions at every point. Will this affect the reconstruction quality a lot?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "It provides a way of balancing the usually high-cost transformer-based 3D reconstruction model and reconstruction quality and consistency over long sequences. This enables the model to operate at real-time with streaming input. The sliding window attention effectively avoids the full casual attention cost. It maintains the simplicity of feedforward neural reconstruction models without heavy keypoint feature matching in traditional SLAM pipelines.", "weaknesses": "* It is unclear for long sequences how well the camera token pool can handle and preserve the global geometry for frames that are far apart. \n* For the efficiency and scalability claim, It lacks some experiments on demonstrating the ability of the model on long sequences and the corresponding runtime statistics. \n* The experiments on the selected scenes are mostly all indoor scenes (e.g. 7 Scenes, NRGBD). It lacks some experiments on more varying conditions such as outdoor scenes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "orWFvwR3c9", "reviewer": "Reviewer_K247", "summary": "This paper presents WinT3R, a feed-forward model for online 3D reconstruction from streaming images. It aims to solve the critical trade-off between reconstruction quality and real-time performance. The method introduces two key contributions: an online sliding window mechanism, which allows image tokens of adjacent frames to interact directly, improving geometric quality , and a compact camera token pool, which maintains global information from all historical frames to enhance the reliability of camera pose estimation.  Experiments demonstrate its effectiveness.", "questions": "Refer to the Weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThis paper presents a novel framework, WinT3R, for online 3D reconstruction, which effectively tackles the critical trade-off between reconstruction quality and real-time performance. The two core contributions, the sliding window mechanism and the camera token pool, are technically sound and directly address clear limitations in prior streaming methods. \n2.\tExtensive experiments validate the effectiveness of the proposed method. \n3.\tThis paper is written and organized well.", "weaknesses": "1.\tThe paper's entire motivation is to close the quality gap between \"suboptimal\" online methods and high-quality offline methods (like the cited VGGT or FLARE). However, all quantitative tables (Tables 1-3) only compare against other online methods. Without including at least one offline advanced baseline, it is impossible to quantify how much of the quality gap has actually been closed, which is essential for contextualizing the paper's contribution. It will be more convincing to add these comparisons.\n2.\tThe paper states the camera head queries the entire historical pool. This implies an attention mechanism with O(T^2) computational complexity, where T is the total number of frames. This quadratic complexity is not scalable, contradicts the real-time (17.2 FPS) claim for any reasonably long video, and is a fundamental design flaw for a streaming system. It will be interesting to discuss about it.\n3.\tThe paper states it uses the final pose from the subsequent window for overlapping frames（L209）. However, the model's token update rule adds the tokens from the initial window to the pool(Eq.5). This raises a problem that the final chosen pose does not match the token stored in the global pool. This problem undermines the pool's global consistency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ek7h4rDy3u", "reviewer": "Reviewer_6j5J", "summary": "This paper proposes a feed-forward reconstruction model WinT3R, which can output precise camera poses and high-quality point maps.\nThe key design is a sliding window mechanism to ensure sufficient information exchange among adjacent frames, and a global camera token pool to enhance the reliability of camera pose estimation without sacrificing efficiency.\nExperimental results show the effectiveness of the proposed method.", "questions": "Please refer to the Weaknesses.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The presentation is clear and easy to follow.\n2. The author conducted extensive experiments to support their statements. And results show that WinT3R outperforms other online baselines in some benchmarks.", "weaknesses": "1. The architecture is highly similar to CUT3R and VGGT. Actually, it seems like a combination of these two methods (state tokens-CUT3R, interaction within window-VGGT). And the training costs are high (compared with CUT3R's 8GPUs).\n2. I want to know the relationship between the global and local tokens (image tokens and camera tokens) for each frame. From Sec.3.3, it seems that this work only supervise the point map in the local coordinate system and the relative poses between frames (following pi3). In this case, is it redundant to maintain both local and global tokens for each frame simultaneously?\n3. With the window based design, I want to know the robustness of WinT3R when faced with unordered inputs. Besides, please use very long sequences (200 frames and more) to compare the reconstruction quality, as it is important in this online settings.\n\nPlease answer the above questions carefully and provide more thorough discussion and comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)