TTT3R: 3D Reconstruction as Test-Time Training
Authors: Xingyu Chen
Yue Chen
Yuliang Xiu
Andreas Geiger
Anpei Chen
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Reconstruction
Structure from Motion
Recurrent Neural Networks
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A simple state update rule to enhance length generalization for CUT3R.

Abstract: Modern Recurrent Neural Networks have become a competitive architecture for 3D reconstruction due to their linear-time complexity. However, their performance degrades significantly when applied beyond the training context length, revealing limited length generalization. In this work, we revisit the 3D reconstruction foundation models from a Test-Time Training perspective, framing their designs as an online learning problem. Building on this perspective, we leverage the alignment confidence between the memory state and incoming observations to derive a closed-form learning rate for memory updates, to balance between retaining historical information and adapting to new observations. This training-free intervention, termed TTT3R, substantially improves length generalization, achieving a 2 $\times$ improvement in global pose estimation over baselines, while operating at 20 FPS with just 6 GB of GPU memory to process thousands of images. Code is available in https://rover-xingyu.github.io/TTT3R.

Published as a conference paper at ICLR 2026
TTT3R: 3D RECONSTRUCTION AS TEST-TIME TRAINING
Xingyu Chen
Yue Chen
Yuliang Xiu
Andreas Geiger
Anpei Chen?
'Zhejiang University
Westlake University
Uni of Tübingen, Tübingen Al Center
6,000 images
Test-Time Training
8,
3.V.
8,
3. V.
CUT3R
CUT3R
Figure 1: Left: CUT3R [89] encodes observations into observation X, and retrieves information by reading out the output token Y.. However, it suffers from the forgetting problem and degrades significantly as the number of input views increases. Right: We treat the state S, fast weight updated via gradient descent, where the learning rate 3, and the gradient V, predicted by the frozen slow weights. These slow weights are learned from training datasets and act as meta-lcarner enabling the fast weight serve as an associative memory. addition, TTT3R makes online state updates by balancing the retention historical information S:-1 with a confidence-awan learning rate 3e. This visualization also incorporates a state reset pracess, please see Appendix B for details.
(memory) S.-1, then interacts with new
ABSTRACT
Modern Recurrent Neural Networks have become a competitive architecture for 3D reconstruction due their linear-time complexity. However, their performance degrades significantly when applied beyond the training context length, revealing limited length generalization. In this work, we revisit the 3D reconstruction foundation models from a Test-Time Training perspective, framing their designs an online learning problem. Building on this perspective, we leverage the alignment confidence between the memory state and incoming observations to derive closed- form learning rate for memory updates, to balance between retaining historical information and adapting to new observations. This training-free intervention, termed TTT3R, substantially improves length generalization, achieving a 2 improvement in global pose estimation over baselines, while operating 20 FPS with just 6 GB of GPU memory to process thousands of images. Code is available in zover- github. 10/TTT3R.
INTRODUCTION
3D reconstruction foundation models aim to predict camera poses and scene representations from a set of input RGB images. Building on the sequence modeling [76), recent advances [87, 91, 101] successfully map sequences of images into pixel-aligned pointmaps [11, 12]. Among these methods, the Transformer [82] has emerged as the dominant architecture, owing to its training efficiency and ability to capture long-range dependencies. However, fundamental limitation lies in the quadratic growth of computational and memory costs with respect to sequence length. Despite various engineering optimizations, such as KV-cache compression [44] and flash attention
[22], the softmax attention remains unchanged and continues to face limited scalability for long contexts [45].

---

Published as a conference paper at ICLR 2026
Real-world applications often require handling an arbitrary number of images. As Fig- ure 2 shows, recent feed-forward methods (e.g.. VGGT [87], Point3R [95]) suffer from high memory consumption. Notably, only CUT3R [89] achieves constant memory usage with RNN-based design. However, as illustrated in Figure 1, CUT3R fails to generalize to long sequences due to training on most 64-frame se- quences. Motivated by these observations, we ask ourselves if there are lessons from modern RNNs that can be used as design principles for 3D reconstruction.
Our GPU memory upper bound (48GB)
50
40-
GPU Usage (GB)
TTT3R
CUT3R
30-
Poim3R
VGre
20-
VGGT (uffline)
10-
3000
Number of Input Views
Figure 2: GPU memory cost for inference.
Recent advances in Recurrent Neural Networks (RNNs) demonstrate performance par with Transformers on language tasks [34, 37J. Recurrent architectures compress the history context into a fix-length memory state, with each output depending solely on the current state and the incoming observation. This recurrent mechanism offers two benefits: efficient processing of long sequences with linear computational complexity, and the ability to scale to longer sequences by simply rolling out the state. Nevertheless, these benefits often come at the cost of substantial performance degradation, particularly when the sequence length exceeds the training context [62, 84].
This naturally raises two questions: (1) why do these models fail to provide robust length gener- alization? and (2) how can length generalization be achieved? To answer these questions, several studies [8, 62, 99] have investigated the length generalization of RNNs, identifying correlations with state overfitting [90]. state forgetting [17, 36], and unexplored state distributions [62]. Solutions such as training on longer sequences and employing Truncated Backpropagation Through Time (TBTT) [62, 75, 94] have been proposed to improve length generalization. While these techniques have been incorporated into recent 3I) reconstruction foundation models, such as CUT3R [89], they still struggle to generalize to sequences comprising hundreds of images.
In this work, revisit the state update rule of recurrent 3D reconstruction models through the lens of Test-Time Training (TTT) [7, 73, 88), and systematically investigate the factors that hinder their ability to generalize across varying sequence lengths. Specifically, inspired by recent findings that recurrent models struggle with length generalization due to state overfitting [62], we reformulate state updating as a TTT-style online learning process [7, 45, 88]. In framework, the historical information is compressed into a state online. We interpret the state as a fast weight [64, 65] learned at test time from the input in-context tokens, rather than from the training dataset. This perspective provides a principled understanding of state overfitting, suggesting that associative recall [9, 60] over long contexts, combined with gradient-based updates using adaptive learning rates balance forgetting and learning [5, 6, 34, 45, 64, 74, 97], can substantially enhance length generalization.
Furthermore, we find that CUT3R [89] can be interpreted as a test-time training mechanism, whereas simply extending the sequence length during training leads to extremely low FLOPs utilization. Therefore, we propose a simple yet effective inference-time state update rule, termed TTT3R, derived as a closed-form state transition for online associative recall in CUT3R. This transition function explicitly defines the learning rate required to update the state at test time, thereby enabling length generalization. Our approach exploits internal confidence signals to selectively suppress low- quality state updates. This yields a stable, training-free free gating mechanism that mitigates catastrophic forgetting [38] without requiring fine-tume fine-tuning additional parameters.
We evaluate TTT3R on standard 3D reconstruction benchmarks, which are typically configured with short- sequence inputs. In this setting, TTT3R performs competitively with state-of-th -he online reconstruction models [89, 95, 106] and demonstrates significant improvements with long-sequence inputs. More importantly, these gains in length generalization come at NO additional computational cost over the bascline, thanks to the proposed state update rule.
Overall, we introduce new TTT-based framework analyze the behavior of stateful 3D reconstruc- tion models. Based on this, we propose a simple, empirical state update rule to enhance sequence length generalization for CUT3R.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
SfM and SLAM. Structre-from-Motion (SIM) [2, 39, 57, 58, 66, 69, 70] and Simultaneous Lo- calization and Mapping (SLAM) [23, 29, 43, 50, 52, 104] have long been the foundation for 3D structure reconstruction and camera pose estimation. These methods rely associating 2D corre- spondences [4, 25, 47, 50, 63] or minimizing reprojected photometric errors [29, 30J. followed by bundle adjustment (BA) [1, 12, 77, 79, 80, 86] for structure and motion refinement. Although highly effective when assembled into comprehensive systems [50, 66], these approaches often struggle in conditions of small camera parallax or ill-posed conditions (e.g., dynamic or textureless). leading to performance degradation. Recent work, such as MegaSaM [43] and VIPE [35], has demonstrated progress in adapting traditional SLAM paradigms to dynamic scenes by integrating semantic seg- mentation [35, 39], optical flows [35, 43, 104, 105], and geometric constraints [35, 39, 43, 48, 104]. Concurrently, methods like VGGT-SLAM [49] and VGGT-Long [24] seek improved robustness by integrating learned front-ends [51, 87, 91]. However, these methods require iterative optimization based on off-the-shelf estimation, where synchronization barriers often lead to cumulative errors and high computational overhead. This reliance hinders real-time online inference and learning scalability (e.g., the 'tabula rasa" blank slate limitation [89]). In this work, we investigate data-driver models with generalizable priors to enable dense 3D reconstruction even from dynamic and textureless video sequences.
Offline Reconstruction Foundation Model. The pioneering feedforward 3D reconstruction method DUS(3R [91] introduced an end-to-end formulation that directly predicts two pixel-aligned pointmaps [11, 12, 68] from an image pair. By leveraging Transformer-bes architecture [28] and direct point supervision on large-scale 3D datasets, DUSt3R inherently accounts for image match- ing [3, 16] and pose estimation [27, 46], resulting in a reconstruction foundation model. Although some follow-up methods [16, 102] extend DUSt3R for robust dynamic scene reconstruction, they inherit the limitation of DUSt3R, requiring costly global alignment when the number of input views exceeds two. To address this issue, Fast3R [96] and VGGT [87] propose to use a large feedforward transformer with global attention that handles multiview inputs and predicts per-view pointmaps simultaneously, without the need for post-processing. leading to state-of-the-ie art 3D point and camera pose reconstruction. However, relying on the full attention [82] causes a quadratic increase in compu- tational and memory cost, and results in offline process that requires re-running inference over all images whenever a new frame arrives. Instead, we aim for compute-efficient, on-the-fly streaming inference that supports long-sequence real-time interactive, and compute-efficient applications.
Online Reconstruction Foundation Model. To improve the reconstruction efficiency, several works introduce memory mechanisms to maintain information from past frames, enabling incremental reasoning and add pointmaps to a canonical 3D space. StreamVGGT [106] concurrently caches historical keys and values memory in causal transformer framework, allowing incremental processing. However, similar to full attention in VGGT [87], the computation and GPU memory usage in Stream VGGT grow redundantly. A promising alternative is the use of recurrent neural network architectures [14, 19, 85], such as CUT3R [89], which maintain a constant-sized memory state, while incrementally integrating new observations by simultaneously updating the state with the newly added view and retrieving historical information from the state. Although the recurrent formulation effectively reduces computational complexity and keeps inference memory usage consistently low, the memory- based methods suffer from the forgetting problem from earlier frames, leading to significant performance degradation as the number of input views increases. To mitigate the forgetting issue, Point3 [95] proposes explicit boset memory, where the history tokens are anchored in the reconstructed 3D point positions. While the explicit memory cache mitigates the forgetting, causes memory cost that grows linearly the number of views increases because the reconstructed points accumulate. In this work, we take an opposite path, exploring a closed-form state update rule that enhances the length generalization of implicit state memory to reasoning over thousands of views, while keeping memory and computation costs consistently low and unchanged as input view growth.
Modern RNN. Recent developments in RNN layers, serve more efficient alternatives to quadratic complex full-attention layers [82], have demonstrated competitive performance in language modeling tasks. One line of research originates from a recurrent variant of attention [37, 64], known as linear attention [37], which uses the standard inner product between the query and key rather than the exponential softmax, allowing the output to be recurrently computed in linear time. However, linear
3

---

Published as a conference paper at ICLR 2026
1
ap V.)
S.
S
a(A)S)-1 X,
S
S
(S,,,,
S,
X,
X
X
(a) Full Attention
(b) Vanilla RNN
Test-Time Training
Figure 3: Sequence Modeling Layers. Full attention appends states, which incurs a quadratic cost. In contrast, vanilla RNNs use fixed-size state with linear complexity, but they suffer from the forgetting problem. Our approach adopts Test-Time Training (TTT), treating the state fast weights leamed during test time via gradient descent with adaptive learning rates, which improves length generalization.
attention equally compresses all key value pairs into its finite-sized state, resulting in performance degradation as the sequence length increases. To address this limitation, various works [5, 54, 74, 97], such as Mamba [21, 34], have proposed adding forgetting gates, in which previous values attenuated by a factor before the new memory stored, to prevent the state from diverging over time. Recently, many of these models have been cast into framework of test-time training/regressionn [7, 73, 88), which views the recurrent update of state as online learning [45] from context [26, 83], balancing between retaining historical memory and adapting new information, shown in Figure 3. The iterate states are also known as fast weights [64, 65], as they change in-context with each timestep, rapidly adapting the input tokens. In contrast to the slow weights in neural networks which act as meta- meta-leerrers [31] and are only adjusted during training ffat weights are learned to function as associative memory [9, 60]. Recent examples of such layers include DeltaNet [64, 98], TTT [73, 103], and Titans [6]. each of these layers was derived from specific choice of retention and adaptation. The idea of learning at test time [10, 72] has also been explored in 3D reconstruction, where methods such CVD [48] and Test3R [100] fine-tune pre-trained model on the test sequence to minimize a self-supervised geometric consistency loss, thereby adapting the model to that particular scene. Inspired by their success, we propose introduce general test-time training framework that enables 3D reconstruction models to achieve both view scalability and memory retention.
3 METHOD
Our method processes a continuous stream of images received online. For each incoming image I, E W we aim to estimate, in real time and the fly: the camera pose T, 3 the camera intrinsic C, E T3x3 and the canonical point cloud P, E TR x We begin in Section 3.1 by introducing sequence modeling to understand and compare different prominent classes of methods that address the pointmap regression problem. Section 3.2 then reformulates recent incremental 3D reconstruction methods from Test-Time Training perspective. Finally, in Section 3.3, we propose TTT3R, showing how the cross- cross-attention between memory and observation can be leveraged as confidence-guided state update rule for online associative recall.
3.1 SEQUENCE MODELING FOR POINTMAP REGRESSION
To transform sequence of images to pixel-aligned pointmaps that lie in unified global coordinate space, a generic formulation can be written as:
X, = Tokenize(I:)
S, (S.ate(S..1,X,,
(I)
Y, = Read(S,, X,)
P, De- ne-tokenize(Y))
where the input image I, is patchified into a set of image tokens X, E (hx througgg an image tokenizer [28], such DINO [15, 53] and CroCo [92, 93]. The image tokens X, update the previous state S into the current state S, using new information. The model then retrieves information stored in the updated state S, by reading out the output token Y, E (hxxw)xx,, Following the readout operation, the corresponding pixel-aligned 3D pointmaps de-tokenizers, such as linear layers with pixel shuffle [67] and a DPT head [61]. The camera pose T, and the camera intrinsic C₁, can either be solved from pixel-aligned 3D pointmaps using the
extracted via dense prediction
4

---

Published as a conference paper at ICLR 2026
PnP [41] and Weiszfeld [56] algorithms, or regressed from image tokens X, through the MLP or trunk attention layers [86, 87].
This sequence formulation offers unified perspective for interpreting pointmap-oriented 3D recon- struction foundation models, where the update and read operations serves as the core distinction among different methods. They fall into two categories: full attention-based based and RNN-based, each introducing specialized designs to the update rules of sequence modeling layers.
For full attention braeet methods, such as Fast3R [96] and VGGT [87], all frames interact through global all-to-all self-attention, which can be interpreted as progressive state concatenation with growing state length:
Update(S.-1, X,) S...aappen Vx,)
(2)
Read(S,, X,) X, + softmax Qx, Qx,
where state - [Kxi,, Vx,). (Kx.-:; vx is list of key-value pairs. Each key-value pair (Kx,, Vx,) ) and query Qx, are transformed from the input token X, via linear layers, and Ks, concat ........ k Vs. concat [V,,..... X,...., Vx,] This modeling requires ((l2) computing complexity, since all output tokens Yi.... Y, must be updated upon receiving X, To efficiently process streaming input, Stream VGGT [106] uses causal attention architecture model the causal nature of streaming data, which restricts each frame to attend only to itself and preceding tokens, allowing only Y, to be updated given X, Causal attention enables incremental processing and reduces the computational cost to (t). However, it shares a similar limitation with full attention: the state represented as key-value list that grows redundantly at Ot), leading increasing memory consumption as the number of views increases.
For RNN-based methods [14, 19, 85, 89, 95], each incoming frame interacts with the state via one-to-one cross-attention, allowing for fixed-length state:
Update(S.-1, X.) = S,- softmax
Vx,
33)
where the state S E Raxe, consisting of n tokens with channel dimension c, encodes the scene with constant length. Qs._, denotes the query projection obtained by applying linear transformation to the state S, Although this recurrent formulation effectively reduces the computational complexity ((1) and inference memory usage constant at (1)), it suffers from forgetting and exhibits significant performance degradation as the number of input views increases.
3.2 REVISIT RNN-BASED RECONSTRUCTION THROUGH TTT
Test- Time Training (TTT) [73] introduces fast weights [65] as rapidly adaptable states that are updated during both training and inference to dynamically capture context. In contrast, slow weights (i.e., model parameters) remain frozen during inference. Formally, TTT represents the state fixed-length fast weight S. E gaxe and updates it via gradient descent:
(S. S.-1 3 V(S.,..
where VSS X,) is a learned gradient function of the previous state S and the current observa- tion X,, aiming encourage the network associate the current observation with the state, and 3r is the learning rate. Intuitively, this online learning process encodes the K K-ccchhe from the current observation into a fixed length of memory (i.e., state) as accurately as possible [88].
For example, linear TTT (or DeltaNet [64, 98]) minimizes the reconstruction error ||[S,||KK _xx Vx, 112 by optimizing the state S to accurately reconstruct the observation value Vx,- This objective yields an analytical gradient:
(S.1 Kx. Vx,
(S. = S. (V
(5)
Here, the key Kx, serves as an index into previous state S.-1, identifying the entries to be updated with the value Vx,- Conceptually, this mechanism treats the state dynamic associative memory, the key specifies where to write, the value specifies what to write, and the learning rate acts as a gating mechanism that controls the memory plasticity by weighting the intensity of the state update.
5

---

Published as a conference paper at ICLR 2026
(Em Q8 Kx
softmax (Qs.. Vx.
S
S
softmar (Quia ( Vx.
S
S
S S
& (S,1,, X,)
S
Update operation
Update operation
K.
K.V
X
Read
Point map, Camera
X
Read
Point map, Camera
Q
(a) CUT3R
(b) TTT3R
Figure 4: TTT3R Illustration, We present a training-free solution for scalable online 3D reconstruction that mitigates forgetting issue in CUT3R. (a) Vanilla CUT3R [89] pipeline. (b) Our reformulation from a test-time training perspective introduces a confidence- guided state update, where alignment confidence between memory and observations serves as per-token learning rates. See Eq. 8 for more details.
Next, analyze the learning rate term which serves the most critical hyperparameter and has been extensively studied in recent advances. It is typically represented as: 1) learnable scalar parameter IR in RetNet [74]; 2) an input-dependent scalar function 3r a (Lg (X.)) R in DeltaNet [64, 98], TTT [73], and Mamba-2 [21]: and 3) a per-token function Br = a (Eg (Xx)) E
Rax1 in Gated Linear Attention [97], which enables token-wise adaptive learning rates across all n
state tokens. Up this point, we reformulate the Eq. 3 using the above TTT formulation:
1.0
softmax (Qs, KX Vx
S + softmax ,K. S.- - (. X,)
(6)
The gradient is defined a linear combination of the observation values Vx, R[hxw)xe, weighted by the softmax alignment scores softmaxe (aammaxxQs.... Kx.) E (ax((xw) between the state query Qs,,, E Raxe and the observation key Kx, E (hxw)xe Conceptually, the gradient function leverages cross-attention aligmment between state query and observation key to determine where to write, assigning the corresponding observation value as what to write to each state token. This formulation has been demonstrated to be effective [91] for learning emergent functional 3D/4D) alignment by implicitly matching cross-view context [16].
However, the softmax operation limits CUT3R's ability to balance retaining historical information with incorporating new inputs, as it forces the model to fully adapt to the latest observations. Specifically, because softmax weights are normalized to sum to 1.0 along the observation-toke dimension, the model always prioritizes new information X, over the historical state S,-1, leading to catastrophic forgetting. This forgetting also reflects the structural discrepancy relative to standard TTT: the lack of flexible learning rate (effectively, a constant 3r = 1.0). Consequently, we are motivated to introduce a state update weight that serves as a gating mechanism to explicitly control memory plasticity.
3.3
SRRIDENCE-GUIDDE STATE UPDATE RULE
Our core idea is to utilize alignment confidence between memory and observation to guide state updates. Figure 4 provides an overview of TTT3R. This confidence constitutes an adaptive per-token state update weight, serving as the per-token learning rate function in the TTT formulation [97].
Recall that the cross- attention (i.e., Qs.,, B aggregates information along the spatial dimension m [......4} 1/3 x {L...., w of the image into n state tokens. This process yields normalized attention weights for each state token, which are then used to compute weighted sum over the value tokens Vx, To address the forgetting issue, we retain the original attention formulation but introduce a per-token learning rate B E Rnxl, derived from the alignment confidence between the state queries Qs... and the observation keys Kx
B. - > Qs., K
To simplify notation, define the summation Σm to be normalized, thus representing mean: Σm 1 E This learning rate act a soft gate in gated attention, incorporating it into the

---

Published as a conference paper at ICLR 2026
attention output allows for better long-context extrapolation [59]. Consequently, the full closed-form state update rule is given by:
-softmax QQ.. (x,
Update 1S = S Be V
(8)
Note that, rather than ignoring quality variations and updating all state uniformly which find leads to suboptimal performance due to low- quality state updates (e.g., textureless regions, see Figure 5)- we leverage cross-attention statis- tics to estimate the alignment confidence state updates and accordingly assign per-token learn- ing rates Be. That is, higher alignment con- fidence in state updates generally indicates stronger match between the state and observa- tion with lower uncertainty. leading to a larger update step in our formulation. By aggregating token-level statistics, we suppress low-quality state updates to enhance performance. similar principle leveraging internal confidence signals selectively filter updates has been explored in concurrent work [32] to improve test-time reasoning for large language models.
1.0
1.0
CUTTR nec.
1.0
Inguts
Image altettion TITER & CUTER Fi
TITTR roc.
Figure 5: By incorporating image attention (i.e.,
Qs KX, B (hxw)) as per-token learning rates
Be R=x1 TTT3R mitigates catastrophic forgetting and facilitates online loop closure.
This formulation enables training free, plug-an -ad--aaddayyyy play intervention for CUT3R, which can be directly applied to downstream tasks without additional fine-tuning.
4 EXPERIMENTS
We evaluate our method on a variety of tasks, including camera pose estimation (Section 4.1), video depth estimation (Section 4.2), and 3D reconstruction (Section 4.3).
Baselines. We first compare TTT3R with the state-of-the-art online 3D reconstruction method CUT3R [89], which performs on-the-fly reconstruction with an RNN-based architecture over stream- ing images. We also evaluate against Point3R [95] and StreaWVGGT [106], which extend CUT3R and VGGT [87], respectively, to longer sequences by fine-tuning with explicit pointmap memory or KV-cache-ber based state representations. In contrast, our approach introduces general sequence modeling framework and an adaptive state learning rate, enabling a training-free solution. In the following experiments, we compare these methods in terms reconstruction accuracy, GPU memory usage, and inference speed. See Appendix C.I and Appendix C.2 of Sup.Mat for more details.
4.1 CAMERA POSE ESTIMATION
Following prior works [89, 102), evaluate camera pose accuracy TUM dynamics [71] and ScanNet [20] datasets. We adopt the standard metric, Absolute Translation Error (ATE), computed after applying Sim(3) alignment [81] between the estimated and ground-truth camera trajectories.
The results of the long-sequence evaluation are shown in Figure 7. For reference, we also in- clude VGGT, an offline method that can be con- sidered as an upper bound for online methods, since its full attention mechanism preserves the entire history context without forgetting. We further evaluate inference efficiency in Figure and Figure 2, reporting two metrics: frames per second (FPS) and peak GPU memory us- age (GB). All models are evaluated on a single 48GB NVIDIA GPU, with the number of input views varied from 50 to 1000 and early termina- tion if oft-of-mentee -e-mmoy occurs.
26.0
15.0
TTT3R
CUT3R
Point3
StraaVGGT
VGGT (offline)
7.5
00M
OOM
00M
200
450
800
820
1000
Number Input Views
Figure 6: Runtime comparison ScanNet [20]. 00M
denotes the method out-of-memory beyond this point.
7

---

Published as a conference paper at ICLR 2026
0.5
0.315
OOM
OOM
0.125
ATE (m)
0.5
(1 11000
0.4
TTEE
U.80
CUTER
AOM
VGGE todiine)
0.1 8-800M
PonnR
COM
StremMGGT
0.000
Number of Input Views
Number of Input Views
Figure 7: Comparison of Camera Pose Estimation on ScanNet [20] (left) and TUM-D [71] (right). OOM denotes out-of-memory. VGGT serves as an offline upper bound by preserving full history, yet full-attention methods (including Stream suffer from high latency and memory exhaustion. Conversely, CUT3R is efficient but drifts long sequences, while Point3R improves accuracy but hits OOM beyond 700 frames. Our method achieves a accuracy improvement over CUT3R while retaining its real-time efficiency. See Appendix C.3 Mup.Mar. for qualitative results.
0.890
0.085
ROwM
0:95
Abs [tee
<<<1 1.225
TTT3R
CUT
DOM OOM
Poior3R
StmmmVGGT
VGGT (offline
0:1
230
300
100
200
500
400
5%
Number of Input Views
Number of Input Views
(a) Scale-invariant relative depth evaluation on Bonn [55] dataset.
0.50
0.20
0.35
0.31
TTT3R CUT3R
Abs Rel
1.25
0.80
Point3R
0.75
0.12
0.20
0.30
200
200
303
500
Number of Input Views
Number of Input Views
(b) Metric depth evaluation KITTI [33], excluding VGGT-based methods that don't support metric depth.
Figure 8: Comparison of Video Depth Estimation. 00M denotes out-of-memory. Full-attention methods (VGGT, StreamVGGT) serve upper bounds for relative depth but hit OOM at > 150 frames. For metric depth, compare the only online metric predictors: CUT3R, Point3R. and TTT3R. While Point3R achieves strong scale-invariant accuracy on short sequences (S 300 frames), it suffers from degradation on longer sequences and inaccurate metric prediction. In contrast, our approach consistently achieves the best overall performance without the need of fine-tuning.
As expected, VGGT and Stream VGGT, both based on full attention, are relatively slow and prone to memory exhaustion. CUT3R, in contrast, maintains consistently low GPU usage and real-time inference but struggles to retain information over long sequences, leading to inaccurate pose estima- tion. Point3R achieves improved accuracy over CUT3R by trading off GPU usage and runtime, but inference is slow and memory runs out beyond 700 frames. By reformulating CUT3R, method achieves accurate pose estimation (with 2 improvement) while preserving the same inference speed and memory efficiency CUT3R.
8

---

Published as a conference paper at ICLR 2026
TTT3R CUT3R Poin3R
Normal Consistency 1
OOM
Chamfer Distance
0.00
SGGT
VGGT (affline)
0.08
0.38
0.00
0.58
COM
50
100
10
200
250
100
303
430
50
330
150
700
250
303
330
010
Number of Input Views
Number of Input Views
Figure 9: Comparison of 3D Reconstruction on 7-scene [68]. We evaluate geometric accuracy (Chamfer Distance 1) and surface quality (Normal Consistency 1) as the number of input views increases. Full-attention methods (VGGT, StreamVGGTT quickly exhaust memory (OOM). While CUT3R suffers from mance degradation due to forgetting. TTT3R maintains robust outperforming CUT3R significantly and achieving lower Chamfer Distance than Point3R. Figure 10 and Ap- pendix C.5 in Map.Mat. presents more qualitative results of 3D reconstruction.
perfor- consistent performance over long sequences,
Please refer Appendix C.3 in Sup.Mat. for qualitative comparisons of camera trajectory estimation.
4.2 VIDEO DEPTH ESTIMATION
Following common practice [89, 102], we evaluate video depth estimation the KITTI [33] and Bonn [55] datasets, which cover dynamic/static as well as indoor/outdoor scenes. We adopt standard metrics: absolute relative eтror (Abs Rel) and 8 1.25 (percentage of predicted depths within 1.25-factor of true depth). Video depth estimation measures both per-frame quality and inter-frame consistency, by aligning predicted depth maps ground truth with per-sequence scale, thereby evaluating relative depth accuracy. For methods that predict metric pointmaps (i.e., outputs in meters with absolute scale), we also report results without scale alignment, evaluating predictions directly in metric units to assess absolute-scale accuracy.
Figure shows quantitative comparison against online baselines. VGGT and Stream VGGT run out of memory after about 150 frames due their reliance full attention. Nonetheless, they serve upper bound in sequenee relative depth evaluation. For metric depth estimation, we report only CUT3R, Point3R, and TTT3R, these are the only online methods predicting metric pointmaps.
Point3R achieves strong scale-invariant accuracy on short sequences (< 300 frames) due to its explicit pointmap memory, but suffers from forgetting and degraded metric-scale accuracy longer sequences. contrast, our approach consistently improves over baselines and achieves the best overall performance, without the need of fine-tuning. See Appendix C.4 of Sup.Mat. for more results.
4.3 3D RECONSTRUCTION
We follow previous work [85, 89] evaluate 3D reconstruction on the 7-scene [68] dateset by measuring the distances between estimated pointmaps and ground-truth point clouds. As in prior work [18, 85, 89], use chamfer distance and normal consistency evaluation metrics. Chamfer distance is computed as the average of accuracy (nearest Euclidean distance from a reconstructed point ground truth) and completeness (the reverse). Unlike prior approaches [85, 89], which sparsely sample 3-5 frames per scene, we evaluate performance on long image sequences to assess memorization capability of different models.
Figure 9 shows that our method significantly outperforms other online approaches such CUT3R [89] and Stream VGGT [106], and achieves results comparable to the top offline, full-attention method VGGT, while operating online in real-time with only 6GB GPU memory. This highlights the effectiveness of our method for 3D reconstruction. Figure 10 presents a qualitative comparison with accurate reconstructions, whereas CUT3R suffers from catastrophic
CUT3R. TTT3R achieves forgetting, leading to drifted camera poses, broken geometry, severe distortions, and ghosting artifacts. For more 3D reconstruction results, please refer to Appendix C.5 in Sup.Mat.

---

Published as a conference paper at ICLR 2026
CUT3K
CUT3R
TTT3R
Figure 10: Qualitative Results for 3D Reconstruction. Compared CUT3R, TTT3R improves sequence length generalization, mitigates forgetting, and enables online loop closure. Other basclines (c.g., VGGT, Point3R) are omitted due to OOM on long sequences. A Check our website to see more vidco comparisons.
5 DISCUSSION
This paper presents TTT3R, providing Test-Time Training perspective for recent 3D reconstruction foundation models, and proposes a simple yet efficient modification to CUT3R that enhances its length generalization. Our experiments demonstrate that TTT3R achieves robust long-sequence 3D reconstruction and outperforms state-of state-of-the-ar methods in most cases. The update is performed during the forward pass without model fine-tuning, making lightweight, plug-am -nd---nd-aayy solution.
Limitations. TTT3R mitigates but does not resolve state forgetting, and it has not yet matched strong offline methods (e.g.. VGGT) in reconstruction accuracy, where full attention despite being slower and more memory denoory-demnndig - preserves the entire history context. This behavior aligns with the unexplored states hypothesis [62], which posits that models trained on short contexts fail to generalize to longer sequences because their recurrence drives the state into dustr---dsti regions not encountered during training. To address this, we explore an optional TTT3R State Reset variant (detailed in Appendix B in Sup.Mat.): by resetting the state to its initial value periodically, we effectively prevent state overfitting. These chunks are then aligned using global metric poses without additional optimization, offering plug-and-pla solution that retains the inference speed and memory efficiency of CUT3R.
Future Work. While TTT3R shows clear boost of test-time regression for associative recall, its design space remains largely unexplored. Recent work [6, 7, 88, 103] highlights a vast opportunity to develop more effective, stable, and parallelizable recurrent architectures. We hope findings will motivate future research to revisit the foundations of 3D reconstruction models and further improve the reconstruction accuracy and length generalization.
10

---

Published as a conference paper at ICLR 2026
Acknowledgments. We thank the members of Inception3D and Endless Al Labs for their help. Xingyu Chen and Yue Chen are funded by the Westlake Education Foundation, Xingyu Chen is also supported by the Natural Science Foundation of Zhejiang province, China (No. QKWL25F0301,, Andreas Geiger are supported by the ERC Starting Grant LEGO-3D (850533) and DFG EXC numher 2064/1 project number 390727645.
REFERENCES
[1] Sameer Agarwal, Noah Snavely, Steven M Seitz, and Richard Szeliski. Bundle adjustment in the large. In ECCV, 2010.
[2] Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M Seitz, and Richard Szeliski, Building rome a day. ACM Communications, 2011.
[3] Honggyu An, Jinhyeon Kim, Sconghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong. and Seungryong Kim. Cross-view completion models zero-shot correspondence estimators. arXiv, 2412.09072, 2024. 3
[4] Herbert Bay, Andreas Ess, Tinne Tuytelaars, and Lue Van Gool. Speeded-up robust features (surf). Computer vision and image understanding. 2008. 3
[5] Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prud- nikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory. NewrIPS, 2024. 2,
[6] Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. Titans: Learning to memorize at test time. arXiv preprint arXiv:2501.0066 2024. 2, 4, 10
[7] Ali Behrouz, Meisam Razaviyayn, Peilin Zhong, and Vahab Mirrokni. It's connected: journey through test-time memorization, attentional bias, retention, and online optimization. arXiv preprint arXiv:2504. 2025. 2, 4,
[8] Assaf Ben-Kish Itamar Zimerman, Shady Abu-Hussein, Nadav Cohen, Amir Globerson, Lior Wolf, and Raja Giryes. Decimamba: Exploring the length extrapolation potential of mamba. arXiv preprint arXiv:2406. 14528, 2024.
[9] Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, and Leon Bottou. Birth of transformer: memory viewpoint. In NeurIPS, 2023. 2, 4
[10] Léon Bottou and Vladimir Vapnik. Local learning algorithms. Neural computation, 1992.
[11] Eric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, and Carsten Rother. Dsac-differentiable ransac for camera localization. In CVPR, 2017. 1,3
[12] Eric Brachmann, Jamie Wynn, Shuai Chen, Tommaso Cavallari, Áron Monszpart, Daniyar Turmukhaabetov, and Victor Adrian Prisacariu. Scene coordinate reconstruction: Posing of image collections via incremental learning of a relocalizer. In ECCV, 2024. 1,
[13] Daniel Butler, Jonas Wulff, Garrett B Stanley, and Michael Black. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012. 22, 23, 24
[14] Yohann Cabon, Lucas Stoffl, Leonid Antsfeld, Gabriela Csurka, Boris Chidlovskii, Jerome Revaud, and Vincent Leroy. Must3r: Multi-view network for stereo 3d reconstruction. In CVPR, 2025. 5,
[15] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021. 4
[16] Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger, and Anpei Chen. Easi3r: Estimating disentangled motion from dust3r without training. arXiv preprint a334412503.24391, 2025. 3, 6, 18, 22, 24
11

---

Published as a conference paper at ICLR 2026
[17] Yingfa Chen, Xinrong Zhang. Shengding Hu, Xu Han, Zhiyuan Liu, and Maosong Sun. Stuffed mamba: Oversized states lead to the inability to forget. arXiv preprint arXiv:2410.07145, 2024. 2
[18] Yue Chen, Xingyu Chen, Anpei Chen, Gerard Pons-Moll, and Yuliang Xiu. Feat2gs: Probing visual foundation models with gaussian splatting. arXiv, 2412.09606 2024. 9
[19] Zhuoguang Chen, Minghui Qin, Tianyuan Yuan, Zhe Liu, and Hang Zhao. Long3r: Long sequence streaming 3d reconstruction. arXiv preprint arXiv:2507.18255 2025. 3,5
[20] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 7, 8, 20, 22
[21] Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprini arXiv:2405.21066 2024. 4, 6, 17
[22] Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. In NeurlPS, 2022.
[23] Andrew Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. PAMI, 2007.
[24] Kai Deng, Zexin Ti, Jiawei Xu, Jian Yang, and Jin Xie. Vggt-long: Chunk it, loop it, align it- pushing vggt's limits on kilometer-scale long rgb sequences. arXiv preprint arXiv:2507.16443, 2025.
[25] Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In CVPRW, 2018. 3
[26] Benoit Dherin, Michael Munn, Hanna Mazzawi, Michael Wunder, and Javier Gonzalvo. Learning without training: The implicit dynamics of in-context learning. arXiv preprint arXiv:2507.16000 2025.
[27] Siyan Dong, Shuzhe Wang, Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. arXiv, 2412.08376, 2024. 3
[28] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 1929, 2020. 3,
[29] Jakob Engel, Thomas Schöps, and Daniel Cremers. Lsd-slam: Large-scale direct monocular slam. In ECCV, 2014. 3
[30] Jakob Engel, Vladlen Koltun, and Daniel Cremers. Direct sparse odometry. PAMI, 2017.
[31] Chelsea Finn, Picter Abbecl, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017. 4
[32] Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence. arXiv preprint arXiv:2508.
2025.
[33] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The international journal of robotics research, 2013. 8, 9, 18, 19, 20, 23, 24
[34] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752. 2023. 2, 4
[35] Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, et al. Vipe: Video pose engine for 3d geometric perception. arXiv preprint arXiv:2508. 10934, 2025.
12

---

Published as a conference paper at ICLR 2026
[36] Samy Jelassi, David Brandfonbrener, Sham M Kakade, and Eran Malach. Repeat after me: Transformers are better than state space models at copying. arXiv preprint arXiv:2402.01032, 2024. 2
[37] Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are rnns: Fast autoregressive transformers with línear attention. In ICMI, 2020. 2,
[38] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinske et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 2017. 2
[39] Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Robust consistent video depth estimation. In CVPR, 2021. 22
[40] Yushi Lan, Yihang Luo, Fangzhou Hong. Shangchen Zhou, Honghua Chen, Zhaoyang Lyu, Shuai Yang, Bo Dai, Chen Change Loy, and Xingang Pan. Stream3r: Scalable sequential 3d reconstruction with causal transformer. 2025. 22, 24
[41] Vincent Lepetit, Francese Moreno-Noguer, and Pascal Fua. Epnp: accurate o(n) solution to the pnp problem. In ICCV, 2009. 5
[42] Vincent Leroy, Yohann Cabon, and Jerome Revaud. Grounding image matching in 3d with MASI3R. In ECCV, 2024. 22, 24
[43] Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. MegaSaM: accurate, fast, and robust structure and motion from casual dynamic videos. In CVPR, 2025. 3
[44] Aixin Liu, Bei Feng, Bin Wang. Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434 2024. 1
[45] Bo Liu, Rui Wang, Lemeng Wu, Yihao Feng, Peter Stone, and Qiang Liu. Longhorn: State space models are amortized online learners. arXiv preprint 2rXiv:2407.14207 2024. 1, 2,
[46] Thibaut Loiseau, Guillaume Bourmaud, and Vincent Lepetit. AlligatOr: Pre-training through co- visibility segmentation for relative camera pose regression. arXiv preprint arXiv:2503.075611 2025. 3
[47] David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 2004. 3
[48] Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation. ACM Trans. on Graphics, 2020. 3,4
[49] Dominic Maggio, Hyungtae Lim, and Luca Carlone. Vggt-slam: Dense rgb slam optimized on the sl (4) manifold. arXiv preprint arXiv:2505. 12549, 2025. 3
[50] Raul Mur- Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: versatile and accurate monocular slam system. IEEE Iransactions on robotics, 2015. 3
[51] Riku Murai, Eric Dexheimer, and Andrew Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 16695-16705, 2025.
[52] Richard A Newcombe, Steven J Lovegrove, and Andrew J Davison. Dtam: Dense tracking and mapping in real-time. In ICCV, 2011. 3
[53] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo. Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint 07X 2023.
13

---

Published as a conference paper at ICLR 2026
[54] Antonio Orvieto, Samuel L Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascamu, and Soham De. Resurrecting recurrent neural networks for long sequences. In ICML, 2023. 4
[55] Emanuele Palazzolo, Jens Behley, Philiip Lottes, Philippe Giguere, and Cyrill Stachniss. Refusion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In IROS, 2019. 8, 9, 20, 23, 24
[56] Frank Plastria. The weiszfeld algorithm: proof, amendments, and extensions. Foundations of location analysis, 2011. 5
[57] Mare Pollefeys, Reinhard Koch, and Luc Van Gool. Self-calibration and metric reconstruction inspite of varying and unknown intrinsic camera parameters. International Journal of Computer Vision, 1999. 3
[58] Marc Pollefeys. Lue Van Gool, Maarten Vergauwen, Frank Verbiest, Kurt Cornelis, Jan Tops, and Reinhard Koch. Visual modeling with hand-held camera. International Journal of Computer Vision, 2004. 3
[59] Zihan Qiu, Zekun Wang, Bo Zheng, Zeyu Huang, Kaiyue Wen, Songlin Yang. Rui Men, Le Yu, Fei Huang, Suozhi Huang, al. Gated attention for large language models: Non-linearity, sparsity, and attention-sink-fres arXiv preprint arXiv:2505.06778. 2025. 7
[60] Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008. 02217, 2020. 2, 4
[61] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense predic- tion. In ICCV, 2021.
[62] Ricardo Buitrago Ruiz and Albert Gu. Understanding and improving length generalization in recurrent models. arXiv preprint arXiv:2507.02782 2025. 2, 10, 20
[63] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Super- glue: Learning feature matching with graph neural networks. In CVPR, 2020. 3
[64] Imanol Schlag, Kazuki Irie, and Jürgen Schmidhuber. Linear transformers are secretly fast. weight programmers. In ICML, 2021. 2, 5, 6, 17
[65] Jürgen Schmidhuber. Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 1992. 2, 4,5
[66] Johannes Lutz Schönberger and Jan-Michael Frahm. concture-fom-mmotee revisited. In
CVPR, 2016. 3
[67] Wenzhe Shi, Jose Caballero, Ferene Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, 2016. 4
[68] Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in RGB-D images. In CVPR, 2013. 3, 3.9
[69] Noah Snavely. Steven M Seitz, and Richard Szeliski. Photo tourism: exploring photo collec- tions in 3d. In SIGGRAPH, 2006.
[70] Noah Snavely, Steven M Seitz, and Richard Szeliski. Modeling the world from internet photo collections. International journal of computer vision, 2008. 3
[71] Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A benchmark for the evaluation of rgb-d slam systems. In 2012 IEEE/RS international conference on intelligent robots and systems, 2012. 7, 8, 18, 19, 20, 22
[72] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In ICML. 2020. 4
14

---

Published as a conference paper at ICLR 2026
[73] Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang. Sanmi Koyejo, et al. Learning to (learn at test time): Rnns with expressive hidden states. arXiv preprint arXiv:240.006620 2024. 2, 4. 5, 6, 17
[74] Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor to transformer for large language models. arXiv preprint arXiv:230..0622,, 2023. 2, 4, 6, 17
[75] Ilya Sutskever. Training recurrent neural networks. PhD thesis, University of Toronto, 2013.
[76] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NeurlPS, 2014.
[77] Chengzhou Tang and Ping Tan. Ba-net: Dense bundle adjustment network. arXiv preprint arXiv:1806.0400) 2018.
[78] Aether Team, Haoyi Zhu, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Chunhua Shen, Jiangmiao Pang, et al. Aether: Geometric-aware unified world modeling. arXiv preprint arXiv:2503.18945, 2025. 22, 24
[79] Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. In NeurlPS, 2021. 3
[80] Bill Triggs, Philip F McLauchlan, Richard Hartley, and Andrew W Fitzgibbon. Bundle adjustment-a modern synthesis. In Vision Algorithms: Theory and Practice: International Workshop on Vision Algorithms Corfu, Greece, 2000. 3
[81] Shinji Umeyama. Least-squares estimation of transformation parameters between point patterns. PAMI, 1991. 7,
[82] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017. 1,3
[83] Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In ICML, 2023. 4
[84] Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, al. An empirical study of mamba-based language models. arXiv preprint 07Xhv:2406.07888, 2024, 2
[85] Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv, 1606, 2024. 3, 5, 9, 22,
[86] Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In CVPR, 2024. 3,5
[87] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In CVPR, 2025. 3,
22, 24
[88] Ke Alexander Wang, Jiaxin Shi, and Emily B Fox. Test-time regression: a unifying framework for designing sequence models with associative memory. arXiv preprint arXiv:2501.12352 2025. 2, 4, 5, 10
[89] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. CVPR, 2025. 1, 2, 3, 22, 23, 24
[90] Shida Wang. Longssm: On the length extension of state-space models in language modelling. arXiv preprint 240.:2400002000, 2024.
[91] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: geometric 3d vision made easy. In CVPR, 2024. 6, 22, 24
15

---

Published as a conference paper at ICLR 2026
[92] Philippe Weinzaepfel. Vincent Leroy, Thomas Lucas, Romain Brégier, Yohann Cabon, Vaibhav Arora, Leonid Antsfeld, Boris Chidlovskii, Gabriela Csurka, and Jérôme Revaud. Croco: Self-supervised pre-training for 3d vision tasks by cross-view completion. In NeurlPS, 2022.
[93] Philippe Weinzaepfel, Thomas Lucas, Vincent Leroy, Yohann Cabon, Vaibhav Arora, Romain Brégier, Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, and Jérôme Revaud. Croco v2: Improved cross-view completion pre-training for stereo matching and optical flow. In ICCV, 2023. 4
[94] Ronald Williams and Jing Peng. An efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural computation, 1990. 2
[95] Yuqi Wu, Wenzhao Zheng, Jie Zhou, and Jiwen Lu. Point3r: Streaming 3d reconstruction with explicit spatial pointer memory. arXiv preprint arXiv:2507. 02863,, 2025. 2, 3, 5, 7, 22, 24
[96] Jianing Yang, Alexander Sax, Kevin J. Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. In CVPR, 2025. 3,5
[97] Songlin Yang. Bailin Wang. Yikang Shen, Rameswar Panda, and Yoon Kim. Gated linear attention transformers with haffware-efficient training. arXiv preprint arXiv:3312.06635, 2023. 2, 4, 6, 17
[98] Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, and Yoon Kim. Parallelizing linear transformers with the delta rule over sequence length. In NeurIPS, 2024. 4, 5, 6, 17
[99] Zhifan Ye, Kejing Xia, Yonggan Fu, Xin Dong, Jihoon Hong, Xiangchi Yuan, Shizhe Diao, Jan Kautz, Pavlo Molchanov, and Yingyan Celine Lin. Longmamba: Enhancing mamba's long con- text capabilities via training-free receptive field enlargement. arXiv preprint arXiv:2504. 16053, 2025. 2
[100] Yuheng Yuan, Qiuhong Shen, Shizun Wang. Xingyi Yang, and Xinchao Wang. Test3r: Learning reconstruct 3d at test time, 2025. 4
[101] Jiahui Zhang, Yuelei Li, Anpei Chen, Muyu Xu, Kunhao Liu, Jianyuan Wang, Xiao-Xiao Long, Hanxue Liang. Zexiang Xu, Hao Su, et al. Advances in ferwaf 3d reconstruction and view synthesis: A survey. arXiv preprint 1XXi:2507.14501 2025. 1
[102] Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. MonST3R: simple approach for estimating geometry in the presence of motion. In ICLR, 2025. 3, 7, 9, 22, 23, 24
[103] Tianyuan Zhang, Sai Bi, Yicong Hong, Kai Zhang, Fujun Luan, Songlin Yang, Kalyan Sunkavalli, William T Freeman, and Hao Tan. Test-time training done right. arXiv preprint arXiv:2505.2388
2025. 4, 10
[104] Zhoutong Zhang, Forrester Cole, Zhengqi Li, Noah Snavely, Michael Rubinstein, and William T. Freeman. Structure and motion from casual videos. In ECCV, 2022. 3, 22
[105] Wang Zhao, Shaohui Liu, Hengkai Guo, Wenping Wang, and Yong-Jin Liu. Particlesfm: Exploiting dense point trajectories for localizing moving cameras in the wild. In ECCV, 2022. 3
[106] Dong Zhuo, Wenzhao Zheng, Jiahe Guo, Yuqi Wu, Jie Zhou, and Jiwen Lu. Streaming 4d visual geometry transformer. arXiv preprint arXiv:2507. 11539, 2025. 2, 3, 7, 9, 22, 24
16

---

Published as a conference paper at ICLR 2026
Appendix
Table of Contents
More Experimental Analysis
A.1
Comparison with Standard Learnable Gating Mechanisms
A.2
Finetuning TTT3R
18
A.3
TTT-derived Update Rule Non-T TTT Baselines
B State Reset
20
C More Results
21
C.1 Experimental Settings
21
C.2 Basclines
22
C.3 Camera Pose Estimation
22
C.4 Video Depth Estimation
23
C.5 3D Reconstruction
24
Use Large Language Models
25
A MORE EXPERIMENTAL ANALYSIS
This section provides detailed experimental analysis to validate the design choices of TTT3R. Specifically, we conduct three primary studies: first, we perform a Comparison with Learnable Gating Mechanisms to contrast TTT3R against existing test-time training baselines that employ different learnable gating mechanisms for modeling the state update learning rate; second, through the TTT3R Finetuning Analysis, we investigate whether applying our derived update rule and confidence confidence-guided learning rate during the training process leads further performance gains; and finally, we present a Comparison with Non-TTT Baselines to evaluate the core effectiveness of our Test-Time Training (TTT) reformulation by comparing it against of strong non-TTT methods.
A.1 COMPARISON WITH STANDARD LEARNABLE GATING MECHANISMS
Conceptual Relation to Common Gating Mechanisms. To further analyze the relationship between our proposed confidence-guided learning rate and existing standard gating mechanisms, we compare how the learning rate is modeled in recent advances:
1. ScalarLR: In RetNet [74], the learning rate represented single learnable scalar parameter, 3 E F. However, the constraint that all observations share the same learning rate inherently limits its representational flexibility.
2. ConditionLR: In models such as DeltaNet [64, 98], TTT [73]. and Mamba-2 [21]. the learning rate is modeled as an input-dependent scalar function: 3 a (Eg (X,)) e R². This provides the capability to condition the learning rate on the current observation X.-
3. TokenLR:
Gated Linear Attention [97] further proposed 3r that both input-conditioned and per-token: 3, = a (Eg (X,)) e g=x1, This enables adaptive assignment of individual learning rates for the state tokens, conditioned on the current observation.
Our core idea also models conditioned pok-token learning rate 3 E R following the TokenLR paradigm. However, in contrast to previous gating mechanisms that rely additional learnable parameters to model the learning rate (e.g.. a scalar in ScalarLR, or a hyper-network (x(-) in ConditionL.R RR and To) we derive training-free, closed-form learning rate. This is motivated by
17

---

Published as a conference paper at ICLR 2026
the explainable doss-attention mechanism [16] and is derived directly from the alignment confidence between the state queries Qs.-. and the observation keys Kx,: : 3 = (Em 1 This formulation achieves per-token adaptivity without introducing any additional parameters, training overhead, or computational cost.
Practical Differences between Common Gat- ing Mechanisms. To further analyze the ef- fectiveness of our proposed TTT-derived update rule and confidence-guue confidence-guided guided learning rate, we dis- cuss the practical differences between TTT3R and simply incorporating standard learnable gating module into the CUT3R state update. As detailed in Table 1 and Figure 11, we in- troduce ScalarLR, ConditionLR, and TokenLR into the state update of CUT3R. In this abla- tion study, we freeze the encoder, decoder, and output heads, and train only the newly added learnable gating module. The training is con- ducted using the same dataset CUT3R, with sequences ranging from 4 to 64 views. The results clearly demonstrate that CUT3R + TokenLR is the most effective among the learnable gating mechanisms. However, it still significantly underperforms TTT3R in both camera pose estimation and video depth estimation. In our observation, the perfor- mance of the learnable gating mechanisms is primarily limited by the training sequence length (i.e. 64 frames). We find that the longer the training sequence, the better the learnable gating mechanism performs. Unfortunately, training on sequences longer than 64 frames is prohibitively costly and becomes infeasible given our hardware constraints (48GB NVIDIA GPU). In contrast, our approach derives the learning rate directly from context without requiring expensive unn-sequence training.
Camera Pose
Video Depth
Method
ATE
RPE
Abs Rel
d <1.25
CUT3R
0.173
0.494
0.152
80.2
CUT3R SealarLR
0,165
0.502
0.151
80.6
CUT3R ConditionI.R
0. 0,166
0.509
0.149
81.0
CUT3R TokenLR
0.154
0.497
0,148
81.5
TTT3R
0. 106
0.431
0.131
86.9
TTT3R Finetune
0.091
0.434
0.133
86.3
Table 1: Evaluation camera pose estimation (1000
frames on TUM-dynamic [71]) and video depth esti- mation (500 frames on KITTI [33]). TTT3R achieves state-of-the-art performance across all compared learn- able gating mechanisms. Finetuning TTT3R provides marginal benefits in pose estimation.
0.15
TTTER
0.38
CUT3R
CUT3R ScaiueLR
0.12
CUT3R CondtiouLR
ATE (m)
0.10
CUT3R
TITER
0.08
0.84
TITER
0.66
CUT3R
CUTER StalaLR
CUT3R Cond
CUTER TokmnR
Number of Input Views
Number Input Views
(a) Camera pose evaluation TUM-D [71].
(b) Metric depth evaluation on KITTI [33].
Figure 11: Quantitative Comparison of Camera Pose and Video Depth Estimation. TTT3R achieves state-of-the-art performance compared to all baseline methods incorporating learnable gating mechanisms. Finetuning TTT3R yields marginal benefits in pose estimation.
A.2 FINETUNING TTT3R
We then experiment with finetuning TTT3R by applying our derived update rule and confidence- guided learning rate during the training process. The training is conducted using the same dataset as CUT3R, with sequences ranging from 4 to 64 views.
As shown in Table and Figure 11, finetuning leads to better performance in camera pose estimation, concurrently degrades the video depth estimation results. One possible reason for this outcome, illustrated in the CUT3R work, is that once the model is finetuned on 4-64 views, the model tends to prioritize global alignment over per-view prediction accuracy, which introduces forgetting and overall performance degradation of video depth estimation.
While incorporating the TTT3R update rule during training helps pose estimation, it does not provide significant benefits if the training sequences short (e.g., 64 frames). As 7, TTT3R shows only minimal performance gains compared to CUT3R on short sequences. We therefore hypothesize that scaling up the training sequence length could lead to better performance.
see from Figure
18

---

Published as a conference paper at ICLR 2026
A.3 TTT-DERIVED UPDATE RULE VS. NON-TTT BASELINES
To evaluate the effecttiveness of our Test-Time Training (TTT) reformulation, compare TTT3R against strong non- non-TTT baselines, including periodic state reset, Exponential Moving Average (EMA) shrinkage to the initial state, and burn-in mechanism with keyframes. We first establish the optimal hyperparameters for these baselines:
1) CUT3R Reset. This mechanism performs a periodic state reset, where the state is reset to its initial value every frames. The resulting state chunks
Baselines
ATE 4 RPE trans
RPE rot 4
Reset 50
0.129
0.007
0.406
then globally aligned using metric camera poses. We first ablate the choice of the reset period {50, 100, 1503. As shown in Table 2, n 100 yields the best results, which we adopt for subsequent experiments.
Reset 100
0.126
0.007
0.403
Reset 150
0.145
0.008
0.416
Table 2: Ablation on the periodicity of the State Reset baseline.
2) CUT3R + EMA. This method introduces shrinkage towards the initial state Su using an Exponential Mov- ing Average (EMA) during inference. The state update formulated as: S, (1 a)SS.1 + aSn, where is the shrinkage rate. We ablate the choice of a E {0.0001, 0.001, 0.013. As shown in Table 3, a = 0.001
Baselines
ATE
RPE trans
RPE
EMA 0.0001
0.169
0.007
0.574
EMA 0.001
0.164
0.007
0.525
EMA 0.01
0.191
0.009
0.673
Table 3: Ablation on the shrinkage rate of the EMA baseline.
provides the optimal performance.
Baselines
ATE L
RPE trams 1 RPE rot
3) CUT3R + BurnIn. The burn-in mechanism updates the state exclusively using keyframes, leaving the state changed for intermediate frames. We ablate the keyframe interval n E (50, 100, 150) frames. As shown in Table 4, an interval of n = 100 yields the best results, which we utilize for CUT3R BurnIn.
BurnIn
0.150
0.0384
3.784
BurnIn 100
0.144
0.0309
3.093
BurnIn 150
0.156
0.0360
3.693
Table 4: Ablation on keyframe interval for the Burn-In baseline.
As shown in Table 5 and Figure 12, TTT3R significantly outperforms all non-TTT baselines (Reset, EMA, and BurnIn) in both camera pose estimation and video depth estimation, thereby validating the effectiveness of our TTT-derived update rule. note that these non-TTT mecha- nisms can also be integrated into TTT3R. Specif- ically, we observe that the Reset mechanism is highly effective for preventing state overfitting. Thus, we integrate Reset into TTT3R with the same optimal hyperparameter (n 100), lead- ing to the variant TTT3R + Reset. This combi- nation further boosts TTT3R to achieve better performance, We provide a detailed analysis of this phenomenon in Section B.
Camera Pose
Video Depth
Method
ATE +
rot
Abs 5 <1.25
CUT3R
0.494
0.152
80.2
CUT3R Reset
0.126
0.403
0.128
84.9
CUT3R EMA
0.164
0.525
0.150
80.7
CUT3R BurnIn
0.144
3.093
0.151
80.2
TTT3R
0.106
0.431
0.131
86.9
TTT3R Reset
0.093
0.375
0.115
88.5
Table 5: Evaluation of camera pose estimation (1000
frames on TUM-dynamic [71]) and video depth estima- tion (500 frames on KITTI [33J). TTT3R achieves better performance across all compared non-TTT baselines. Furthermore, the integrated variant, TTT3R Reset, achieves the best performance.
TTTRR
0.16
TTTRR Reset
0.16
CUTER
CUT3R Reset
EMA
ATE (n))
0.10
CUT3R Bunalo
1.25
0.86
0.88
0.54
TIT3R + Reset
0.86
CUTSR
012
CUT3R Reset
CUTSR
012
CUT3R Bumin
0.90
206
400
400
sio
180
160
391
300
500
Number of Iaput Views
Number of Input Views
(a) Camera pose evaluation on TUM-D [71].
(b) Metric depth evaluation KITTI [33].
Figure 12: Quantitative Comparison of Camera Pose and Video Depth Estimation. TTT3R achieves state-of-the-art performance compared to all TTT baselines. Furthermore, the integrated variant, TTT3R Reset, achieves superior performance in both camera pose and video depth estimation.
19

---

Published as a conference paper at ICLR 2026
08-
TTT3R
100M
FTTSR
0.150
CUTER
00M
66-
CUTRR Reset
PoinE
ATE (m)
4.5
SmamvVGG
0.100
0.4-
VGGT (ottine)
022
w080-
0.1
0.25
OOM
0OM
0.60
Number of Input Views
Number of Input Views
(a) Camera pose evaluation on ScamNet [20].
(b) Camera pose evaluation on TUM-D 171].
TIT3R
0.15
TTT3R
Reset
0.4
CUT3R
0.34
0:92
Abs Rel
0.33
Roin3R
1.25
0:50
0.30
0.09
200
200
Number of Input Views
Number of Input Views
(c) Metric depth evaluation on Bonn [55], excluding VGGT-based methods that don't support metric depth.
0.22
0.23
0.58
TTTER
TTT3R + Reser
Abs Rel
CUT3R
1.25
0.50
CUT3R Reset
0,34
Point3R
0.52
0.20
0.70
1003
200
302
400
1203
1000
2003
300 400
500
Number of Input Views
Number of Input Views
(d) Metric depth evaluation on KITTI [33], excluding VGGT-based methods that don't support metric depth.
Figure 13: Comparison of Camera Pose and Video Depth Estimation. For the sake of simplicity, with the exception of Fig. 1, we do not use State Reset in any experiments or analyses reported in the main paper. For readers interested in the improvements achievable with State Reset, we provide comprehensive results here.
B STATE RESET
As discussed in the limitation subsection (Sec. 5), TTT3R only mitigates but does not fully resolve state forgetting, resulting in failure beyond 1000 frames (as shown in Fig. 14, middle). This ob- servation is consistent with the unexplored states hypothesis [62], which posits that models fail to generalize to longer sequences when their recurrence, applied to extended sequences, produces state distributions not encountered during bhining--uggestinng that these models overfit to states produced early in the sequence when trained on short contexts. Based on this hypothesis, we incorporate State Reset mechanism: the state is reset its initial value every 100 frames, thereby preventing state overfitting (as shown in Fig. 14, right). The resulting chunks are then aligned using the global metric camera poses without any optimization, making TTT3R + State Reset a plug-and-play solution that preserves CUT3R's 's inference speed and memory footprint.
Note that, for the sake simplicity, with the exception of Fig. 1, we do not employ State Reset in any experiments or analyzes reported in the main paper. The State Reset is only used for visualization demonstrations that exceed 1000 frames. Specifically, we only apply the State Reset in Fig. 1 and Fig. where we visualize the final result and also augment CUT3R with the State Reset-to to allow for a fair and intuitive comparison. For readers interested the improvements achievable with State Reset, we provide quantitative results in Figure 13.
20

---

Published as a conference paper at ICLR 2026
CUT3R
TTT3R
TTT3R + State Reset
Figure 14: State Reset for sequences beyond 1000 frames. As discussed in the limitation subsection,
TTT3R only mitigates but does not fully resolve state forgetting, resulting in failure beyond 1000 frames, For visualization demonstrations exceeding 1000 frames (Fig. and Fig. 15), further augment TTT3R with State Reset mechanism: the is reset every 100 frames, and the global metric camera pose used cue align the resulting chunks, Note that, for simplicity, we do not employ State Reset any quantitative experiments and analyses in the main paper. State Reset is applied only in Fig. and Fig. 15 (where we visualize the final outcome. and also augment CUT3R with State Reset to enable a fair and intuitive comparison), and Fig. 13 (to report quantitative improvements achievable with State Reset).
CUT3R
CUT3R+ State Reset
TTT3R+ State Reset
Figure 15:
In-the-wild Video Reconstruction -Sequences beyond 1000 frames. While CUT3R + State Reset still suffers from drifting, TTT3R + State Reset enables robust long-sequence 3D reconstruction beyond 1000 frames. The TTT3R State Reset is performed in the forward pass without any optimization, making plug-and and-play solution that preserves CUT3R's inference speed and memory footprint.
C MORE RESULTS
C.1 EXPERIMENTAL SETTINGS
We present two experimental settings: (1) long sequence evaluation, compare TTT3R with online methods that could handle hundreds of images, which challenging setting by measuring the state capability to memorize entire sequences, rather than short video clips; and (2) short sequence evaluation, to compare the performance of our method to wide range of baselines (since these baselines, hindered by out-of-memoryy are infeasible to handle long sequences). Please refer to the main paper for the long-sequence results. We report the shoutnee evaluation in the following section of the supplement.
21

---

Published as a conference paper at ICLR 2026
Sintel (50 frames)
TUM-dynamies (90 frames)
ScanNet (90 frames)
Method
Online ATE L
RPE trans
RPE rot
ATE 4
RPE trans
RPE rot
ATE
RPE trans + RPE
Robust-CVD [39]
0.360
0.154
3,443
0.153
0.026
3.528
0.227
0.064
7.374
CasualSAM [104]
X
0.141
0.035
0.615
0.071
0.010
1.712
0.158
0.034
1.618
DUS3R [91]
X
0.417
0.250
5.796
0.083
0.017
3.567
0.081
0.028
0.784
MASOR [42]
0.185
0.060
1.496
0.038
0.012
0.448
0.078
0.020
0.475
MonST3R [102]
0.111
0.044
0.869
0.098
0.019
0.935
0.077
0.018
0.529
Basi3R [16]
0.110
0.042
0.758
0.105
0.022
1.064
0.061
0.017
0.525
AETHER[78] [78]
X
0.189
0.054
0.694
0.092
0.012
1.106
0.176
0.028
1.204
VGGT [87]
0.062
0.471
0.012
0.010
0.035
0.015
0.377
Spann3R [85]
0.329
0.110
4,471
0.056
0.021
0.591
0.096
0.023
0.661
CUT3R [89]
0.2113
0.066
0.621
0.046
0.015
0.473
0,099
0.022
0,600
Point3R [95]
0,351
0.128
1.822
0.075
0.029
0.642
0.106
0.035
1,946
Strc [106]
0,251
0.149
1.894
0.061
0.033
3.209
0.161
0.057
3.647
STream3R [40]
0.213
0.076
0,868
0.026
0.013
0.330
0.052
0.021
0,850
TTT3R
0.201
0.063
0.617
0.028
0.012
0.379
0.064
0.021
0.592
Table 6: Evaluation Camera Pose Estimation- Short Sequence on Sintel [13], TUM-dynamies [71]. and
ScanNet [20] datasets. TTT3R achieves the best overall performance among online methods, while accuracy has not yet matched strong offline methods (e.g.. VGGT), where full attention - despite being slower and more demory-demanding preserves the entire history context.
C.2 BASELINES
We first compare TTT3R with pairwise 3D reconstruction foundation models, including DUSt3R [91]] MASt3R [42], MonST3R [102], and Easi3R [16], which takes pair of views as input and requires extra global alignment stage to consolidate the pairwise predictions. We also compare with AETHER [78] and VGGT [87], which could predict all pointmaps simultaneously, without the need for post-processing leading to Mfate-of-the-an 3D point and camera pose reconstruction. However, relying on global alignment or full attention limits all the aforementioned methods to handling only short image sequences, in an offline reconstruction manner, where it needs to rerun inference of all images whenever new frame arrives. For online methods, we compare TTT3R with Spann3R [85] and CUT3R [89], which operates online with RNN-based architectures and could handle streaming images on the fly. For concurrent works that are most similar to our method, we compare TTT3R with Point3R [95], StreamVGGT [106] and STream3R [40], which aim to extend CUT3R and VGGT to handle long image sequences, but take different approach by fine-tuning CUT3R and VGGT with explicit pointmap memory and KV cache as state representation, respectively. Unlike these works, our method introduce sequence modeling from the Test-Time Training (TTT) perspective. As a result, TTT3R achieves online associative recall by deriving closed-form update rule, without requiring fine-tuning CUT3R or training extra parameterized components.
general framework and reformulate CUT3R
C.3 CAMERA POSE ESTIMATION
Following prior works [89, 102], we evaluate camera pose estimation accuracy on Sintel [13], TUM dynamics [7]]. and ScanNet [20] datasets. We use standard error metries: Absolute Translation Error (ATE), Relative Translation Error (RTE), and Relative Rotation Error (RRE), after applying the Sim(3) alignment [81] on the estimated camera trajectory to the ground truth. We compare with both 3D reconstruction foundation models and prior per-sequence optimize approaches, such as RobusiCVD [39] and Ca [104], which jointly optimize camera parameters and dense depth maps to fit each sequence.
The results of the long-sequence evaluation are presented in Figure 7. Since many baselines can only process short sequences due to out-of-memory, also show the short sequence evaluation in Table 6, and separately highlight the leading approaches for methods that operate offline (i.c., require additional optimization global attention) and those that do not (i.e., online). Although gap persists between offline and online methods, our approach achieves the best overall performance among online methods, particularly in TUM- -UM-dynamics and ScanNet datasets.
We show qualitative comparisons of the estimation of the camera trajectory in Figure 16. TTT3R demonstrates more accurate and robust camera pose estimation over CUT3R, effectively leveraging
22

---

Published as a conference paper at ICLR 2026
GT
CUTER
54
2.4
TTT3R
CUTSR
CUTTR 11154
51
GTT
OUTER
CUTER TTT3R
OUTER
TITER
VTT3R
cumR
2.8
TITR
CUT3R
2.0 TTT3R
CUTER
TIT
65
Figure 16: Visualization of Estimated Camera Trajectories Long Sequence. The trajectories are plotted along the two axes with the highest variance to capture the most significant motion. Our estimated camera trajectory TTT3R deviates less from the ground truth GT compared to the baseline CUT3R.
the inherent knowledge the 3D reconstruction foundation model with just a few lines plug-and- play code by proposing general sequence modeling formulation and deriving novel state transition rule of CUT3R from TTT pespective.
C.4 VIDEO DEPTH ESTIMATION
Following common practice [89, 102], we evaluate video depth estimation on KITTI [33], Sintel [13], and Bonn [55] datasets covering dynamic and static, indoor and outdoor scenes. We absolute relative error (Abs Rel) and < 1.25 (percentage of predicted depths within 1.25-factor of true depth) as metries. Video depth estimation evaluates per-frams depth quality and inter-frame depth consistency by aligning predicted depth maps ground truth using a per-sequence scale, which measures the relative depth accuracy. For methods that predict metric pointmaps (i.e., outputs in meters with absolute scale), also report results without scale alignment, evaluating predictions directly in metric units to assess absolute-scale accuracy.
Figure 8 presents the quantitative comparison between our method and the online baselines. Our approach delivers the best overall performance without ANY fine-tuning. The results in Table show that even for short sequences, online methods, leading in KITTI dataset for both metric and scalc-invariant evaluations and ranking two in Sintel and Bonn datasets.
method still achieves state-of-the-as competitive performance in
23

---

Published as a conference paper at ICLR 2026
1rammn
400 300frams
Out of Memory
150 150mmms
400
Out of Memory
GT
VGGT
CUT3R
TTT3R
Figure 17: Qualitative Results of 3D Reconstruction. TTT3R improves length generalization over CUT3R while preserving its speed memory efficiency. Offiine methods (e.g.. VGGT) achieve accurate reconstruction on short sequences (150 frames) fail on longer sequences (400 frames) due to memory constraints.
Sintel (50 frames)
BONN (110 frames)
KITTI (110 frames)
Alignment
Method
Online
Abs Rel | 1.25
Abs Rel
d<1.25 f
Abs Rel
<1225
DUSI3R [91]
0.656
0.155
0.144
81.3
MAS13R [42]
0.641
0.252
0.183
74.5
MonST3R [102]
0.378
55.8
0.067
96.3
0.168
74.4
Fasi3R [16]
0.377
55.9
0.059
97.0
0.102
91.2
AETHER [78]
0.324
50.2
0.273
59.4
0.056
97.8
Per-sequence Scale
VGGT [87]
0.287
66.1
0.055
97.1
0.070
96.5
SpannäR [85]
0.622
42.6
81.3
0.198
73.7
CUT3R [89]
0.421
47.9
93.7
0.118
88.1
Point3R [95]
0.452
48.9
96.0
0.136
84.2
Stream VGGT [106]
0,323
0.059
97.2
0.173
72.1
STream3R [40]
51.1
0.116
89.6
TTT3R
0.404
50.0
0.068
95.4
0.113
90.4
MASI3 [42]
1.022
14.3
0.272
70.6
0.467
15.2
CUT3R [89]
1.029
23.8
0.103
88.5
0.122
85.5
Metric Scale
Point3R [95]
0.777
17.1
0.137
94.7
0.191
73.8
STream3R [40]
1.041
21.0
0.084
94.4
0.234
57.6
TTT3R
0.977
24.5
0.090
94.2
0.110
89.1
Table Evaluation Video Depth Estimation Short Sequence. We report scale-invariant relative depth (aligned by per-sequence scale) and metric scale absolute depth accuracy on Sintel [13], Bonn [55], and KITTI [33] datascts. TTT3R achieves statc-of-the- oott----ttt-ttt--- competitive performance among online methods, leading KITTI for both metric and scale-invariant evaluations and ranking first second in Sintel and Bonn.
C.5 3D RECONSTRUCTION
The results are presented in Figure 18, Figure 15 and Figure 17. TTT3R supports both video sequences and sparse photo collections, across static and dynamic scenes, and performs online 3D reconstruction by estimating camera parameters and dense geometry for each incoming frame. TTT3R is simple modification to CUT3R that improves length generalization via a closed-fon state update, enabling robust long-sequence 3D reconstruction. The update is performed in the forward pass without any model fine-tuning. making it a plug-and and-play solution, while preserving CUT3R'S ' inference speed and memory footprint and operating online at realtime and only cost 6GB GPU memory.
24

---

Published as a conference paper at ICLR 2026
Figure 18: In-the-Wint Wild Video Reconstruction Short Sequence. TTT3R performs online 3D reconstruction by estimating camera parameters and dense geometry for each incoming image. It supports varying-lenge image inputs, either video streams sparse photo collections, across both static and dynamic scenes.
D USE OF LARGE LANGUAGE MODELS
We used large language model assist with copy editing-grammar checking, wording suggestions, and minor style and clarity improvements- after the scientific content, methodology, analyses, and conclusions had been written by the authors.
25


Reviews:
[{"review_id": "omO4UPaUQD", "reviewer": "Reviewer_yvT7", "summary": "This paper addresses the critical issue of catastrophic forgetting and poor length generalization in recurrent 3D reconstruction foundation models. It re-frames the recurrent state update mechanism as an online, test-time training (TTT) problem. The primary contribution is TTT3R, a novel, training-free framework that derives a closed-form adaptive learning rate based on the alignment confidence between the memory state and new observations. This dynamically balances memory retention and adaptation, enabling robust and accurate 3D reconstruction and pose estimation over long image sequences without incurring additional computational or memory costs.", "questions": "Please see the weakness part.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The re-interpretation of the RNN state update (specifically in CUT3R) as a test-time training process is a novel and insightful conceptual contribution. The derivation of a closed-form, confidence-guided learning rate (Eq. 7) from this perspective is an elegant solution to the problem of balancing state retention and adaptation.\n\n2. The method is training-free and acts as a plug-and-play module for the CUT3R baseline. It achieves significant performance gains, including a 2x improvement in global pose ATE on long sequences. It consistently outperforms other online methods (CUT3R, Point3R) on multiple challenging benchmarks (ScanNet, TUM-D) while, crucially, maintaining the same low memory footprint (6GB) and real-time inference speed (20 FPS) as the baseline.\n\n3.   The paper tackles a highly significant and practical bottleneck in the field. Length generalization is a major limiting factor for applying efficient, streaming 3D reconstruction models to real-world, long-term video sequences for applications like SLAM or AR.", "weaknesses": "1. Clarity of Final Update Rule: The presentation of the final, implemented update rule is confusing. Equation (8) merely restates the generic TTT formula. The actual update rule, which appears to be $S_t = S_{t-1} + \\beta_t \\cdot [softmax(Q_{S_{t-1}} K_{X_t}^\\top) V_{X_t}]$ (where $\\beta_t$ is from Eq. 7), is only implied by diagrams (Fig. 4, 5) and the analysis of CUT3R (Eq. 6). The paper would be much clearer if this final equation was explicitly stated.\n\n2. Relation to Gating Mechanisms: The proposed confidence-guided learning rate $\\beta_t$ is functionally very similar to a standard gating mechanism. The paper could be strengthened by a more thorough discussion of the conceptual and practical differences between its TTT-derived update rule and simply adding a more traditional, or even learnable, gating module to the CUT3R state update.\n\n3. Another relevant work, Test3R, also present test-time training based on dust3r series with self-consistency guided adaptation. A discussion and comparison with this method would be necessary to better contextualize the paper's contribution. \n\n[1] Test3R: Learning to Reconstruct 3D at Test Time, NeurIPS 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "coLYaLoDcC", "reviewer": "Reviewer_jkPo", "summary": "This paper, TTT3R, introduces a training-free method to improve long-sequence generalization in recent recurrent 3R models (in particular, CUT3R). The task is feed-forward point-cloud reconstruction from a sequence of RGB frames. Prior work on this task includes DUSt3R, VGGT, CUT3R, et al. For long video, feed-forward reconstruction, recurrent transformers like CUT3R are generally considered competitive.\n\nThe contribution can be summarized in one sentence: rather than updating every state token as in CUT3R, TTT3R redesigns a per-token update weight (a “learning rate” in the paper’s test-time-training view) so that only those state tokens that are highly relevant to the current observations receive large updates.\n\nIn summary, the paper presents a training-free, neat idea with useful insights into how to weight state updates. The experiments show that this approach improves CUT3R’s generalization on long sequences. Nice idea, and nice results.\n\nThat said, I think the presentation would benefit from revision. Personally, I wouldn’t frame the method as “test-time training.” The same idea could be described more directly and precisely without introducing new terminology. See details below.\n\nOverall, this is a good paper with a somewhat complex presentation. I would consider accepting it if the authors rephrase certain terms.", "questions": "- I am very interested to know whether the authors have tried applying this new updating rule to fine-tune the CUT3R model. Could the performance be further improved?\n- The authors present an insightful design for the weight updating rule. Have they explored alternative approaches to weight updating? I would be interested to learn about that as well.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- A neat and elegant idea. Only a small amount of code modification is needed to alleviate the forgetting problem of CUT3R on long-sequence inputs.\n- The results are impressive. Figures 7, 8, and 9 clearly show that this method achieves significantly better reconstruction quality than CUT3R.\n- The visualizations are well done. Thank you for including Figure 10 and the supplementary webpage.", "weaknesses": "The main weakness lies in the presentation.\n\nI would not frame the method as “test-time training.” The same idea could be described more directly and precisely without introducing new terminology. For example, I find Equation (5) unhelpful in conveying the main idea. It introduces gradient notation for a reconstruction error, which requires readers to spend time understanding the notation and the context of TTT. Once readers parse what TTT and Equation (5) mean, the paper essentially presents Equation (6), which is not really related to TTT. I would describe the approach instead as “per-token adaptive weighting of state updates,” which is much more intuitive.\n\nPerhaps I misunderstood something, but in Equation (6), is it truly necessary to introduce the gradient notation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XjxycBTsKS", "reviewer": "Reviewer_dBBg", "summary": "TTT3R reframes 3D reconstruction from image sequences as a test-time training problem to address the limited length generalization of recurrent models like CUT3R. It introduces a confidence-guided state update rule, deriving a closed-form learning rate from cross-attention alignment scores. This training-free intervention balances historical retention with new observations, mitigating catastrophic forgetting. Evaluations show TTT3R improves performance on long sequences (e.g., 2× better pose estimation) while maintaining CUT3R's efficiency (20 FPS, 6 GB memory for thousands of frames).", "questions": "Please see the weakness section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The strengths of this paper lie in:\n1) Good motivation, sequence-length generalization is a real failure mode for stateful online 3D models. Tackling it at test time without retraining is practical.\n2) It is very interesting to see the training-free, plug-and-play intervention design. Requires no fine-tuning or extra parameters; preserves CUT3R’s O(1) memory and O(1) per-step compute while adding only a cheap per-token gate; practical and implementation-friendly.\n3) We can see the obvious gains in length generalization (e.g., 2× ATE improvement over CUT3R on ScanNet/TUM-D at ≥200–1000 views; Fig. 6) without sacrificing throughput (20 FPS; Fig. 7) or memory (6 GB; Fig. 2).\n4) The presentation of this paper has clear technical reformulation casts CUT3R’s recurrent state update as test-time training with fast weights; derives a closed-form per-token learning-rate gate beta_t from alignment confidence, Eq. 7, yielding an explicit update rule (Eq. 8) that mitigates softmax-induced overwriting.", "weaknesses": "May major concerns lie in:\nThis paper proposes a simple, empirical state-update rule to enhance sequence-length generalization for CUT3R. It seems to be a straightforward extension of CUT3R. Specifically,\n1) From the TTT3R HTML, the core proposal for a state update appears to be a gradient step on a per-timestep loss with respect to the state, i.e., Update(S_{t-1}, X_t) = S_{t-1} − beta ∇_S L(S_{t-1}, X_t). This is a clean, causal mechanism aligned with the TTT literature but adapted to “fast weights”/state instead of parameters.\n2) The update form given in TTT3R is easy to integrate with CUT3R,  that is to treat the persistent scene state as a latent variable and make a local corrective step using the current-frame loss. If the loss L uses only X_t (and not future frames), this is strictly causal and compatible with streaming.\n3) CUT3R maintains a persistent state via state tokens and a learned update/readout (ViT decoders). CUT3R trains with a fixed context length and acknowledges resets of the state on single-view datasets, which suggests the usual risk of degradation when inference sequences are longer than the training horizon. A test-time state-adaptation rule that damps drift and improves long-horizon stability is therefore a relevant add-on.\n\nSome other technical problems:\n1) What is the test-time loss L(S_{t-1}, X_t)?\nIf it uses only self-supervised signals from a single frame (e.g., priors, regularizers) it may be weak/noisy. If it uses multiview photometric/reprojection consistency, you must ensure it’s causal (only past frames). Please make explicit exactly which losses are used at test time and why they are valid surrogates for “state correctness.”\nIf supervised losses are used in any evaluation, that would not be a realistic test-time condition—please clarify.\n2) Stability analysis vs “framework to analyze dynamics”\nThe claim of “a TTT-based framework to analyze the dynamics” suggests more than proposing an update rule. Do you provide any formal analysis (e.g., linearization around a fixed point, Jacobian spectral radius of the state dynamics, contraction conditions w.r.t. beta, or Lyapunov-style arguments)?\nIf not, please temper the language or add at least a local linear analysis that shows how the beta step acts as a stabilizer/damping term and under what conditions it improves stability.\n3) Evidence that the update improves sequence-length generalization in CUT3R\nPlease report metrics as a function of time t on sequences substantially longer than the training horizon (e.g., 2–4×). Plot error growth with/without TTT state updates, and show that the slope is reduced. Include both static and dynamic scenes since CUT3R handles both.\nProvide ablations on: number of gradient steps per frame, beta schedule, gradient clipping, choice of loss, and whether backprop-through-time is disabled (it should be, for causality/cost).\nCompare to strong non-TTT baselines: periodic state reset, EMA/shrinkage to s0, learned gating/forget gates, and “burn-in” with keyframes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SubM6VsUaq", "reviewer": "Reviewer_6nKa", "summary": "This paper proposes TTT3R, a test-time training framework for online 3D reconstruction tasks, including depth estimation and camera pose estimation from long video sequences. While prior offline reconstruction models (e.g., VGGT, Point3R) often run out of memory when processing long sequences, TTT3R offers a training-free and memory-efficient solution. Built upon the recurrent architecture of CUT3R, it introduces a confidence-guided state update rule that adaptively balances memory retention and adaptation to new observations, leading to improved performance and generalization on long video sequences.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The idea is simple, intuitive, and well-motivated, reinterpreting 3D reconstruction as a test-time training problem and providing a fresh, elegant, and training-free plug-and-play solution that requires no additional parameters or fine-tuning, making it easy to integrate into existing 3D reconstruction pipelines.\n\n2. The results are clearly better than previous online methods in long sequences, where most offline methods run out of memory.", "weaknesses": "1. The contribution is somewhat incremental, as the proposed method mainly builds upon CUT3R with a modified state update rule. Since TTT3R is presented as a general training-free framework, it would be valuable to include or discuss additional baselines beyond CUT3R to better demonstrate generality.\n\n2. The performance is still lower than offline methods such as VGGT on short sequences, indicating that the proposed test-time mechanism, while efficient, does not fully close the performance gap with global-attention architectures.\n\n3. It would also strengthen the paper to compare or at least discuss more recent approaches, such as MegaSaM, VIPE, VGGT-SLAM, and VGGT-Long. Even though some of these may not yet be peer-reviewed, acknowledging them would help position this work within the rapidly evolving landscape of long-sequence 3D reconstruction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)