IncVGGT: Incremental VGGT for Memory-Bounded Long-Range 3D Reconstruction
Authors: Keyu Fang
Changchun Zhou
Yuzhe Fu
Hai Helen Li
Yiran Chen
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Reconstruction
memory efficient
long range
point cloud
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We present IncVGGT, a training-free incremental variant of VGGT that makes transformer-based 3D reconstruction feasible for long sequences in real-world applications. Vanilla VGGT relies on dense global attention, which causes memory to grow quadratically and requires excessive computation, making it impractical for long-sequence scenarios. Even evolved streaming variants, such as StreamVGGT, still suffer from rapidly growing cache and latency. IncVGGT addresses these challenges from two orthogonal directions: (1) register and fuse overlapping frames into composite views, reducing duplicate tokens, and (2) history-side pruning retains only the top-$k$ most relevant/maximum slots together with the most recent one, bounding cache growth. This incremental and memory-efficient design minimizes computation and memory occupation across arbitrarily long sequences. Compared to StreamVGGT, IncVGGT sustains arbitrarily long sequences with large efficiency gains (e.g., on 500-frame sequences, 58.5$\times$ fewer operators, 9$\times$ lower memory, 25.7$\times$ less energy, and 4.9$\times$ faster inference) while maintaining comparable accuracy. More importantly, unlike existing baselines that directly run out of memory beyond 300 (VGGT)–500 (StreamVGGT) frames, IncVGGT continues to operate smoothly even on 10k-frame inputs under an 80GB GPU, showing that our design truly scales to ultra-long sequences without hitting memory limits. These results highlight IncVGGT’s potential for deployment in resource-constrained edge devices for long-range 3D scenarios.

Published as a conference paper at ICLR 2026
INCVGGT: INCREMENTAL VGGT FOR MEMORY- BOUNDED LONG-RANGE 3D RECONSTRUCTION
Keyu Fang
Changchun Zhou'
Yuzhe Fu Duke University yuzhe. fu@duke. edu:
Duke University
Duke University
keyu. fangeduke. edu
changchun. zhou@duke edu
Hai "Helen" Li
Yiran Chen
Duke University hai. li@duke.edu
Duke University
yiran. chen@duke. edu
ABSTRACT
We present VGGTG training- incremental variant of VGGT that makes transformer-based 3D reconstruction feasible for long sequences in real-world ap- plications. VGGT relies on dense global attention, which causes memory to grow quadratically and requires excessive computation, making impractical for long- sequence scenarios. Even evolved streaming variants, such as StreamVGGT, still suffer from rapidly growing cache and latency. VGGT addresses these chal- lenges from two orthogonal directions: (1) register and fuse overlapping frames into composite views, reducing duplicate tokens, and (2) history-side pruning re- tains only the top-k most relevan//hihhest-soorng slots together with the most re- cent one, bounding cache growth. This incremental and memory-efficien design minimizes computation and memory occupation across arbitrarily long sequences. Compared to Stream VGGT, IncVGGT sustains arbitrarily long sequences with large efficiency gains (e.g., on 500-frame sequences, 58.5x fewer operators, 9x lower memory. 25.7x less energy, and 4.9x faster inference) while maintaining comparable accuracy. In addition, unlike existing baselines that directly run out memory beyond 300 (VGGT)-500 (StreamVGGT) frames, VGGT contin- ues operate smoothly even 10k-frame inputs under an 80GB GPU, demon- strating that our design scales to ultra-long sequences without hitting memory limits. These results highlight IGCVGST's potential for deployment in resource- constrained edge devices for long-range 3D scenarios.
INTRODUCTION
3D reconstruction from videos for long-range 3D space is long-standing challenge with broad impact across vision and robotics (Zhuo et al. 2025). Applications span immersive VR/AR/XR experiences (Hong al. 2024 Zheng et al. 2024). real-time robotics and manipulation (Sünderhauf et al. 2018), autonomous navigation (Geiger et al. 2012). In many of these settings, models must run under strict memory and compute limits on edge or mobile hardware (Howard al. 2017). where repeatedly processing overlapping content in long or redundant streams quickly becomes impractical.
Feed-forwad transformer backbones have recently demonstrated strong 3D performance without per-scene optimization (Wang et al. 2024b Leroy et al., 2024). VGGT (Wang et al. 2025) is prominent example that predicts depth, pose, point maps, and tracks in single pass. However, its global self-attention scales quadratically with the total token length (Vaswani al. 2017). and naive streaming still accumulates an ever-growing cache. In practice, both behaviors become bottlenecks on long sequences, limiting real-time use in VR/AR devices, robots, and vehicles.
Our key observation is: video streams exhibit substantial redundancy along two axes YYoon & Choi 2023 Input-side redundancy: Adjacent frames repeatedly cover the same regions, so incurring
*Corresoonding author.

---

Published as a conference paper at ICLR 2026
the full token cost every step leads to redundant ieppuuttatiooHaabbbiii al. 2021 History- side redundaney: Once relevance is known, many cached keys/values contribute little to the next step (Adnan et al. 2024). We instantiate these insights in VGGT, which retains the backbone's 's feed-forward nature while eliminating redundancy from two orthogonal directions. Input side: We register and compose short windows into compact composite views using simple span gate: when viewpoints are coherent, frames are aligned and merged (Brown & Lowe, 2007). When they drift, the window is bisected with one shared overlap frame preserve stitchability. This eliminates pixel-level overlap before tokenization, reducing the token load that enters attention. History side: We bound the key-value cache by relevance rather than sequence length: only the top-k historically important slots together with the most recent one are retained for the next step. The result replaces dense "all tokens vs. history" interactions with sparse "few tokens vs. few slots" computation, delivering large savings in memory and operators while preserving geometric stability.
The design targets real deployments of 3D/4D perception: long recordings, stabilized capture, telep- resence, and robotics all benefit from processing each region once and carrying forward only what remains useful. The sestrrationnnd-compptttt artifacts (voids and seams) so that tokenization remains stable (Li et al. 2013). The cache rule is a minimal change to the attention layer (Saxena et al.) (024)). No iterative optimization or heavy post-processee poatpsocessing is required, and both components run online. In this paper, have the following two main contributions:
step includes lightweight handling of warping
Input-side registration and composition. We align and merge short windows into com- posite views, collapsing overlapping regions so that the transformer processes far fewer tokens (Sec. 3.1.
Global-local cache pruning. We retain only fixed number of globally high-score slots plus the most recent one, converting a growing cache into a constant-size set (Sec. 3.2)
BACKGROUND
Structure-from-Mootoon (SIM) and Multi-view Stereo (MVS). Classical 3D reconstruction ap- proaches are rooted in geometry-basee based pipelines such as fructure-from--mee view stereo (MVS) (Ozyeşil et al.) 2017 Wang et al.) 2024a). These methods detect and match sparse image features, estimate camera poses through bundle adjustment, and recover dense geome- try using photometric consistency. While highly accurate and theoretically wrol-ggounded, they rely on iterative optimization and careful initialization, which makes them computationally demanding and often brittle in challenging scenarios (Zhang et al.). As a result, conventional SIM/MVS sys- tems struggle to scale to long video sequences real-time applications, often requiring hours of optimization for a single sequence. This motivates learning- baaning-based ser--fowward approaches that can offer faster inference.
(SfM) and multi-
Learning-based 3D Reconstruction. Neural representations such as Neural Radiance Fields (NeRF) (Mildenhall et al., 2020) introduced new paradigm for scene reconstruction, but typi- cally require per-scene optimization and heavy computation, limiting their use in large-scale or long-sequence settings. To overcome these limitations, recent feed-forward methods directly pre- dict geometry and pose from input images. For example, DUSt3R and MASt3R estimate point maps or depth maps in a single pass, while extensions like CUT3R (Wang et al.) attempt incorporate memory mechanisms for longer sequences. These approaches improve efficiency and generaliza- tion. but still struggle with scalability, as frames are largely processed independently without ex- ploiting temporal redundancy. This gap has motivated transformer-based designs that process all frames jointly.
Transformer-based 3D Vision. The success of transformers in 2D vision (Dosovitskiy et al.) 2021) has inspired their application to multi-view 3D reconstruction. representative example is VGGT, which employs pure transformer backbone to jointly predict depth, camera pose, point maps, and feature tracks within a single feed-forward framework. By treating all input views as tokens in global self-attention mechanism, VGGT achieves state-of-the-art accuracy across diverse 3D benchmarks and demonstrates the potential of transformers as a foundation model for geometry understanding with strong cross-dataset generalization.
2

---

Published as a conference paper at ICLR 2026
Defirent Virea
Frame
Canara
Frans Akgeomn
Point
Frane
Depth
Figure I: Overview of the VGCT pipeline. Input frames are aligned into common reference before tokenization. Self- and global attention then operate with bounded cache, where low-score keys/values are pruned. Outputs are predicted jointly by camera, point, and depth heads.
However, this design comes with significant scalability costs. The quadratic complexity of attention makes memory usage grow rapidly with the number of frames (Dao) 2 instance, a 24GB GPU can only process few dozen images before running out of memory. In addition, the model processes all frames jointly in one shot, which prevents incremental extension to longer sequences and causes resource usage to grow superlinearly as input length increases. As result, VGGT achieves high accuracy on short sequences but cannot be directly applied to continuous or large- scale real-world videos. Addressing this limitation requires redundancy- strategies for both inputs and cached history.
4D and Long-Sequence Reconstruction. Extending transformer-based 3D models long tinuous sequences has recently gained sigmificant attention. VGGT-Long (Deng et al. 2025) tackles the scalability issue by dividing input videos into overlapping chunks and performing alignment across chunks, enabling large-scale reconstructions such as kilometer-long trajectories. However, this strategy tailored for offline processing and requires additional post-hoc alignment. In contrast, Stream VGGT adapts VGGT to online video by introducing causal attention and key-value caching, allowing frames to be processed sequentially while reusing history rather than re-encoding all views jointly. Stream VGGT preserves the forward nature of VGGT, supports real-time streaming inference, and currently represents the state of the art for long-sequence 4D reconstruction. Nev- ertheless, caching every key and value across time inevitably causes memory usage to grow with sequence length, so although VGrT is effective in practice, efficiency degrades over long videos, underscoring the need for redundancy-awe designs that scale gracefully with sequence length.
METHOD
We pursue scalability on high-overlap visual streams by removing redundancy at two complemen- tary stages. First, an input-side registration and composition module collapses pixel-level overlap within short window, yielding compact composite views whose token count we denote by T Zitováá & Flusser 2003). Second, a global-local cache pruning module retains only S historical slots (high-scoring plus recency) for attention, both compute and memory remain controlled as sequences grow (Xiao et al. 20244 Kwon et al. (2023). Throughout we use L for the total number of tokens in dense global tormulation; avoid overloading notation, the span gate in Sec. 3.1 uses separate threshold X. Fig. gives an overview of the pipeline.
3.1 REGISTRATION-BASED REDUNDANCY REDUCTION
Adjacent frames in long video streams often contain heavy pixel-level overlap, so processing each independently causes the transformer to tokenize the same regions multiple times. Our method mitigates input redundancy before attention by registering and composing a window of K frames into single composite view in reference domain (Huang et al.).
ienitraatio-anddcommpiiii pipeline is illustrated in Fig. 2 We adopt a hierarchical composition strategy controlled by a span gate: first attempt to align and merge all K frames; if the normalized coverage of the warped supports remains within a threshold X, we generate one composite view; otherwise, the window is recursively bisected into two sub-windows until each satisfies the span criterion. To ensure frames remain
3

---

Published as a conference paper at ICLR 2026
stitchable, adjacent sub-windows share one overlap frame. By collapsing overlapping pixels. the transformer ultimately fed with a token count that scales with the composed support area rather than the raw number of frames. A statistical comparison of token growth under the baseline and our method is presented in Fig. 3
Fodtara
Hemegraphy
Matiiing
Span
Inpat looen Count input Size
Toan Cate
Spen
Heveersty
Pedure aeocing
10AS 203--50--20
Figure 2: Registration-an--oompiiii pipeline (Sec. 3.1b. short window of frames aligned via feature matching and homography, warped into a reference domain, and com- posed through the span gate (split/merge) with lightweight craetiig/croppinnnnnaiiii posite view.
1pp durrion
Figure 3: Input token count vs. input size (N denotes tokens of one standard view). Results are from tandt_cb truck with our method.
to produce compact com-
Feature detection and registration. Given a short window of K frames { we first align all frames to common reference before composition. Specifically, we select To as the reference, extract local features (e.g., ORB (Rublee et al., (01T) or SIFT (Lowe) 20044) on each frame, and establish candidate correspondences via k-nearest neighbors (kNN) matching. To reduce mismatches, apply ratio tests and cross-checking, retaining only high-confidence correspondences, thus obtaining homographies H,-ii- between adjacent frames (e.g. via DLT RANSAC) (Agarwal al. 2005). From these correspondences, we estimate pairwise homographies H that map frame to frame i- 1. To propagate alignment the reference, cumulative warps obtained by composition:
H4-00 = H1-0 H2-1 Hi-i-1,
i= i --,,
with Ho-vo = I. This procedure provides lightweight yet effective alignment that tolerates outliers
and mild parallax, while keeping all frames within consistent reference domain (see the left block of Fig. 2] Although homographies cannot fully account for strong 3D parallax. this approximation proves sufficient for the short temporal windows considered. Notably, it allows efficient streaming implementation: once H;-i-- is estimated, subsequent frames can be incrementally aligned to the reference without reprocessing earlier frames.
Band-limited warping, Direct projection of all warped frames onto a global canvas can cause the canvas size grow unbounded, producing excessive extrapolation and large unsupported regions the periphery. Such instability not only wastes tokens but also disrupts the subsequent span compu- tation. To address this, we restrict composition to a narrow vertical band centered the reference support. Concretely, a global translation is applied so that all warped coordinates remain nonneg- ative within this band-limited canvas. This design bounds the effective canvas size, suppresses unsupported margins, and ensures that span evaluation in the next stage remains well-conditioned.
Normalized span and recursive bisection. decide whether K frames can be composed one shot, we define normalized span that measures the horizontal coverage of the union of warped supports units of a single-frame width. Let S; = HQ,, be the warped support frame (restricted to the band mind-iimtedd canvas) and let Wo be the width of 2g. Using (.)-) for the horizontal projection (the minimal interval covering a set along the x-axis),
span span((H::o)) BB UKK-" S.)
Wo
If span < X. viewpoints coherent and proceed with single-shot composition. Otherwise split into two sub-windows of size [K/2] and [K/2] that share the middle frame (e.g., [0: K/2]]

---

Published as a conference paper at ICLR 2026
Og
0:
C2
My
Oe
W.
(M]
OK'
We
Wa
w/
Wer
Inc. Inf.
(NTT))
01
We
We
Ma
Wa
KV Cache
Wa
(N(E)
(1.E5
OKT
We
We
w/
Global-Local
Wk
Who
Pruning l.
Gur
W- Wei
[W
N: #Points
E: Embedding size
O: Attention output I: Attention Input
Cached, No Comp. Tim.
Max and V R Newn Nearest K and V
Figure 4: Top: Naive recomputation, where each new frame attends to tokens (L X L). I: In- cremental inference, which updates only the queries from the new frame. 2.1: KV caching, which reuses stored keys/values so new queries attend directly to the accumulated history. 2.2: IncVGGT which selects only the top-k maximum slots plus the most recent one, yielding constant-size cache.
and K)) and recurse independently. The overlap frame serves as an alignment anchor that guarantees stitchability if higher-level fusion is needed; practice, span decreases quickly with window size under typical motions.
Blending and void mitigation. Once (sub-)window passes the span gate, its warped frames fused by mask-aware distance-transffrm feathering (Allène et al. (200)). For each warp, we compute a binary validity mask and assign per-pixel weights that increase smoothly toward the interior, so that the normalized accumulation
Σ. W.(x) T((x)
î(x)
8>0.
Σ, W.(2))
suppresses seams without introducing halo artifacts. To mimic streaming input, later frames over- write earlier ones, ensuring that newly arrived content fills small residual gaps.
Planar warps may still produce unsupported voids, especially at disocclusions aries. We apply a lightweight clean-up that operates only in these regions: persistent unsupported margins the horizontal extremes are trimmed, and strict voids (all-zero pixels) are inpainted with a small-radius fill to remove speckles while preserving valid texture (Telea 2004 Quan et al.) 2024). Optionally cropping to the reference-framt height further regularizes the token grid and improves batching efficiency. The output examples in Fig, iillustrate typical seam and void artifacts together with their removal.
band bound-
3.2 GLOBAL-LOCAL CACHE PRUNING
Simply storing all past keys and values, as done in vanilla streaming transformers (Dai et al. 2019) leads to ever-growing cache size and rising cost, making long sequences infeasible. In VGGT, each incoming frame triggers full recomputation of global attention across all tokens, producing a dense L matrix where = F p is the total token length for frames with p tokens per frame (Fig. 4 top). This quadratic cost requires both heavy computation and memory to maintain
5

---

Published as a conference paper at ICLR 2026
the key-value states. Incremental inference mitigates this by processing only the queries from the new frame rather than recomputing all past ones (Fig. 4 1). KV caching further improves efficiency by storing historical keys/values that new queries can directly attend to them (Fig. 4 2.1) (Zaao et al. 2024). However, the per-step cost O H Tnia dp.) still grows linearly with cache length. Once the cache accumulates 7-800 frames, memory already exceeds 80 GB and per-frame inference time keeps rising with sequence length. This motivates a more selective strategy: instead of retaining all slots, we keep only a compact set of globally high-score slots plus the most recent one, while low-score slots often arising from overlapping or repetitive views are pruned redundant (Fig. 4 2.2).
A100
Global-local pruning rule. Real-world video streams for 3D reconstruction exhibit strong frame- to-frame continuity and overlap. Slots with the highest scores at step are likely to be reused at step 1+1. We exploit this property by propagating historical relevance and preselecting only the top-k slots, together with the most recent one to capture sudden scene changes (Kim Jung 2025
et al. 2024). Formally, let
Qu Vdn
A softmax
(3)
be the attention over cached keys at time l. Reducing A, across queries/heads yields a per-key relevance vector s!0 E IR Lai Before processing frame 1+,, we preselect
S TopK(s(0, Rc) (most recent),
and restrict attention to this (k+1) (k+1)-sized) set:
Qu+1 K ) Vsut1 Vān
O++1 = softmax
(4)
This avoids scoring against the entire cache and yields constant-sin history per step.
Complexity and memory. Restricting attention to Se+1 bounds the per-step score computation to Q H (k+1)-dy)) dy) and the KV footprint to O (A++1)-d, per head, both independent of sequence length. Over steps, the cost is linear in T for fixed k, while retaining enough temporal context due to inter-frame redundancy.
Combined effect. Let S denote the number of retained historical tokens (5= k++1) and Î the composed token count from Sec. 3.1. The dense baseline over L tokens costs O(B H L2 d,) per layer. With registration reducing inputs to T and the cache limited to S, the per-step cost becomes O(B-H- H. -.. and stored key-value memory is bounded by ( dp.) per head. Together, input- side registration and history-side pruning convert quadratic growth into much smaller S < L significantly reducing both operator count and memory footprint the method level.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
All experiments are conducted on single NVIDIA A100 GPU (80GB). The input image height is fixed at 518 pixels while preserving the original aspect ratio, ensuring consistent resolution across datasets. During inference, inputs are constructed as sliding windows of 10 consecutive frames, with each prediction aligned to the first frame of the window. For history compression, we retain the top-k = 5 cached frames, and for multi-view selection, the top-20 camera tokens, corresponding to roughly five neighboring views. Unless otherwise specified, models are evaluated using official Stream VGGT weights with our inference modifications. Efficiency measured in terms of infer- ence latency, GPU memory (allocated and reserved), GEMM GFLOPs, and hardware-level metrics including average power and energy per output. Accuracy is assessed on multi view reconstruction and long-sequence depth estimation tasks.

---

Published as a conference paper at ICLR 2026
Memory Usage
Sequence Length
X
Average Latency per va Sequence Length
30
WGl (Merrc))
StEEHWOST (Remary)
Car Ind
70
SouunivG2T [ncrumanc
VGGT COM
Labenty per frome (mr))
StramtvGG DGN
300
200
25
53
100
200
300
130
Sequence Langrn
Sequence Length (trames)
Figure 5: Per-frame latency vs. sequence length on tandtzband ITT datasets..
Figure 6: Reserved and incremental memory vs. sequence length. OOM indicates out-of- memory.
Table 1: Inference time (s) across different sequence lengths (OOM indicates out-of-memory).
Method
5
10
20
50
100
200
300
500
1k
10k
Ours (Inc)
0.57
1.04
2.09
4.17
8.20
16.28
25.50
38.12
82.00
797.31
Stream StreamVGGT VGGT
0.50
1.06
2.15
7.23
20.09
68.22
136.72
185.11
OOM
OOM
VGGT
0.51
0.63
1.09
2.37
5.78
17.34
36.32
OOM
OOM
OOM
4.2 EFFICIENCY EVALUATION
Latency. Latency is evaluated on Tanks-and-Temples (train and truck) for sequences up to 300 frames, and on KITTI for longer sequences (Knapitsch et al.) 2017). Per-frame latency is computed as total inference time divided by sequence length; for sequences longer than 20 frames, both the merging step and the forward pass are included. Figure[JJshows per-frame latency (ms/frame). while Table 1 indicates total sequence time (s). Our method remains nearly constant once the sequence exceeds 100 frames, whereas StreamVGGri rises sharply and VGGT also grows with length. For example, from 10 to 300 frames, our method decreases slightly from about 104 ms to 85 ms per frame, while EtreaMGGGT increases from 106 ms to 455 ms and VGGT from 63 ms to 117 ms. This demonstrates that our incremental design successfully prevents latency per frame from scaling with sequence length.
At the sequence level, Table I shows the same trend. At 100 frames, method completes in 8.20 s, compared to 20.09 s for Stream VGGT (2.4x faster) and 5.78 s for VGGT. At 300 frames, the difference is substantial:
method finishes in 25.54 s, whereas StreamVGGi requires 136.72 (5.4x), and VGGT 35.18 s. These results highlight that our incremental design delivers substantially faster inference than StreamVGG1 while maintaining comparable accuracy to VGGT and even surpassing it in some cases.
Memory Usage. Since our focus is on memory bottlenecks, we present reserved memory through- this section, it directly reflects the peak footprint that determines CUDA out-of-memory (OOM) failures. Here, OOM denotes the runtime termination when the GPU cannot allocate further memory. We further define incremental memory as the additional reserved usage incurred per for- ward step. As shown in Figure 6, both VGGT and Stream VGGT exhibit steep growth in reserved memory with longer sequences. VGGT already exceeds 60 GB 300 frames and fails thereafter, even without bundle adjustment. Stream VGGT behaves similarly, reaching more than 78 GB and crashing at 1 frames. In contrast, our method maintains nearly flat reserved memory profile across all tested lengths, staying below 9 GB even Ik frames. This stability indicates that redundancy-awwat state compression effectively breaks the quadratic growth trend that makes exist- ing approaches impractical for long sequences.
We further examine incremental memory, defined as the additional reserved usage incurred per for- ward step. Again in Figure 6, both baselines show rapidly increasing increments. with VGGT and Stream VGGT surpassing 5-20 GB once the input length exceeds 100 frames. Our method, how- ever, keeps the incremental cost tightly bounded around 2-3 GB regardless of sequence length. This

---

Published as a conference paper at ICLR 2026
Table 2: Theoretical attention operator counts (in k) computed on the KITTI dataset.
#Frames
VGGT
Stream VGGT
Ours (Inc)
5
7.19k
6,38k
6.38k
50
286.52k
163.42k
11.05k
100
951.58k
548.13k
27.39k
200
3603.63k
1981.39k
70.26k
300
7956. 14k
4299.78k
126.33k
500
OOM
11591.97K
198.88k
Table 3: Energy consumption (J) across different sequence lengths.
The last column presents the
average power consumption (W).
Method
5
20
50
100
200
300
500
1k
Avg Power (W)
Ours (Inc)
64.7
238.0
113.7
202.8
455.8
811.0
1183.2
2885.3
135.7
VGGT
62.2
105.8
513.4
1269.3
3926.9
8078.7
OOM
OOM
178.3
Stream VGGT
125.6
309.2
1119.2
3360.9
11228.2
23516.1
30503.5
OOM
153.8
bounded slope is critical: ensures that even under long streaming inputs, memory growth remains predictable and sustainable. Together, these findings demonstrate that approach is not only ef- ficient in terms of total footprint but also robust scaling behavior, enabling reliable operation on GPUs where existing models would immediately of low reserved footprint and bounded incremental cost highlights that our method is well-suited for edge cases and commodity GPUs, where both absolute capacity and memory headroom are highly constrained.
out of memory. More broadly, the combination
Computation efficiency. To further analyze computational efficiency, estimate the number of attention operators, which contribute over 95% of the total FLOPs (Sreedhar et al., 2022). We present theoretical values derived from matrix dimensions, focusing only on the attention compo- nent. As shown in Table [ our method reduces operator counts dramatically compared to VGGT and VGGT. For example, 300 frames, VGGT requires about 7,956k attention operators and Stream VGGT about 4,300k, whereas our method uses only 126k-over 60x fewer than VGGT and 30x fewer than Stream This significant reduction directly explains the improvements in latency and energy, since far fewer redundant attention computations are executed while reconstruc- tion quality is preserved.
Energy Efficiency. Beyond memory and latency, further examine hardware-lewe efficiency in terms of energy consumption (Cao et al. 202T). As summarized Table 3 our method consistently requires substantially less energy than both VGGT and Stream VGGT across all sequence lengths. While the average power draw is slightly lower than competing baselines (135.7 W vs. 153.8 and 178.3 W), the key advantage comes from significantly shorter runtimes, which translate into much lower energy pergg--pe--eeuence. For instance, at 300 frames our method consumes only 811 J, compared 8079 for VGGT and over 23,000 J for VGGT. This oft-oordrrof--maniieeeee as the sequence length increases. Since all methods are evaluated on the same sequence lengths, total energy directly comparable; equivalently, dividing by frame count yields consistent J/frame trends. These results indicate that our redundancy-awert also reduces wasted GPU cycles, As a result, this energy efficiency makes the approach especially appealing for resource-constrained settings, such as commodity GPUs edge devices, where both memory and power budgets are limited. In practice, this means our model can sustain Long-sequence reconstruction tasks hardware platforms where other methods would either exhaust memory or exceed acceptable power envelopes, broadening the applicability of foundation models in 3D vision.
gap widens
design not only conserves memory but
4.3 ACCURACY EVALUATION
We evaluate accuracy on video depth and multi-view reconstruction, as single-frame and camera pose estimation nearly identical Stream VGGT due shared pretrained weights. Our goal is therefore to confirm that the efficiency gains described in Section a.2 achieved while maintaining accuracy at comparable level, without incurring significant degradation.
8

---

Published as a conference paper at ICLR 2026
Table Video depth evaluation on Sintel, Bonn, and KITTI datasets. Results for Stream VGGT and obtained from our evaluation, while others are derived from SGGT 8<1.22 is reported as a percentage.
Method
Type
Sintel
Bonn
KITTI
Abs Rel
1.225 t
Abs Rel
8<1.25
Abs Rel 4
8< 1.22
VGGT
Dense-view
0.298
68.1
0.057
96.8
0.061
97.0
Spann3R
Streaming
0.622
42.6
0.144
81.3
0.198
73.7
CUT3R
Streaming
0.421
47.9
0.078
93.7
0.118
88.1
Point3R Stream VGGT
Streaming
0.452
48.9
0.060
96.0
0.136
84.2
Streaming
0.328
64.9
0.059
97.2
0.173
72.2
Ours (Ine)
Streaming
0.341
63.1
0.064
97.0
0.176
71.9
Table Multi-view reconstruction on 7Scenes and NRGBD. Include median accuracy (Acc), com- pleteness (Comp), and normal consistency (NC1).
Method
7Scenes
NRGBD
Acemed +
Compmed + NC1 med
Accmed +
Compmedd
NC1 Iee
VGGT
0.0055
0.0067
0,948
0.0139
0.0147
0.993
Stream VGGT
0.0241 0.0266
0.0203
0.915
0.0520
0.0340
0.988
Ours (Ine)
0.0203
0.901
0.0516
0.0345
0.987
Video Depth. Table shows video depth results Sintel (Butler al. 2012), Bonn (Palazzolo et al. 2019). and KITTI. Across datasets, our accuracy remains very close to StreamVGGT1 with relative differences typically within 1-3%. For example, on Sintel the Abs Rel differs by only about 4%, and on KITTI the gap is below 2%. On Bonn, both methods achieve nearly identical 1.25 scores around 97%. These small gaps indicate that the efficiency gains of our method are realized with only minimal loss in accuracy, while still outperforming other streaming baselines such as Spann3R, CUT3R, and Point3R by a clear margin.
Multi-view Reconstruction. Table results 7Scenes (Shotton et al. 2013) and NRGBD (Azinović et al. 2022). VGGT achieves the lowest errors overall, but is a non-streaming baseline and cannot scale beyond short sequences. Among streaming models, the accuracy (Acc) metric shows complementary strengths: on 7Scenes, Stream attains slightly lower Acc, while on NRGBD method achieves marginally better Acc. Other metries remain largely comparable, and in all cases the differences are within 0.3%, indicating that our approach matches the reconstruc- tion quality of the best streaming baseline, while uniquely offering the efficiency and scalability demonstrated in Section 4.2
DISCUSSION
We focus on a training-free, redundancy-aware inference framework that remains efficient in ex- tremely long sequences. This design offers strong scalability and memory savings but also intro- duces natural trade-off. When the overlap ratio becomes very small or decreases rapidly, such as in scenes with large viewpoint changes or limited texture, the reduced geometric redundancy can influ- ence the stability of the registration module. We also discuss two possible modes for long-sequence reconstruction: training-free and finetuning. We choose the training-free mode because VGCVGTT already preserves comparably high level of accuracy, leaving limited room for finctuning to pro- duce substantial gains, and because the training- free setting keeps the method architecture-agnose and directly applicable to arbitrarily long inputs. Finetuning remains compatible option through existing VGGT and Stream VGGT pipelines, but it is orthogonal to the core focus of this work. These choices reflect practical compromises that allow the system to stay lightweight and highly memory efficient.
CONCLUSION
We presented VGGT an incremental and memory-efficient visual geometry transformer for long- range 3D reconstruction. Our design collapses redundant input views into compact composites

---

Published as a conference paper at ICLR 2026
and prunes the history cache with pop-k-plus-reeeccy rule. This ensures both operator count and memory footprint remain bounded while preserving awareness of global and local spatial context. Experimental results show that VGGT outperforms Sfateoof-the-an methods by nearly an order of magnitude. VGGT scales effectively to long sequences while preserving accuracy, avoiding the memory and latency limitations of existing baselines. More broadly, VGGT demonstrates that redundancy-awarr design can make transformer-based reconstruction practical for VR/AR, robotics, and edge devices, enabling reliable deployment real-world long--equent sequence scenarios.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant Nair, llya Soloveychik, and Pu- rushotham Kamath. Keyformer: Kv cache reduction through key tokens selection for efficient generative inference. Proceedings of Machine Learning and Systems, 7, 2024.
Anubhav Agarwal, C. V. Jawahar, and
survey of planar ho- mography estimation techniques. Technical Report IITT/T//005/12, Centre for Vi- sual Information Technology, International Institute of Information Technology. Hyderabad, 2005. URL A s_survey_oo of
Narayanan.
planar Lareveyy___plaaaa_____
Cédric Allène, Jean- Jean-Philippe Pons, and Renaud Keriven. Seamless image-based based texture atlases using multi-band blending. In Proceedings of the International Conference on Pattern Recognition (ICPR), 2008.
Dejan Azinović, Ricardo Martin Martin-Brualla, Dan B Goldman, Matthias Nießner, and Justus Thies. Neural rgb-d surface reconstruction. In CVPR, 2022.
Matthew Brown and David G. Lowe. Automatic panoramic image stitching using invariant features. volume 74, 2007. doi: 1007// 1263-006-0002-3.
D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In A. Fitzgibbon et al. (eds.), European Conference Computer Vision (ECCV), LNCS 7577, Part IV, pp. 611 611-525. Springer-Verlag. October 2012.
Qingging Cao, Yash Kumar Lal, Harsh Trivedi, Aruna Balasubramanian, and Niranjan Bala- subramanian. Irene: Interpretable energy prediction for transformers. In ACL. Association for Computational Linguistics, 2021. doi: 10.18653/v 1/2021..cl-ooon..666 URL https:
//aclanthology
org//o2l...........o htt.hhhooooyyr.gg/....... acl- long.
Zihang Dai, Zhilin Yang. Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019. URL
https: /ttp::///axvvv.oo/ org/ abs /tbs/1901.. 02860
Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08699/ 2023.
Kai Deng, Zexin Tí, Jiawei Xu, Jian Yang, and Jin Xie. Vggt-long: Chunk it, loop align it pushing vggt's 's limits on kilometer-scale long rgb sequences. arXiv preprint arXiv:2507. 16443, 2025.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021. arXiv preprint arXiv:2010.11929
Qichen Fu, Minsik Cho, Thomas Merth, Sachin Mehta, Mohammad Rastegari, and Mahyar Na- jibi. Lazyllm: Dynamic token pruning for efficient long context 11m inference. arXiv preprint arXiv:2407.14057 2024. doi: 10448550/arXiv.2477.4453
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012.
Amirhossein Habibian, Davide Abati, Taco S. Cohen, and Babak Ehteshami Bejnordi. Skip- convolutions for efficient video processing. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2021.
Jin Gyu Hong, Seung Young Noh, Hee Kyung Lee, Won Sik Cheong, and Ju Yong Chang. 3d clothed human reconstruction from sparse multi-view images. In Proceedings of the IEEÉ/CVF Conference Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.

---

Published as a conference paper at ICLR 2026
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications, 2017. arXiv preprint
Xiaoshui Huang, Guofeng Mei, Jian Zhang, and Rana Abbas. A comprehensive survey on point cloud registration. URL wtttttss://aaattt org/ abs / 2103. 02690
Heekyum Kim and Yuchul Jung. Entropy-guided kv caching for efficient Ilm inference. Mathemat- ics, (3(),) 2025. doi: 0..3390/math33155355
Arno Knapitsch, Jaesik Park, Qian-Y Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions Graphics, 36(4), 2017. doi: 10.1 3072959.3073599.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng. Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles (SOSP), 2023. doi: 10.1 00.11455/360000033611166. URL https://arxix:
Skg/aba/2300 06180
Vincent Leroy, Yohann Cabon, and Jerome Revaud. Grounding image matching in 3d with mast3r. In Proceedings of the European Conference on Computer Vision (ECCV), 2024.
Dongping Li, Kaiming He, Jian Sun, and Kun Zhou, geodesic-preserving method for image warping. In CVPR, 2015.
David G. Lowe. Distinctive image features from ocale-invarian keypoints. International Journal of Computer Vision, 60(2), 2004.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. In ECCV, 2020.
Onur Ozyeşil, Vladislav Voroninski, Ronen Basri, and Amit Singer. A survey of structure from motion. Acta Numerica, 26, 2017. doi: 10. 10.1017/50024999110000
Emanuele Palazzolo, Jens Behley, Philipp Lottes, Philippe Giguère, and Cyrill Stachniss. Refu- sion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), 2019. doi: 10. N0.1109//R0S4408970000 201199998997209999797999
Weize Quan, Jiaxi Chen, Yanli Liu, Dong-Ming Yan, and Peter Wonka. Deep learning-based image and video inpainting: survey. International Journal of Computer Vision, 2024. doi: 10.1007/ sl 111223-023-01977--
Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. Orb: An efficient alternative to sift or surf. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2011.
Utkarsh Saxena, Gobinda Saha, Sakshi Choudhary, and Kaushik Roy. Eigen attention: Attention in low-rank space for ky cache compression. arXiv preprint arXiv:2408.05646. 2024.
Jamie Shotton, Ben Glocker, Shahram Izadi, and Antonio Criminisi. Scene coordinate regression forests for camera relocalization in rgb-d images. In CVPR, 2013.
Kavya Sreedhar. Jason Clemons, Rangharajan Venkatesan, Stephen W. Keekler, and Mark Horowitz. Vision transformer computation and resilience for dynamic inference. arXiv preprint arXiv:2212.02687, 2022.
Niko Sünderhauf, Oliver Brock, Walter Scheirer, Raia Hadsell, Dieter Fox, Jürgen Leitner, Ben Upcroft, Pieter Abbeel, Wolfram Burgard, Michael Milford, and Peter Corke. The limits and potentials of deep learning for robotics. arXiv preprint arXiv:1804.06553 2018.
Alexandru Telea An image inpainting technique based on the fast marching method. Journal of Graphics Tools, 9(1), 2004.
12

---

Published as a conference paper at ICLR 2026
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurlPS, 2017.
Fangjinhua Wang, Qingtian Zhu, Di Chang, Quankai Gao, Junlin Han, Tong Zhang, Richard Hartley, and Marc Pollefeys. Learning-based multi-view stereo: survey, arXiv preprint arXiv:2408. 15235, 2024a.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. Proceedings of the IEEE/CVF Con- ference Computer Vision and Patlern Recognition (CVPR), 2025.
Qianqian Wang al. Cut3r: Continuous 3d perception model with persistent state. URL https:
//
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024b.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. International Conference on Learning Representations (ICLR), 2024.
Jihun Yoon and Min-Kook Choi. Exploring video frame redundancies for efficient data sampling and annotation in instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2023.
Jiahui Zhang, Yuelei Li, Anpei Chen, Muyu Xu, Kunhao Liu, Jianyuan Wang, Xiao-Xiao Long, Hanxue Liang, Zexiang Xu, Hao Su, Christian Theobalt, Christian Rupprecht, Andrea Vedaldi, Hanspeter Pfister, Shijian Lu, and Fangneng Zhan. Advances in forward reconstruction and view synthesis: A survey.
/507 14501
Youpeng Zhao, Di Wu, and Jun Wang. Alisa: Accelerating large language model inference via 17312
sparsity-aware kv caching, 2024. URL htt.atts//aaaaa..orr////s/////
Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, and Yebin Liu. Gps-gaussian: Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Dong Zhuo, Wenzhao Zheng, Jiahe Guo, Yuqi Wu, Jie Zhou, and Jiwen Lu. Streaming 4d visual geometry transformer. arXiv preprint 213., 2025.
Barbara Zitová and Jan Flusser. Image registration methods: survey. Image and Vision Computing, 21(11), 2003.
13

---

Published as a conference paper at ICLR 2026
A ADDITIONAL QUALITATIVE VISUALIZATIONS
Figures 7 and provide additional qualitative comparisons between the proposed IncVGGT the prior Stream PereamVGGT. VGGT, and VGGT-Long on long-sequence reconstruction (On KITTI dataset). As shown, Inc VGGT produces more complete scene geometry with clearer high-frequency structures such as trees, lamp posts, and roadside details. This improvement mainly comes from reducing the large amount of near-duplicate frames present in real long streams; our registration module merges these highly redundant views, which reduces conflicting signals, prevents the backbone from generating scattered outliers, and avoids the loss of geometry that can occur when the KV-cache becomes overloaded.
Figure 7: Qualitative reconstruction produced by IncVGGT.
Figure 8: Reconstruction produced by StreamVGGT
14

---

Published as a conference paper at ICLR 2026
Figure 9: Reconstruction produced by VGGT-Long (The visualization is taken from the original VGGT-Long paper).
15


Reviews:
[{"review_id": "WwbSAMW0Zp", "reviewer": "Reviewer_AxtP", "summary": "This paper introduces IncVGGT, a training-free, incremental variant of VGGT designed for memory-bounded, long-range 3D reconstruction. The method tackles the severe memory and computational scaling issues of existing transformers through a dual strategy: (1) it registers and fuses overlapping input frames into \"composite views\" to reduce input token redundancy , and (2) it implements a global-local cache pruning rule, retaining only the top-k most relevant slots plus the most recent one to bound history cache growth. This approach enables the model to process arbitrarily long sequences (e.g., 10k frames) where baselines fail due to out-of-memory errors , achieving substantial gains in efficiency (memory, speed, and energy) while maintaining comparable accuracy.", "questions": "- The global-local cache pruning retains a very small, fixed-size cache ($k=5$). While this bounds memory, how does it affect the model's ability to handle long-range dependencies, such as loop closure or revisiting a previously mapped area? Does this small $k$ effectively limit the model to only local, short-term consistency?\n- The method is applied as a training-free modification to StreamVGGT weights. Have the authors investigated whether fine-tuning the model *with* the proposed input-merging and cache-pruning mechanisms could recover the accuracy lost to this aggressive, non-differentiable pruning? It seems plausible that this would yield a model that is both efficient and as accurate as the baseline.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The primary strength of this work is that it directly addresses the most significant barrier to using transformers for real-world 3D reconstruction: prohibitive memory and computational cost on long sequences. The ability to run on 10k-frame sequences while state-of-the-art baselines fail at 300-500 frames is a practical achievement.\n\n- The efficiency improvements are not marginal but substantial, with orders-of-magnitude reductions in operator count, memory usage, and energy consumption. The paper provides thorough experimental validation for these claims, including latency, memory, computation, and energy metrics , which strongly support the method's value.\n\n- The approach is \"training-free\", allowing it to be applied directly to existing models like StreamVGGT. This significantly lowers the barrier to adoption. Crucially, the paper demonstrates that these massive efficiency gains are achieved while \"maintaining comparable accuracy\", with only minimal performance drops in the reconstruction tasks.", "weaknesses": "-  The method introduces two critical, hard-coded hyperparameters: the span gate threshold $\\lambda$ and, more importantly, the cache size $k=5$. The entire efficiency-accuracy trade-off hinges on $k$. The paper provides no ablation study to justify $k=5$ or to show the sensitivity of the model's accuracy and memory footprint to this parameter.\n- While the efficiency gains are dramatic, the accuracy tables show a consistent, albeit small, degradation in quality compared to the StreamVGGT baseline. The paper presents this as \"comparable\", but it is a clear trade-off. This accuracy drop is likely a direct result of the \"training-free\" approach, as the model was never trained to be robust to such aggressive input and cache pruning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YjeeRb9Axg", "reviewer": "Reviewer_EqRK", "summary": "This work presents IncVGGT, a training-free incremental variant of VGGT designed to address the quadratic memory growth and excessive computation issues in large-scale visual geometry transformers. The method tackles redundancy from both the input and history sides: on the input side, it performs registration and composition to merge short temporal windows into compact composite views; on the history side, it retains only the top-k historically important slots together with the most recent one for the next step. Experimental results demonstrate notable improvements in both inference time and memory efficiency.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well written and easy to follow, with clear figures and publicly released code.\n\n- It proposes a simple yet intuitive solution with solid technical contributions.\n\n- The method offers practical value by effectively reducing memory and computation overhead.\n\n- IncVGGT operates in a training-free manner, making it efficient and convenient for deployment.", "weaknesses": "- The registration-based redundancy reduction component has been widely applied in the community (e.g., [1]). Moreover, introducing a separate registration-and-composition preprocessing module somewhat compromises the clean and unified design philosophy of VGGT, which originally aimed to eliminate all explicit priors.\nA more elegant approach would be to integrate the registration and composition mechanism directly into VGGT’s internal architecture, preserving its end-to-end structure.\n\n- Limited Novelty: While the contributions toward improving memory and speed are meaningful for industrial and practical applications, the registration-based redundancy reduction and global-local cache pruning techniques are incremental relative to VGGT. The authors themselves describe IncVGGT as an “incremental variant” of VGGT, suggesting that it contributes less conceptual novelty to academic research, even though it provides clear engineering value.\n\n[1] Xiaoshui Huang, Guofeng Mei, Jian Zhang, and Rana Abbas. A Comprehensive Survey on Point Cloud Registration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NyZrZUlvGD", "reviewer": "Reviewer_bfom", "summary": "This paper introduces IncVGGT, an incremental and training-free inference framework for theVGGT to enable memory-bounded, long-range 3D reconstruction. The core contribution is a novel two-pronged approach to mitigate the quadratic complexity and ever-growing memory footprint of standard and streaming Transformers. First, an input-side registration and composition module merges overlapping frames into compact composite views, significantly reducing the number of input tokens. Second, a history-side pruning mechanism for the KV cache retains only a fixed-size set of the top-k most relevant historical slots plus the most recent one, effectively converting a linearly growing cache into a constant-size one. \nExperimental results demonstrate that IncVGGT achieves order-of-magnitude improvements in efficiency over state-of-the-art baselines like StreamVGGT, while maintaining comparable accuracy.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well writen and intuitive.\n2. Inc-VGGT proposes an elegant two-pronged solution combining input-side frame composition and history-side KV cache pruning.\n3. Inc-VGGT achieves massive efficiency gains with only negligible degradation in accuracy, maintaining performance comparable to the state-of-the-art streaming baseline.", "weaknesses": "1. Lack of Qualitative Visualizations: The paper relies solely on quantitative metrics to evaluate reconstruction accuracy. For a vision task like 3D reconstruction, this is insufficient as metrics can sometimes fail to capture important visual artifacts. Providing qualitative visualizations comparing the reconstructed scenes against baselines and ground truth would be essential for a comprehensive assessment of the model's precision.\n\n2. Absence of a Limitations Analysis: The paper does not include a dedicated section or discussion on the limitations of the proposed method. A thorough analysis of potential failure modes—such as scenes with significant 3D parallax where homography is fragile, or low-texture environments where feature matching may fail—would make the paper more complete and provide a clearer picture of its applicability boundaries.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tVtMMT0a0t", "reviewer": "Reviewer_dTAF", "summary": "This paper addresses VGGT's quadratic memory growth that limits it to 300-500 frames for transformer-based 3D reconstruction. IncVGGT proposes a training-free solution with two strategies: (1) input-side registration fuses overlapping frames into composite views using homography-based alignment before tokenization, and (2) history-side pruning retains only top-k relevant cached slots plus the most recent frame. At 500 frames, this achieves 58.5× fewer operators, 9× lower memory, 4.9× faster inference, and scales to 10k frames with only 1-4% accuracy degradation.", "questions": "- Can you provide ablations showing accuracy vs. efficiency tradeoffs for different cache sizes k ∈ {1, 3, 5, 10, 20}, sensitivity to span threshold \\lambda, and contribution of each component (registration alone vs. pruning alone vs. combined)?\n- You cite VGGT-Long as explicitly addressing kilometer-scale sequences but provide no quantitative comparison. Can you directly compare on the same datasets/sequences and explain why your streaming approach is preferable to their chunk-based method?\n- For 10k-frame sequences, can you provide accuracy metrics plotted against sequence length (e.g., at 1k, 2k, 5k, 10k frames) to demonstrate that errors don't compound over time? How do homography drift and cache pruning affect long-term geometric consistency?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses memory scalability in transformer-based 3D reconstruction, a genuine problem that limits the deployment of foundation models in real-world applications (VR/AR, robotics, autonomous systems).\n- The quantitative gains are substantial and well-documented across multiple dimensions: 58.5× fewer operators, 9× memory reduction, 25.7× energy savings, and demonstrated scalability to 10k frames.\n- The key insight that video redundancy exists at both input (overlapping pixels) and history (cached attention) levels is conceptually clean.", "weaknesses": "- Both core components rely on well-established techniques, with little innovation. Input-side registration uses standard homography-based panorama stitching (Brown & Lowe, 2007), while history-side pruning applies straightforward top-k selection to KV caches, similar to existing work in LLMs (H2O, StreamingLLM, and Keyformer).\n- The homography-based registration fundamentally assumes planar scenes or negligible parallax, yet this critical limitation receives only brief acknowledgment (line 190: \"cannot fully account for strong 3D parallax\") without rigorous analysis. The paper provides no quantitative evaluation of when/how this assumption breaks down, no testing on datasets with significant depth variation or non-planar geometry, and no failure case examples or degradation analysis.\n- Tables 4-5 show 1-4% accuracy drops that are dismissed as \"minimal\" without investigating root causes or conditions under which degradation occurs. The paper lacks ablation studies on critical hyperparameters, including span threshold \\lambda, cache size k (only k=5 tested), and window size K, making it impossible to understand the sensitivity to design choices or to optimize accuracy-efficiency trade-offs. Most critically, there is no analysis of error accumulation over the claimed 10k-frame sequences. Do homography estimation errors compound over time, and how does cache pruning affect long-term geometric consistency?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VDNoAEVyhA", "reviewer": "Reviewer_iPad", "summary": "The paper presents IncVGGT, a training-free, streaming variant of VGGT for very long videos under tight memory/compute budgets. It improves scalability via (1) input-side registration & composition, which aligns short windows and fuses overlapping frames into composite views, and (2) history-side cache pruning, retaining only the top-k most relevant cached slots plus the most recent one to avoid key–value growth. Built from StreamVGGT pretrained weights, IncVGGT has substantial latency, memory, and energy reductions compared to StreamVGGT/VGGT. On camera pose and multi-view reconstruction, it performs on par with StreamVGGT but remains below VGGT in accuracy.", "questions": "*  After composite-view inference, how do you recover per-frame camera poses and per-pixel depths in each original image? \n* What happens when inputs are out-of-order or have limited overlap so composites can’t be formed—do you fall back to StreamVGGT, and with what cost/accuracy?\n* Could authors provide long-sequence accuracy benchmarks (not just efficiency) to show performance at the scales your method targets (1K, 10K frames)?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "* **Scales to very long videos**: Handles sequences up to thousands of frames under tight memory/compute, enabling long-range reconstruction without retraining.\n* **Simple approach**: (i) Registration + composite views cut redundant tokens before attention; (ii) Top-k + recent cache pruning bounds KV growth.", "weaknesses": "* **Strong assumptions for input-side composition**. The composition step implicitly requires frames to be in temporal order and to have large inter-frame overlap; if a long sequence contains only partial overlaps (or out-of-order frames), composites cannot be formed and the method effectively falls back to using all frames (akin to StreamVGGT).\n* **Unspecified “un-stitching.”** After composing views, the paper does not explain how to map predictions back to original frames—e.g., recovering per-pixel depth and per-frame camera poses in the native image coordinates—which is critical for practical use.\n* **Lack of long-sequence accuracy benchmarks**. While the paper shows large latency/memory reductions on long sequences, it reports quantitative accuracy only on short-sequence settings; there is no long-range accuracy evaluation (camera pose, video depth estimation), leaving the real effectiveness on very long videos unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)