VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold
Authors: Dominic Rosario Maggio
Hyungtae Lim
Luca Carlone
Venue: NeurIPS 2025 poster ()
Keywords: RGB SLAM
Feed Forward Mapping
Pose Graph Optimization on Manifold
Primary area: applications
TL;DR: A dense RGB SLAM system constructed by aligning submaps from VGGT, optimized on the SL(4) manifold.

Abstract: We present VGGT-SLAM, a dense RGB SLAM system constructed by incrementally and globally aligning submaps created from the feed-forward scene reconstruction approach VGGT using only uncalibrated monocular cameras. While related works align submaps using similarity transforms (i.e., translation, rotation, and scale), we show that such approaches are inadequate in the case of uncalibrated cameras. In particular,  we revisit the idea of reconstruction ambiguity, where given a set of uncalibrated cameras with no assumption on the camera motion or scene structure, the scene can only be reconstructed up to a 15-degrees-of-freedom projective transformation of the true geometry. This inspires us to recover a consistent scene reconstruction across submaps by optimizing over the SL(4) manifold, thus estimating 15-degrees-of-freedom homography transforms between sequential submaps while accounting for potential loop closure constraints. As verified by extensive experiments, we demonstrate that VGGT-SLAM achieves improved map quality using long video sequences that are infeasible for VGGT due to its high GPU requirements. Our code is available at https://github.com/MIT-SPARK/VGGT-SLAM.

VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold
Dominic Maggio Hyungtae Lim" Luca Carlone Massachusetts Institute of Technology (drmaggio, shapelim, Icarlone] @mit edu
Abstract
We present VGGT-SLAM, dense RGB SLAM system constructed by incre- mentally and globally aligning submaps created from the feed-forward scene reconstruction approach VGGT using only uncalibrated monocular cameras. While related works align submaps using similarity transforms (i.e., translation, rotation, and scale), show that such approaches ibrated cameras. In particular, we revisit the idea of reconstruction ambiguity, where given a set of uncalibrated cameras with no assumption on the camera motion or scene structure, the scene can only be reconstructed up to 15-degrees- of-freedom projective transformation of the true geometry. This inspires recover consistent scene reconstruction across submaps by optimizing over the SL(4) manifold, thus estimating 15-degrees-of-freee of-edeeoomoo homography trans- forms between sequential submaps while accounting for potential loop closure constraints. As verified by extensive experiments, we demonstrate that VGGT- SI.AM achieves improved map quality using long video sequences that are in- feasible for VGGT due to its high GPU requirements. Our code available https://githinn com MIT - SPARK/ VGGT-SLAN
inadequate in the case of uncal-
Introduction
One of the most fundamental tasks in computer vision is that of simultaneous localization and mapping (SLAM) where given multiple monocular (or stereo) images, the task is to generate 3D
reconstruction of the scene and estimate the b-degres--offfreedoo (DOF) pose of the cameras. Most
approaches for this have traditionally leveraged classical multi-view geometry constraints 47
25 80
81. data association 38 37), and backend optimization such as bundle adjustment 52
53 57
[
58
Recently, new paradigm of using simpler, feed-forward networks, which produce point clouds from uncalibrated input images, has gained increasing popularity. In this thrust, the seminal work DUS(3R (73] takes pair of images and estimates dense point clouds both images in the reference frame of the first camera, thus creating a dense scene reconstruction and allowing the camera poses estimated easily with a 3-point RANSAC solver
59
40)
To extend feed-forwar reconstruction to multiple images, VGGT (Visual Geometry Grounded Transformer) [7I] takes in an arbitrary number of images, and in addition estimating dense point clouds of each camera frame, also estimates depth maps, feature tracks, and camera poses and intrinsics. However, VGGT limited in the number of images that can be processed by GPU memory. For example, in the case of an NVIDIA GeForce RTX 4090 with 24 GB, this is limited to approximately 60 images, making larger reconstructions requiring hundreds or thousands of images infeasible.
"Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Sim
SL(4)
Figure I: VGGT-SLAM alignment of submaps created from VGGT using Sim(3) alignment (top) and (L) alignment (bottom). Here, Sim(3) insufficient to align submaps due to projective ambiguity, motivating our SL(4)- SLAM. Experiments performed on a segment of the Clio [39] apartment and cubicle seenes.
One may suspect that simple, trivial solution would be create multiple submaps with VGGT where each submap contains at least one overlapping image, and solve for the scale parameter between submaps (as the reconstruction does not capture metric scale), with VGGT's estimated poses being used to align rotation and translation (i.e., estimating a Sim(3) transformation between submaps). While we demonstrate Sim(3) optimization shows impressive reconstructions in many cases, we empirically observe that the feed-forward nature of VGGT with uncalibrated cameras introduces projective ambiguity, which in addition the Si) DOF includes shear, stretch, and perspective DOF, especially when the disparity between images becomes small. This ambiguity cannot be fully resolved through a similarity transformation alone.
Unveiling why a similarity transformation is sometimes insufficient for this recent transformer-base scene reconstruction method causes us return to classical computer vision for answers, specifically the notion of projective ambiguity. To rectify projective reconstruction to a metric reconstruction requires computing 4 x 4 homography matrix (26) which can be mapped to the Special Linear, (4(,), Lie group. Since this is a Lie group, we can formulate the submap alignment problem as factor graph optimized on the (4) manifold globally align an arbitrary number of submaps given both estimates of relative homographies between sequential submaps and added constraints from detected loop closures.
Contributions Firstly, we present the first SLAM system that leverages the feed-forward scene re- construction capabilities of VGGT (71), extending it to large-scale scenes that cannot be reconstructed from a single inference of VGGT. Our system operates entirely with monocular RGB cameras and does not require known camera intrinsics consistent calibration across frames. Importantly, achieves this without any additional training.
Secondly, while Sim(3) optimization is often sufficient, we identify and analyze scenarios where projective ambiguity arises, as presented in Fig. [ In these cases, conventional similarity transforms do not fully resolve scale and alignment issues. We highlight this limitation and demonstrate how incorporating projective constraints addresses the problem.
Finally, we propose the first factor graph formulation that operates directly on the SL(4) manifold to address projective ambiguity. Even in practical scenarios, where projective ambiguity is less dominant, we show that ST.(()- optimization achieves performance competitive with or superior to other sfate-of-the-arr learning- SLAM approaches, offering a principled framework for handling cases where similarity transformations are insufficient.

---

2 Related Work
Classical Scene Reconstruction Classical scene reconstruction methods typically rely on geometric
features to estimate camera poses and reconstruct 3D scenes from multi-view images [45] S 19 I1
56 using 32 75] bundle adjustment [58 50], by performing sparse feature extraction, matching, and robust pose estimation by optimizing for SE(3) transformations. Several works have also performed
dense SLAM [00 76 41 777, and O provides a survey on SLAM. Multiple works have also
performed classical projective scene reconstruction using uncalibrated cameras [68] (27). Recently, Sim-Sync (78 introduced a certifiably optimal algorithm that jointly estimates camera poses and per-image scaling factors that leverage pretrained monocular depth predictions.
Feed-forward Scene Reconstruction The seminal work DUSt3 73 has spawned multiple followup works in feed-forward scene reconstruction. DUSt3R takes in a pair of images and for each image, outputs dense point map in the reference frame of the first camera. From the point maps, the camera focal lengths can be estimated using the Weiszfeld algorithm [5]] and poses can be recovered using multiple methods such as 3-point RANSAC [48 40]. MASt3R follows a similar design but also outputs descriptors that can be used to generate pairwise correspondences between the two frames. MASt3R-SFIT [15] demonstrates global optimization of multiple images using MASt3R but computation scales quickly with the number of frames.
To extend the idea of DUS:3R to multiple frames, Spann3R 77 leverages a learned memory module and Cut3R 772 uses a recurrent state model. Both can incrementally reconstruct scene using multiple images, but are each limited to short sequences. Recently, Pow3R (2 extends the DUSt3R framework to optionally take in any estimates of any combination of camera intrinsics, poses, and depth (which may be sparse or dense) and demonstrates substantial improvement in scene reconstruction and pose estimation given the added inputs. Splatt3R [22 extends the DUSt3R idea to Gaussian Splatting [31] by directly outputting the Gaussian Splatting parameters given two views, and PreF3R 8 extends this to multiple views using similar memory framework as Spann3R. Reloc3r [4] modifies the DUS:3R framework to directly output relative camera poses and uses motion averaging to recover absolute poses with respect map database.
Most similar to ours is MASUR--SAA 46). MAS03R-SLAN leverages MAS(3R construct an impressive real time dense monocular SLAM system that does not require known calibration. Their pipeline also includes efficient optimization over (3) poses and loop closures. Since MASt3R is limited to two input frames at a time, here, we desire to build on top of the more powerful VGGT architecture for SLAM system which can leverage broader information of the scene by taking an arbitrary number of frames for feed-forward reconstruction (bounded by computational limits) and provides direct estimates of camera poses. However, as mentioned, fusing submaps from VGGT goes beyond traditional point cloud registration problem as alignment cannot be effectively performed with only a similarity transformation. Unlike MASGR--LAM,, as will be discussed Sec. 4.2 we do not need to estimate correspondences between frames.
An alternative paradigm, scene coordinate regression, with works such as ACE 3 and DSAC* estimates world points from images with respect to a global scene frame by using a scene specific trained network.
Optimization over the Special Linear group To the best of our knowledge, we are the first work create a factor graph optimization for point cloud alignment on the SL((4 manifold. Prior works use optimization on the SL(3) manifold (corresponding to the 8-DOF homography matrix commonly used in image alignment) for aligning multiple images for panoramic stitching (2
and dense SLAM [35]. The 15-DOF homography matrix is used for classical tasks such as auto- and good practices for estimating homography are extensively studied in 24
calibration 23
3 Review: VGGT
Here, we provide the relevant preliminaries of VGGT [71]. VGGT takes as input an image set I= (M..... MM which consists of u images, tokenizes them with a fine-tuned DINO [49] backbone, and then applies Eneematiig-AAtentiion (alternating between applying global and frame- wise attention). The output tokens can then be passed camera head to estimate intrinsics and camera poses (defined with respect to the first frame), or to Dense Prediction Transformer (DPT)
3

---

heads [54], which outputs dense depth maps for each image, dense point map (where the points of each camera are defined with respect to the first camera), and dense features for point tracking, with confidence estimates provided for each.
In this paper, (C,,-.., J J (as they are fully dense, the width and height of corresponding components in M. D, and C are the same). We refer to the outputs from each Z as submap S, which will correspond to node in pose graph optimization for VGGT-SLAM. We do not run the 3D point DPT head as it was observed in Wang et al. [7]] that more accurate point clouds can be achieved by inverse projecting D using projection matrices from the camera head, giving us a dense point cloud which is defined with respect to the coordinate frame of the first camera in I. We denote this point cloud as X To filter unreliable points, we prune points whose associated confidence values in the confidence maps are less than Teeal of the average confidence across C.
the dense depth maps D= (D
DDa) and confidence score maps C
4 VGGT-SLAM
Here we describe the design of our VGGT-SLAM system. In Sec. 4.1 we determine how to generate a list of images that will be passed to VGGT to produce a local submap, S. In Sec. 4.2 we provide a discussion of projective ambiguity and show how we can align two overlapping submaps by estimating relative 15-DOF homography matrix between sequential submaps, and in Sec. 4.3 describe the process of adding loop closure constraints between non-sequential submaps. Finally, in Sec. 4..4 we show how can globally optimize all submap alignments into consistent map by optimizing on the SL(4) manifold.
4.1 Incremental submap- based keyframe selection and generation
First, we begin by describing how to incrementally construct submaps and organize keyframes within each submap from sequentially incoming images. For this, we construct an image set Tiatest- As is typical in visual SLAM [57 46, 644, we select an image as a keyframe if disparity (which we estimate using Lucas- Kanade (38) with respect to the previous keyframe is larger than a user-defined threshold Tdisparity. Even though VGGT demonstrates monocular depth capabilities [71] from learned priors, utilizing images with sufficient disparity improves relative depth estimation performance as adds multi- view information and additionally reduces the number of images process.
If sufficiently high disparity is estimated, the current image is designated a keyframe and added to a list of images, Iaatest until the size of the list reaches set limit w. In addition to Iiatest each submap's s associated image set is constructed by concatenating two additional sets of images. The first set includes single image chosen as the last non-loop-closure image from the previous submap, denoted as Mprior- Up to Whoop images to be used for loop closures (discussed in Sec. (4.)) may also be appended at the end of the collection, forming the final image set for the submap as Diatest + [Mprior) } UJjtestt TToop. This image set is then passed to VGGT to generate the submap, Sjstest.
4.2
Local submap alignment addressing projective ambiguity
which have point clouds S and x in their respective local submap frames, our objective is to solve for a transfor- mation, H E 4x4 that aligns the two submaps such that for any noise-free corresponding points XS.
Given two overlapping submaps S; and Sj generated as described in Sec. 4.1
R.3 the following relation holds:
X
HX
where for simplicity, we overload notation such that X is in homogencous coordinates when multiplied by the 4 x homography matrix. Under a typical 3D point cloud alignment problem, for example from LIDAR SLAM [33), H would represent translation and rotation in SE(3). If the point clouds additionally differ in scale, then H would be on Sim(3), the group of similarity transformations. However, here we do not have typical point clouds as x is constructed by uncalibrated cameras. Thus, we recall the Projective Reconstruction Theorem (26 Chapter 10.3], which in summary

---

states that if correspondences between two from uncalibrated cameras uniquely determine the fundamental matrix, then the correspondences may be used to reconstruct the corresponding 3D points up to 15-DOF homography transformation. This transform is the sume for any such corresponding points, except those on the line connecting the camera centers as these points cannot be reconstructed uniquely. Relevant to our setup, the Projective Reconstruction Theorem also applies to a reconstruction with more than two cameras [26]. Thus, in the most general case, the reconstruction computed using set of uncalibrated cameras differs from metrically correct reconstruction by a projective transformation (i.e., homography) H. The matrix H has 15 DOF and can be mapped uniquely to the special linear group, SL(4), by normalizing with the determinant. The (4) group consists of all real-valued 4 x 4 matrices with unit determinant. Note this is not the same as the more common 8 DOF homography matrix commonly used in planar computer vision tasks such as image warping, which belongs to ST(3). The reconstruction can be transformed to an affine reconstruction (i.e. parallel lines are preserved) when scene priors available, for example if points are known to lie on parallel lines. If further priors are known, such as lines in the scene are orthogonal, then the reconstruction can be converted to metric reconstruction (differing by only similarity transform to the true Euclidean reconstruction). VGGT is thus able to leverage learned scene priors potentially estimate metric reconstruction, but as we have shown in Fig. | in the most general case when estimates of scene priors are unreliable, the reconstruction differs by projective ambiguity, requiring a 15-DOF homography matrix to rectify. We will now estimate such homography.
By our construction of the submaps that they share a same image, we have an atypical advantage in solving for H as we have a dense set of correspondences without needing to estimate associations.
As is well known by the direct reconstruction method (66], the optimal homography in [ can be solved in closed form as solution to the following homogeneous linear system:
Agh =0
(2)
with h E IR 16 containing the flattened parameters of the homography and A₁ contains constraints for
a particular pair of 3D points. minimum solution requires 5 points (i.e., E {1: 5}), and to build in robustness to incorrect depth measurements from VGGT, we solve 2 using RANSAC [18] with 5-point solver. As the homography matrix is estimated up to scale, we scale by the fourth root of the determinant such that the determinant is unity and the resulting matrix belongs to SL(4).
Transformation of camera poses via homography Using a homography between reference
frames and j. Hj. the camera poses can be corrected using the following 26 P, H, where E 3 is the camera matrix created from the poses and intrinsic estimates from VGGT. We can then decompose P to recover the camera pose.
4.3 Loop closures
Our procedure for creating loop closures for VGGT-SLAM consists of two steps: (i) performing image retrieval (i.e., setting Tooo in Sec. p.1, and (ii) estimating relative homographies, which are then added to the factor graph as loop closure constraints (Sec. 4.4). First, for image retrieval, when constructing submap, we compute and store an image descriptor for each keyframe using SALAD 28). Then, once Liatest reaches its size threshold w, we search over the image descriptors in the previous submaps S; Vi E [1 lastest Tittrval } to fetch set of frames of size Whoep that have the highest similarity (using the L2 norm) to any of the keyframes in Zistest, and also exceed a user-defined similarity threshold Tácse to reduce false positive matches. These frames make up Iloop- which is added to the list of keyframes for the current submap, and then all frames are sent to VGGT as described in Sec. 4.1
Next, given the estimated submap, Statest, from VGGT, we estimate the relative homographies between the loop closure frames in Statest and the submaps, S,, retrieved during the image retrieval process described above. As in Sec. [42 we again have the benefit of not requiring an estimate of correspondences to compute homographies for loop closures; thus, we can directly use (2) between the frames in Tooop and their respective identical frames in the submap where they originated. This then provides Whoo loop closure constraints between Statest and the corresponding submaps.
5

---

Note that a potential alternative is to get the descriptor using the output tokens from VGGT's fine- tuned DINO backbone. This alleviates using a separate descriptor module and storing the physical images. However, this requires storing larger features in memory compared to the relatively small SALAD features, and the system memory needed store the images our base approach is relatively low.
4.4 Backend: Nonlinear factor graph optimization on the SL(4) manifold
Given all relative homographies H;, between submaps S; and Sj. our goal is to compute the absolute homographies H2 that transform all submaps into a common global reconstruction. To achieve this, we formulate nonlinear factor graph optimization problem based on Maximum A Posteriori (MAP) estimation [2] [17 20]. Specifically, we estimate the absolute homographies by minimizing the following cost function under Gaussian noise assumptions on the relative homographies:
H argginn HESL(() (i.j)EC
Σ
Loog HH (H))
where Log(:) () the mapping function that transforms group element to (vectorized) element of the corresponding Lie algebra, C denotes an index set of constraints that includes odometry and loop closures, and we set 2. E 125x15 to the identity matrix.
To solve (3), we iteratively compute state increments by solving a lincarized least squares problem. To this end, we define & IR 15 the tangent-span parameterization of SL(,). the mapping function (og(Exp
and Exp (E) exp($*) H. In particular, e Lie algebra element of st(4), computed by summing the k-th component of & with
Exp- 15 SL.(4), which satisfies Log [Exp (<)
its k-th corresponding generator G₁ vk: : (1: 15) (i.e., e = =1= Ex G [6]. More details can be found in Appendix A
Next, defining the measurement function as (E:,E)) = Log HH,',,), the incremental update of each pose can be approximated using Taylor's expansion [3] as follows:
1 ① d.. E, @ 65) E,),,))) (J,6; + J,ō,), & 5 = Hxp (6)
where J, AdH, HH and J, 115115 Here, Adи is the adjoint map, defined Adæ
B B- & H [16). where B = (vec (G1) vec (Gg) ... vec (G1s)] R 166115 in the case of the (4) manifold [9] and @ denotes the Kronecker product, which forms block matrix by multiplying each element of the first matrix with the entire second matrix.
Finally, formulate the linearized residuals and the resulting local problem at the linearization point Hj as follows:
D argmin
Σ
llei J,ō, J,J,+ JJ5ō ei Log (H,'H, (H;)
(5)
RED
(KJ)EC
To solve ()) we use the Aevenberg-Marquuan optimizer [55] and each iteration, the poses are updated on the Lie group as H +- H Exp(0) 63].
Experiments
We follow similar experiments as MAS:3--SAA to evaluate camera pose estimation and dense reconstruction in Sec. 5.2 and Sec. 5.3 respectively, demonstrate qualitative results Sec. 5.5 and
finally perform ablations in Sec. [.6
In our case, this is an extension of pose graph optimization, where we estimate absolute poses from pairwise pose measurements.

---

5.1 Experimental setup
We evaluate VGGT-SLAM on standard RGB SLAM benchmarks assess both camera pose es- timation accuracy and dense mapping quality. For evaluation of pose estimation, we employ the 7-Scenes [60] and TUM RGB-D [65] datasets, and report root mean square error (RMSE) of the absolute trajectory error (ATE) using evo [2I]. Since 7-Scenes [60] provides scene ground truth, this dataset is also used evaluate dense mapping quality in terms of accuracy, completion, and Chamfer
distance 46
As baseline approaches, we primarily compare VGGT-SLAM with DROID-SLAM 67 and MAS SLAM (46) as the state-of-the-art learning-based SLAM approaches in uncalibrated scenarios (and Spann3R 70 for dense evaluation). We use reported numbers from MASC3--SAAM 46] for baselines, except for the uncalibrated version of DROID-SLAN Although DROID-SLAM, requires camera intrinsics, we also evaluate it in an uncalibrated setting by estimating intrinsics with an automatic calibration pipeline [69]. as is suggested by Murai er al. [46]. While our approach operates without camera calibration, we also include comparison with state-of-the-arr methods 7 34 79 66
12 82
provided with camera intrinsics. Due to potential randomness in our approach caused by RANSAC, we report the average performance over five runs, which have low spread (small standard deviation) shown in Sec. 5.6
We refer to a simpler (im(3) version of VGGT-SLAM as Ours (Sim(3)), for which we follow similar structure as our SL(4) pipeline except we align relative rotation and translation between submaps using pose estimates from VGGT and estimate scale correction by comparing the estimated point clouds of the overlapping images. Loop closures and relative factors are added to the factor graph as SE(3) factors.
We use an NVIDIA GeForce RTX 4090 GPU with AMD Ryzen Threadripper 7960X CPU. For parameters, we set Whoop Taispurity 50 pixels, Tinterval = 2, Taesc 0.8, and Teonf 25%. We also use 300 RANSAC iterations with a threshold of 0.01. We show evaluations of both the SL(4) and S3m(3) version of VGGT-SLAM with different submap sizes (i.e., different values for w).
5.2 Pose estimation evaluation
As shown in Tables ! and 2 VGGT-SLAM performs comparable to the top performing uncalibrated
baselines on 7-Scenes and T TUM RGB-D. On 7-Scenes for instance, VGGT-SLAM has approximately the same average APE as the top performing baseline MAS13R-SLAM. On the TUM dataset, the SI.(4) version of VGGT-SLAM performs the best overall with average error of 0.053 This demonstrates that we are able to extend VGGT to multiple sequences while introducing a new category of SLAM system by optimizing submap alignment as SL(4) factors. Here, observe that our (3) version also performs well, as these scenes are generally cases where VGGT is able to leverage strong priors for metric reconstruction. Thus, while have shown cases where (L) is needed (Fig. D. the addition of higher degrees of freedom with our novel SLAM formulation maintains competitive performance, while improving some more challenging cases.
One particular scene where our method underperforms is on the TUM floor scene. This highlights challenge of estimating homography, which is the presence of degeneracy in the case of a planar scene. The floor scene contains several images that only view the flat floor leading to non-unique solutions for the homography matrix, which causes the overall reconstruction diverge. Building robustness for the planar case is an important component for SL(4) SLAM, which we leave as an exciting direction for future work. The TUM 360 scene is particularly challenging for smaller submap sizes (although handled well with w = 32) because smaller submaps are more likely to encounter approximately pure rotation in this scene, which can have reduced depth accuracy and hence higher outlier ratio when running 5-point RANSAC to estimate homography.
7

---

Table I: Root mean square error (RMSE) of absolute trajectory error (ATE) on 7-Scenes 600 (unit: m). The gray rows indicate the results using the calibrated camera intrinsics and the symbol indicates that the baseline evaluated in the uncalibrated mode. Green is best and light green is second best.
Sequence
Method
Avg
heads
office
pumpkin
kitchen
stairs
Calib.
NICER-SLAM
3
0.033
0.069
0.042
0.108
0.200
0.039
0.108
0.086
DROID SLAM
0.036
0.027
0.025
0,066
0.127
0.040
0.026
0.049
MASH3R ALAM
0.053
0.025
0.015
0.097
0.088
0.041
0.011
0.047
DROID-SLAM
07
0.047
0.038
0.034
0.136
0,166
0.080
0.044
0.078
Uncalib,
MAS MASGR-SLAM
46
0.063
0.046
0.029
0.103
0.114
0.074
0,032
0.066
Ours (3im
32)
0.037
0.026
0.018
0.104
0.133
0.061
0.093
0.067
Ours (SI.(4), o 32)
0.036
0.028
0.018
0.103
0.133
0.058
0.093
0.067
Table 2: Root mean square error (RMSE) of absolute trajectory error (ATE) on TUM RGB-D 6 (unit: m). The gray rows indicate the results using the calibrated camera intrinsics and the * symbol indicates that the baseline evaluated in the uncalibrated mode. Green is best and light green is second best.
Soquence
Method
Avg
360
Geak
deak2
Iloor
plant
room:
kPy
teddy
xy
ORB DRB-SLLAM3
21
X
0.017
0.210
X
0.034
X
x
0.009
N/A
DeepV2D
0.243
0.166
0.379
1.653
0.203
0.316
0.064
0.375
DeepFactios
1
0.159
0.170
0.253
0.169
0.305
0.364
0.043
0.601
0.035
0.233
Calib.
DPV-SLAM
0.112
0.018
0.029
0.057
0.021
0.330
0.030
0.084
0.010
0.076
DPV-SLAM-
R51
0.132
0.018
0.029
0.050
0.022
0.096
0.032
0.098
0.010
0.054
GO-SLAM
1V
0.089
0.016
0.028
0.025
0,026
0.052
0.019
0.048
0,010
0.035
DROID SLAM
0.111
0.018
0.042
0.021
0.016
0,049
0.026
0.048
0.012
0.038
MAS:3R-SIAN
0.049
0.016
0.024
0.025
0.020
0.061
0.027
0.041
0.009
0.030
DROID-SLAN
0,202
0.032
0.091
0.064
0.045
0,918
0.056
0.045
0.012
0.158
Uncalib.
MASGR-SLAM
46
0,070
0.035
0.055
0.056
0.035
0.118
0,041
0.114
0,020
0.060
Ours (Simu(3)
32)
0.123
0.040
0,055
0,254
0.022
0.088
0,041
0.032
0.016
0.074
Ours (SI, vo 32)
0.071
0.025
0.040
0.141
0.023
0.102
0.030
0.034
0.014
0.053
5.3 Dense reconstruction evaluation
Table 3: Root mean square error (RMSF) reconstruction
evaluation on 7-Scenes (60) (unit: m). @n indicates keyframe every n images.
Following the protocol of MAS-RR-LAA,, we provide dense reconstruction performance 7-Scenes; see Table 3 Here, we observe that while performance is comparable across meth- ods, VGGT-SLAM achieves the best performing accuracy and Chamfer distance, demonstrating the high accuracy of our dense point cloud re- construction.
Method
7-Soomes
ATE L Acc. & Complet. J. Chamfer J
DROID-SLAM
69
0.049
0.141
0.048
0.094
Calib
MASCRR-SAAM
16
0.047
0.089
0.085
0.087
Spunn3R 6020
70
N/A
0.069
0.047
0.058
Spunn3R @2
NIA
0.124
0.043
0.084
Uncalib
MASCR-SLAMM
4M
0.066
0.068
0.045
0.056
Ours (Sim (3), w
32)
0.067
0.052
0.062
0.057
5.4 Timing Analysis
Ours (SL(4) (1)
32)
0.067
0.052
0.058
0.055
In Table we show approximate timing results comparing the primary components of the Sim and (L( versions of VGGT-SLAM. As expected, the total time used to run optical flow and determine keyframes for all frames in the submap (keyframe detection), the total time to run SALAD and perform image retrieval for all frames the submap (loop closure detection), and the time to run the VGGT model (VGGT inference) is identical up to random variations. The time to optimize the factor graph using GTSAM (backend optimization) is very fast, taking only about half a millisecond for both variants, since the factor graph in VGGT-SLAM, is relatively small compared to factor graphs used in typical real time visual odometry problems [56]. The primary runtime difference between the Sim(3) and S4( variants is the time compute relative edge constraints (relative transformation estimation) since the 5- 5-point RANSAC estimation takes about 17 ms longer than the time to compute the S3) transformation. However, this increase is comparatively small, only about 2.5 percent of the time needed to run VGGT.
Memory needed to run VGGT for VGGT-SLAM is independent of the total number of images in the scene since VGGT run with fixed size submaps. An analysis of memory usage for running VGGT with varying numbers of images can be found in [10].
8

---

Table 4: Runtime comparison between (3) and SL(4) versions of VGGT-SLAM the office_ sequence with window size w (unit: msec). Runtime is averaged over five runs.
Stage
VGOT-SLAM Sim
VGGT-SLAM SL(4)
Keyframe detection
176 ms
176 ms
VGGT inference
662
Loop closure detection
105 ms
105 ms
Relative transformation estimation
11 ms
28 ms
Backend optiiizttion
0.4
0.5
5.5 Qualitative results
We present qualitative results to illustrate the mapping fidelity of VGGT-SLAM using SL(4) opti- mization. In Fig. longer 55 meter trajectory that loops inside an office corridor. In addition to the dense reconstruction, also show all mapped camera poses, where different colors indicate the submap associated with each image. In particular, the office corridor loop clearly shows 22 different submaps which have been joined into globally consistent map with loop closure at the end of the trajectory.
show example reconstruction from the office scene in 7-Scenes and from
Figure 2: Reconstruction and pose estimates from VGGT-SLAM on the office scene from 7-Scenes showing 8 submaps and from custom scene showing 55 meter loop around office corridor with 22 submaps. Both 16. Different image colors indicate the submap associated with each image.
In Fig. [ we show two select examples where using only Sim(3) is unable to align overlapping submaps while our SL.(4) alignment strategy able to rectify the projective ambiguity between submaps. Thus, while we have shown that Sim(3) generally achieves accurate performance across our quantitative experiments, in the general case where a feed-forward reconstruction method like VGGT is unable to estimate metric reconstruction (for reasons discussed in Sec. 4.2) due to the computational limits, our introduction an SL(4)-base) SLAM system shows promise in leveraging the potential of high accuracy, dense, Searning-based SLAM system. For Fig, []] Tdisparity is set to 0 to highlight the impact of projective ambiguity, which degrades the performance Sim alignment and affects overall map quality.
5.6 Ablations
In Fig. we provide multiple ablation studies which show the following for three different submap sizes (w 8, 16, 32): (a) improved pose accuracy of VGGT-SLAM when loop closures leveraged along with showing tight statistical spread of results from averaging 5 runs from our experiments, (b) that loop closures generally lead increasing reduction in pose error as the number of submaps increases since there are an increased number of loop closures, (c) the effect of different values of Toonfs which expected, larger values lead higher accuracy on dense reconstruction and lower completion, with our default value of 25% showing an appropriate balance.
6 Limitations
We have presented new type SLAMM system that addresses the issue of projective ambiguity from an underlying feed-forward scene reconstruction method (in case, VGGT), As creating factor

---

0.085
Complee jm]
0,08
E 0008
ATE
0.075
Chanfer
00.4
0.01
0.06
0.08
40,6
0.063
0.05
20
25
50
10
25
50
50
10
25
50
Window
of submaps
Conf. the, Teal
Conf.
thr, Toolf
Conf. thr, Toun!
Conf.
thr, Tool
(a)
(b)
Figure 3: Ablation studies: (a) Effect of loop closure (LC) on absolute trajectory error (ATE) across different window sizes, in TUM 65 where * annotations indicate measurements with 10- < p-value < 10 after paired I-test. (b) Reduction in ATE achieved by incorporating loop closure in TUM, highlighting that the number of submaps increases, our SL(da-based optimization leads to greater reductions ATE. (c) Performance changes with respect to the confidence threshold Teonf in ATE, accuracy, completion, and Chamfer distance under varying window sizes in 7-Scenes [60]
graph that optimizes the (4) manifold is new paradigm for the SLAM problem, it leaves much ground for further improvements. In particular, the estimation of the full 15-DOF homography matrix degenerate in the case of planar points, which can lead to unstable solutions the planar floor scene of the TUM dataset. Our current implementation of homography using points from VGGT is also vulnerable outliers. While use 5-point RANSAC to reduce this issue, the presence of a high outlier ratios or adversarial outliers (which are present due to local consistency of points in VGGT) can cause incorrect homography estimates discussed in Sec. 5.2 The ray-based matching in MASt3R-SLAM provides robustness to errors in depth measurements, and a similar method can potentially be adapted for homography estimation. Additionally, 15 DOF give rise to added opportunity of scene drift. While our addition of loop closures substantially corrects drift, an inaccurate relative homography estimate or long time between loop closures can cause not just drift in scale, rotation, and translation seen in classical SLAM, but also in scene perspective, which opens up an interesting area of research into further optimization into SL(()- SLAM. Finally, lens distortion is not rectified by the homography matrix since projective transformation preserves straight lines, and thus images are assumed to be undistorted when running VGGT-SLAM.
have observed in
7 Conclusion
In this study, we have leveraged VGGT, feed-forward reconstruction model, to incrementally construct a dense map from uncalibrated monocular cameras, proposing a novel SLAM framework called VGGT-SLAM, which locally and globally (through loop closures) aligns submaps from VGGT. By exploring VGGT's geometric understanding through the lens of classical multi-view computer vision, we have shown that in the general case, these submaps must be aligned with projective transformation, and in doing so we have created the first factor graph SLAM system optimized on the (4) manifold. In future work will further investigate conditions under which Sim(3) optimiza- tion suffices and investigate techniques to actively employ both (i) and SL(4) optimization in a unified system to enable more robust SLAM system for real-time performance.
Acknowledgments
This work is supported in part by the NSF Graduate Research Fellowship Program under Grant 2141064, the ONR RAPID program, and the National Research Foundation Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2024-0046 140)). The authors would like to gratefully acknowledge Riku Murai for assisting us with benchmarking.
References
[1] M. Abate, Y. Chang. N. Hughes, and L. Carlone. Kimera2: Robust and accurate metric-semaatic SLAM in the real world. In Infl. Sym. on Experimental Robotics (ISER), 2023.
[2] Jose Luis Blanco. A tutorial on se(3) transformation parameterizations and on-manifold optimization. 09
2010.
[3] Eric Brachmann, Tommaso Cavallari, and Victor Adrian Prisacariu. Accelerated coordinate encoding: Learning to relocalize in minutes using rgb and poses. In CVPR, 2023.
10

---

[4] Eric Brachmann and Carsten Rother. Visual camera re-localization from rgb and rgb-d images using dsac. pami, 44(9):5847-5865 2021.
[5] D. Bradley, T. Boubekeur, and W. Heidrich. Accurate multi-viow reconstruction using robust binocular stereo and surface meshing. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages
1-8. 2008.
[6] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza. J. Neira, I. Reid, and J.J. Leonard. Past, present, and future of simultaneous localization and mapping: Toward the robust- pebust-perception age. IEEE Trans. Robotics, 32(6): (166)13000 1309-1332, 2016. arxiv preprint: 1606.05830, (pdr)
[7] Carlos Campos, Richard Elvira, Juan J Gómez Rodríguez, José MM Montiel, and Juan D Tardós. ORB- SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM. IEEE Truns.
Robotics, 2021.
[8] Zequn Chen, Jiezhi Yang, and Heng Yang. Pref3r: Pose- free ferw-forward 3d gaussian splatting from variahle-length image sequence. arXiv preprint 1rXiv:2411.6687) 2024.
Saddennndoo Marston Conder, Edmund Robertson, and Peter Williams. Presentations for 3-dimensional special linear groups over integer rings. Proceedings of the American Mathematical Society. 115(1):19-26, 1992. [10] Wenyan Cong. Yiqing Liang, Yancheng Zhang, Ziyi Yang. Yan Wang, Boris Ivanovic, Marco Pavone, Chen Chen, Zhangyang Wang, and Zhiwen Fan. E3d-bench: A benchmark for end-to-end 3d geometric foundation models. arXiv preprint vrXiv:2506.01953 2025.
[9]
[11] Andrei Cramariuc, Lukas Bernreiter, Florian Tschopp. Marius Fehr. Victor Reijgwart, Juan Nieto, Roland Siegwart, and Cesar Cadena. maplab 2.0-A modular and multi-modal mapping framework. IEEE Robotics and Automation Letters, 8(2):520-527, 2022.
[12] J. Czarnowski, T. Laidlow. R. Clark, and A. Davison. DeepFactors: Real-time probabilistic dense monocular SLAM. IEEE Robotics and Automation Letters, 5(2):711-728, 2020.
[13] Frank Dellaert. Factor graphs and GTSAM: A hands-on introduction. Technical Report GT-RIM-CP&R
2012-002, Georgia Institute of Technology, September Septembbr 2012.
[14] Siyan Dong. Shuzhe Wang. Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3R: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization. arXiv preprint FrXiv:2412.08377 2024.
[15] Bardienus Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, and Jerome Revaud. MAS13R-SMM: a Fully-Integrated Solution for Unconstrained Structure-from-Motionn arXiv 2024.
preprint arXiv:2409.1911
[16] Ethan Eade. Lie groups for 2D and 3D transformations. URL http://ethaneade. com/lie. pdf. revised Dec.
117:118, 2013.
[17] K. Ebadi. L. Bernreiter, H. Biggie. G. Catt, Y. Chang, A. Chatterjee, C.E. Denniston, S-P. Deschênes, K. Harlow. S. Khattak, L. Nogueira, M. Palieri, P. Petráček, P. Petrlik, A. Reinke, V. Krátký, S. Zhao, A. Agha-mohammadi, K. Alexis, C. Heckman, K. Khosoussi, N. Kottege, B. Morrell, M. Hutter, F. Pauling, F. Pomerlcau, M. Saska, S. Scherer, R. Sicgwart, J.L. Williams, and L. Carlone. Present and future of SLAM in extreme underground environments. IEEE Trans. Robotics, 40:936 109936-959, 2024. (pdf)
[18] M. Fischler and R. Bolles Random sample consensus: a paradigm for model fitting with application to image analysis and automated cartography. Commun. ACM, 24:381-395, 1981.
[19] Y. Furukawa. B. Curless. S. M. Seitz, and R. Szeliski. Towards internet-scale multi-view stereo. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 1434-1441, 2010.
[20] Giorgio Grisetti, Rainer Kümmerle, Cyrill Stachniss, and Wolfram Burgard. A tutorial on graph-based
SLAM. IEEE Intelligent Transportation Systems Magazine, 2(4):31-43, 2010.
[21] Michael Grupp. evo: Python package for the evaluation of odometry and SLAM. /nttps://githin
com/Mi /Mi chae Grupp/evo, 2017.
[22] Tarek Hamel, Robert Mahony, Jochen Trumpf. Pascal Morin, and Minh- Minh-Due Hua. Homography estimation on the special linear group based on direct point correspondence. 2011 50th IEEE Conference on Decision and Control and European Control Conference, pages 7902-7908. 2011.
[23]
Richard Hartley, An algorithm for self calibration from several views. In cvpr, pages 908-912. 1994,
[24] R. Hartley A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press,
2000.
[25]
R. I. Hartley. defense of the eight-point algorithm. IEEE Trans. Pottern Anal. Machine Intell., 19(6):580-
593, June 1997.
[26] R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press,
second edition, 2004.
[27] Je Hyeong Hong. Christopher Zach, Andrew Fitzgibhon, and Roberto Cipolla. Projective bundle adjustment from arbitrary initialization using the variable projection method. In European Conf. Computer Vision
(ECCV), pages 477-493. Springer, 2016.
[28] Sergio lzquierdo and Javier Civera. Optimal transport aggregation for visual place recognition. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2024.
[29] Wonbong Jang, Philippe Weinzaepfel, Vincent Leroy. Lourdes Agapito, and Jerome Revaud. Pow3R:
Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors. arXiv preprint
arXiv:2503.17316 2025.
[30] Krishna Murthy Jatavallahhula, Ganesh Iyer, and I iaam Paull, gradslam: Dense slam meets automatic diffcrentiation. In IEEE Intl. Conf. on Robotics and Automation (ICRA), pages 2130-2137. IEEF, 2020.
11

---

[31] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler. and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 43(44):33-1, 2023.
[32] J.J. Koenderink and A.J. vanDoorn. Affine structure from motion. Journal of the Optical Society of America A, 8(2):377-385, 1991.
[33] Hyungtae Lim, Beomsoo Kim, Daebeom Kim, Eungchang Mason Lee, and Hyun Myung. Quatro++:
Robust global registration exploiting ground segmentation for loop closing in LiDAR SLAM. Intl. J. of Robotics Rexearch, pages 685-715, 2024.
[34] Lahav Lipson, Zachary Teed, and Jia Deng. Deep patch visual SLAM. In European Conf. on Computer Vision (ECCV), pages 4244440, 2024.
[35] Steven Lovegrove, Parametric dense visual SLAM. PhD thesis, 2012.
[36] Steven Lovegrove and Andrew Davison. Real-time spherical mosaicing using whole image alignment. In eccv, pages 73-86. Springer, 2010.
[37] D.G. Lowe. Distinctive image features from scale-invariant keypoints. Inil. J. of Computer Vision, 60(2):11-100, 2004.
[38] B. D. Lucas and Takeo Kanade. An iterative image registration technique with an application in stereo vision. In Intl. Joint Conf. on Al (IJCAI), pages 674-679, 1981.
[39] D. Maggio. Y. Chang. N. Hughes. M. Trang. D. Griffith, C. Dougherty, E. Cristofalo, L. Sehmid, and L. Carlone. Clio: Real-time task-driven open-set 3D scene graphs. IEFF Robotics and Automation Letters (viden)
(RA-L.) 9(10):891188228, 2024.
[40] J. Matas and O. Chum. Randomized RANSAC with sequential probability ratio test. In Intl. Conf. on Computer Vision (ICCV), pages 1727-1732, 2005.
[41] Hidenobu Matsuki, Riku Murai, Paul HJ Kelly, and Andrew Davison. Gaussian splatting SLAM. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 18039-18048, 2024.
[42] C. Mei, S. Benhimane, E. Malis, and P. Rives. Constrained multiple planar template tracking for central catadioptric cameras. In British Machine Vision Conf. (BMVC), September 2006.
[43] C. Mci, S. Benhimane, E. Malis, and P. Rives. Homographyybasee tracking for central catadioptric cameras.
In JEEE/SS Inil. Conf. on Intelligent Robots and Systems (IROS), October 2006.
[44]
C. Mei, S. Bennhimane, E. Malis, and P. Rives. Efficient homography-basee based tracking and 3-D reconstruction
for single-viewpoint sensors. IEEE Trans. Robotics, 24(6); 1352- 1364, Dec. 2008
[45] E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd. 3d reconstruction of complex structures
with bundle adjustment: incremental approach. In IEEE Inrl. Conf. on Robotics and Automation (ICRA), pages 3055-3061, May 2006.
[46] Riku Murai, Eric Dexheimer, and Andrew Davison. MASAR-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors arXiv preprint #xix:2412.12399 2024.
[47] David Nistér. An efficient solution the five-point relative pose problem. IEEE Trans. Pattern Anal. Machine Intell., 26(6):756-770. 2004.
[48] D. Nistér. A minimal solution to the generalised 3-point pose problem. In IEEE Conf, on Computer Vision and Pattern Recognition (CVPR), pages 560-567, 2004.
[49] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual
features without supervision. arXiv preprint arXiv:2304, 230.00330 2023.
[50] Linfei Pan, Daniel Barath, Marc Pollefeys, and Johannes Lutz Schönherger. Glohal Stracturefrrmm-Motion
Revisited, In European Conf. on Computer Vision (ECCV), 2024.
[51] Frank Plastria. The Weiszfeld Algorithm: Proof, Amendments, and Extensions, pages 357-389". Springer
US. New York, NY, 2011.
[52] Tong Qin, Shaozu Cao, Jie Pan, and Shaojie Shen. A general optimization-based framework for global pose estimation with multiple sensors. arXiv preprint: 1901.03644 2019.
[53] Tong Qin. Peiliang Li. and Shaojie Shen. Vins-mono A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics, 34(4):1004-102) 2018
[54] R. Ranftl, A. Bochkovskiy, and Koltun. Vision transformers for dense prediction. In Inrl. Conf. on
Computer Vision (ICCV), pages 2179-12188. 2021.
[55] D.M. Rosen, M. Kaess, and J.J. Leonard. An incremental trust trust-region method for robust online sparse beast-squares estimation. In IEEE Intl. Conf. on Robotics and Automation (ICRA), pages 1262-1266 St. Paul, MN, May 2012.
[56] A. Rosinol, M. Abate. Y. Chang, and L. Carlone. Kimera: an open-source library for real-time metric- semantic localization and mapping. In IEEE Intl. Conf. on Robotics and Automation (ICRA), pages 1689-1696. 2020. arXiv preprint: 1910.02490. (video) [57] A. Rosinol, A. Violette, M. Abate, N. Hughes, Y. Chang, J. Shi, A. Gupta, and L. Carlone. Kimera: from
1910.02490.((vieee)) (code) (pdf))
SLAM spatial perception with 3D dynamic scene graphs. Inti. J. of Robotics Research, 40 1011441))
1546, 2021. arXiv preprint: 1001.06894,[/. (pdr).
[58] Johannes Schonberger and Jan- Michael Frahm. Structure-froo fronee-ffommmmmoo from-motiooooon motion revisited. In IEEE Conf. on
Computer Vision and Pattern Recognition (CVPR), pages 4104-4115 4114, 13, 2016.
[59] Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Mare Pollefeys. Pixelwise view selection for unstructured multi-view stereo. In European Conf. on Computer Vision (ECCV), pages 501-518. Springer. 2016.
[60] J. Shotton, B. Glocker, C. Zach. S. Izadi, A. Criminisi, and A. Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgh-d images. In IEEE Conf. on Computer Vision and Pattern
12

---

Recognition (CVPR), pages 2930-2937, 2013.
[61]
Heung- Young Shum and Richard Szeliski. Systems and experiment paper: Construction of panoramic image mosaies with global and local alignment. International Journal of Computer Vision, 36:101-10 130. 2000.
[62]
Brandon Smart, Chuanxia Zheng. Iro Laina, and Victor Adrian Prisacariu. Splatt3R: Zero-shot gaussian
splatting from uncalibrated image pairs, arXiv preprint arXiv:2408. 1108.13917 13912, 2024.
[63]
Joan Sola. Quaternion kinematics for the error-state kalman filter. arXiv preprint arXiv:/711.02506 2017. Seungwon Song. Hyungtae Lim, Alex Junho Lee, and Hyun Myung. Dyna VINS: A visual-inertial SLAM for dynamic environments. IEEE Robotics and Automation Letters. 7(4): 1552--11300 2022. Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Damiel Cremers. benchmark for the evaluation of RGB-D SI.AM systems. In JEFP/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), pages 573-580. IEEF, 2012.
[64]
[65]
[66] Zachary Teed and Jia Deng. DEEPV2D: Video to depth with differentiable structure from motion. Inil. Conf. on Learning Representations (ICLR), 2018.
[67] Zachary Teed and Jia Deng. DROID-SLAM: Deep visual SLAM for monocular, stereo, and RGB-d cameras. In A. Beygelzimer, Y. Dauphin. P. Liang, and J. Wortman Vaughan, editors, Advances in Neural
Information Processing Systems (NIPS), 2021.
[68] B. Triggs. Factorization methods for projective structure and motion. In IEEE Conf. Computer Vision and Pottern Recognition (CVPR), pages 845-851, 1996.
[69] Alexander Veicht. Paul-Edouard Sarlin, Philipp Lindenberger, and Mare Pollefeys. GeoCalib: Learning
Single-image Calibration with Geometric Optimization. In European Conf. on Computer Vision (ECCV), pages 1-20. Springer, 2024.
[70] Hengyi Wang
Lourdes Agapito. 3D reconstruction with spatial memory. arXiv preprint arXiv:2408. 16061, 2024.
[71] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny.
VGGT: Visual geometry grounded transformer. arXiv preprint urXiv:2503. 11651, 2025.
[72] Qianqian Wang. Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous
3D Perception Model with Persistent State. arXiv preprint urXiv:2501. 2500.12387. 12387, 2025.
[73] Shuzhe Wang. Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d
vision made easy. In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pages 20697-20709, 2024.
[74] Wenshan Wang. Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang. Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. 2020.
[75] Juyang Weng, Thomas Huang, and Narendra Ahuja. Motion and structure from line correspondences: Closed solution, uniqueness, and optimization. IEEE Trans. Pattern Anal. Machine Intell., 14(3), 1992.
[76] T. Whelan, R.F. MoresoM, B. Glocker, A. J. Davison, and S. Leutenegger. ElasticFusion: Real-Time
Dense SLAM and Light Source Estimation. 2016.
(77) Chi Yan, Delin Qu, Dan Xu, Bin Zhao, Zhigang Wang, Dong Wang, and Xuelong Li. GS-SLAM: Dense visual slam with 3d gaussian splatting. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
pages 19595-19604, 2024.
[78] X. Yu and H. Yang. SIM-Syne: From certitiably optimal synchronization over the 3d similarity group
scene reconstruction with learned depth. IEEE Robotics and Automation Letters, 9(5):471 (55:4471-4478, 4478, 2024. [79] Youmin Zhang, Fabio Tosi, Stefano Mattoccia, and Matteo Poggi. GO-SLAM: Global optimization for consistent 3D instant reconstruction.
Intl. Conf. on Computer Vision (ICCV), pages 3727-3737. 2023. [80] Yinqiang Zheng, Yubin Kuang, Shigeki Sugimoto, Kalle Astrom, and Masatoshi Okutomi. Revisiting the PnP problem: A fast, general and optimal solution. In Int.. Conf. on Computer Vision (ICCV), pages 1144-2351,20 2013.
[81] Yinqiang Zheng, Shigeki Sugimoto, Imari Sato, and Masatoshi Okutomi. A general and simple method for camera pose and focal length determination. In IEEE Conf. on Computer Vision and Pattern Recognition
(CVPR). 2014.
[82] Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R Oswald, Andreas Geiger, and Mare Pollefeys. NICER-SLAM: Neural implicit scene encoding for RGB SLAM. In IEEE International Conference on 3D Vision (3DV), pages 42-52, 2024.
13

---

datasets where A Tangent space of SL(4)
Here, we provide the explicit generators, G2 Ak {1 15), of SL(,). which allow us relate the Lie algebra s4) to the Lie group SL(4).
The tangent space of SL(() consists of all 4 real matrices with zero trace. Thus, there 15 generators, Ga. where 12 of them are defined as Esh for a + where is in the (a, b) entry and 0, elsewhere. The remaining three generators are B diag(1, -1,
1, 0, 0), B2 diag(O, 1, -),
B3 = diag(0, (0, O, 1, -11) Explicitly, the generators are as follows:
1
0
0
0
G Eo1
G2 Esg
G3 Ess
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
1
0
0
G E10
G5 E12
G. E
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
G E20
G. E21
Go Egs
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
G1 Ego
Gn Es1
G12 Es2
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
G13
G., B.
G1 B,
0
0
0
-1
0
0 1
0
0
0 0
-1
Thus, as briefly explained in Sec. group H E (SI.(4) is given by:
4.4 the relation between the Lie algebra, 5*h E s((()), and the Lie
15 Σ E. G+
exp = exp
B Extra Quantitative Results
We provide addition results of evaluating on the 7-Scenes 60 and TUM RGB-D 655 we experiment with different submap sizes (Appendix B. and show the number of submaps and loop closures per scene (Appendix B.22)
B.1 Evaluation with different submap sizes
Here show results for the 7-Scenes and TUM RGB-D datasets in Tables and with different submap sizes (w = 8, 16, 32). For 7-Scenes, we also include results for 1. Recall that is the size of new images in the submap, so the case w 1, each submap has one new image, one image from the prior submap, and up to one extra image from loop closures. For small submap size of w 1, the backend becomes numerically unstable for some TUM scenes (consistently floor and 360) preventing an estimated alignment, and thus we do not include the
for TUM. This is due to reasons discussed in Sec. 6 Particularly, for the floor scene there are a large portion of images which only view planar scene which makes the estimation of the full 15-DOF homography matrix
14

---

degenerate, and for the 360 scene, using small submap size such as w 1 is likely to encounter pure rotation which can result in less accurate depth measurements from VGGT and hence reduced accuracy in our estimate of relative homography.
Table 5: Root mean square error (RMSE) of absolute trajectory error (ATE) on 7-Scenes (umit: m). The symbol indicates that the baseline is evaluated in the uncalibrated mode, all VGGT-SLAM configurations are evaluated in uncalibrated mode. Green is best and light green is second best.
Sequence
Method
Avg
chess
fixe
heads
office
pumpkin
kitchen
stairs
DROID-SLAM+ 67
0.136
0,078
MASER SLAM 46
0.063
0.046
0.029
0.103
0.114
0.074
0.032
0.066
(3im(3), 1)
0.047
0.113
0.050
0.070
Uncalih
Ours (3im(3),
8)
0.039
0.027
0.020
0.108
0,144
0.053
0.080
0.067
Ours (Sim(3); or 16)
0.037
0.027
0.021
0.107
0.135
0.051
0.093
0.067
Ours (Sim(3),
32)
0.037
0.026
0,018
0.104
0,133
0.061
0.093
0.067
Ours (SL 1), w = 1)
0.089
0.046
0,072
0.119
0,147
0.055
0,100
0,090
(SL,(4,
85
Ours (SI,(4) 16)
0.036
0.065
0.037
0.107
0.139
0.050
0.093
0.075
Ours (SL( 32)
0.036
0.028
0.103
0.133
0.058
0.093
0.067
Table 6: Root mean square error (RMSE) of absolute trajectory error (ATE) on TUM RGB-D 65 (unit: m).
The symbol indicates that the baseline is evaluated in the uncalibrated mode, all VGGT-SLAM configurations are evaluated in uncalibrated modc. Green is best and light green is second best.
Sequence
Method
Avg
desk
desk2
plant
2oom
rpy
teddy
xyz
DROID SLAM* 67
0,202
0.032
0.091
0.064
0.045
0.918
0.056
0.045
0.012
0.158
MASCA--SLAM 4M
0.070
0.035
0.055
0.056
0.035
0.118
0.041
0.114
0.020
0.060
Ours (Sim(3), (3), w 5)
0.070
0.026
0.030
0.048
0.026
0.081
0.024
0.035
0.015
0.040
Unealib.
(Sim(
Ours (Simn (3), 32)
0.123
0.040
0.055
0.254
0,022
0.088
0.041
0.032
0,016
0.074
(SI,(4), 8)
0.046
0.095
0.130
0.031
0.115
Ours (4L(4), 166
0,147
0.032
0.087
0.158
0.027
0.150
0.037
0.069
0.035
0.083
Ours (4), vo 32)
0.071
0.025
0.040
0.141
0.023
0.102
0.030
0.034
0.014
0.053
Table 7: Dense reconstruction evaluation on 7-Scenes 660
(unit: m).
Method
7. 7cSenes
ATB
Acc.
Complet. J Chamfer 1
MAS SLAM* 46
0.056
0.068
0.045
0.056
Ours (Sim(3), w = 1)
0.070
0.066
0.051
0.059
Uncalih.
Ours (Sim(3), (3),
0.067
0.054
0.056
0.055
Ours (Sim (3), 16)
0.067
0.054
0.058
0.056
(Sim(33)
0.067
0.052
Ours (ST. tc 1)
0.090
0.080
0.068
0.074
Ours (SL.(4) wr = 8)
0.084
0.067
0.065
0.066
Ours (4), 16)
0.075
0.061
0.063
0.060
Ours (4), w. 32)
0.067
0.052
0.058
0.055
B.2 Number of submaps per scene
As reference, in Tables g and 9 we show the number of total submaps in each scene for 7-Scenes and TUM RGB-D for different values of experimented submap size, and also show the number of loop closures in each scene.
B.3 Evaluation of Focal length Consistency
To provide quantitative results showing that VGGT can produce an estimate of the scene which differs by more than similarity transformation to the true scene, this section we show inconsistencies in estimates of camera intrinsics from VGGT. Here, a single camera is used per scene and different scenes can use different cameras. We observe that even though the true intrinsics of the camera should be approximately constant within a scene, VGGT has varying estimate of the instrines both inside a submap and across different submaps. This provides further demonstration that the VGGT reconstruction of submap can differ from the true scene by more than similarity transformation
15

---

Table 8: Window size and corresponding submap and loop closure counts when Whop 1, shown submaps (# of loops)".
Sequences in 7-Scenes (65
Window size,
cheas
rice
beads
office
pumpkin
kitchen
sLaiis
29 (11)
50 (46)
62 (49)
58
43 437337
43 (38)
14 (4
400)
(4)
(3)
(4)
6(0)
(
2
4(1)
((2)
(
3(0)
32
(00))
20)
2(0
(
2.00)
(
(0)
Table 9: Window size and corresponding submap and loop closure counts when Whopp "# of submaps (# of loops)".
1. shown as
Soquences in TUM-RGB-D
Window size,
36C
deak
desk2
floor
plant
room
rpy
teddy
xyz
168 (151)
(42)
98 (84)
(87)
102 (92)
186 (162)
95 (89)
146 (125)
(54)
(3)
(10)
16
(2)
(2)
(4)
(
7(2)
(2)
64)
10 (4)
4 (.)
(1)
(0)
(2)
(()))
4
(
3())
(2)
2.00)
and contain affine and projective degrees of freedom which can be resolved using the homography alignment. Table we summarize the standard deviation, range, and average of all focal length estimates for four scenes. We observe that for both the office loop scene and 7-Scenes, our Sim (3) variant of VGGT-SLAM performs comparable to the SL(4) variant while SL(4) performs significantly better than (3m the Tabletop and Bollards scene. Consistent with this observation, in Table I0 we notice that the later two have much larger intrinsic error (larger standard deviation and larger range) than the former two.
445
435
420
835-
415
410
Submaa
Figure 4: VGGT estimates of the focal length (fx) of every keyframe in the office loop scene from Fig. 2 for all 22 submaps.
720
050
640
630
Submap
Figure 5; VGGT estimates the focal length (fx) every keyframe in tabletop scene from Fig for all suhmaps.
16

---

Table 10: Statistics of VGGT Focal length (fx) estimates. All values in pixels.
Soene
Sid
Range
Average
Oflice Loop (Fig
7.3
36.9
429.0
7-Scenes
9.0
59.7
435.1
Tabletop (Fig
37.1
122.8
669.1
Bollards (Fig
51.8
177.3
738.9
C Extra Qualitative Results
C.1 Extra examples of SL(4) versus Sim
While have mentioned that the Sim(3) version of VGGT-SLAM often provides high quality reconstructions, here we provide additional examples of cases where Sim(3) is not sufficient and (4) is necessary achieve consistent alignment across submaps.
Sim(3)
SL(4)
Figure 6: Example tabletop scene showing Sim(3) is unable to align the submaps while SL(() is able to
correct for projective ambiguity. Here 32 and Tiisprrtty 50.
17

---

Sim(3)
R
SL(4)
Figure 7: Example on tabletop scene showing Sim(3) is unable to align the submaps while SL(4) is able to correct for projective ambiguity. The true scene only has one banana, but the Sim(3) reconstruction shows hallucination of two caused by misalignment. Camera pose estimates colored by submap. Here 16 and
Teisuriy 50.
Sim(3)
SL(4)
Figure 8: Example outdoor scene with yellow bollards surrounding tanks showing Sim is unable to align the submaps while SL(4) is able correct for projective ambiguity. The true has single bollards spaced around the tanks while the Sim(3) scene hallucinates clusters bollards due misalignment. Here w 16 and Feisparity 25.
18

---

C.2 7-Scenes Qualitative Results
Here we provide additional visualizations of scene reconstructions from the 7-Seenes dataset experi-
ments for VGGT-SLAM with SL(.). We use the default parameters from Sec. 5
Figure 9: Visualization of reconstruction on 7-Scenes fire scene with 2 submaps. Camera pose estimates colored by submap.
Figure 10: Visualization of reconstruction on 7-Scenes heads scene with 2 submaps. Camera pose estimates are colored by submap. Part of the scene in cropped for visual clarity.
19

---

C.3 TUM RGB-D Qualitative Results
Here we provide additional visualizations of scene reconstructions from the TUM RGB-D dataset experiments for VGGT-SLAN with SL(4). We use the default parameters from Sec.
Figure 11: Visualization of reconstruction on TUM room scene with submaps. Camera pose estimates are colored by submap. Part of the scene in cropped for visual clarity.
Figure 12: Visualization of reconstruction TUM 360 scene with 6 submaps. Camera pose estimates colored by submap. Part of the scene in cropped visual elarity.
20

---

Figure 13: Visualization of reconstruction on TUM xyz scene with 2 submaps. Camera pose estimates are colored by submap.
21

---

C.4 Additional Outdoor Qualitative Results
While our method is primarily tested on indoor scenes, here we provide an additional example of VGGT-SLAM on an outdoor scene from the TartanAir dataset [74]. Here w 16, Tdisparity = 50, and
Teoaf 50.
Figure 14: Visualization of reconstruction Tartan (scene Neighborhood Easy, POOS, left camera) with submaps. Camera pose estimates colored submap.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The paper includes the claims in both the abstract and in a contributions section of the introduction, which reflect the scope of the paper's contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We provide both limitations section and further discussion of limitations in our experimental analysis.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [Yes]
Justification: Our paper does not include new theorems. All equations are numbered with citations provided when they are applied from prior work.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All parameters are mentioned in the paper and we benchmark using publicly available datasets.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
24

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We public datasets for all baseline experiments and major results and will include our code as a zip file in the supplementary following the code submission guidelines.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https://ni cc/
public/ http://uuideeeeeeeeeeeeeeee
ss onn ollic) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See NeurlPS code and data submission guidelines (https
/ //nips. httccc/pbbbt wtcc/pbblcc/gge wttp//uwieeeeeee// SCoee ss onPol icyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We mention all parameters used in our experiments with justification for the major parameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, we provide eиror bars and show the statistical significance of our main experimental results with an added study in our ablations section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We mention that the experiments are performed on an NVIDIA GeForce RTX 4090 GPU with AMD Ryzen Threadripper 7960X CPU.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't fidn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htwp:///wss...eee...ccc//aaaaaaa
Answer: [Yes]
Justification: We have followed all guidelines in the NeurlPS Code of Ethics and our experiments do not include any human subjects and our experiments use common open source SLAM benchmark datasets.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
As SLAM system, paper does not have significant societal impacts.
Guidelines:
26

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our method does not require training and use open source weights and datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide citation to all use of prior datasets and the foundation model that our SLAM system use and follow the terms of use of each.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
27

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperss woperswithhoos com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
We include our code in the supplementary and include relevant licenses with the code.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do human subjects in our paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not use human subjects in our paper.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
28

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The core method development does not involve the use of LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy (https: https:///eurrpae httprrppps.ccccooeeeenneeee////s /002/ LLM) for what should or should not described.
29


Reviews:
[{"review_id": "d5CWF6Duwx", "reviewer": "Reviewer_enTd", "summary": "This paper presents VGGT-SLAM, the first dense RGB SLAM system based on SL(4) manifold optimization. To address the projection ambiguity problem of uncalibrated monocular cameras, the authors use VGGT feedforward reconstruction to generate subgraphs, achieving global alignment by estimating a 15-degree-of-freedom homography matrix and combining it with factor graph optimization. Experiments show that this method outperforms traditional Sim(3) optimization in long sequence scenarios, particularly in cases of significant projection ambiguity, although it exhibits degradation issues in planar scenes.", "questions": "1.The 15 degrees of freedom in SL(4) optimization increase the difficulty of solving. In SL(4) optimization, how do you balance computational efficiency with accuracy? Have you considered a dynamic switching strategy between Sim(3) and SL(4) optimization?\n2.If there are a large number of outliers in the depth map generated by VGGT (such as in low-texture areas), this situation is possible. Even with the RANSAC algorithm, the estimation of the homography matrix may still be erroneous.  Have you conducted research addressing this specific issue?\n3.The paper mentions GPU limitations for VGGT (~60 frames). How does VGGT-SLAM perform in terms of memory usage and running speed when processing long sequences?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.Introduces SL(4) manifold optimization to SLAM for the first time, addressing projection ambiguity in uncalibrated cameras and filling the gap in feedforward reconstruction methods for large-scale scenes.\n2.Combines classic projection reconstruction theorems to prove the necessity of SL(4) transformations, achieving global consistency through factor graph optimization.\n3.Validates the advantages of SL(4) over Sim(3) on the 7-Scenes and TUM datasets, with ablation studies analyzing the effects of loop closure and confidence thresholds.\nWeaknesses:\n1.Limited performance gain on 7-scenes dataset. Compared to using Sim(3)-based optimization, SL(4)-based optimization does not achieve noticeable improvement according to Table 1 and 3. The paper mentions that the homography estimation can easily degrade in planar scenes like TUM floors, leading to reconstruction divergence. However, the performance gain on TUM-RGBD dataset is higher. The analysis appears to be inconsistent.\n2.Limited experimental evaluation. While paper emphasizes dense mapping, it is necessary to evaluate 3D mesh reconstruction quality quantitively and qualitatively. Importantly, the paper lacks computational cost comparison, such as tracking and mapping speed.", "was_revised": true, "final_justification": "The authors' rebuttals adequately addressed my concerns. I suggest borderline accept of the paper.", "rebuttal": "We thank the reviewer for positive comments such as the significance of using SL(4) optimization to address projective ambiguity. We address each of the reviewer’s questions and concerns below: \n\n[SL(4) performance on benchmarks] We emphasize that while our proposed simpler Sim(3)  baseline approach is sometimes able to perform well, showing good performance on 7-Scenes and TUM RGB-D overall, there are multiple important cases where Sim(3) is insufficient to align VGGT submaps due to a projective ambiguity which we reconcile with our full SL(4) method. To highlight this, we include visual examples on scenes where SL(4) is needed, such as in Figure 1, and include three additional examples in the Supplementary Material. On the nine TUM RGB-D scenes, the scene where the SL(4) version of VGGT-SLAM has the largest error is on the floor scene which from analyzing the optimization steps occurs due to planar degeneracies when estimating the 15 DOF homography, which is why we mention that scene as an example limitation of planar degeneracies. This is consistent with the well-known issue that estimating the full 15 DOF homography matrix is degenerate in the case of planar points (such as in Multi-View Geometry [25], chapter 10) since 5 unique points are required for a unique solution.\n\nTo further show the benefits of SL(4), in our revised paper we will also include an additional plot showing the fluctuations in VGGT’s estimate of the camera intrinsics. These fluctuations can be attributed to the projective ambiguity motivating the need for a projective alignment. We summarize some of the results of VGGT’s estimate of the focal length (fx in pixels) of all keyframes in a scene, where a single camera is used for all images in a scene and thus should have constant focal length. 7-Scenes - avg: 435.1, std: 9.0, range: 59.7; Tabletop (Figure 5 in Supplementary Material) - avg: 679.4, std: 37.7, range: 117.3; Outdoor (Figure 6 in Supplementary Material) - avg: 738.9, std: 51.8, range: 177.3; Office Loop (Figure 2) - avg: 429.4, std: 6.9, range: 33.4. In particular, for the Tabletop and Outdoor scenes (for which in our paper we qualitatively demonstrate improvement with SL(4) over Sim(3)), there is substantial discrepancy in the focal length estimates from VGGT. \n\n[Computational Cost evaluation] We thank the reviewer for mentioning this important evaluation that should be included. We have conducted runtime analysis for VGGT-SLAM and will add the timing results to the paper. On a 4090, VGGT-SLAM with SL(4) for submap size 16 takes 991.6 ms per submap with the following timing breakdown (all in milliseconds). VGGT: 820.2 ms, keyframe detection: 90.7, loop closure detection and SALAD computation: 41.8 ms, relative homography estimation: 31.4 ms, and backend optimization: 7.4 ms. For Sim(3), all steps are the same except the relative estimation takes 11.3 ms instead of 31.4 ms. For mesh reconstruction, our method produces point clouds which we evaluate following the dense evaluation protocol of MASt3R-SLAM in Table 3. We also provide qualitative results in Figures 1 and 2 and in Figures 4–11 in the Supplementary Material.\n\n[Balancing SL(4) computation and accuracy] We thank the reviewer for mentioning this important detail about balancing computation between SL(4) and Sim(3). Both Sim(3) and SL(4) use a nonlinear backend optimization (using GTSAM), where the difference is on which manifold the optimization occurs. The relative estimation of SL(4), while more involved that Sim(3), is still a linear optimization problem and after conducting timing analysis, we observe the computation cost for SL(4) only uses about 20.1 ms more than Sim(3) which is relatively small compared to the time to run VGGT. The reviewer raises an interesting idea to dynamically switch between Sim(3) and SL(4) and this is something we have previously experimented with during development by comparing the relative inliers and outliers from each method and picking the optimized solution accordingly. This is an idea which is used in classical structure from motion, for example COLMAP uses it to switch between 3x3 homography and essential matrix estimation, and it is possible that better tuned inlier detection could lead to future improvements in this direction.\n\n[Handling outliers] To handle outlier points, particularly in challenging low texture regions as mentioned by the reviewer, during development we did experiment with different RANSAC parameters (such as outlier thresholds and number of iterations) and with confidence thresholds to filter points from VGGT, the latter of which helps to increase the inlier ratio. We keep the selected parameters constant across all quantitative experiments to show our default parameters are sufficient to handle outliers in most cases. A 5-point RANSAC is known to statistically require only 145 iterations (although we use 300 max iterations) for a 99% success probability with 0.5 outlier ratio and scales well for higher outlier ratios. VGGT has high resilience in low texture environments compared to prior methods, but it is expected that in extreme cases with high outliers most SLAM systems will not be able to produce a valid solution. To improve RANSAC estimation, we also experimented with more complex point sampling such as weighted sampling based on the VGGT confidence estimate and weighted sampling based on the level of non-coplanarity between points. However, we empirically found these approaches did not bring significant improvement to justify the added complexity. \n\n[Performance on long sequences] We thank the reviewer for raising this important question. Since VGGT-SLAM works by aligning submaps, computation time is roughly constant as scene size increases. From the timing breakdown of VGGT-SLAM’s components discussed earlier, SALAD feature extraction and VGGT are run over constant number of frames (the size of the submap), the relative estimation of SL(4) is between two submaps which is constant, and the backend is a small factor graph (typically, real time mapping systems with GTSAM have on the order of 10s of thousands of factors for bearing measurements while we just have factors between adjacent submaps and submaps with loop closures). Memory of the core part of VGGT-SLAM with the exception of SALAD retrieval vectors is also approximately constant.  In practice, we do store the point clouds of each submap in working memory for visualizing map construction in real-time and this does increase with map size. Since the visualization is not essential for VGGT-SLAM and can be disabled, the point clouds could either be stored in system memory or engineered to be more memory efficient depending on the application."}, {"review_id": "PPMys4AY8H", "reviewer": "Reviewer_wLsM", "summary": "This paper addresses the alignment of local reconstructions and camera poses produced by feed-forward transformer models—specifically VGGT—into a globally consistent map and trajectory. Since VGGT reconstructs each submap from uncalibrated video, these local reconstructions can suffer from projective distortions that make simple similarity (Sim(3)) alignment inaccurate. To overcome this, the authors propose aligning every pair of local submaps with a 4×4 homography on the SL(4) manifold. Relative homographies are estimated both between temporally adjacent submaps and between submaps linked by loop-closure detection. These pairwise homographies form the edges of an SL(4) pose graph, whose global optimization yields absolute 4×4 transformations. Finally, the global camera poses are recovered by decomposing these absolute homographies into intrinsic and extrinsic parameters. Experiments on the 7-Scenes and TUM-RGBD datasets—evaluating both global trajectory accuracy and dense point-cloud alignment—show that this SL(4)-based approach outperforms previous methods when camera calibration is unavailable.", "questions": "- Please refer to weakness.\n- Eq.2: The paper only describe a minimal solution with RANSAC, is there any non-linear refinement after that? If not, what is the performance difference between w/ and w/o non-linear refinement.\n- What is the efficiency / fps of the full pipeline?\n- To distinguish between the insufficiency of SIM(3) transformation itself and the estimation of SIM(3), what if we initialize SIM(3) pose (both relative and global) with decomposed Homography and then refine?", "limitations": "The authors have discussed the limitations of planar scenes. However, the author did not discuss the potential limitation when the distortion of the VGGT local reconstruction is more than a projective transformation.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- This paper is well organized and written. The problem of projective ambiguity in VGGT is clear and motivates the use of a $4\\times4$ homography transformation.\n\n- The method itself is simple, effective, and presented with sufficient detail. The open-sourced code further enhances reproducibility. The implementation is also comprehensive, including relative homography estimation using DLT and global homography estimation as a pose graph.\n\n- The proposed solution is agnostic to local 3D reconstruction methods and can be combined with other uncalibrated 3D reconstruction models.\n\n### Weaknesses\n- The major assumption motivating the paper is that when Sim(3) is insufficient, the relative transformation between local VGGT reconstructions is primarily a projective transformation due to uncalibrated cameras. However, this assumption is insufficiently justified in the current version of the paper. Specifically, the following questions merit further clarification:\n\n  - How frequently are VGGT’s reconstructions distorted?\n\n  - When a reconstruction is distorted, how well can a projective transformation handle this distortion?\n\n  - How reliable is the decomposition itself—i.e., how the accuracies of decomposed K, R, and t related to the accuracy of H and the magnitude of shearing, stretching, and perspective components in H?\n\n- The method is non-learnable. To engage a broader audience at NeurIPS, it would be worthwhile to discuss and better demonstrate how the proposed method could be integrated into a learning pipeline. Both the estimation of local and global homographies could be made differentiable with reasonable effort.\n\n- A discussion of classic projective reconstruction (e.g., Factorization Methods for Projective Structure and Motion, CVPR ’96; Projective Bundle Adjustment from Arbitrary Initialization using the Variable Projection Method, ECCV ’16) is missing.\n\n- The datasets used in evaluation are limited, only TUM-RGBD and 7 Scenes. While other standard benchmarks such as Tartan Air and KITTI is missing. Therefore, it is uncertain whether the assumption still holds for outdoor scenes with forward motion.", "was_revised": true, "final_justification": "This paper proposes to aligns projectively distorted point clouds from VGGT both locally and globally using 4\\times4 homography transformation. The motivation is to address the camera intrinsic ambiguity of VGGT's prediction, and the corresponding solution is simple and proven to be effective on limited small scale scenes. During the rebuttal, the authors have indicated that they have verified the advantage of SL(4) homography transformation over SIM(3) similarity transformation is still valid when the distortion is more than projective, where the conclusion is independent of specific estimation method of SIM(3). \nIn addition, the pipeline can be made differentiable with reasonable effort. Therefore, I recommend to accept the paper.", "rebuttal": "We thank the reviewer for an accurate summary and positive comments, and we address all questions and concerns as follows: \n\n[Frequency that VGGT’s reconstructions are distorted] In addition to the two visual examples in Figure 1 of scenes where Sim(3) is insufficient for submap alignment and SL(4) is needed, we also include three additional examples in the Supplementary Material. Since Sim(3) is sometimes sufficient, we include our simpler Sim(3) version of VGGT-SLAM as well in the paper and in our attached code (where a single command line option allows switching to Sim(3)). However, because we notice multiple important cases where Sim(3) is not sufficient, we show that a higher dimensional projective alignment is needed and hence develop the full SL(4) version of VGGT-SLAM. \n\nWhile it can be difficult to precisely quantify how often SL(4) is needed, to better clarify the benefits of SL(4), in our revised paper we will include a plot showing the fluctuations in VGGT’s estimate of the camera intrinsics. The fluctuation can be attributed to the projective ambiguity.  We summarize some of the results of VGGT’s estimate of the focal length (fx in pixels) of all keyframes in a scene, where a single camera is used for all images in a scene and thus should have constant focal length. 7-Scenes - avg: 435.1, std: 9.0, range: 59.7; Tabletop (Figure 5 in Supplementary Material) - avg: 679.4, std: 37.7, range: 117.3; Outdoor (Figure 6 in Supplementary Material) - avg: 738.9, std: 51.8, range: 177.3; Office Loop (Figure 2) - avg: 429.4, std: 6.9, range: 33.4. In particular, for the Tabletop and Outdoor scenes (for which in our paper we qualitatively demonstrate improvement with SL(4) over Sim(3)), there is substantial discrepancy in the focal length estimates from VGGT. \n\n[How well can a projective transformation handle the distortion] We thank the reviewer for the insightful question which lets us clarify an important component of the theory behind VGGT-SLAM. According the Projective Reconstruction Theorem (section 4.2 of the manuscript and in [25]), if a set of uncalibrated cameras satisfies multi-view constraints (which can be given by the Fundamental matrix for corresponding points), then the difference between the estimated reconstruction of the scene and the true reconstruction can be explained by a 15 DOF projective transformation. Thus, a projective transformation is sufficient and in addition to theoretical justification, we experimentally observe that the SL(4) method is able to align submaps in the cases where Sim(3) is insufficient. If the reviewer is referring to camera lens distortion, we do not correct for camera distortion and distortion would violate the premise of the Projective Reconstruction Theorem since straight lines would not be preserved. VGGT also assumes negligible lens distortion. \n\n[Decomposition of K,R, and t] To clarify, we do not estimate the poses from scratch. Instead, we get the pose estimates from VGGT and apply a transforming homography to the poses. For this, we use the procedure outlined in the last section of 4.2.  However, we apologize that there was a typo in the equation in this section. The correct procedure to transform a camera matrix $P_j$ to $P_i$ should be $P_i=P_j H^{-1}$ where $P_i$ and $P_j$ are 3x4 matrices. Our code has the correct method but the typo in the paper likely caused confusion that a 4x4 matrix had to be decomposed and we apologize for that confusion. We have fixed the error in the revised manuscript. The decomposition of the matrix $P_i$ is then simply (we use decomposeProjectionMatrix in OpenCV) extracting the translation component, an SO(3) component, and an upper triangular component, for which we have not observed instabilities in practice. \n\n[Method is non-learnable] In this work, we extend VGGT to a SLAM system and in doing so we leverage theory from classical multi-view reconstruction of a scene with uncalibrated images to show why aligning submaps from VGGT cannot always be done effectively with a similarity transform and instead needs a projective transform. It would be straightforward to use an existing AutoDiff library for SLAM like PyPose or Theseus for the linear relative homography estimation and nonlinear global optimization, allowing these steps to either be learned or form a bilevel optimization problem to also support VGGT training. However, since it would be straightforward to do so, we do not believe it would add to the theoretical contribution of the work and would be out of scope. Additionally, with the popularity of geometric foundation models and of VGGT, many faster and more capable versions of VGGT are likely to become available and these can easily be integrated into VGGT-SLAM since no fine tuning is required with our current approach.\n\n[Missing references] We thank the reviewer for mentioning relevant prior works and we will add a discussion of these into our revised manuscript.\n \n[Outdoor Scenes] The reviewer mentions a concern that assumptions may not hold in the case of outdoor scenes or scenes with forward motion. Our response is five-fold. First, if the assumptions are referring to whether submaps can always be aligned with a projective transformation, we refer to our earlier remark that the Projective Reconstruction Theorem does not place requirements on the type of motion and thus the alignment procedure still works in the case of forward motion. Second, as an experiment of forward motion, we include results in a loop around an office floor in Figure 2 where multiple submaps are aligned under forward motion. Third, for outdoor scenes, our Supplementary Material in Figure 6 includes results of running VGGT-SLAM outside a building viewing a curbside with bollards and large holding tanks. Fourth, as discussed in the limitations section, we do not expect VGGT-SLAM with SL(4) to work well on long driving datasets like Kitti because as discussed in the limitation section, SL(4) can have excessive drift if there is a long period between loop closures as is the case in driving datasets. Lastly, in response to the reviewer, we have run a scene from TartanAir with VGGT-SLAM to provide an additional outdoor example and will include the results in the revised Supplementary Material. \n\n[Non-linear refinement] To clarify, in case there is confusion around our use of the word “minimal” to describe our RANSAC solver, we use a minimum solver for optimizing a local estimate of homography, meaning that we use the minimum number of points required for a unique solution (which is 5 in the case of a 4x4 homography). This can be solved with a linear solver and we employ a 5-point RANSAC to add robustness to outliers. Using a minimum number of points is well-known to be the most efficient way to use RANSAC since the fewer points required to obtain a solution, the less number of iterations needed to statistically reach a solution built with a set of inliers. After getting a solution from RANSAC, we attempted an additional optimization over all inliers, which is a common practice, but we did not find an accuracy improvement and thus removed the extra computation step. Our global optimization in the backend to optimize the factor graph does use a non-linear solver. \n\n[FPS of the full pipeline] We thank the reviewer for raising an important issue that we should have included timing results in the paper. We have added a full timing breakdown table to the revised manuscript which includes the following results: On an RTX 4090, VGGT-SLAM with SL(4) for submap size 16 takes 991.6 ms per submap with the following timing breakdown (all in milliseconds). VGGT 820.2 ms, keyframe detection: 90.7 ms, loop closure detection and SALAD computation: 41.8 ms, relative homography estimation: 31.4 ms, and backend optimization: 7.4 ms. For Sim(3), all steps are the same except the relative estimation takes 11.3 ms instead of 31.4 ms. Thus, VGGT-SLAM with SL(4) runs at approximately 16 FPS. \n\n[Distinguishing between Sim(3) error and Sim(3) insufficiency] We are not sure if we understand the question here and perhaps this is due to our typo of correcting the pose estimates with $P_i=P_j H^{-1}$ as discussed earlier.  If a Sim(3) (7DOF) transformation is not sufficient to align submaps, then either an affine (12 DOF) or a projective (15 DOF) transformation is needed. To start with Sim(3) and then refine the transformation in a way that could align the submaps would imply upgrading it to a higher dimensional transform (affine or projective). Our current formulation of Sim(3) estimation minimizes the potential for a false estimate of Sim(3) because we are aligning the last frame of one submap with the first frame of the subsequent submap, where the two frames are identical. In other words, the two frames have the same pose and we can therefore just set the rotation and translation of the first frame to be that of the last pose of the prior submap, and we only need to solve for the scale factor. We are happy to discuss this question further if this response has not fully addressed the question. \n\n[VGGT reconstruction with more than a projective ambiguity] As discussed in an earlier comment, the Projective Reconstruction Theorem from classical multi-view geometry says that if a set of uncalibrated cameras satisfies multi-view constraints, then the difference between the estimated reconstruction of the scene and the true reconstruction can be explained by a 15 DOF projective transformation. Points that do not satisfy multi-view constraints would be outlier points. To handle outliers, we filter out points using VGGT’s estimated confidence value for each point and use RANSAC in our estimation of relative homography estimates. We will also add to the limitations section that lens distortion is not handled by our projective rectification."}, {"review_id": "rUGi9eilo3", "reviewer": "Reviewer_BmpF", "summary": "This paper introduces VGGT-SLAM, an innovative dense RGB SLAM system that cleverly leverages VGGT's feed-forward scene reconstruction capabilities to construct dense maps from uncalibrated monocular cameras incrementally. The author proposes the first factor graph formulation that operates directly on the SL(4) manifold to address projective ambiguity. Even in practical scenarios, where projective ambiguity is less dominant, 62 we show that SL(4)-based optimization achieves performance competitive with or superior to other 63 state-of-the-art learning-based SLAM approaches, offering a principled framework for handling cases where similarity transformations are insufficient.", "questions": "1. Based on Table 3 showing Root Mean Square Error (RMSE) reconstruction evaluation on 7-Scenes, the performance differences between Sim(3) and SL(4) optimization appear to be minimal. This raises a fundamental question about the necessity of the complex SL(4) manifold optimization. Can the authors provide a comprehensive ablation study comparing:\n   - SE(3) pose graph optimization\n   - Sim(3) similarity transformation\n   - Full SL(4) homography optimization\nThis would help clarify whether the SL(4) formulation is truly necessary or if simpler approaches could achieve comparable results.\n2. What is the computational cost difference between these approaches, and does the marginal accuracy improvement justify the increased complexity? The similar performance between Sim(3) and SL(4) on 7-Scenes raises suspicions about potential training data contamination. Can the authors explicitly confirm whether the 7-Scenes data was used in VGGT's training? If so, what percentage of the training data comes from 7-Scenes?\n3. How does VGGT-SLAM perform on scenes that are fundamentally different from the training distribution (e.g., outdoor vs. indoor, different lighting conditions, different camera intrinsics)?\n4. How does the computational efficiency of VGGT-SLAM compare to Traditional SLAM methods (ORB-SLAM3, DROID-SLAM)? Can the author provide a detailed timing breakdown that shows reasonable computational costs? This clarification is crucial for understanding the true contribution and practical value of the proposed approach.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- **Large-Scale Scene Extension**: VGGT was originally limited by GPU memory constraints to processing only a small number of frames. The authors successfully extended it to large-scale scenes by creating multiple submaps and performing global alignment. This breakthrough enables the reconstruction of scenes requiring hundreds or even thousands of frames.\n\n- **SL(4) Manifold Optimization**: To address the projective ambiguity issue in uncalibrated camera reconstructions, the paper proposes optimization over the SL(4) manifold. This approach estimates 15-degree-of-freedom homography transforms between consecutive submaps, not only accounting for potential loop closure constraints but also providing more accurate alignment compared to traditional similarity transformations (Sim(3)).\n\n- **Extensive experiments** on standard RGB SLAM benchmarks demonstrate the effectiveness of VGGT-SLAM. The system exhibits robust performance across various scenarios, including long video sequences that are infeasible for VGGT due to its high GPU requirements.\n\n## Weaknesses\n\n- **Degenerate Cases**: The paper acknowledges that the estimation of the full 15-DOF homography matrix can be degenerate in the case of planar points, leading to unstable solutions. This is observed in the planar floor scene of the TUM dataset, where the performance of VGGT-SLAM degrades due to the presence of degenerate cases.\n- **Outlier Sensitivity**: The current implementation of homography using points from VGGT is vulnerable to outliers. While the use of a 5-point RANSAC helps reduce this issue, high outlier ratios or adversarial outliers can still cause incorrect homography estimates, affecting the overall performance of the system.\n- **Computational Efficiency**: The paper does not extensively discuss the computational efficiency of the proposed method. While the experiments are performed on a high-end GPU, the scalability and real-time performance of VGGT-SLAM in resource-constrained environments are not fully explored.\n\n- **Complexity of Nonlinear Optimization**: The paper introduces nonlinear factor graph optimization on the SL(4) manifold to compute absolute homographies and align submaps globally. While this approach is mathematically rigorous and provides a principled framework for handling projective ambiguity, it introduces additional complexity and potential robustness issues. Specifically, the iterative computation of state increments through solving linearized least squares problems can be computationally intensive and sensitive to initialization and noise. Integrating a nonlinear optimization system with the end-to-end VGGT model may compromise the overall simplicity and elegance of the approach. The additional optimization steps could complicate system implementation and deployment, potentially reducing accessibility for practitioners. The authors might consider exploring more streamlined and elegant ways to combine VGGT's strengths with global alignment, without relying on complex nonlinear optimization frameworks.", "was_revised": true, "final_justification": "While the authors' responses have addressed most of my concerns, I raise my original rating to AC.", "rebuttal": "We thank the reviewer for positive feedback and for insightful questions which we address in the following:\n\n[Degenerate Cases and Outlier Sensitivity] The reviewer mentions two important areas of improvement for VGGT-SLAM, which are discussed in detail in Sec. 6 Limitations. While these issues exist, we are still able to show state-of-the-art results on a variety of datasets. For example, even though we have lower accuracy on the floor scene from TUM RGB-D due to the planarity of the scene, our average across all TUM RGB-D scenes is still the most accurate compared to all uncalibrated baselines. Solving the planar degeneracy issue would further boost performance and reduce potential failure cases. In practice, our 5-point RANSAC procedure with estimating homography is shown to be effective throughout our experiments, but we see potential to use inspiration from ideas such as ray-based matching from MASt3R-SLAM to further add robustness to outliers. We believe these each are interesting and non-trivial areas and we plan to explore them in future work to further improve performance. \n\n[Computational Efficiency] The reviewer raises an important point about including timing results for VGGT-SLAM. On an RTX 4090, VGGT-SLAM with SL(4) for submap size 16 takes 991.6 ms per submap with the following timing breakdown (all in milliseconds).  VGGT: 820.2 ms, keyframe detection: 90.7 ms, loop closure detection and SALAD computation: 41.8 ms, relative homography estimation: 31.4 ms, and backend optimization: 7.4 ms. For Sim(3), all steps are the same except the relative estimation takes 11.3 ms instead of 31.4 ms. We will include a table showing a breakdown of timing results in the revised manuscript. We will also include results showing GPU memory usage compared to submap size, of which the majority of memory is used by VGGT. The overall framerate of VGGT-SLAM is also comparable to MASt3R-SLAM on the same hardware. The majority of VGGT-SLAM’s compute requirements are for VGGT and given the popularity around VGGT, it can be expected that there will be future lighter weight version of VGGT that can be plugged into VGGT-SLAM; although, running on lightweight devices such as edge compute is beyond the scope of the current paper.\n\n[Complexity of nonlinear optimization] We thank the reviewer for raising this important concern about the computational complexity of the nonlinear backend optimization. The computation time of the nonlinear solver (built on top of GTSAM) is about 7.4 ms per submap which uses less than 1 percent of the total submap computation time. Typically, real time mapping systems with GTSAM have on the order of 10s of thousands of factors for bearing measurements while we just have factors between adjacent submaps and submaps with loop closures. As discussed, to clarify this important point, we will include a timing table which shows the time used by each major component of VGGT-SLAM. To minimize the amount of user complexity, we chose to integrate our backend optimization as part of a popular optimization library (GTSAM) which creates a simpler python API interface between the core VGGT-SLAM code and the backend optimization steps. Future work could potentially leverage foresight of projective ambiguity discussed in VGGT-SLAM as a non-trivial but exciting direction to improve training, and as the reviewer mentions, this can further reduce the number of components of the SLAM system.\n\n[Table 3 Results] Since Sim(3) optimization is sometimes sufficient with submap alignment, we include both the Sim(3) and SL(4) versions of VGGT-SLAM. While results in Table 3 are comparable between Sim(3) and SL(4), we include examples of cases where Sim(3) is not sufficient due to projective ambiguity. To reconcile this limitation of Sim(3), we develop the full SL(4) version of VGGT-SLAM while concurrently using theory from classical computer vision to justify why there can be a projective ambiguity in the submap alignment problem. Our manuscript provides two examples in Figure 1 and we include three more examples in the Supplementary Material. For an ablation study of Sim(3) compared to SL(4), we show both pose evaluation and dense reconstruction evaluations of both the Sim(3) and SL(4) variants of VGGT-SLAM on 7-Scenes and TUM RGB-D datasets. Furthermore, we include additional quantitative results on 7-Scenes and TUM RGB-D in the Supplementary Material for varying submap sizes, along with qualitative examples. An SE(3) baseline would not be expected to show reasonable results since VGGT does not try to estimate the scale of the scene and thus scale must be estimated between submaps to get a reasonable overall map.\n\nTo include more detailed analysis,  in our revised paper we will also include an additional plot showing the fluctuations in VGGT’s estimate of the camera intrinsics. The fluctuation can be attributed to the projective ambiguity.  We summarize some of the results of VGGT’s estimate of the focal length (fx in pixels) of all keyframes in a scene, where a single camera is used for all images in a scene and thus should have constant focal length. 7-Scenes - avg: 435.1, std: 9.0, range: 59.7; Tabletop (Figure 5 in Supplementary Material) - avg: 679.4, std: 37.7, range: 117.3; Outdoor (Figure 6 in Supplementary Material) - avg: 738.9, std: 51.8, range: 177.3; Office Loop (Figure 2) - avg: 429.4, std: 6.9, range: 33.4. In particular, for the Tabletop and Outdoor scenes (for which in our paper we qualitatively demonstrate improvement with SL(4) over Sim(3)), there is substantial discrepancy in the focal length estimates from VGGT. \n\n[Computational Cost Difference] We will clarify this important question about the computation cost difference between SL(4) and Sim(3) by including the timing breakdown table discussed earlier. Specifically, for Sim(3) optimization, all steps of VGGT-SLAM are the same except the relative estimation takes 11.3 ms instead of 31.4 ms, leading to approximately 20.1 ms of increased cost per submap for SL(4) with our current implementation of the VGGT-SLAM code. \n\n[7-Scenes dataset] Based on the VGGT paper, the following datasets were included in training and are similar to those used by MASt3R (which is the Geometric Foundation Model used by one of the compared methods, MASt3R-SLAM). The datasets used in training are: Co3Dv2, BlendMVS, DL3DV, MegaDepth, Kubric, WildRGB, ScanNet, Hyper- Sim, Mapillary, Habitat, Replica, MVS-Synth, PointOdyssey, Virtual KITTI, Aria Synthetic Environments, Aria Digital Twin. The 7-Scenes dataset is not included in the VGGT training. We also include visual results on custom captured scenes such as the office loop (Fig 2) and several cell phone captured scenes in both indoor and outdoor in Appendix C. The observed similarity between Sim(3) and SL(4) performance is because Sim(3) is a subset of the SL(4) group and if a submap alignment can be fully explained by a Sim(3) transformation, then the SL(4) transformation should be nearly identical to the Sim(3) transformation. \n\n[Using VGGT-SLAM on out-of-distribution data] We use the frozen weights from VGGT and experiment on both popular benchmarking datasets and custom datasets which have been collected by multiple cameras such as a cell phone camera and a RealSense D455 camera. Our custom datasets include both indoor scenes and an outdoor scene (Figure 6 in the Supplementary Material). In developing VGGT-SLAM, we experimented with other scene challenges such as an outdoor scene at night being illuminated by flood lights and in general had good results. However, while VGGT-SLAM can correct projective ambiguity from VGGT, which may be in part due to challenging scenes, avoiding an otherwise completely incorrect scene reconstruction in an out-of-distribution scene would be a credit to VGGT not VGGT-SLAM. In practice, we observe impressive generalizability of VGGT, which has made it a powerful foundation model for a SLAM system. \n\n[Comparison to Traditional SLAM methods] Our manuscript includes comparisons to DROID-SLAM, ORB-SLAM, and other traditional SLAM approaches in Table 2 in terms of accuracy. For timing, we will include a timing breakdown of VGGT-SLAM as discussed. For reference, the timing of VGGT-SLAM per frame is comparable to that of MASt3R-SLAM and we will include timing results of MASt3R-SLAM as well in the revised manuscript. ORB-SLAM is much lighter in computation than all of VGGT-SLAM, MASt3R-SLAM, and DROID-SLAM. However, these works are trying to accomplish different ends by also using different means. ORB-SLAM is designed to be a lightweight CPU-only visual odometry system that requires calibrated cameras and many complex components. In contrast, methods like VGGT-SLAM and MASt3R-SLAM are dense SLAM systems which can work with uncalibrated cameras and remove multiple degrees of complexity by extending recent Geometric Foundation Models to a SLAM system. Due to recent popularity around Geometric Foundation Models such as VGGT, it is likely that lighter weight versions will soon be available, and since we do not require fine tuning, we look forward to easily integrating future more efficient VGGT models into VGGT-SLAM."}, {"review_id": "zC5EwPUEbU", "reviewer": "Reviewer_Wbeo", "summary": "This paper proposes an approach to monocular SLAM by leveraging the output of VGGT, combined with a homography-based graph optimization strategy. The method aims to resolve projective ambiguities between submaps and improve global consistency.", "questions": "Section 4.2 (Transformation of camera poses via homography): A standard camera matrix is a 3×4 matrix, and if extended to a 4×4 form, its last row should be (0,0,0,1). After applying a general homography matrix, how do you ensure that this structure is preserved? Shouldn't the transformation between two camera matrices be expressed as Pi= Pj H^{-1}, where both Pi and Pj are 3×4 matrices?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n- This is the first work to use VGGT as a front-end for monocular SLAM, and also the first to utilize the Lie algebra of SL(4) for pose graph optimization.\n- It introduces a new method for aligning submaps generated by feed-forward 3D models, addressing projective inconsistencies.\n- The pose graph optimization is formulated using second-order Levenberg–Marquardt (LM) optimization over the Lie algebra of SL(4).\n- The proposed method achieves better or competitive performance compared to current SOTA methods in terms of camera trajectory accuracy and reconstruction quality.\n\n\n**Weaknesses**\n- The paper assumes that the main reason of the global disalignment is the inaccurate intrinsics, but it lacks analysis on the quality of intrinsics from the output of VGGT \n- The paper lacks an evaluation of runtime efficiency and GPU memory consumption, both of which are critical for SLAM applications.\n- The 4×4 homography matrix used in the paper actually belongs to the projective general linear group PGL(4), not SL(4), although both groups have 15 degrees of freedom.\n- There is no direct comparison between the proposed method and the original VGGT. While VGGT alone may not handle large-scale scenes, comparisons on smaller datasets would still provide valuable insights.\n- Minor typo: Line 147 — “a single frames” should be “a single frame.”", "was_revised": true, "final_justification": "The authors provide additional details on the runtime analysis and clarify their choice of using SL(4) instead of PGL(4). Although the proposed SL(4) optimization yields only marginal improvements for SLAM performance, the underlying concept of projective ambiguity is  quite interesting, offering valuable inspiration to the research community.", "rebuttal": "We thank the reviewer for providing detailed feedback and positive discussion of the paper. We address all of the reviewer comments in the following:\n\n[Analysis of intrinsics accuracy] We thank the reviewer for raising an important point about including analysis of the accuracy of intrinsic estimates from VGGT. In preparing VGGT-SLAM, we studied the estimated intrinsics from VGGT and noticed that for images taken from the same camera with constant calibration, focal length estimates from VGGT can have significant variation. We will include plots showing intrinsics estimates compared to the true intrinsic parameters in the revised manuscript for multiple scenes.  We summarize some of the results of VGGT’s estimate of the focal length (fx in pixels) of all keyframes in a scene, where a single camera is used for all images in a scene and thus should have constant focal length. 7-Scenes - avg: 435.1, std: 9.0, range: 59.7; Tabletop (Figure 5 in Supplementary Material) - avg: 679.4, std: 37.7, range: 117.3; Outdoor (Figure 6 in Supplementary Material) - avg: 738.9, std: 51.8, range: 177.3; Office Loop (Figure 2) - avg: 429.4, std: 6.9, range: 33.4. In particular, for the Tabletop and Outdoor scenes (for which in our paper we qualitatively demonstrate improvement with SL(4) over Sim(3)), there is substantial discrepancy in the focal length estimates from VGGT. \n\n[Analysis of runtime and memory] The reviewer raises an important point about including timing results for VGGT-SLAM. On an RTX 4090, VGGT-SLAM with SL(4) for submap size 16 takes 991.6 ms per submap with the following timing breakdown (all in milliseconds): VGGT: 820.2 ms, keyframe detection: 90.7 ms, loop closure detection and SALAD computation: 41.8 ms, relative homography estimation: 31.4 ms, and backend optimization: 7.4 ms. For Sim(3), all steps are the same except the relative estimation takes 11.3 ms instead of 31.4 ms. We will include a table showing a breakdown of timing results in the revised manuscript. Will will also include results showing GPU memory usage compared to submap size, of which the majority of memory is used by VGGT. \n\n[PGL(4) vs SL(4)] We thank the reviewer for detailed feedback on the usage of the PGL(4) group compared to the SL(4) group. There is an important but subtle distinction between the two which justify why we use SL(4) as the manifold for VGGT-SLAM optimization. The 4x4 homography matrix is only up to scale and the reviewer is correct that it belongs to PGL(4). However, the key point is that to optimize multiple homographies in the factor graph, we map each homography to a matrix in SL(4) and perform the optimization on the SL(4) manifold. This is because when computing the residual, it is necessary to compute the distance between homography matrices in a unique manner. For this reason, the matrices are scaled to have determinant one, which maps them to a unique SL(4) matrix, and the optimization occurs on the SL(4) manifold. This can be seen in prior works which perform optimization with the 3x3 homography using SL(3), such as [21] “Homography estimation on the special linear group based on direct point correspondence” - in particular Section II C discusses associating homography matrices to SL(3) . Our manuscript includes a larger set of references in this line of work in the last paragraph of Sec. 2 as follows: “Prior works use optimization on the SL(3) manifold (corresponding to the 8-DOF homography matrix commonly used in image alignment) for aligning multiple images for panoramic stitching [21, 59, 41, 40, 42, 34] and dense SLAM [33].” To increase clarity about this important point, we will change our discussion in the revised manuscript to state that the 4x4 homography matrix belongs to PGL(4) and that we associate the homography matrix to a unique matrix on SL(4) by normalizing it to have determinant one. We believe these revisions will increase the precision of the paper’s description of the homography. \n \n[Direct comparison to VGGT] To fully address the concern raised by the reviewer about not directly comparing to VGGT, we will add comparison results of VGGT compared to VGGT-SLAM. As the reviewer mentions, this will not be possible on larger scenes due to VGGT memory constraints, but we can include results on smaller scenes. Specifically, using the same images selected as keyframes by VGGT-SLAM to downsample the number of frames, we will include comparison results on 7-Scenes for VGGT. Doing this strategy on the TUM RGB-D dataset would exceed memory limits.\n\n[Typo line 147] We thank the reviewer for bringing this to our attention and will fix the typo in the revised manuscript. \n\n[Transformation of camera poses via homography] The reviewer is correct that there is a mistake in the original manuscript and $P_i$ and $P_j$ should be 3x4 matrices where $P_i= P_j H^{-1}$. Our attached code in the Supplementary Material uses the correct equation and we will fix the mistake in the paper. We thank the reviewer for pointing out this important fix."}]

Author rebuttal:

Meta-review: This paper introduces VGGT-SLAM, a dense RGB SLAM system that incrementally and globally aligns submaps using uncalibrated monocular cameras. Unlike prior methods relying on similarity transforms, it leverages SL(4) manifold optimization to estimate 15-DOF homography transformations, addressing reconstruction ambiguity inherent to uncalibrated setups.

The initial concerns of the reviewers can be summarized as follows:

1. Limited experiments (e.g., computational analysis, inaccurate intrinsics) and inconsistent conclusions from experiments (R-enTd, R-wLsM, R-Wbeo).
2. Questionable approach stating that SIM(3) are insufficient (R-wLsM); and sensitivity to outliers and degenerate cases (R-BmpF).

After the discussion between authors and reviewers, most reviewers felt that the rebuttal and discussion clarified most of their concerns. Because the paper presents interesting contributions, and reviewers are supportive of accepting this work, I am recommending acceptance. We encourage the authors to include clarifications from the discussion with the reviewers to a final version.
Decision: Accept (poster)