CogniMap3D: Cognitive 3D Mapping and Rapid Retrieval
Authors: Feiran Wang
Junyi Wu
Dawen Cai
Yuan Hong
Yan Yan
Venue: ICLR 2026 Poster (2026)
Keywords: Feed-Forward 3D; Dynamic Scene Understanding
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We present CogniMap3D, a bioinspired framework for dynamic 3D scene understanding and reconstruction that emulates human cognitive processes. Our approach maintains a persistent memory bank of static scenes, enabling efficient spatial knowledge storage and rapid retrieval. CogniMap3D integrates three core capabilities: a multi-stage motion cue framework for identifying dynamic objects, a cognitive mapping system for storing, recalling, and updating static scenes across multiple visits, and a factor graph optimization strategy for refining camera poses. Given an image stream, our model identifies dynamic regions through motion cues with depth and camera pose priors, then matches static elements against its memory bank. When revisiting familiar locations, CogniMap3D retrieves stored scenes, relocates cameras, and updates memory with new observations. Evaluations on video depth estimation, camera pose reconstruction, and 3D mapping tasks demonstrate its state-of-the-art performance, while effectively supporting continuous scene understanding across extended sequences and multiple visits.

Published as a conference paper at ICLR 2026
COGNIMAP3D: COGNITIVE 3D MAPPING AND RAPID RETRIEVAL
Feiran Wang's Junyi Wu', Dawen Cai', Yuan Hong", Yan Yan University of Illinois Chicago, Universitt of Michigan, University of Connecticut
Depth Maps Initial Poses:
Joint Optimization
Time
Motion Cue
Update
Updated Scene
Static Scene
Memory Bank
Scene Memory.
Refined Poses
Figure 1:
CapDD maintains a cognitive mapping system that recalls, stores, and updates memories. Given an imput video, it outputs camera poses and point clouds by isolating static scenes through motion cues, interacting with its memory bank, and optimizing across multiple visits.
ABSTRACT
We present CogniMap3D, bioinspired framework for dynamic 3D scene un- derstanding and reconstruction that emulates human cognitive processes. Our ap- proach maintains persistent memory bank of static scenes, enabling efficient spa- tial knowledge storage and rapid retrieval. CogniMap3D integrates three core ca- pabilities: multi-stage motion cue framework for identifying dynamic objects, a cognitive mapping system for storing, recalling, and updating static scenes across multiple visits, and a factor graph optimization strategy for refining camera poses. Given an image stream, our model identifies dynamic regions through motion cues with depth and camera pose priors, then matches static elements against its memory bank. When revisiting familiar locations, CogniMap3D retrieves stored scenes, relocates cameras, and updates memory with new observations. Evalu- ations on video depth estimation, camera pose reconstruction, and 3D mapping tasks demonstrate its state-of ofate-of-the-ar performance, while effectively supporting continuous scene understanding across extended sequences and multiple visits.
INTRODUCTION
Humans exhibit remarkable ability to process dynamic visual scenes: our attention naturally prioritizes moving objects while simultaneously constructing persistent spatial representations of static environments (Abrams & Christ 2003 Franconeri & Simons] 2003). For instance, during an equestrian performance shown in Fig. observers unconsciously notice the motion of horse and rider, while extracting depth cues and motion parallax to separate moving objects from static back- grounds (Rogers & Graham 1979: Born & Bradley 2005). Based on spatial representations, the hippocampus constructs internal "cognitive maps" in an egocentric reference frame (Eichenbaum, 2015 Burgess] 2006). When revisiting familiar environments, humans reliably recall static scenes even when dynamic elements have changed, facilitating efficient navigation and spatial reasoning with minimal cognitive load (O (O'keefe & Nadel 1979 Epstein et 2017
"Corresponding Author
"The code is available
cogn woognimap3i

---

Published as a conference paper at ICLR 2026
Inspired by these human cognitive processes, aim build 3D cognitive mapping systems that can similarly distinguish dynamic objects from static backgrounds while maintaining persistent mem- ory of statie 3D environments. The challenge lies in developing systems that can simultancously: (1) distinguish between static and dynamic scene elements in monocular videos, (2) construct and maintain persistent representations of static environments and efficiently recall and update these representations when revisiting familiar scenes, and (3) establish stable camera pose estimates that remain geometrically consistent despite the presence of dynamic objects.
Recent advances in 3D reconstruction have made significant progress in related areas. Monocular
depth estimation (MDE) works (Bian et al. 2021] Ranfil et al. 2021 Yin et al.) 2022 Bhat et al. 2023
Godard et al.) 2019 Yang et al. [2244] Li & Snavely 2018) estimate precise 3D information
but fail to localize camera poses. Visual SLAM approaches (Agarwal et al. 201 Campos et al.
2021
Schonberger & Frahm 2016 Davison et al. 2007 Pollefeys et al. 2008 Mur-Artal et al.,
2015
achieve accurate camera poses but typically require additional camera intrinsics and precise
initialization Visual foundation models (VFM) (Wang et al., 2024 Zhang et al. 2024; Duisterhol
ct al. 2024; Wang et al. 2025bla a directly regress 3D geometry and camera poses from RGB images, establishing solid foundation for dynamic scene reconstruction. However, these approaches lack
the cognitive mapping capabilities needed for persistent memory and scene revisitation.
Building human cognitive mechanisms and recent advances in visual foundation models, present CogniMap31:: Cognitive 3D Mapping and Rapid Retrieval, a comprehensive framework for dynamic scene understanding that emulates human cognitive processes with three key capabilities: 1) A multi-stage motion cue framework that accurately separates dynamic objects from static back- grounds through progressive refinement of 2D-3D motion cues; 2) A cognitive mapping system that creates, recalls, and updates memory of static environments, enabling efficient scene recognition and relocalization across multiple visits; 3) A geometrically consistent camera pose optimization strat- egy that stabilizes predicted parameters through factor graph optimization focused on static regions.
Specifically, given an image stream, CogniMap3D first predicts initial camera parameters and depth information through Visual Foundation Model (Wang et al. (2025a). Our motion cue framework then progressively identifies dynamic objects through a coarse-to-fing approach combining optical flow clustering, geometry-based motion analysis, and 3D keypoint refinement. With the accurate static scene representation, our system efficiently matches hybrid features from 2D keyframes and 3D static geometry against the memory bank, verifying potential matches through geometric align- ment of static point clouds. Upon recognizing a familiar environment, CogniMap3D recalls the stored static scene, relocates the camera pose, and updates the memory with new observations, enabling continuous scene refinement. To enhance geometric consistency, we implement a factor graph optimization that jointly refines camera poses using constraints from both newly observed static regions and updated memory.
We evaluate CogniMap3D on various 3D tasks, including consistent video depth estimation, camera pose reconstruction, and 3D reconstruction, achieving competitive or state-of-the-ar performance. Our experiments demonstrate the system's 's ability to efficiently recall previously stored environ- ments, update them with new observations, and maintain a coherent memory bank that supports continuous scene understanding across extended sequences and multiple visits to the same scene.
2 RELATED WORK
Foundation Models for 3D Reconstruction. Directly predicting 3D geometry from RGB images
offers significant flexibility for real-world applications. Monocular depth estimation (MDE) (Bian)
al. 2021 Ranfil et al. 2021 Yin al. 2022 Bhat et al. 2023 Godard al., 2019 Yang al. has demonstrated robust generalization across diverse scenes but lacks camera pose information and temporal consistency in videos. DUSL3R (Wang et al., 2024) pioneered pointmap representation for scene-level 3D reconstruction, implicitly inferring both camera pose and aligned point clouds from image pairs. Subsequent approaches (Lu et al.) 2024 Zhang et al.) 2024 Sucar et al. 2025 Wang & Agapito| 2024 Duisterhof et al. 2024) extended this framework but required processing videos as numerous image pairs with time cimee-oonuminng optimization. Re- cent advances toward online processing include CUT3R (Wang al. 2025b), which implements stateful recurrent network for incremental pointmap refinement, and VGGT (Wang et al.) (025a)),
2024
Li & Snavely] 2018) 2011)
which employs feed-forwart model for joint prediction of camera poses and 3D) geometry without
2

---

Published as a conference paper at ICLR 2026
Flow Geometry Cue
Robust
Optical Flow
Background Foo
Festure Motch
GMM
Oleu
Points Move
Mean Move
O
Input Video
Depth&Pose
Coarse Mesk
Fine Mask
Robust Mask
Prompl Point
Refine&Trane Track
Multi-stage Motion Cue for Locating Dynamic Area. Given pair of images in video, we first predict the initial depth and camera pose through VFM to establish 3D prior. Our pipeline then processes three specialized motion cues through progressive 2D-3D interaction effectively iso- lates robust dynamic regions, enabling accurate refinement and tracking across subsequent frames.
Figure 2:
post-processing. However, these visual foundation models focus mainly on immediate observations without mechanisms for persistent scene understanding.
Dynamic Scene Reconstruction. Visual foundation models (VFM) for dynamic scene reconstruc- tion (Zhang et al. 2024 Team et al. 2025 Ravi et al.
2024 Chen et al. 2025) aim to recover
3D geometry when both camera and scene elements are in motion. Recent approaches exhibit dis- tinet trade-offs: MonST3R (Zhang al. 2024) extends DUS:3R through optical flow but employs threshold-based detection with limited generalization; MegaSAM (Li et al.] 2024) utilizes neural networks for motion prediction but suffers from domain transfer issues; AETHER (Team et al,, 2025) leverages SAM2 (Ravi et al. 2024) for segmentation but struggles with elements out of preset category; and BA-Track (Chen al. (2255) implements 3D tracking-based decoupling but requires camera intrinsic priors. Our sulti-stage motion cue framework achieves robust dynamic-statie sep- aration through progressive refinement, combining optical flow clustering, geometry-bast based motion analysis, and 3D keypoint refinement.
Structure from Motion and Visual SLAM. Classical SLAM methods (Agarwal al. 2011 Cam
pos et al. 2021 Schonberger & Frahm 2016 Davison et al. 2007 Pollefeys et al. [0088 Mur- AuraArtal
et al.) 2015) estimate camera poses through feature matching and bundle adjustment but struggle with textureless regions. Learning- based approaches like Droid-SLAM (Teed & Deng. 202 ad- vance differentiable optimization yet exhibit limited generalization to dynamic environments. Re- cent developments for handling dynamic scenes include MegaSAM's (Li al., 2024) probability predictions, DPVO's (Teed et al.) (003) patch-based features, MASt3R-SIM (Duisterhof et al.] 2024) learned feature integration, Anycam's (Wimbauer et al.) 2025 depth-optical flow combina- tion, and BA-Track's (Chen et al. 2025) point decoupling. However, these systems typically require camera intrinsics and maintain internal states that conflict with VFM's predictions. Our factor graph optimization framework refines VFM "FM-predicted camera poses using multi- view constraints on static regions, yielding consistent trajectories compatible with foundation model outputs.
METHOD
Our approach takes monocular videos input achieve dynamic understanding with per- sistent spatial memory. The pipeline consists of three integrated components: a multi-stage motion cue framework that identifies dynamic objects, a cognitive mapping system that creates, recalls, and retrieves memories, and factor graph optimization strategy that refines camera trajectories.
3.1 MULTI-STAGE MOTION CUE FRAMEWORK
We propose efficient pipeline for identifying dynamic objects across scenes in monocular videos with moving cameras, as illustrated Figure 2 Given initial depth maps D, camera poses E estimated by VGGT (Wang et al. 2025a), we implement progressive refinement process.
3

---

Published as a conference paper at ICLR 2026
Input
Ground Truth
Monst3R
Ours
Figure 3: Dynamic Mask Comparison. We visualize dynamic regions as white overlays on input images. Compared with MonST3R, method achieves more complete and precise masks.
Flow Motion Cue. To identify potential dynamic regions, we first compute the optical flow field fte-t" F. flow (I',1") I and partition into K distinct components via Gaussian Mixture Model clustering. We then get coarse mask by excluding the component with minimal motion magnitude:
M (z,8) = ((((f (z,8)) +
arg min
Σ (u,r)EC,
1 | v)))
(1)
ie sE(
Cil
where E(.) assigns cluster labels and C, = {u,, v) l(f (u, u)) if represents pixels in cluster.
Geometry Motion Cue. To distinguish moving objects from optical flow caused by camera movement, we follow similar principle as MonST3R (Zhang et al. 2024) but with a more ro- bust implementation. We unproject images into 3D pointmaps ps and pt,, where P(x, y) ((I,))(K*) y, _]". By transforming P through the relative camera pose and projecting onto It's image plane, we compute the expected flow for static scene elements. Subtracting this static scene flow prediction from the observed optical flow reveals motion caused by dynamic objects:
F Fite (x, v)
F t- Row
(x,
[ (K" E" (E"))- pt.z)) (z, v) w)]l|
where >--) denotes projection camera extrinsic parameters, and Pt (z, y) the back-projected 3D point through corresponding depth
the image plane, Kt is the intrinsic calibration matrix, E the
D(x, v). We derive the geometry motion cue M Ma-" 24-1" (a, method to determine threshold automatically.
11 [F res (z. T) using Otsu's s
Robust Motion Cue. Leveraging dynamic region candidates from previous stages (Mite and
geo we further refine dynamic object localization by matching keypoints between frames and analyzing their correspondenices. After transforming matched keypoints into world coordinates, compute mean 3D displacement vector. Within candidate regions, keypoints whose displacement significantly deviates from this mean in either magnitude direction are classified dynamic. The final dynamic mask (Mitn corresponds regions containing these outlier keypoints.
Dynamic Areas Tracking. After precisely identifying dynamic areas, uniformly extract prompt points within these regions, using SAM2 (Ravi et al., (024) to refine dynamic masks and track- ing dynamic areas across subsequent frames. Throughout the video, we continuously monitor the geometry motion cue MMgo for new moving objects, executing our pipeline when changes are de- tected and updating Mayn accordingly. As shown in Fig. 3 our method provides accurate dynamic segmentation that serves as a foundation for both cognitive mapping and camera pose optimization.
3.2 COGNITIVE MAPPING SYSTEM
Inspired by the human ability to retain and recall static elements familiar scenes, we design a cognitive map system that stores, recalls, and updates 3D scene representations, shown in Fig.
4

---

Published as a conference paper at ICLR 2026
Update
Create
Mumory Bunk
Keyframes
DINOY
Time
Poinilets
Pointmap
Features
Sample
Input Video
Dypa &Confidence
Static Scene
Scene in Memory
Update Scene
Cognitive Mapping System. Given the input video, we estimate per-frame dynamic mask with prior of the depth, confidence, camera pose. DINOv2 and Pointnet+ encode selected static images and static scene into latent features respectively. We then match features with global feature table, failed, new memory slot is created; otherwise the corresponding memory is up- dated, enabling fast recall, relocalization, and refinement of the current scene.
Figure 4:
Memory Bank Creation. We construct scalable memory banks using dual representation strategy that integrates 3D geometric and 2D visual features from static scenes. For 3D) features, we filter point clouds retain only static regions with high confidence, then structure them using an octree hierarchy with adaptive voxel downsampling. These point clouds are encoded with PointNet++ (Qi al., 7a) balancing memory efficiency with geometric fidelity. For 2D features, we extract global visual embeddings from static image regions using DINOv2 (Oquab et al. (023)) creating compact representations of each viewpoint. We select representative keyframes by maintaining consistent feature distances between consecutive frames, ensuring balanced visual coverage.
The resulting memory bank is a hierarchical structure, with each scene assigned unique identifier (map_id). Each map stores a static point cloud 'Pstatic E R x3 and its associated 2D and 3D features. For efficient retrieval, we implement global feature table using hash-based approximate nearest neighbor principles and a companion mapping file. This two-tier design separates fast visual search from subsequent loading of geometric data, enabling rapid scene matching and recall.
Memory Recall and Relocalization. For observations. employ two-stage approach to determine the location has been previously mapped. First, we match each query frame's 's static features against memory bank by computing L.2 distances in feature space. Each successful match casts a vote for its corresponding map, allowing us to identify candidate environments through sequence-level consensus rather than relying on single-frame comparisons.
For the highest-voted candidate map, we conduct geometric verification by aligning its stored point cloud (Pmup) with the query sequence's static point cloud (Pquery) using ICP. We focus the ab- solute count of inlier correspondences and their RMSE, enabling robust matching even with partial scene overlap. This approach effectively distinguishes true relocalization opportunities from visu- ally similar but geometrically distinct environments. Upon successful verification, we obtain the precise 6-DoF camera pose within the map coordinate system, enabling immediate reuse of previ- ously optimized scene representations for subsequent mapping and tracking operations.
Memory Update. Following successful relocalization, we update two core components of our mem- ory system: First, we enhance visual recognition by extracting features from new keyframes and adding them both the global feature table and the matched map's 's feature set. This enriches visual references for future recognition from multiple viewpoints. Second, refine geometric representa- tion by transforming the current static point cloud into the matched map's coordinate system using the obtained camera pose. We then merge the aligned data with the existing map and apply consistent voxel downsampling to eliminate redundancy while maintaining resolution quality.
This dual update strategy extends coverage to previously unobserved regions while improving curacy in overlapping areas. The updated static scene in the memory bank serves three critical functions: enriching the persistent environmental model, refining the current point cloud through
5

---

Published as a conference paper at ICLR 2026
integration of prior knowledge, and providing stronger constraints for subsequent camera trajectory optimization. Each revisit creates a progressive cycle where better recognition enables more precise updates, completing the cognitive loop of storage, recall, and refinement.
3.3 CAMERA TRAJECTORY OPTIMIZATION
We propose a factor graph optimization approach to refine camera trajectories, enhancing global geometric consistency through static scene constraints from both current observations and memory.
Initial Landmark Selection. Reliable landmark selection is crucial for effective optimization. To ensure high-quality geometric constraints, extract candidate landmarks exclusively from static regions (1 (1-Mayn) with high confidence values. For scenes recognized from memory, we transform stored static points into the current coordinate frame using the alignment transformation computed during memory recall, serving as additional landmarks with established 3D positions, providing stronger geometric constraints. Landmark association employs two-step verification process with an adaptive threshold Tdiat = max( maax(mmin- dscene a) that scales with scene dimensions.
Factor Graph Optimization. Our approach jointly optimizes camera poses T, SE(3) and 3D landmarks L E R by minimizing:
X argminx x Σ ΣIC(E)1
JEF
represents the optimal state. We incorporate three complemen- tary constraints: To establish accurate geometric correspondence, define projection factors that ensure 3D landmarks align with their 2D observations, as well as prior factor which anchors the coordinate system:
where X = ([T)) -ررر
Sproy
(T) ziy; Sprior (To) = To e To
(4)
where is the projection function, 2ij is the observed 2D point, To is the initial camera pose estimate, and E represents the difference in the SE(3) manifold. To encourage physically plausible motion, we enhance trajectory smoothness with inter-frame motion constraints:
( = (T: @ (I
(5)
which naturally penalizes deviations from initial relative transformations between consecutive frames. The complete cost function uses the Huber loss function p for robustness:
C(X) = 1fprior prion
Σ
Σ
Ifmotion 12 motson
pros
(iJ)EO
When revisiting familiar scenes, landmarks retrieved from memory assigned higher confidence by downscaling their projection covariance (Eproj <- Eppoo 0 < a < 1). We minimize C(X) using the Revenberg-Marquuare algorithm with a Huber loss, and apply standard stability enhancements including rotation re-orthogonalization via SVD and adaptive thresholding for outlier rejection.
EXPERIMENTS
CogniMap3D processes monocular videos of dynamic scenes, providing accurate depth estimates, camera poses, and persistent scene memory. We evaluate against specialized methods for depth estimation, camera tracking and 3D reconstruction, while also demonstrating our system's 's unique capabilities for scene recognition and memory updates across multiple visits.
Baselines. We compare CogniMap3D against -fate-of-the-ar methods that approach different as- pects of dynamic scene understanding. Our primary set of basclines includes Spann3R (Wang & Agaaito 2024), MonST3R (Zhang et al. 2024), CUT3R (Wang et al., (0255), and VGGT (Wang et 2025a). MonST3R extends DUSBR (Wang et al., (024) to handle dynamic scenes by in- tegrating optical flow analysis for motion segmentation, while Spann3R employs spatial memory mechanisms to process variable-length sequences. CUT3R implements stateful recurrent model for continuous scene refinement with each new observation. VGGT serves as our foundation model baseline for direct regression of geometric information. While these methods provide strong base- lines for depth and pose estimation, our approach uniquely integrates long-term scene memory ca- pabilities for efficient recognition and update of previously visited environments.

---

Published as a conference paper at ICLR 2026
Table 1: Video Depth Evaluation. We report scale&ssiff--mvariant depth accuracy and FPS. Methods requir-
ing global alignment are marked "GA", while "Optim." " and "FF" indicate optimization and online methods.
Stnte Butter et al. 2012
BONN Paluzoolo BONNPPaluzolo et dl
2019 KETT/ et 2013
Category
Mechod
Optim. FF Abs Rel
6:c1.25
Abs Rel
6:c1
Abs Rel
6=1125
FFS
Depth Extimation
ChrunoDepit Slan et
DepthsAnytlinn- V2 Yasp et 2024
0.687
0.367
48.6
55.4
0,100
0.106
92.1
91.1
0.167
0.140
80.4
75.9
3.13
1.89
model
DeptCrrteee
0.242
0.075
58.1
DUSISR-GA Wane
81.8
MASIBR GiLaor
0.167
0.137
33.6
Vision Roundation Model
MouST3R.CM00 COASTT33..A//2ppP
0.333
0.066
73.8
Spunn Wine A 470
0.508
50.8
0.157
82.1
0.207
73.0
13.55
CUT3R Wine ar 232250
0.454
55.7
0.174
94.5
11100
86.7
10.58
VEGT Wang (0250
01799
62.4
0.054
07.1
0.072
96.4
21.5
Ours
0.295
68.6
0.858
97.9
0.069
96.2
14.32
Table 2: Quantitative Results of 3D reconstruction. Evaluation on 7-Scenes dataset shows our
approach achieves comparable results to SOTA methods.
7-Scemes 7-Scenes/hootton Shotton et ul. 2013
Method
Optim,
FF
Acci, Mean
Acc| Med.
Compl Mean
Compl Med.
NC1 Moan
NC1 Med.
FPS
DUSGR- GA Wang al.
0.1
0.077
0.181
Mon8 F3R-GX Zhuin et 2024
0.248
0.185
0.266
0.167
0.672
0.759
0.39
CUT3R Wane aL
0.126
0.047
0.154
0,031
0.727
0.831
17.0
Wang
0.088
0.092
0.040
21.5
Ours
0
0.041
0.089
0.039
0.751
0.863
14.3
4.1
VIDEO DEPTH ESTIMATION
Datasets and Metries. Following the benchmarks of previous works (Hu et al. 2024 Zhang al. 2024 Wang et al. 20255), our evaluation uses Sintel (Butler et al. 2012). KITTI (Geiger et al, 2013 and Bonn (Palazzolo et al. 2019] datasets, covering synthetic and real-world environments across indoor and outdoor settings. We report absolute relative error (Abs Rel) and percentage of inlier points with 8 < 1.25, applying per-sequence scale and shift alignment. We denote "FF" as methods that obtain predictions from a single forward pass of a vision foundation model without pairwise multi-vies reconstruction or test-time gradient-bass optimization, and "Optim." as meth- ods that perform explicit pairwise reconstructions over image pairs.
Results. Table shows CogmiMap3D achieves competitive performance across most datasets. Our method outperforms models designed for static scenes like DUSL3 and Spann3R, while match- (Yang et al. 2024). Though our memory system introduces slight computational overhead, CogniMap3D remains faster than optimization- approaches, effectively balancing accuracy and efficiency.
ing VGGT and surpassing specialized depth estimation networks like
4.2 3D RECONSTRUCTION AND ANALYSIS
Qualitative Analysis on Dynamic 3D Reconstruction. We compare the reconstruction quality of CagDDMap3D with MonST3R (Zhang et al. (2024) and CUT3R (Wang et al. 2025b) the DAVIS (Perazzi et al. 2016) and KITTI (Geiger et al.] 2013) datasets, as shown in Fig. MonST3R processes image pairs for dynamic scene reconstruction but lacks global consistency, resulting in visible noise. CUT3R improves upon this with a state-based approach to integrate observations, yet still suffers from error accumulation that causes progressive misalignments across sequences. In contrast, our method builds upon and enhances VGGT's foundation through our multi-stage mo- tion cue analysis, accurately separating dynamic elements from static backgrounds and producing cleaner, more consistent reconstructions across all test scenarios.
Reconstruction with Memory. key advantage of CogniMap3D is its cognitive mapping sys- tem, demonstrated in the bottom rows of Fig. 5 When revisiting environments, our system recog- nizes familiar scenes by matching current observations against stored visual and geometric features, recalls the corresponding memory, and integrates new static information. We visualize this capa- bility by rendering stored memory scenes with higher brightness to distinguish them from newly observed elements. The visualization reveals how subsequent visits integrate additional static ele- ments while maintaining overall scene structure, enabling long-term environmental understanding that more closely mimics human cognitive spatial memory.
Quantitative Analysis on 3D Reconstruction. We evaluate our method on the 7-Scenes dataset using accuracy (Acc), completion (Comp), and normal consistency (NC) metrics. Following prior
7

---

Published as a conference paper at ICLR 2026
Input Video
Monst3R
CUT3R
Ours
Recall Memory
Memory
Memory
Memory
Update Scene
Figure 5: Qualitative Results of Dynamic 3D Reconstruction. We compare our method with concurrent works MonST3R (Zhang al. 2024) and CUT3R (Wang al. 2025b). Our method achieves cleaner reconstructions with better preservation of both static and dynamic elements. The bottom rows demonstrate CogniMap3D's 's unique capability to store previous scenes in memory and recall them upon revisitation. We render stored memory scenes with higher brightness to distinguish them from newly observed scene.
works (Wang al. 2025b Zhu al. 2022 Wang al.) 2024 Wang Agapito 2024). we evaluate
using 3-5 frames per scene. As shown in Table 2 our method achieves comparable results, especially on accuracy and completion metries.
4.3 CAMERA POSE ESTIMATION
Datasets and Metrics. To rigorously evaluate CogniMap3D's s camera pose estimation capabili- ties, employ three complementary datasets that present distinct challenges: Sintel (Butler al.) 2012) with its elaborate dynamic content, TUM-dynamics (Sturm et al.) 2012) featuring real-world dynamic scenes with ground truth trajectories, and ScanNet (Dai et (017)) assess general- ization to static environments with diverse architectural layouts. Following previous works (Wang
al. 20256, Chen al. 2024 Zhang al. 2024), our quantitative assessment utilizes three key metrics: Absolute Translation Error (ATE) for global trajectory consistency, and Relative Pose Error (RPE) in both translation (RPE trans) and rotation (RPE rot) measure incremental positional and rotational accuracy over standardized distances.
Results. Table presents comprehensive comparison of camera pose estimation methods across three distinct categories. The first category comprises SLAM-based approaches designed for camera pose estimation, which demonstrate high accuracy but require ground truth camera intrinsics as input. The second category includes optimization-base vision foundation models, which achieve impressive results through scene reconstruction via feature matching but at the cost of computational efficiency. Most notably, CogniMap3D excels in the third category of achieving superior performance across all datasets with an ATE of 0.176 on Sintel and 0.012 on TUM-dynamies, outperforming competing approaches such DUSE3R, Spann3R, and CUT3R.
methods (FF),
8

---

Published as a conference paper at ICLR 2026
Table 3: Camera Pose Estimation Evaluation on Sintel, TUM-dynamic, and ScanNet datasets. We group methods into (I) SLAM-based methods requiring intrinsics, (II) optimization-basee VFM methods, and (III) cerd-forward VFM methods.
Sinte Butler et al.
2012
TUM-dynamic Sturm al. 2012
ScanNo Dai cl
2017
Category Method
ATE L RPE trans
RPE rot
ATE RPE trans
RPE rot
|TEE RPE trans
RPE rot
DROID-SLAM Teol & Dong
2021
0.175
0,084
1,912
DPVO
0.115
0.072
LEAP Ten 2224
0.089
0.066
1.250
0.068
0.008
1,686
0.070
0.018
0.535
Robust-CVD Kopf
0.360
0,154
0.227
0.064
CasualSAM Zhang et al. 20c9
0.141
0.035
0.615
0.071
0.010
1.712
0.158
0.034
1.618
11
DUSE3 GA Winz of
20.24
0.417
0.250
5.796
0.083
0.017
3.567
0.081
0.028
0.784
MASI3R-GA
0.185
0,060
0
MorST3R Gk Grizning cral.
2021
0.111
0.044
0.869
0.098
0.019
0.935
0.077
0,018
0.529
DUSE3R Wanr
2024
0.290
0,132
7,869
0.140
0.106
3,286
0.246
0.108
8,210
Spann3t Want & Armioo
2024
0.329
0.110
4,471
0.056
0.021
0.591
0.096
0.023
0.661
III
CUT3R Wang eral 20231
0.213
0,066
0.621
0.046
0.015
0.473
0.099
0.022
0,600
VGGT Wrnn
0.189
0.069
0.529
0.028
0.020
0.350
0.023
0:015
0.326
Ours
0.176
0.068
0.600
0.012
0.010
0.311
0.019
0.011
0.331
Table Memory Recall Analysis.
Table Camera Pose Analysis.
Table Memory Size.
Method
Acc|
Compl
NC1
Method
ATE|
RPE,
Number Accuracy (%)
MonST3R-GA
0.248
0.266
0.672
Baseline
0.024
0.334
100
Ours
0.086
0.089
0.751
PaP-RANSAC
0.025
0.510
50
96
Ours Update
0.082
0.085
0.789
DPVO
0.019
0.510
100
97
Ours
0.012
0.311
200
97.5
4.4 ABLATION STUDY
Memory Recall Analysis. Our model continuously recalls, updates, and stores 3D scenes in mem- ory. When processing image streams from previously visited environments, it retrieves stored rep- resentations to assist current scene understanding. We demonstrate this capability on the 7-Scenes first initialize the memory bank with single
dataset as shown in Table 4 For evaluation, randomly selected frame from each scene. Leveraging memory recall, our method with updated memory outperforms both baseline methods and our model without memory.
Camera Pose Methods. We evaluate multiple methods for stabilizing initial camera poses from VFM. Our baseline is established without camera refinement, memory recall, or any pose adjustments beyond static scene estimation. Tab. [5] indicate that existing methods struggle: PaP+RANSAC (Gao al.) (003) suffers from poor temporal consistency, while learning-base based methods like DPVO (Teed et al., (023) maintain internal states incompatible with VFM's prior. Our factor graph optimization jointly refines camera extrinsics and landmark positions, reducing trajectory error and maintaining rotation precision.
Memory Matching, We evaluate CogniMap3D's memory matching on DAVIS (Perazzi et al. 2016) by dividing 50 scenes into thirds and performing 200 pairwise matches between segments. As Table shows, our system maintains high accuracy increasing memory sizes, demonstrating robust feature-based matching for effective scene recognition and camera pose refinement.
CONCLUSION
We presented CogniMap3D,, bioinspired framework for dynamic scene understanding that emu- lates key aspects of human cognitive processing through three complementary capabilities; a multi- stage motion cue framework that progressively distinguishes dynamic objects from static back- grounds, a cognitive mapping system that creates and maintains persistent environmental memory, and a camera pose refinement strategy that establishes reliable coordinate frames through factor graph optimization. Our comprehensive evaluation demonstrates superior performance depth es- timation, camera pose estimation, and 3D reconstruction across diverse datasets.

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our research targets scientific exploration of dynamic scene understanding and 3D scene memory systems, with potential benefits for autonomous navigation, augmented reality, and assistive tech- nologies. We also recognize risks such as inadvertent privacy breaches and misuse for surveillance tracking. Any deployment should follow transparent usage protocols and comply with applicable privacy regulations and ethical guidelines. This work is developed for academic research purposes, and we discourage applications that could infringe individual privacy.
REPRODUCIBILITY STATEMENT
We aim make COGNIMAP3D fully reproducible. The main paper (Sec. 3) specifies the end- to-end pipeline (motion cues, memory, factor-graph refinement) and evaluation protocols (ATE, RPE trans/rot). The supplementary material (Sec. B provides implementation details, including model backbones and third-party components, all hyperparameters, random seeds, dataset splits and preprocessing, and configuration specifications. The code repository is linked on the first page.
ACKNOWLEDGEMEET
This research is supported by NSF IIS-2525840, CNS-2432533, CNS-2432534, ECCS-2514574, NIH IRF IRFIMH133664-00 and Cisco Research unrestricted gift. This article solely reflects opinions and conclusions of authors and not funding agencies.
REFERENCES
Richard Abrams and Shawn E Christ. Motion onset captures attention. Psychological science, 14 (5):427-432, 2003.
Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M Seitz, and Richard Szeliski. Building rome day. Communications of the ACM, 54(10):105-112, 201
Shariq Farooq Bhat, Reiner Birki, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero- shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302.12288 2023.
Jia- Wang Bian, Huangying Zhan, Naiyan Wang, Tat-Jun Chin, Chunhua Shen, and Ian Reid. Auto- rectify network for unsupervised indoor depth estimation. IEEE transactions on pattern analysis and machine intelligence, (((12)):802-9811)) 2021.
Richard T Born and David C Bradley. Structure and function of visual area mt. Annu. Rev. Neurosci., 2871):157-189 2005.
Neil Burgess. Spatial memory: how egocentric and allocentric combine. Trends in cognitive sci- ences, 10(12):551-557, 2006.
Daniel Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In Computer Eision-ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12, pp. 611-625. Springer, 2012.
Carlos Campos, Richard Elvira, Juan Gómez Rodríguez, José MM Montiel, and Juan D) Tardós. Orb-slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. IEEE transactions robotics, 37 2021.
Weirong Chen, Le Chen, Rui Wang, and Mare Pollefeys. Leap-vo: Long-term effective any point tracking for visual odometry. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19844-19853, 2024.
Weirong Chen, Ganlin Zhang, Felix Wimbauer, Rui Wang, Nikita Araslanov, Andrea Vedaldi, and Daniel Cremers. Back on track: Bundle adjustment for dynamic scene reconstruction. arXiv preprint arXiv:2504. (0l,:22044.4516,,, 2025.
10

---

Published as a conference paper at ICLR 2026
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 582555828253 2017.
Andrew J Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. IEEE transactions on pattern analysis and machine intelligence, 29(6): 1052-1067,
2007.
Bardienus Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, and Jerome Revaud. Mast3r-sfm: a fully-integrater solution for unconstrained structure-from pnncuurefroo-mmotionn motion. arXiv preprint arXiv:2409. 19152, 2024.
Howard Eichenbaum. The hippocampus as a cognitive map... of social space. Neuron, 87 2015.
Russell A Epstein, Eva Zita Patai, Joshua B Julian, and Hugo J Spiers. The cognitive map in humans: spatial navigation and beyond. Nature neuroscience, 20(1 I): 150--1113,, 2017.
Steven L Franconeri and Daniel J Simons. Moving and looming stimuli capture attention. Perception & psychophysics, 65(7):999- 1010, 2003.
Xiao-Shan Gao, Xiao-Rong Hou, Jianliang Tang, and Hang-Fei Cheng. Complete solution clas- sification for the perspective-three-pontt problem. IEEE transactions pallern analysis and machine intelligence, 25(8):930-943 2003.
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The 2211....,,.013.
kitti dataset. The international journal robotics research,
Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel Brostow. Digging into self- supervised monocular depth estimation. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pp. 3828-3838, 2019.
Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open- ppendworld videos. arXiv preprint arXiv:2409.02099 2024.
Johannes Kopf, Xuejian Rong, and Jia-Bin Bin Huang. Robust consistent video depth estimation. Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1611- 1621, 2021.
Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision, pp. 71-91. Springer, 2024.
Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp-
2041-2050, 2018
Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast, and robust struc- ture and motion from casual dynamic videos. arXiv preprint arXiv:2412.04455 2024.
Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wenping Wang, and Yuan Liu. Align3r: Aligned monocular depth estimation for dynamic 2024.
videos. arXiv preprint arXiv:2412.088
Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: A versatile and accurate monocular slam system. IEEE transactions on roboties, 3145):1147-1163, 2015.
John O'keefe and Lynn Nadel. Précis of o'keefe nadel's the hippocampus as cognitive map. Behavioral and Brain Sciences. 2(4):487-494, 1979.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arX1v:2304.007/93, 2023.
11

---

Published as a conference paper at ICLR 2026
Emanuele Palazzolo, Jens Behley, Philipp Lottes, Philippe Giguere, and Cyrill Stachniss. Refu- sion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7855-7862. IEEE, 2019.
Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Lue Van Gool, Markus Gross, and Alexander Sorkine- Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmen- tation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 724-732, 2016.
Mare Pollefeys, David Nistér, J-M Frahm, Amir Akbarzadeh, Philippos Mordohai, Brian Clipp. Chris Engels, David Gallup, S-J Kim, Paul Merrell, al. Detailed real-time urban 3d reconstruc- tion from video. International Journal of Computer Vision, 18:143-167,,000
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++:: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017b.
René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international conference compuler vision, pp. 12179-12 188, 2021.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma. Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714. 2024.
Brian Rogers and Maureen Graham. Motion parallax independent cue for depth perception. Perception, 8(2): 134, 1979.
Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motiie revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104-4113, 2016.
Jiahao Shao, Yuanbo Yang. Hongyu Zhou, Youmin Zhang, Yujun Shen, Vitor Guizilini, Yue Wang, Matteo Poggi, and Yiyi Liao. Learning temporally consistent video depth from video diffusion priors. arXiv preprint arXiv:2406. 01493, 2024.
Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb-d images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2930-2937, 2013.
Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A bench- mark for the evaluation of rgb-d slam systems. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 573-580. IEEE, 2012.
Edgar Sucar, Zihang Lai, Eldar Insafutdinov, and Andrea Vedaldi. Dynamic point maps: A versatile representation for dynamic 3d reconstruction. arXiv preprint arXiv:2503. /63///03...3..... 2025.
Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector- free local feature matching with transformers. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 8922-8931, 2021.
Aether Team, Haoyi Zhu, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Chunhua Shen, Jiangmiao Pang. al. Acther: Geometric-awee unified world modeling. arXiv preprint arXiv:2503. /8945, 2025.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision- ECC- 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pp. 402-419. Springer, 2020.
12

---

Published as a conference paper at ICLR 2026
Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems, 14115588-15569 2021.
Zachary Teed, Lahav Lipson, and Jia Deng. Deep patch visual odometry. Advances in Neural Information Processing Systems, 36:39033-3901,, 2023.
Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv preprint arXiv:2408. /6061, 2024.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vegt: Visual geometry grounded transformer. arXiv preprint arXiv:2503. 11651, 2025a.
Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Con- tinuous 3d perception model with persistent state. arXiv preprint arXiv:2501. 12387, 2025b.
Shuzhe Wang. Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 20697-20709, 2024.
Felix Wimbauer, Weirong Chen, Dominik Muhle, Christian Rupprecht, and Daniel Cremers. Any- cam: Leaminng to recover camera poses and intrinsics from casual videos. arXiv preprint arXiv:2503.23282, 2025.
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10371-10381, 2024.
Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Simon Chen, Yifan Liu, and Chunhua Shen. Towards accurate reconstruction of 3d scene shape from single monocular image. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(5):6480-649,, 2022.
Junyi Zhang, Charles Hermaann Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, De- qing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arXiv:2410.03825. 2024.
Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Rubinstein, Noah Snavely, and William T Freeman. Structure and motion from casual videos. In European Conference on Computer Vision, pp. 20-37. Springer, 2022.
Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin Os- wald, and Mare Pollefeys. Nice-slami Neural implicit scalable encoding for slam. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 77786-1796,, 2022.
13

---

Published as a conference paper at ICLR 2026
Appendix: Cognitive 3D Mapping and Rapid Retrieval
The appendix provides limitations and future directions (Sec. A), detailed implementation specifi- cations including model backbones, and configurations (Sec. B), visualizations of our multi-stage motion cue pipeline (Sec. C), and supplementary results covering robustness analysis, large-scale reconstruction, ablation studies, qualitative comparisons, and memory footprint evaluation (Sec. D).
A LIMITATION
Although we have successfully implemented a memory bank system for 3D scene recall with a high matching success rate when revisiting similar environments, mismatching incidents occasionally troduce noise into the system. Further optimization of our matching algorithm is required to ensure more stable and consistent performance across varied environmental conditions. Additionally, our point cloud registration accuracy is contingent upon the quality of static scene representation, which can lead to potential misalignments between point clouds captured from identical scenes under dif- ferent conditions or perspectives. Factors such lighting variations, occlusions, and viewpoint changes can impact registration accuracy. We plan to address these limitations through robust fea- ture extraction techniques and adaptive registration algorithms in our future research initiatives.
B IMPLEMENTATION DETAILS
We incorporate VGGT (Wang et al., 2025a) provide initial depth estimation and camera pose pri- ors for our system. Our method is fully implemented in PyTTrch and all experiments are conducted on an NVIDIA A6000 GPU with 48GB VRAM. For optical flow computation between consecutive frames, employ RAFT (Teed & Deng 2020) with resolution of 840x480 pixels, while feature matching across non-consecutive images is performed using LoFTR (Sun et al. 2021) with shared self and cross attention mechanisms. To efficiently encode downsampled point clouds into compact 3D feature representations, we implement a modified version of PointNet++ (Qi et al. (017b) with three abstraction layers and feature dimensions of 128, 256, and 512 respectively. For visual fea- ture extraction, we utilize the backbone architecture of VGGT, specifically leveraging DINOv2 with its self-supervised training paradigm to encode keyframes into rich 3D feature representations with a dimensionality of 1024. Our implementation of Perspective-n-Pots Point (PnP) and Random Sample Consensus (RANSAC) algorithms closely follows the methodology outlined in (Gao et al., (003), with an inlier threshold of 2.0 pixels and maximum of 1000 iterations. Finally, we implement DPVO (Teed et al.) 2023) using the same camera intrinsic parameters as VGGT to maintain consis- tency in our visual odometry pipeline.
For each incoming frame, we perform full VGGT forward pass to obtain depth and camera pose, and reuse its intermediate DINOv2 features as 2D descriptors, invoked. RAFT and SAM2 are both applied downsampled resolution to estimate optical flow and track masks. The memory-related back-end is invoked every 20 frames rather than at every frame, since neighboring frames observe very similar content. At these keyframes, we first perform 2D memory recall over the feature bank; only when high-confidence candidates are found do activate 3D geometric validation and a mactor-grapt update. The 3D validation uses PointNet++ features and ICP on downsampled sparse point clouds of retrieved static landmarks before fusing them into the global map.
additional DINOv2 model is
C VISUALIZATION OF MULTI-STAGE MOTION CUE
Our multi-stage motion cue framework processes video sequences to accurately segment dynamic objects complex scenes with moving cameras. As illustrated in Fig. and Fig. 7 this progressive refinement approach consists of four key stages:
First, we compute optical flow between consecutive frames using RAFT TTed Deng 2020 capturing motion information throughout the scene. The resulting flow fields (shown in the top-left of each figure) contain motion vectors for both dynamic objects and background affected by camera movement.
14

---

Published as a conference paper at ICLR 2026
Frame 1
Frame
Frame 10
Frame 12
Optical Flow
Flow Motion Cue
Robust Motion Cue
Dynamic Mask
Background Flow
Flow Res
GeometryMotion Cue
Prompt Point for Refine
Frame 1
Frame
Frame 10
Frame 12
Figure 6:
Motion Cue Process of the Train Scene in DAVIS dataset.
Frame
Frame 4
Frame
Frame
i
Optical Flow
Flow Motion Cue
Robust Motion Cue
Dynamic Mask
Background Flow
Flow Res
GeometryMotion Cue
Prompt Point for Refine
Frame
Frame 4
Frame 10
Frame 12
Figure 7: Motion Cue Process of the Street Scene in KITTI dataset.
Second, we implement two parallel motion cue extraction processes. The Flow Motion Cue isolates potential dynamic regions by partitioning the optical flow field into distinct components via Gaussian Mixture Model clustering, excluding components with minimal motion magnitude. Simultaneously. we compute the Background Flow by transforming pointmaps through relative camera poses and projecting them onto image planes.
Third, we generate the Flow Residual by subtracting the expected background flow from the ob- served optical flow, effectively highlighting motion caused exclusively by dynamic objects. This residual is thresholded using Otsu's 's method to produce the Geometry Motion Cue, which more accurately distinguishes genuine object motion from samera-induced apparent motion.
Finally, we derive the Robust Motion Cue by analyzing keypoint correspondences between frames and identifying outlier displacements. This refined information produces our Dynamic Mask, with
15

---

Published as a conference paper at ICLR 2026
red dots indicating prompt points for further refinement using SAM2. The bottom row of each figure demonstrates how our approach maintains consistent dynamic object segmentation across multiple frames (1, 10, and 12), effectively handling diverse scenarios ranging from model train sets with predictable motion patterns to real-world street scenes with cyclists and pedestrians.
D SUPPLEMENTARY RESULTS AND ANALYSES
D.1 ROBUSTNESS TO INITIALIZATION ERROR
To further assess robustness to initialization enrors, we add zero-mean Gaussian noise with rotation standard deviation nn and translation standard deviation a. to the initial poses, and compare tra- jectory errors on the -'Mmicymmiic dataset. The comparison between the VGGT baseline and our method under different levels pose perturbation is reported in Table
Table 7: Robustness to noisy pose initialization on TUM-dynamic.
Method
Noise level n///o,)
ATE
RPE trans
RPE rot Perturbation level
VGGT
0.028
0.020
0,350
Ours
0.0%/0.00 m
0.012
0.010
0,311
none
Ours
1.0%/0.01 m
0.013
0.011
slight 0.318
Ours
3.0%/0.03 m
0.015
0,012
0,331
moderate 0.343
Ours
5.0%/0.05
0.019
0,014
severe
Under slight and moderate perturbations, CogniMap3D maintains performance close to the clean initialization. Even with severe perturbations of 59, it still outperforms the VGGT baseline, indicat- ing robustness to reasonably large pose initialization noise.
D.2 RECONSTRUCTION IN LARGE-SCALE OUTDOOR SCENES
CogniMap3ID designed maintain stable memory of static structure that be efficiently reused when scenes are revisited, so that repeated long-term visits can enhance geometric consis- tency instead of accumulating drift. To evaluate this ability in more challenging, open-world settings, we consider KITTI, a standard large-scale outdoor benchmark with street and highway driving se- quences, containing repeated viewpoints, moving vehicles, pedestrians, and strong viewpoint and illumination changes.
Table 3D reconstruction on KITTI (outdoor driving sequences).
Method
Acc| Mean
Acc| Med.
Compl, Mean
Compl, Med.
NC1 Mean
NC1 Med.
CUT3R
0.089
0.058
0.108
0.078
0.895
0.912
VGGT
0.071
0.045
0.089
0.061
0.913
0.935
Ours
0.052
0.036
0.073
0.049
0.942
0.951
As shown in Table 8 CogniMap3D attains lower accuracy and completeness errors and higher nor- mal consistency than CUT33 and VGGT on KITTI. Within this benchmark, these results indicate that the proposed memory mechanism can be beneficial not only on smaller indoor settings evalu- ated in the main paper but also in large-scale, long-term outdoor sequences with dynamic objects and revisits.
D.3 ABLATION OF 2D, 3D FEATURES, AND GEOMETRIC VALIDATION
In CogniMap3D, memory recall and validation are implemented three-stage pipeline rather than as independent modules. First, 2D visual features are used for coarse candidate retrieval from the memory bank. Second, the candidates based 3D geometric validation decides whether a candidate is accepted and fused into the map, or rejected so that
refined using 3D geometric features. Finally, ICP-
memory entry is created.
To evaluate the contribution of components, we perform an ablation study on DAVIS dataset under the scene-matching setting of Table T with memory bank of size 200. We compare: (i) 2D-only,
16

---

Published as a conference paper at ICLR 2026
Table 9: Memory ablation on DAVIS (scene matching).
Variant
3D features
ICP validation
Accuracy t (%)
Throughput t (q/s)
2D-only
93.5
18.2
2D+3D, no ICP Full (2D+3D+ICP)
/
X
96.8
10.5
/
/
97.5
6.8
which directly accepts the top-1 match from coarse candidates; (ii) 2D+3D, no ICP, which augments with 3D features but omits geometric validation; and (ii) Full, which uses the complete 2D+3D+ICP recall accuracy and matching throughput (queries per second):
pipeline. We report
As shown Table 9 2D-only matching offers the highest throughput but lower accuracy. Adding 3D features improves recall with moderate reduction in speed. The full three-stage stage variant further increases accuracy at the cost of additional computation, illustrating the trade-off between robustness and efficiency in the memory recall module.
D.4 QUALITATIVE COMPARISON ON DYNAMIC SCENES
To illustrate the performance of Cag on dynamic scenes, we provide a qualitative compari- son with VGGT on a KITTI sequence in Fig. g The scene contains a moving eyclist and pedestrians, both of which induce strong motion relative to the static background.
In the VGGT reconstruction, the moving cyclist appears with smeared and partially duplicated ge- ometry, and the wheels and upper body are less clearly delineated. The pedestrians on the right also exhibit blurrier shapes and less coherent structure. In contrast, the CogniMap3D reconstruc- tion shows more complete and visually consistent shape for the cyclist, including better-defined wheels and torso, and provides sharper, more coherent geometry for the pedestrians. this exam- ple, these visual differences suggest that the proposed dynamic-mann pipeline helps produce cleaner reconstructions around moving objects while preserving the static background.
VGGT
CogniMap3D
Figure 8: Qualitative comparison KITTI sequence. We compare reconstructed point clouds rendered over the input image for VGGT (top) and CogniMap3D (bottom). The red boxes highlight regions with moving area (cyelist and pedestrians). In this example, CogniMap3D yields more complete and visually coherent reconstructions of the moving regions.
17

---

Published as a conference paper at ICLR 2026
D.5 MEMORY FOOTPRINT
We examine the memory footprint of CogniMap3D. In our implementation, the persistent scene memory is stored the CPU downsampled static landmarks with compact feature descriptors. indexed by a hash table. This memory is maintained in host RAM and can grow with the number of revisited scenes.
At inference time, only the subset of landmarks that are relevant to the current frame is transferred the GPU for recall and optimization. As result, GPU usage largely dominated by the VGGT forward pass rather than by the size of the stored memory. Measured on our setup, VGGT alone uses about 11.7 GB of GPU memory, while CogniMap3D uses about 11.9 GB due the additional memory and optimization modules. This small increase indicates that the GPU memory consump- tion does not grow proportionally with the total size of the scene memory and remains close that of rumning VGGT alone.
D.6 EFFECT OF MEMORY ON LONG-SEQUENCE POSE ESTIMATION
To analyze how the proposed memory mechanism affects camera pose estimation on long trajecto- ries with natural revisits, we evaluate CogniMap3D on several KITTI driving sequences of extended length. We compare three variants: (i) the VGGT backbone, which provides the initial depth and pose estimates; (ii) CogniMap3D without memory, which applies motion cues and a static-only factor graph but does not recall or fuse past scenes; and (iii) the full CogniMap3D with memory- enabled recall and map fusion. We report absolute trajectory error (ATE), translational relative pose error (RPE trans), and rotational relative pose error (RPE rot), averaged over the selected sequences.
Table 10: Long-sequence pose evaluation on KITTI.
Method
Memory
ATE
RPE trans |
VGGT
0.092
0.034
0.421
Ours w/o mem
0.068
0.026
0.367
Ours w/ mem
0.060
0.023
0.352
As shown in Table [0] both CogniMap3D variants yield lower ATE and RPE than VGGT these long outdoor trajectories, and enabling memory provides further improvements on all three pose metrics. Within this evaluation, the results suggest that, these sequences, the memory module can provide additional benefits for pose accuracy and consistency beyond the VGGT backbone and the no-memory variant.
E
LLM USAGE
Large language models (LLMs) were used only aid in wording and polishing the writing. They were not involved in the research design, methodology, experiments, or analysis.
18


Reviews:
[{"review_id": "IRHSratTiy", "reviewer": "Reviewer_GYbG", "summary": "This paper proposes CogniMap3D, a cognitively inspired framework for long-term 3D scene understanding from monocular videos. The method emulates human spatial cognition by maintaining a persistent memory of static environments while filtering dynamic elements and refining camera poses through recalled geometric anchors. It integrates multi-stage motion cues for dynamic object suppression, a dual 2D–3D embedding memory for scene recall and update, and a factor-graph–based trajectory optimization constrained by static landmarks. Experiments on diverse datasets, including Sintel, KITTI, and 7-Scenes, demonstrate that CogniMap3D achieves state-of-the-art or comparable performance in depth estimation, camera pose reconstruction, and 3D mapping, offering a robust and efficient solution for continual scene understanding under dynamic conditions.", "questions": "CogniMap3D shares conceptual similarities with prior methods such as Neural Map, iMAP, and NICE-SLAM, all emphasizing scene memory and revisitation-based localization. While its “cognitively inspired” design is reasonable, the paper does not clearly explain how the proposed dual-modality memory differs from these works in map representation, retrieval, or update strategy. A clearer articulation of these distinctions would better highlight the paper’s unique contribution.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Conceptual originality. The paper introduces a cognitively inspired formulation of 3D scene understanding that explicitly models long-term memory, recall, and update—bridging human cognitive mapping theories with modern video foundation models. This conceptual framing is both novel and timely for the community’s growing interest in continual, memory-based perception.\n2. Technical coherence. The pipeline is well structured: multi-stage motion cues, a dual-modality memory, and factor-graph optimization are tightly integrated, yielding a clear causal chain from dynamic-object filtering to stable pose refinement. Each component is motivated by a specific limitation in existing VFMs and validated through comprehensive experiments.\n3. Empirical thoroughness. Experiments span multiple datasets and tasks (depth, pose, reconstruction) with detailed ablations demonstrating the contribution of each module. The method achieves state-of-the-art or comparable performance while maintaining efficiency, highlighting both the practicality and scalability of the proposed framework.", "weaknesses": "While the paper is conceptually strong and empirically well-supported, several technical limitations remain that constrain its general applicability and robustness.\n\n1. The proposed multi-stage motion cue framework heavily relies on the accuracy of the underlying Video Foundation Model (VFM), particularly the depth and pose priors obtained from VGGT. Since these priors are directly used to compute geometric residuals and dynamic masks, any failure of the VFM in low-texture or high-illumination-variance regions could propagate errors through the entire pipeline, suggesting limited robustness to imperfect geometric initialization.\n2. The memory recall mechanism lacks explicit safeguards against false-positive retrievals. Although the paper mentions geometric verification via ICP, it does not clarify how mismatched recalls are detected or handled. In the absence of clear rejection or fallback strategies, incorrect memory associations could introduce erroneous landmarks into the factor graph optimization, potentially leading to catastrophic trajectory drift.\n3. The experimental validation focuses mainly on small- to medium-scale or short video sequences, such as 7-Scenes and Sintel, which do not fully reflect the challenges of large-scale, long-term, and highly dynamic environments. The paper’s claim of “long-term cognitive mapping” would be more convincing if evaluated under extended temporal settings or realistic outdoor sequences exhibiting significant environmental changes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mZq3I74Znb", "reviewer": "Reviewer_rprw", "summary": "This paper presents CogniMap3D, a biologically inspired framework for the problem of understanding and reconstructing dynamic 3D scenes from monocular videos. The core contribution is the design of an integrated pipeline, including a multi-stage motion cue separation module, a cognitive mapping system, and a camera trajectory optimization strategy. The cognitive system is capable of creating, storing, recalling, and updating persistent \"memories\" of static scenes and is designed to mimic the spatial cognitive ability of humans to revisit familiar environments. Experimental results on several standard datasets such as Sintel, KITTI, TUM-dynamics, and 7-Scenes show that the framework achieves competitive performance on multiple tasks.", "questions": "1. With regard to the reported 14.32 FPS, the authors should clarify exactly what modules are included in the metric and clarify whether it covers the complete, computationally intensive back-end flow, from memory retrieval and geometry validation to factor graph optimization.\n2. The authors need to supplement the discussion on the robustness of the cascaded framework under initial estimation errors. If the system design includes a specific mechanism to mitigate this error propagation, the principle needs to be explicitly stated.\n3. The authors should conduct experiments to quantify the specific contribution of the 3D geometry validation link. For example, how does the system's performance change after using only 2D visual features for scene recall and relocation (i.e., removing 3D validation)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Systematically introducing the concept of \"cognitive memory\" into dynamic scene reconstruction is a novel and visionary attempt, which directly addresses the key challenge of transitioning from processing isolated video clips to achieving long-term, persistent environmental perception.\n2. The paper is clearly articulated, effectively conveying its complex system architecture and core ideas through high-quality illustrations, enabling readers to clearly understand its workflow and contributions.", "weaknesses": "1. Although 14.32 FPS is reported in Table 1, this speed is quite amazing considering the complexity of the whole system (integrating VGGT, RAFT, DINOv2, PointNet++, SAM2, etc.). It is recommended that the authors more clearly state which modules are covered by this FPS test.\n2. The multi-stage, cascaded architecture of the framework raises a key concern that small biases in upstream modules may be amplified later, suggesting that the authors briefly discuss the robustness of the system to initial estimation errors.\n3. The paper claims that its key advantage is to handle long-term sequences and scene revisits, but the relevant quantitative evaluation is only carried out on small-scale relocation datasets such as 7-Scenes. This framework lacks validation in larger scale and more challenging long-term real environments.\n4. Ablation of memory systems has not been adequately studied, and it would be more compelling to provide a more nuanced analysis of what contributions 2D visual features and 3D geometric features make in scene recall and validation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k2E3t6gCV4", "reviewer": "Reviewer_RgAW", "summary": "This paper proposes a feed-forward 3D network for dynamic scenes. It separates static and dynamic regions using dynamic mask and introduces a memory system. The results showed more stable camera pose and depth estimation under motion compared with baselines.\n\nContributions:\n\n1. A multi-stage motion-cue module that progressively refines the dynamic mask using geometry and optical flow, which is more stable than the MonST3R masks.\n2. A memory system for mapping, recall, and update, enabling robust relocalization.", "questions": "1. Can you provide **head-to-head comparisons with raw VGGT outputs** on scenes with **more dynamic objects**, and analyze the module’s concrete gains on moving regions?\n2. Can you report **memory-induced improvements** across **additional benchmarks** (eg. pose esitimation)?\n3. Can you **evaluate memory on longer sequences** to demonstrate practicality and its effect on drift/relocalization beyond VGGT?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper provides clear writing and figures. The dynamic mask pipeline is easy to follow.\n2. Geometry and flow cues plus global mean move refine masks and stabilize tracking. The results showed that 3D reconstruction is more stable than baselines in metrics and visualization.\n3. Using 2D/3D features supports fast indexing, and voting + ICP verifies matches. The memory further supplies constraints that help the global optimization of camera trajectories.", "weaknesses": "1. VGGT comparison gap. The model initializes pose and depth with VGGT, yet lacks direct, controlled comparisons against VGGT in the visualizations; reconstruction metrics are also close. Given VGGT is not specifically optimized for dynamic scenes, should include more dynamic motion benchmarks and report VGGT vs. your method to substantiate the value of dynamic-mask extraction.\n2. Lack evidence for the memory module. While the paper states that memory provides stronger constraints for trajectory optimization, the ablation focuses on 3D reconstruction only. The specific contribution of memory to pose accuracy/consistency remains under-substantiated. Consider testing memory on longer sequences and evaluating improvements on tasks/settings where VGGT might struggle (e.g., long-horizon drift, relocalization).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3127b1cjBE", "reviewer": "Reviewer_tg31", "summary": "This paper introduces CogniMap3D, a bio-inspired framework for dynamic 3D scene understanding that mimics human cognition by maintaining a persistent memory of static environments. The core innovation lies in integrating three capabilities: a multi-stage motion detection system to identify dynamic objects, a cognitive mapping system for storing and recalling static scenes across multiple visits, and a factor graph optimization for refining camera poses. Evaluations show state-of-the-art performance in depth estimation, pose reconstruction, and 3D mapping, enabling continuous scene understanding over extended periods and repeated visits to the same location.", "questions": "In Table 1, the proposed method belongs to 'FF' category. However, different from CUT3r and VGGT which are eng-to-end, the proposed method requires an optimization process to refine camera trajectory. Is the optimization feed-forward?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The cognitive 3D mapping is intuitive and novel, which effectively decouple static and dynamic parts of the scene.\n\n2. The writing quality, experimental results and visualization are good.", "weaknesses": "1. The performance of CogniMap3D is close to VGGT in all experiments, but the inference speed is much lower than VGGT. More discussion on the performance is required.\n\n2. What is the memory footprint of the proposed method. Does the proposed memory mechanism require high GPU memory?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)