# LiDAR as a Geometric Prior: Enhancing Camera Pose Tracking Through High-Fidelity View Synthesis
## 2 Related Works  
Camera pose estimation has been a longstanding problem in robotics and computer vision, typically addressed through visual or visual-inertial odometry pipelines. These approaches are broadly categorized into indirect (feature-based) and direct (photometric) methods. Indirect methods, which our work belongs to, build a sparse map of landmarks and track the camera by matching distinctive keypoints. Direct methods, conversely, operate directly on pixel intensities, minimizing a photometric error over often dense or semi-dense regions of the image, which avoids the feature extraction step but can be more sensitive to illumination changes. A dominant and highly successful approach within indirect methods is visual SLAM (Simultaneous Localization and Mapping). These systems build a map of 3D landmarks and track the camera's pose by matching keypoints detected in incoming images to these landmarks. ORB-SLAM2 [3] is a landmark system in this domain, providing a complete solution for monocular, stereo, and RGB-D cameras using its fast and robust ORB (Oriented FAST and Rotated BRIEF) features. Building upon this success, ORB-SLAM3 [4] introduced a multi-map system with tighter visual-inertial integration, significantly improving accuracy and robustness. Similarly, UcoSLAM [5] offers a versatile solution that integrates natural feature points with artificial square markers, enhancing performance in texture-less environments. These highly optimized systems serve as a crucial benchmark for evaluating any new pose estimation pipeline. The task of localizing within a pre-existing map, central to our second proposed method, is known as visual place recognition and is a critical component of large-scale localization. Modern systems often employ a hierarchical approach: first, an image retrieval stage identifies candidate locations in the map (coarse localization), followed by a fine-grained pose estimation step utilizing 3D-2D correspondences. The incorporation of learned features has significantly advanced this paradigm. For instance, SuperPoint [6] introduced a self-supervised CNN (Convolutional Neural Network) to jointly detect keypoints and compute descriptors, while SuperGlue [7] proposed a graph neural network with attention to perform highly accurate feature matching. These components are often combined in hierarchical localization pipelines [8], demonstrating the power of learned features for robust matching and localization, which informs the design of our own map-based tracking. The fusion of LiDAR with visual and inertial modalities has also produced powerful odometry frameworks [9]. FAST-LIVO [10] and its successor FAST-LIVO2 [1] are notable examples of tightly coupled systems that fuse LiDAR, image, and inertial data. These systems enhance pose accuracy via plane priors and real-time dense mapping, which facilitates downstream tasks such as the rendering process used in our work. FusionGS-SLAM [11] extends this direction by incorporating global structural constraints into the optimization pipeline, improving global consistency and supporting photorealistic dense mapping at scale. Complementing these approaches, PRFusion [12] focuses on place recognition, combining LiDAR and image features to enable robust location matching in visually degraded environments. Numerous works address visual localization by matching camera inputs to previously acquired LiDAR maps. Early approaches include monocular [13] and stereo-based methods [14]. Recent systems, such as LM-LVIL [15], improve upon this by directly projecting LiDAR map points into the image frame and minimizing photometric errors. Learning-based approaches, such as CMRNet [16], also tackle camera-LiDAR registration using convolutional neural networks. A fundamental challenge when using rendered data for training or tracking is the sim-to-real domain gap: keypoint detectors trained on real-world images often perform poorly on synthetic renderings due to differences in texture, lighting, and sensor noise.