# DALLoc: Deep-Learning Accelerated Visual Localization Based on Mesh Representation
## 2 Related Work  
The 6 DoF visual localization has been widely studied in recent years. It can be classified into structure-based [5; 6; 7; 8] and image-based approaches [19; 20; 21; 22; 23]. The structure-based approach normally triangulates feature correspondences from the database images to generate a 3D point cloud and represents the scene using this point cloud and database images. Each 3D point has its corresponding local features in the database images to match. In the test process, the 2D-3D matches are established between the local features in a query image and database point cloud using descriptor matching. Then, a PnP method based on these matches is used to localize the query image. Moreover, a hierarchical approach [5; 8] is often used to handle different illumination conditions, seasonal changes, and large scenes with numerous database images. In most hierarchical localization pipelines, first, an image retrieval [24; 25] stage is used to select the most relevant database image set for feature matching. The selected images range from 10 to 50, depending on different conditions. The 3D points for localization are often restricted to being visible in these images. Second, the 2D-3D matches of query features and database 3D points are established via explicit descriptor matching. Finally, the query pose can be obtained based on these matches. Meanwhile, the image-based approaches represent the 3D geometry implicitly by the weights of a machine learning model. Refs. [26; 27; 28; 29] adopt the scene coordinate regression techniques, which use the neural network to regress the 2D-3D matches instead of computing them via explicit descriptor matching. There are also some techniques use the network to be a pose regressor. Refs. [19; 20; 21; 30; 31] use the network to regress the absolute poses, and [32; 33] choose to regress the relative pose from a known database image. Though these scene coordinate regressors achieve state-of-the-art results for small scenes, the pose calculated from these methods are not competitive with the structure-based methods in more challenging scenes, even when [31; 34] use the new view synthesis technology to generate more images for training the network. Generally, the 3D point cloud used for scene representation is sparse and is generated from Sfm. One 3D point normally has many local features to match in different database images. For a large scene with millions of 3D points and many more 2D features, the mismatch of a 2D feature and 3D point is unavoidable when generating the point cloud. A covisable filtering [35; 36] can reduce the mismatches by removing the 3D points that might not be visible together. Additionally, RANSAC can help to calculate the poses and find the best solution with maximal data support. A dense 3D model [14; 15; 16] is an alternative to sparse point clouds and database images for scene representation. Different dense types ranging from dense multi-view stereo and laser point clouds to colored or textured meshes, have been used to represent the scene. Different from [15], which is mainly focused on coarse image localization (on the level of hundreds of meters), ref. [14] proposed a full pipeline to centimeter-accurate image localization based on mesh representation. This mesh-based method is still structure-based and obtains the 3D points from the depth map rendered from the mesh. Moreover, using the mesh to establish 2D-3D correspondences in runtime is more lightweight for memory usage than storing the whole Sfm model and images [14]. However, no matter which forms represent the scene, the final step of localization is to perform a RANSAC-based PnP solver for pose estimation. In each iteration, it samples a minimal set of 2D-3D correspondences to calculate pose. This step is time-consuming because it requires many iterations to refine the optimal solution from the noisy data. Therefore, the robustness of 2D-3D matches used to calculate relative pose is essential to improve the performance of the localization algorithm.