Improving Map-Free Localization with Depth
Supervision

Hitesh Jain¹ and Sagar Verma¹

Granular AI, Cambridge, MA, USA
{hitesh,sagar}@granular.ai

**Abstract.** We propose an enhancement to the MicKey architecture, a state-of-the-art method for map-free localization, by integrating the Depth Anything network [10] to improve depth estimation. Our approach uses Depth Anything-generated ground truth depth maps during MicKey's training, resulting in more accurate depth predictions, especially in challenging regions like depth discontinuities. Experimental results show that our method maintains competitive performance across key metrics while requiring minimal supervision, thereby improving the versatility and accessibility of the MicKey framework for diverse datasets.
Code: https://github.com/micropilot/dualmapfree

# 1 Proposed Method & Implementation Details

We propose modifying the MicKey architecture, which currently demonstrates exceptional performance on the map-free localization benchmark. MicKey utilizes a feature extractor that partitions the image into patches and computes four different types of features for each patch: 1) a 2D offset (U), 2) keypoint confidence (C), 3) depth value (Z), and 4) descriptor vector (D).

To improve depth learning during MicKey's training, we explored integrating the Depth Anything network [10], which is known for its impressive zero-shot capabilities and reduced generalization error, into MicKey. This network has been trained on a large-scale dataset of approximately 62 million samples and demonstrates lower generalization error. We use this network to generate ground truth depth images from the map-free dataset.

By incorporating Depth Anything-generated ground truth depth maps into MicKey's training, we anticipate several key advantages:

- The supervision provided by these high-quality depth maps can guide MicKey in learning more accurate depth estimations, particularly in regions where traditional feature detectors might struggle, such as areas with depth discontinuities or corners. This additional supervision helps mitigate one of the primary issues with conventional depth estimators, which often fail in such challenging areas due to their independent operation from feature detectors.

- Secondly, the use of Depth Anything reduces the reliance on extensive ground truth depth data, which can be difficult to obtain in certain domains. This aligns with MicKey's design philosophy of requiring minimal supervision,
making it even more versatile and accessible for training across various datasets without the need for comprehensive depth annotations.

<figure><img src="image_1.png" /><figcaption>Fig. 1: Depth Anything network integrated with MicKey training.</figcaption></figure>

Figure 1 illustrates the integration of Depth Anything ground truth during MicKey's training by computing the mean square loss between the ground truth (z') and predicted depth features (Z) and adding this loss to the VCRE loss. To ensure compatibility between the ground truth depth map and the predicted depth map's shape, we partition the ground truth depth map into patches and compute the average depth for each patch. This new term ensures that the network's depth predictions align with the high-quality depth estimates provided by the Depth Anything model. By doing so, we can guide the network to learn more accurate and robust depth representations, leveraging the strengths of the ground truth depth maps without altering the core architecture of MicKey.

## 2 Result

In our evaluation, we compared the modified MicKey architecture with several leading methods in the map-free localization benchmark, as detailed in Table 1.
<table><caption>**Table 1:** Comparison of different methods from map-free localization single frame leaderboard.</caption><thead><tr><th rowspan="2">Method</th><th colspan="2">VCRE&lt;45°</th><th colspan="2">Median Reproj.</th><th colspan="2">Median Error</th></tr><tr><th>AUC Precision</th><th>Error (px)</th><th>AUC Precision</th><th>Trans. (m)</th><th>Rot. (°)</th><th>Err &lt;25cm, 5°</th></tr></thead><tbody><tr><td>MicKey [1]</td><td>0.558</td><td>30.1%</td><td>126.9</td><td>0.283</td><td>12.0%</td><td>1.59</td><td>26.0</td></tr><tr><td>FAR [7]</td><td>0.481</td><td>25.3%</td><td>137.1</td><td>0.392</td><td>17.7%</td><td>1.48</td><td>17.3</td></tr><tr><td>RoMa [4] w/ MicKey</td><td>0.604</td><td>37.8%</td><td>111.9</td><td>0.546</td><td>31.4%</td><td>1.18</td><td>15.6</td></tr><tr><td>SuperGlue [8] w/ MicKey</td><td>0.556</td><td>29.8%</td><td>139.9</td><td>0.490</td><td>23.5%</td><td>1.70</td><td>26.1</td></tr><tr><td>LoFTR [9] w/ MicKey</td><td>0.550</td><td>27.2%</td><td>155.0</td><td>0.467</td><td>20.3%</td><td>1.92</td><td>33.6</td></tr><tr><td>DPT &amp; ASpanFormer [2]</td><td>0.414</td><td>20.8%</td><td>161.8</td><td>0.361</td><td>16.3%</td><td>1.90</td><td>29.2</td></tr><tr><td>DPT &amp; LightGlue w/ DISK</td><td>0.355</td><td>19.5%</td><td>138.8</td><td>0.314</td><td>15.9%</td><td>1.44</td><td>18.5</td></tr><tr><td>DPT &amp; DISK [6]</td><td>0.346</td><td>15.1%</td><td>208.2</td><td>0.264</td><td>10.2%</td><td>2.59</td><td>52.0</td></tr><tr><td>DPT &amp; DeDoDe [3]</td><td>0.325</td><td>16.9%</td><td>167.4</td><td>0.265</td><td>12.5%</td><td>2.02</td><td>30.3</td></tr><tr><td>DPT &amp; SiLK [5]</td><td>0.192</td><td>9.8%</td><td>176.4</td><td>0.157</td><td>7.3%</td><td>2.21</td><td>33.8</td></tr><tr><td>MicKey (GT-Depth)</td><td>0.548</td><td>28.0%</td><td>141.96</td><td>0.273</td><td>10.765%</td><td>1.842</td><td>30.78</td></tr></tbody></table>

Specifically, we compared our approach against the original MicKey [1], FAR [7], RoMa [4] combined with MicKey, SuperGlue [8] integrated with MicKey, and LoFTR [9] paired with MicKey. Additionally, we assessed the performance of combinations of DPT with ASpanFormer [2], LightGlue integrated with DISK, DISK [6] alone, DeDoDe [3], and SiLK [5]. The results demonstrate the competitive performance of our approach, particularly when using Depth Anything-generated ground truth depth maps, highlighting its effectiveness in improving depth estimation within the MicKey framework.

## References

1. Arnold, E., Wynn, J., Vicente, S., Garcia-Hernando, G., Monszpart, Á., Prisacariu, V.A., Turmukhambetov, D., Brachmann, E.: Map-free Visual Relocalization: Metric Pose Relative to a Single Image. In: ECCV (2022)

2. Chen, H., Luo, Z., Zhou, L., Tian, Y., Zhen, M., Fang, T., McKinnon, D., Tsin, Y., Quan, L.: ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer. In: ECCV (2022)

3. Edstedt, J., Bökman, G., Wadenbäck, M., Felsberg, M.: DeDoDe: Detect, Don't Describe — Describe, Don't Detect for Local Feature Matching. In: 3DV (2024)

4. Edstedt, J., Sun, Q., Bökman, G., Wadenbäck, M., Felsberg, M.: RoMa: Robust Dense Feature Matching (2024)

5. Gleize, P., Wang, W., Feiszli, M.: SiLK: Simple Learned Keypoints (2023)

6. Michał J. Tyszkiewicz, Pascal Fua, E.T.: DISK correspondences with DPT-KITTI depth maps (2024)

7. Rockwell, C., Kulkarni, N., Jin, L., Park, J.J., Johnson, J., Fouhey, D.F.: FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimation (2024)

8. Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperGlue: Learning Feature Matching with Graph Neural Networks. In: CVPR (2020)

9. Sun, J., Shen, Z., Wang, Y., Bao, H., Zhou, X.: LoFTR: Detector-Free Local Feature Matching with Transformers (2021)

10. Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., Zhao, H.: Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. In: CVPR (2024)