Feature Query Networks: Neural Surface Description for Camera Pose Refinement (CVPR 2022)
Channel: Dmytro Mishkin
Date: 2022-06-20

Feature Query Networks: Neural Surface Description for Camera Pose Refinement

 by Hugo Germain (Ecole des Ponts ParisTech)x, Daniel DeTone (Facebook), Geoffrey M Pascoe (Facebook), Tanner Schmidt (Facebook Reality Labs), David Novotny (Facebook AI Research), Richard Newcombe (Facebook), Chris Sweeney (Facebook Reality Labs), Richard Szeliski (The University of Washington), Vasileios Balntas (Facebook Reality Labs)

Accurate 6-DoF camera pose estimation in known envi- ronments can be a very challenging task, especially when the query image was captured at viewpoints strongly differ- ing from the set of reference camera poses. While structure- based methods have proved to deliver accurate camera pose estimates, they rely on pre-computed 3D descriptors coming from reference images often misaligned with query images. This discrepancy can subsequently harm downstream cam- era pose estimation tasks. In this paper we introduce the Feature Query Network (FQN), a ray-based descriptor re- gressor that can be used to query descriptors at known 3D locations under novel viewpoints. We show that the FQN is able to model viewpoint-dependency of high-dimensional keypoint descriptors and bring significant relative improve- ments to structure-based visual localization baselines.

CVPR 2022 Image Matching Workshop https://image-matching-workshop.github.io