Abstract. Detecting interest points is a key component of vision-based estimation algorithms, such as visual odometry or visual SLAM. Classically, interest point detection has been done with methods such as Harris, FAST, or DoG. Recently, better detectors have been proposed based on Neural Networks. Traditionally, interest point detectors have been designed to maximize repeatability or matching score. Instead, we pursue another metric, which we call succinctness. This metric captures the minimum amount of interest points that need to be extracted in order to achieve accurate relative pose estimation. Extracting a minimum amount of interest points is attractive for many applications, because it reduces computational load, memory, and, potentially, data transmission. We propose a novel reinforcement-and ranking-based training framework, which uses a full relative pose estimation pipeline during training. It can be trained in an unsupervised manner, without pose or 3D point ground truth. Using this training framework, we present a detector which outperforms previous interest point detectors in terms of succinctness on a variety of publicly available datasets. Fig. 1 . State-of-the art interest point detectors are designed to yield a large amount of repeatable interest points. Instead, we propose to train a network to detect a minimum amount of interest points that are sufficient to obtain accurate relative pose estimates. Previous methods typically extract several hundred interest points for pose estimation. In contrast, our detector extracts less than one hundred interest points. The per-pixel score produced by our detector is shown on the right. Circles indicate extracted interest points (inliers in green, outliers in red). Green lines indicate correct matches.
SIPS REF uses a ranking loss to estimate a concise set of interest points.
SIPS: Unsupervised Succinct Interest Points
IEEE International Conference on 3D Vision (3DV), Qu\'ebec City, 2019
Computer Science