We propose a fundamentally novel approach to real-time visual odometry for a monocular camera. It  Tracking a hand-held camera and recovering the threedimensional structure of the environment in real-time is among the most prominent challenges in computer vision. In the last years, dense approaches to these challenges have become increasingly popular: Instead of operating solely on visual feature positions, they reconstruct and track on the whole image using a surface-based map and thereby are fundamentally different from feature-based approaches. Yet, these methods are to date either not real-time capable on standard CPUs [11, 15, 17] or require direct depth measurements from the sensor [7] , making them unsuitable for many practical applications. In this paper, we propose a novel semi-dense visual odometry approach for a monocular camera, which combines the accuracy and robustness of dense approaches with the efficiency of feature-based methods. Further, it computes highly accurate semi-dense depth maps from the monocular images, providing rich information about the 3D * This work was supported by the ERC Starting Grant ConvexVision and the DFG project Mapping on Demand far close Figure 1 . Semi-Dense Monocular Visual Odometry: Our approach works on a semi-dense inverse depth map and combines the accuracy and robustness of dense visual SLAM methods with the efficiency of feature-based techniques. Left: video frame, Right: color-coded semi-dense depth map, which consists of depth estimates in all image regions with sufficient structure. structure of the environment. We use the term visual odometry as supposed to SLAM, as -for simplicity -we deliberately maintain only information about the currently visible scene, instead of building a global world-model. Feature-based monocular SLAM. In all feature-based methods (such as [4, 8] ), tracking and mapping consists of two separate steps: First, discrete feature observations (i.e., their locations in the image) are extracted and matched to each other. Second, the camera and the full feature poses are calculated from a set of such observations -disregarding the images themselves. While this preliminary abstraction step greatly reduces the complexity of the overall problem and allows it to be tackled in real time, it inherently comes with two significant drawbacks: First, only image information conforming to the respective feature type and parametrization -typically image corners and blobs [6] or line segments [9] -is utilized. Second, features have to be matched to each other, which often requires the costly computation of scale-and rotation-invariant descriptors and robust outlier estimation methods like RANSAC. Dense monocular SLAM. To overcome these limitations and to better exploit the available image information, dense monocular SLAM methods [11, 17] have recently been proposed. The fundamental difference to keypoint-based approaches is that these methods directly work on the images
Semi-dense visual odometry REF is a pioneering work that tracks a monocular camera in real-time using direct alignment algorithm.
Semi-dense Visual Odometry for a Monocular Camera
2013 IEEE International Conference on Computer Vision
2013 IEEE International Conference on Computer Vision
Computer Science