In many robotic domains such as flexible automated manufacturing or personal assistance, a fundamental perception task is that of identifying and localizing objects whose 3D models are known. Canonical approaches to this problem include discriminative methods that find correspondences between feature descriptors computed over the model and observed data. While these methods have been employed successfully, they can be unreliable when the feature descriptors fail to capture variations in observed data; a classic cause being occlusion. As a step towards deliberative reasoning, we present PERCH: PErception via SeaRCH, an algorithm that seeks to find the best explanation of the observed sensor data by hypothesizing possible scenes in a generative fashion. Our contributions are: i) formulating the multi-object recognition and localization task as an optimization problem over the space of hypothesized scenes, ii) exploiting structure in the optimization to cast it as a combinatorial search problem on what we call the Monotone Scene Generation Tree, and iii) leveraging parallelization and recent advances in multi-heuristic search in making combinatorial search tractable. We prove that our system can guaranteedly produce the best explanation of the scene under the chosen cost function, and validate our claims on real world RGB-D test data. Our experimental results show that we can identify and localize objects under heavy occlusion--cases where state-of-the-art methods struggle.
The idea of using search to explain' scenes was popular in the early years of 2D computer vision: Goad @cite_12 promoted the idea of treating feature matching between the observed scene and 3D model as a constrained search while Lowe @cite_8 developed and implemented a viewpoint-constrained feature matching system. Grimson @cite_16 introduced the Interpretation Tree to systematically search for geometrically-consistent matches between scene and model features, while using various heuristics to speed up search. Our work is also based on a search system, but it differs from the aforementioned works in that the search is over the space of full hypothesized rendered scenes and not feature correspondences. In fact, our proposed algorithm does not employ feature descriptors at all.
@cite_16 @cite_12 @cite_8
2068204893 2108808334 2010945382
How an object appears in an image is determined in part by interactions with other objects in the scene. Occlusion is the most obvious from of interaction. Here we present a system which uses 3D CAD models in combination with optical and range data to recognize partially occluded objects. Recognition uses a hypothesize, perturb, render, and match cycle to arrive at a scene-optimized prediction of model appearance. This final scene-optimized prediction is based upon an iterative search algorithm converging to the optimal 3D pose of the object. During recognition, evidence of terrain occlusion in range imagery is mapped through the model into the optical imagery in order to explain the absence of model features. A similar process predicts the structure of occluding contours. Highly occluded military vehicles are successfully matched using this approach. We propose a powerful pipeline for determining the pose of a query image relative to a point cloud reconstruction of a large scene consisting of more than one million 3D points. The key component of our approach is an efficient and effective search method to establish matches between image features and scene points needed for pose estimation. Our main contribution is a framework for actively searching for additional matches, based on both 2D-to-3D and 3D-to-2D search. A unified formulation of search in both directions allows us to exploit the distinct advantages of both strategies, while avoiding their weaknesses. Due to active search, the resulting pipeline is able to close the gap in registration performance observed between efficient search methods and approaches that are allowed to run for multiple seconds, without sacrificing run-time efficiency. Our method achieves the best registration performance published so far on three standard benchmark datasets, with run-times comparable or superior to the fastest state-of-the-art methods. 3D object recognition in scenes with occlusion and clutter is a difficult task. In this paper, we introduce a method that exploits the geometric scale-variability to aid in this task. Our key insight is to leverage the rich discriminative information provided by the scale variation of local geometric structures to constrain the massive search space of potential correspondences between model and scene points. In particular, we exploit the geometric scale variability in the form of the intrinsic geometric scale of each computed feature, the hierarchy induced within the set of these intrinsic geometric scales, and the discriminative power of the local scale-dependent invariant 3D shape descriptors. The method exploits the added information in a hierarchical coarse-to-fine manner that lets it cull the space of all potential correspondences effectively. We experimentally evaluate the accuracy of our method on an extensive set of real scenes with varying amounts of partial occlusion and achieve recognition rates higher than the state-of-the-art. Furthermore, for the first time we systematically demonstrate the method's ability to accurately localize objects despite changes in their global scales. Accurately determining the position and orientation from which an image was taken, i.e., computing the camera pose, is a fundamental step in many Computer Vision applications. The pose can be recovered from 2D-3D matches between 2D image positions and points in a 3D model of the scene. Recent advances in Structure-from-Motion allow us to reconstruct large scenes and thus create the need for image-based localization methods that efficiently handle large-scale 3D models while still being effective, i.e., while localizing as many images as possible. This paper presents an approach for large scale image-based localization that is both efficient and effective. At the core of our approach is a novel prioritized matching step that enables us to first consider features more likely to yield 2D-to-3D matches and to terminate the correspondence search as soon as enough matches have been found. Matches initially lost due to quantization are efficiently recovered by integrating 3D-to-2D search. We show how visibility information from the reconstruction process can be used to improve the efficiency of our approach. We evaluate the performance of our method through extensive experiments and demonstrate that it offers the best combination of efficiency and effectiveness among current state-of-the-art approaches for localization.