A Computational Framework for Attentional 3D Object Detection

A Computational Framework for Attentional 3D Object Detection Germ´an Mart´in Garc´ia and Simone Frintrop {martin,frintrop}@iai.uni-bonn.de Institute of Computer Science III, Universit¨at Bonn, 53117 Bonn, Germany Abstract We present a computational framework for the detection of un- known objects in a 3D environment. It is based on a visual attention system that detects proto-objects which are improved by iterative segmentation steps. At the same time a 3D scene model is built from measurements of a depth camera. The de- tected proto-objects are projected into the 3D scene, resulting in 3D object models which are incrementally updated. Finally, environment- and object-based inhibition of return enables to withdraw the attention from one object and switch to the next. We show that the system works well in cluttered natural scenes and can find and segment objects without prior knowledge. INTRODUCTION Object detection is one of the tasks which are easy to solve for humans but hard for machines. Especially unsupervised ob- ject detection, i.e., finding all objects in a scene without pre- vious learning, is largely unsolved in machine vision. 1 How- ever, a system that is able to localize unknown objects in un- known environments is tremendously useful for robotics. For example, a future robot that shall assist in a household must be able to operate autonomously in a new house and is per- manently faced with new, unknown objects. Since humans are able to solve such tasks easily, a promising approach for technical systems is to mimic the human visual system. 2 In humans as in machines, one of the challenges is to deal with the huge amount of perceptual input. Despite the parallelity of the brain, its capacity is not sufficient to deal with all sensory data in detail and a selection has to take place. Neisser (1967) was the first who proposed a two- stage processing of perception that solves this task: first, a pre-attentive process selects regions of interest in paral- lel, and, second, an attentive process investigates these re- gions sequentially in more detail. This view has since then widely spread and many psychological theories and models build upon this dichotomy (e.g. Treisman & Gelade, 1980; Wolfe, 1994). Rensink (2000) has further developed this idea with his coherence theory of attention. It states that the pre-attentive processing determines structures, which he calls proto-objects, that describe the local scene structure of a spa- tially limited region. After that, focused attention selects a small number of proto-objects which form a coherence field representing a specific object. Here, we present a computational framework that follows Rensink’s idea of proto-objects as pre-processing step for ob- ject detection. Our approach generates proto-objects with a 1 The winner of the latest Semantic Robot Vision Challenge (http://www.semantic-robot-vision-challenge.org) was only able to detect 13 out of 20 objects (Meger et al., 2010), although in this challenge, the target objects were known in advance. 2 However, note that our intention is to obtain an improved tech- nical system rather than to mimic the HVS as closely as possible. bottom-up visual attention system (Klein & Frintrop, 2012) and improves their shape by iterative segmentation steps. In contrast to other attention models, we operate on 3D data from a depth camera and are thus able to obtain 3D object models in space, which are incrementally updated by inte- grating new perceptual data. In computational systems based on bottom-up visual at- tention, the focus of attention is directed to the most salient region in the scene. In order to scan the whole scene, this requires a way to withdraw attention from that region and switch to the next. In human vision, this is performed by in- hibition of return mechanisms (IOR) that inhibit the currently attended region (Tipper et al., 1994). In most computational systems, IOR is implemented by ze- roing values in the saliency map (Itti et al., 1998). This is sufficient in static images, but when acting in a 3D world, the correspondence between spatial locations and image regions is required. This affects also the IOR mechanism, since when the perspective of the observer changes or objects are mov- ing, inhibition has to move with them, preventing attention to re-visit the objects directly. This motivates the use of a 3D map that grounds the perceptions in space and enables to maintain a coherent IOR representation over space and time. Corresponding to human vision (Tipper et al., 1994), our IOR mechanism is both object- and environment-based. The contributions of this paper are threefold. First, instead of operating on 2D images, we perform attention-based ob- ject detection on 3D data; this enables us to situate the at- tention system in a 3D environment, resulting in a coherent representation of objects over time. Secondly, it allows for performing not only an environment-based but also an object- based inhibition of return mechanism that operates in space and time. Finally, the use of salient blobs instead of only fix- ation points for initializing the segmentation process lets us bound the amount of perceptual data to be processed. Related Work Many computational attention systems have been built dur- ing the last two decades, first for the purpose of mimicking and understanding the human visual system (survey in Heinke & Humphreys, 2004), and second to improve technical sys- tems in terms of speed and quality (survey in Frintrop et al., 2010). The general structure of attention systems is based on psychological models such as the Feature Integration Theory (Treisman & Gelade, 1980) and states that features are com- puted in parallel before they are fused to a saliency map. One component of attention systems is the inhibition of re- turn mechanism. While IOR is simple on static images, image sequences introduce the challenge of establishing correspon-