Learning to perceive coherent objects

Learning to Perceive Coherent Objects Nimrod Dorfman 1 (nimrodd@weizmann.ac.il) Daniel Harari 1 (danny.harari@weizmann.ac.il) Shimon Ullman 1 (shimon.ullman@weizmann.ac.il) Department of Computer Science, Weizmann Institute of Science, Rehovot, Israel Abstract Object segregation in a visual scene is a complex perceptual process that relies on the integration of multiple cues. The task is computationally challenging, and even the best performing models fall significantly short of human performance. Infants initially have a surprisingly impoverished set of segregation cues and their ability to perform object segregation in static images is severely limited. Major questions that arise are therefore how the rich set of useful cues is learned, and what initial capacities make this learning possible. Here we present a computational model that initially incorporates only two basic capacities known to exist at an early age: the grouping of image regions by common motion and the detection of motion discontinuities. The model then learns significant aspects of object segregation in static images in an entirely unsupervised manner by observing videos of objects in motion. Implications of the model to infant learning and to the future development of object segregation models are discussed. A B Figure 1: Object segregation, infant to adult capacity. (A): A complex scene, easily segregated by an adult. (B): At 3 months, infants do not appear to divide the figure into two components (after Spelke et al. 1993). start from a surprisingly limited capacity for segregating the world into coherent objects, and reach the capability of the adult system. For computational modeling of vision, an intriguing possibility is to try to surpass the capabilities of current models by following a strategy similar to human development, namely, start with the appropriate set of basic capacities and learning mechanisms and allow the model to develop on its own the final segregation capabilities. In the current study we focus on specific sub-problems within this broad domain. We develop a model that incorporates simple basic capacities, which are known empirically to already exist in young infants. It uses them to segregate familiar objects and to extract and use so-called 'boundary ownership' cues (indicating boundaries as well as figure/background direction) for static object segregation. The model initially has no ability to segregate objects in static images, but it can compute visual motion and motion discontinuities. It is exposed in an unsupervised manner to video sequences containing moving objects. It uses them to segregate familiar objects in static images and to learn local boundary ownership cues. These are used as cues for static object segregation, applied to novel objects. In the next sections we briefly summarize relevant background from developmental studies of object segregation, followed by a presentation of the current model. Keywords: Visual perception; computational modeling; development; object segregation; figure-ground. Background and Goals We naturally perceive the scene around us as containing coherent objects, separated from each other and from their background. Even in a complex image such as Figure 1A, we can count for example the number of distinct cars, delineate their boundaries, etc. The ability to segregate the scene into objects, delineate their boundaries, and determine occlusion relations (termed here 'object segregation'), relies on a complex set of processes, which integrate multiple cues that are only partially understood. Infants' initial ability to segregate scenes into coherent objects is rudimentary and it does not make use of even basic salient 'Gestalt' properties such as uniformity of texture, brightness or color, the smooth continuity of boundary contours, occlusion cues and the like (Spelke et al. 1993). For instance, infants at 3 months of age do not appear to distinguish that the shape in Figure 1B is likely to be composed of two distinct components. The contrast between Figures 1A and 1B illustrates the span of learning accomplished in performing object segregation. The ability to segregate objects based on multiple cues develops quickly already in the first year of life, but the learning process continues over an extended period of time (Kovaks et al. 1999). The process of learning object segregation raises fundamental questions for cognitive development and computational modeling of vision. For cognitive development, it is of basic interest to understand the innate capacities and learning mechanisms that allow the system to Early Development of Object Segregation Initial object segregation by infants is based almost exclusively on dynamic cues, which are then used to learn static object segregation. We focus below on two main