Automatic Visual Integration: Defragmenting the Face

Automatic Visual Integration: Defragmenting the Face Luke Barrington (lbarrington@ucsd.edu) Electrical & Computer Engineering Department, University of California, San Diego La Jolla, CA 92093 USA Garrison W. Cottrell (gary@cs.ucsd.edu) Computer Science & Engineering Department, University of California, San Diego La Jolla, CA 92093 USA visual memory. Since snapshots of the environment are unlikely to be repeated, a straightforward template matching procedure is unlikely to work. For a simple task like face recognition, it is therefore important that there be a matching procedure that probabilistically assesses how well samples drawn from the current environment could have been generated by previous observations. Here we describe how the model developed by Lacroix et al. (2006) can be viewed as a kernel density estimate of the likelihood that new visual patches of faces were generated by our memory. We then show how we can use the same model for face identification. Finally, we explore the issue of the scale- space representation used and show how different spatial frequencies affect the matching process. Abstract We describe a computational model of saccadic visual memory applied to the task of facial recall. Each saccade provides a mixed-resolution, quasi-stationary snapshot of a visual scene to the striate cortical areas yet the brain recreates spatially and temporally smooth perceptions and memories by combining these visual fragments. We build on the work of (Lacroix et al., 2006) to model this integration of saccades in a facial recall task by selecting fixation points based on low- level saliency of a face image. At each fixation point, this model stores discrete visual samples with multiple resolutions as activation patterns without knowledge of their temporal or spatial origin to create a kernel density estimate of the studied faces. These visual fragments are then integrated and compared to new fixations during recall. We replicate Lacroix’s results by demonstrating that the model achieves human levels of performance on the standard psychological memory test of facial recall. We then extend the model to facial identity recognition and examine the task-dependent effects of visual resolution. We begin by presenting an improvement to the model of (Lacroix et al., 2006) that uses a saccade selection routine that uses the same filters that are used for the memory representation. In Experiment 1, we demonstrate that this model can solve the challenge of visual integration by modeling the facial recall task. Experiment 2 extends the model to perform a new task; recognition of facial identity. Finally, Experiment 3 examines the effect of allowing the model to process Gabor filter bands separately and indicates a potential method for task-based control of visual attention. Visual Integration Visual perception begins with retinal sampling information of localized areas of a scene before our eyes. These saccades are discrete in space and time. High-resolution visual information is only available from the fovea, covering only a small portion of the observed scene. In order to perceive all parts of a visual scene with great detail as well as to maintain neural activation in the visual cortex, we repeatedly foveate different areas of the scene, spending the most time fixating on the parts that are most salient or most task-relevant (Yarbus 1967). Visual Memory Model Saccade Selection Given a current fixation point, the choice of where to saccade to next is driven by a number of external cues including motion, peripheral complexity and non-visual stimuli (e.g. sound) as well as top down task-dependant directives such as attention and expectation. Though many methods (Mozer, Shettel & Vecera 2005; Wolfe, 1994; Zelinsky et. al 2005) have been proposed for how to integrate these cues, in this work we concentrate only on bottom-up salience of static images. We model the saccade selection process using an interest operator for determining the scan paths introduced in (Yamada & Cottrell, 1995). This simplified model uses the rotational variance of low- resolution Gabor filter responses to construct a distribution of the contour complexity (read: salience) over all pixels in a given image: It seems reasonable to suppose that the sequences of saccades or scan paths that collect the visual data do not, in general, follow exactly the same path and thus can not reconstruct the exact sequence of stored visual memories when examining a previously-viewed object (although the distribution of saccades tends to be similar, see Henderson, Williams, & Falk, 2005). Yet we can mentally comprehend and recreate spatially and temporally complex constructs using only a combination of these static, non-uniform retinal samples stored in memory. How is this discrete information integrated into a continuous, dynamic construct? The solution to this visual jigsaw puzzle must come from the way that retinal frames are stored in and retrieved from