When Holistic Processing is Not Enough: Local Features Save the Day

When Holistic Processing is Not Enough: Local Features Save the Day Lingyun Zhang and Garrison W. Cottrell lingyun,gary@cs.ucsd. edu UCSD Computer Science and Engineering 9500 Gilman Dr., La Jolla, CA 92093-0114 USA Abstract Is configural information or featural information more important for facial identity recognition? How are the skills for processing these types of information devel- oped? To investigate these issues, Mondloch et al. de- signed three sets of face images based on a single face, “Jane”, to measure featural, configural, and contour processing. These stimuli were tested on human sub- jects of different ages in a same/different task. We test our model [Dailey et al., 2002] of face processing on these stimuli. We find that our model is overly holistic: It finds the configural differences the easiest to detect, while adult human subjects find featural changes the easiest to detect. We then introduce a representation of the important parts of the face (eyes and mouth) to our holistic model. We find that only a relatively small amount of holistic representation, compared to parts representations, is necessary to account for the data. Introduction We have developed a model of face processing that ac- counts for a number of important phenomena in facial expression processing, holistic processing and visual ex- pertise [Dailey and Cottrell, 1999, Cottrell et al., 2002, Dailey et al., 2002, Joyce and Cottrell, 2004]. Here, we investigate the model’s ability to account for human sen- sitivity to variations in faces that are considered theoret- ically important for face identification. Face processing is typically described as holistic or configural. Holistic processing is typically taken to mean that the context of the whole face has an important contribution to pro- cessing the parts: subjects have difficulty recognizing parts of the face in isolation, and subjects have diffi- culty ignoring parts of the face when making judgments about another part. Configural processing means that subjects are sensitive to the relationships between the parts, e.g., the spacing between the eyes. We will use the two terms configural and spacing interchangeably in this paper. Holistic processing can easily be captured by a model that uses whole-face template-like representa- tions as ours does: interference from incongruent halves of a face occurs when making judgments about a different part (e.g, expression on top when a different expression is on bottom [Cottrell et al., 2002]). However, configu- ral effects related to spacing information are attenuated by the alignment procedure that we typically use, which warps the image so the eyes and mouth are always in the same three positions. Diamond and Carey [Diamond and Carey, 1986] were among the first to discriminate between the types of processing involved in face/object perception and recog- nition. Based on studies looking at the inversion ef- fect to faces, landscapes and dogs in both dog novices and dog experts, they proposed that first-order rela- tional information, which consists of the coarse spa- tial relationships between the parts of an object (i.e. eyes are above the nose), is sufficient to recognize most objects. By contrast, second-order relational informa- tion, which is needed for face recognition and recogni- tion of individuals within categories of expertise, is re- served for visually homogeneous categories where slight differences in configuration must be used to distinguish between individuals (e.g. a slight change in the dis- tance between the eyes and the nose). Diamond and Carey [Diamond and Carey, 1986] suggest that experi- ence allows people to develop a fine-tuned prototype and to become sensitive to second-order differences between that prototype and new members of that category (e.g. new faces). One implication of the Diamond and Carey study is that the inversion effect (a large reduction in same/different performance on inverted faces, com- pared to inverted objects) is based on a relative re- liance on second-order relational information, and that perhaps this characteristic distinguishes face/expert- level processing from regular object recognition. Farah et al. [Farah et al., 1995] found that encouraging part-based processing eliminated the inversion effect, whereas allowing/encouraging non-part-based process- ing resulted in a robust inversion effect. Thus Farah et al. conclude that the inversion effect, in faces and other types of stimuli, is associated with holistic pattern perception. However, subjects are also quite sensitive to changes in the features themselves – substitutions of different eyes or mouths can make the face look quite differ- ent. The Thatcher illusion [Thompson, 1980] suggests that parts are processed somewhat independently, and only loosely connected to the representation of the whole face. Recently, a study by Mondloch et al. that varied these different aspects of a face (configuration, feature changes, and changes to contour of the face) found dif- fering levels of sensitivity to the type of manipulation in a same/different paradigm. While the manipulations were not performed parametrically (no equating of the diffi-