Is Facial Expression Processing Holistic?

Is Facial Expression Processing Holistic? Akinyinka O. Omigbodun (aomigbodun@ucsd.edu) Electrical and Computer Engineering, University of California San Diego La Jolla, CA 92093 USA Garrison W. Cottrell (gary@eng.ucsd.edu) Computer Science and Engineering, University of California San Diego La Jolla, CA 92093 USA Abstract Most studies examine holistic processing with respect to fa- cial identity, but at least one study has also looked at holis- tic processing of facial expressions (Calder, Young, Keane, & Dean, 2000). However, this work used the partial composite paradigm, which is known to exhibit bias effects (Richler, Che- ung, & Gauthier, 2011). The complete composite paradigm (Gauthier & Bukach, 2007) provides a bias-proof way to quantitatively measure holistic processing. In this paper, we perform the corresponding experiment in our face process- ing model (EMPATH, (Dailey, Cottrell, Padgett, & Adolphs, 2002)) to predict whether holistic processing of facial expres- sions will be found if the corresponding human experiments are performed, and our prediction is that it will. We also com- pared our model’s performance to the participants in recent ex- periments in facial expression recognition in humans (Tanaka, Kaiser, Butler, & Le Grand, 2012). Tanaka et al. (2012) con- cluded that expression recognition is not always holistic, but our results suggest that it is. Keywords: holistic processing Introduction Is all facial processing holistic? For example, when we are judging whether a person is trustworthy, tired, middle-aged, or happy, is our decision based on a global percept of the face rather than constituent parts? One might suspect that the answer is no, at least in some cases. Holistic process- ing might depend upon the task. The evidence from face in- version, composite, and parts-wholes experiments weighs in favor of holistic over parts-based processing for face iden- tity (Yin, 1969; Tanaka & Farah, 1993; Cheung, Richler, Palmeri, & Gauthier, 2008; Richler, Tanaka, Brown, & Gau- thier, 2008). Less research has focused on understanding the nature of facial expression recognition. One reason to suspect that expression recognition is different is the categorization of expressions into just seven categories – happy, sad, surprised, angry, disgusted, fearful, and neutral – in line with the “six basic expressions” theory (Ekman & Friesen, 1976), at least as practiced in most psychology experiments. Previous work in our lab has shown that the basic level processing of objects (e.g., into an overall category, such as chairs or lamps), does not behave in the same manner as subordinate level process- ing in our models, and does not engage our model’s equiv- alent of the Fusiform Face Area. One consideration is that the holistic processing of faces may be induced if there is any variation in identity, regardless of the task being performed. In our modeling work, we tested the hypothesis that without any variation in identity, the processing of facial expressions is holistic. Holistic processing of facial stimuli is generally mea- sured with composite face paradigms, where chimeric faces are constructed from the top and bottom halves of different “source” faces. Subjects are asked to identify face halves of chimeric faces or to judge whether two halves of a pair of chimeric faces are the same or different. Misalignment of the top and bottom halves generally leads to an increase in the subjects’ accuracy and/or a decrease in reaction time rela- tive to the aligned condition, demonstrating that faces are pro- cessed holistically. We test our model with simulations based on the complete composite paradigm (Gauthier & Bukach, 2007), an improvement upon what Gauthier & Bukach call the partial design, which is what is classically used (Young, Hellawell, & Hay, 1987). The complete composite paradigm eliminates the effects of response bias (for example, a prefer- ence towards answering “same” for misaligned stimuli). our results predict that a facial expression recognition experiment with humans based on the complete composite paradigm will indicate holistic processing. We then use our model to account for experimental results in Tanaka et al. (2012). Tanaka et al. used their own nonstan- dard composite paradigm in one of their experiments that we simulated and a variation on the partial design in another that we also simulated. They inferred from their experimental re- sults that facial expression processing is holistic when there is a clash between parts of a facial expression (e.g. angry-happy composite) but analytic or parts-based when there is little or no conflict between the parts (e.g. normal happy face). They posit an earlier stage where a stimulus is rapidly assessed for parts that clash to determine the processing pathway to be used. However, we achieved similar results with our model which (1) has one processing pathway for all stimuli, (2) does not have an earlier stage for quick appraisals, and (3) is not trained to combine decisions on parts of a stimulus into an overall decision. Our results suggest that one holistic pro- cessing pathway is sufficient to account for their data. Complete Composite Paradigm (CCP) In composite paradigms, participants in experiments are pre- sented with two composite faces, and asked to make same- different judgments about the cued face halves (top or bot- tom) while ignoring the other face halves. The complete composite paradigm (CCP) (Gauthier & Bukach, 2007) pro- vides a bias-proof way to quantitatively measure the inter- action between a subject’s decision and the presence or ab-