A New Angle on the EMPATH Model: Spatial Frequency Orientation in Recognition of Facial Expressions

A New Angle on the EMPATH Model: Spatial Frequency Orientation in Recognition of Facial Expressions Rentao Li (reli@ucsd.edu) Department of Computer Science, 9500 Gilman Drive La Jolla, CA 92093 USA Garrison Cottrell (gary@eng.ucsd.edu) Department of Computer Science, 9500 Gilman Drive La Jolla, CA 92093 USA Abstract Many have investigated the sensitivity of face processing to both spatial frequencies and face orientation, but few have researched the sensitivity of face processing to the orientation of spatial frequencies. One recent exception has been Yu, Chai, & Chung (2011), which investigated facial expression recognition in regards to the orientation of spatial filters and showed that most information is contained in the horizontal orientation. Here, we model the Yu, Chai, & Chung (2011) study using the EMPATH model, a feed-forward neural network that has been used to model facial expression recognition (Dailey, Cottrell, Padgett, & Adolphs 2002). We used the NimStim set of facial expressions, which were the basis for the Yu, Chai, & Chung (2011) experiment, and followed their method of filtering images through different spatial orientations. Our results show that this simple, biologically plausible model produces very similar results to that of human subjects in their study. Keywords: emotions; facial expressions; spatial frequency; neural network; face recognition. Introduction Many studies have been conducted regarding the role of spatial frequencies in human face recognition (Nasanen, 1999; Costen, Parker, & Craw, 1996; Gold, Bennett, & Sekuler, 1999), although the uniqueness of sensitivity of faces to spatial frequency has been debated (Williams, Willenbockel, & Gauthier (2009). However, few have explored how different orientations of spatial frequencies impact recognition of facial images. One such experiment was done by Yu, Chai, & Chung (2011), who passed facial images through orientation filters from -60 to 90 degrees in increments of 30 degrees. They found that the spatial information near horizontal (between -30 and 30 degrees) were the most important for normally-sighted human respondents to recognize facial expressions. Background: Yu, Chai, & Chung’s Experiment The aim of the Yu, Chai, & Chung (2011) experiment was to determine which spatial orientations on the face contained the most information for identifying emotions. The four emotions they tested were the closed-mouth forms of anger, fear, happiness, and sadness. Images were obtained from the NimStim set of facial expressions (Tottenham, Tanaka, Leon, McCarry, Nurse, Hare, Marcus, Westerlund, Casey, & Nelson 2009), and were distorted with an orientation filter of bandwidth 23 o in the Fourier domain, where the center of the filter ranged from -60 o to 90 o in increments of 30 o . Unfiltered images were used for comparison. Their experiment consisted of having 15 normally-sighted human subjects try to recognize the expression displayed by each image under a four-way forced choice. The results indicate that the human observers had the most success with images filtered at orientations near the horizontal (-30 o , 0 o , and 30 o ), suggesting that horizontal spatial information is most important for recognizing facial expressions. One modest exception to this trend is the fearful face; the human subjects tended to be significantly biased towards labeling a face as fearful as the orientation filter approached 90 o , which seems to indicate that much of the information for fear is represented vertically. The purpose of this current experiment is to determine if a neural network model can produce similar results as the human subjects, especially in regards to the increased recognition performance for horizontal orientations and the preference towards fear for vertical orientations. Such evidence would provide greater support for Yu, Chai, & Chung’s (2011) findings and further validate EMPATH’s flexibility and accuracy in modeling human face recognition. Methods The Model The neural network used for this experiment closely followed the EMPATH model developed by Dailey et al. (2002), consisting of a biologically plausible, three-layer, feed-forward perceptron. EMPATH has been shown to have remarkable face recognition performance on aligned, grayscale images from Ekman and Friesen’s POFA (1976). Without being tuned specifically to those images, the network classified the emotions Anger, Disgust, Fear, Happiness, Sadness, and Surprise with 90% accuracy on average, compared to 91.6% for human subjects (Dailey et al., 2002). For this experiment, we kept much of the settings (outlined below) identical to those of the original EMPATH