BEHAVIORAL AND BRAIN SCIENCES (2010) 33, 417 –480
doi:10.1017/S0140525X10000865

The Simulation of Smiles (SIMS)
model: Embodied simulation and themeaning of facial expression
Paula M. Niedenthal
Centre National de la Recherche Scientiﬁque (CNRS) andClermont Université, 63037 Clermont-Ferrand, France
niedenthal@wisc.edu
http://wwwpsy.univ-bpclermont.fr/∼ niedenthal/Martial Mermillod
Centre National de la Recherche Scientiﬁque (CNRS) andClermont Université, 63037 Clermont-Ferrand, France
martial.mermillod@univ-bpclermont.fr
http://wwwpsy.univ-bpclermont.fr/∼ mermillod/

Marcus Maringer
Department of Psychology, University of Amsterdam, 1018 WB Amsterdam,
The Netherlands
m.maringer@rug.nl

Ursula Hess
Department of Psychology, Humboldt-Universität Berlin, 12489 Berlin,
Germany
Hess.Ursula@psychologie.hu-berlin.de
http://www.psychophysiolab.com/uhess/WebUH_fr/UH_fr/index.html

Abstract: Recent application of theories of embodied or grounded cognition to the recognition and interpretation of facial expression ofemotion has led to an explosion of research in psychology and the neurosciences. However, despite the accelerating number of reported
findings, it remains unclear how the many component processes of emotion and their neural mechanisms actually support embodied
simulation. Equally unclear is what triggers the use of embodied simulation versus perceptual or conceptual strategies in determining
meaning. The present article integrates behavioral research from social psychology with recent research in neurosciences in order toprovide coherence to the extant and future research on this topic. The roles of several of the brain’s reward systems, and the amygdala,somatosensory cortices, and motor centers are examined. These are then linked to behavioral and brain research on facial mimicry andeye gaze. Articulation of the mediators and moderators of facial mimicry and gaze are particularly useful in guiding interpretation of
relevant findings from neurosciences. Finally, a model of the processing of the smile, the most complex of the facial expressions, ispresented as a means to illustrate how to advance the application of theories of embodied cognition in the study of facial expression of emotion.
Keywords: amygdala; basal ganglia; Duchenne smiles; eye contact; embodiment; facial expression; mimicry; simulation; somatosensory
cortexA smile is the chosen vehicle for all ambiguities.
— Herman Melville (1852/1996, Pierre, or, The Ambiguities,
p. 84)

Smiles can be simple things. Individuals show very highagreement in their assignment of the label “smile” tophotographs of facial gestures with certain structuralfeatures (Izard 1971). This is true across cultures(Ekman 1994; Haidt & Keltner 1999). Yet, smiles arealso complicated things. Although smiles often communicate that the expresser feels “happiness” or “joy”(Frank et al. 1997; Frank & Stennett 2001; Messinger
et al. 2001), some smiles signal affiliative intent or a
responsiveness to group norms; others express more
complex interpersonal or status motivations (Abel# Cambridge University Press 2010

0140-525X/10 $40.00

2002; Fogel et al. 2000; LaBarre 1947; Keltner 1995;
Tipples et al. 2002).
How do individuals interpret the meaning of a smile? One
possibility is that each smile has its own specific facial morphology, which constitutes slightly different visual features.
Meanings associated with these configurations could thenbe learned. Ekman (2001) identified 18 types of smiles andproposed that there might be as many as 50 in all. If visualfacial features did all of the work in grounding meaning,
we would expect very few errors in interpreting smiles
across individuals and cultures. Yet, even though errors inclassification of a smile as such are not frequent (Haidt &Keltner 1999), errors of interpretation of specific smile
meanings are much more so (Bernstein et al. 2008).
417

Niedenthal et al.: The Simulation of Smiles (SIMS) model
In the present article we argue that observers of smiles
sometimes construct an embodied simulation of thenuanced affective state conveyed by the smile that is supported by the brain’s reward, motor, somatosensory, andaffective systems. They then use this simulation to represent the smile’s intended meaning. Our approach also
outlines the conditions under which simulations, in contrast to other bases of processing, are actually used, andwhy and when these alternative processes may lead toerrors.
To accomplish our goals, we devote the first section of
the article to a review of research on the meaning ofsmiles from the view of the person doing the smiling.
Based on existing functional accounts, we characterize
smiles as produced by positive emotion (enjoyment
smiles), by positive social motives (affiliative smiles), andas a way of communicating and maintaining social status(dominance smiles).
In the second section of the article, we review researchon the neural bases of smile processing in the perceiver ofthe smile. Possible roles of the brain’s reward centers,
orbital prefrontal cortex, amygdala, motor regions, andsomatosensory cortices are outlined, with accounts ofmotor processing linked to research on facial mimicry
from social psychology and the neurosciences.
In a third section, eye contact is discussed. Our novel
proposal is that eye contact automatically triggers anembodied simulation of what a smile means. A large literature in social and developmental psychology supports thisclaim, and we rely on it to draw conclusions from recent
neuroscience findings. Finally, we bring together these

PAULA M. NIEDENTHAL received her Ph.D. at the University of Michigan and was on the faculty of thedepartments of Psychology at Johns Hopkins University and Indiana University (USA). She is currently
Director of Research in the National Centre for Scientific Research and member of the Laboratory in Social
and Cognitive Psychology at Blaise Pascal University inClermont-Ferrand, France. Her areas of research
include emotion– cognition interaction and representational models of emotion. Author of more than 100
articles and chapters, and several books, Dr. Niedenthal is a fellow of the Society for Personality andSocial Psychology.
MARTIAL MERMILLOD is Associate Professor of Psychology at the Blaise Pascal University, CNRS (UMR6024) and Institut Universitaire de France. He has
published over 30 peer reviewed articles and bookchapters in the domain of emotion and perception,neural computation, and cognitive sciences.
URSULA HESS is Professor of Social and Organisational
Psychology at the Humboldt-University, Berlin. She
has published extensively (over 100 peer reviewed
articles and book chapters) in the domain of emotion
communication.
MARCUS MARINGER is a lecturer of Psychology at theHague University for Applied Sciences. He has published several reviewed articles and book chapters in
the domain of emotional influences on perception
and judgment formation.

418

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

summaries to motivate a comprehensive model of smileinterpretation for three smile types: enjoyment, affiliative,and dominance smiles.
1. Recognition and access to meaning of
expressions
Different processes can support people’s ability to recognize smiles and what they mean (Adolphs 2002; Atkinson
2007; Kirouac & Hess 1999). We begin with visual facialcues and then consider a variety of other cues.
The classification of expressions into basic categories
usually relies on a perceptual analysis of the stimuli, sometimes called pattern matching (Buck 1984). Smith et al.
(2005) have shown that distinct facial features can beused to classify facial expressions and that these features
correspond to a de-correlation model. In their view, configurations of muscles have emerged during phylogenetic
development of the human species that maximize differences between the six basic emotional expressions andproduce efficient recognition of these expressions. At theneural level, this process seems to be supported by theoccipito-temporal cortices (Adolphs 2002).
Whereas the analysis of visual facial features may be sufficient to classify prototypical expressions in simple tasks,
this process is unlikely to be sufficient to recognize lessprototypic, perhaps more realistic, emotional expressions,or to represent their subtle meanings. In such cases, we
propose that perceivers must call on various sources of
non-visual information, such as conceptual emotion
knowledge about the expresser and the social situation(Kirouac & Hess 1999; Niedenthal 2008). For example,
faces provide information about the sex, age, and race ofthe other person, and individuals possess expectations
and stereotypes about how members of these social groups
react emotionally (Hess et al. 2005). Such conceptual knowledge about emotion has been shown to exert effects earlyin the processing of ambiguous facial expressions (e.g.,
Halberstadt & Niedenthal 2001; Halberstadt et al. 2009;
Hess et al. 2009a).
Embodied simulation also supports the recognition andaccess to meaning of facial expressions (e.g., Atkinson
2007; Decety & Chaminade 2003; 2004; Gallese 2003;
2005; Goldman & Sripada 2005; Keysers & Gazzola
2007; Niedenthal 2007; Niedenthal et al. 2005b; Winkielman et al. 2009). When we use the term “embodied simulation,” we mean that a facial expression has triggered asimulation of a state in the motor, somatosensory, affective,
and reward systems that represents the meaning of theexpression to the perceiver. In an embodied simulationaccount, the perception of a facial expression isaccompanied by the bodily and neural states associatedwith the expression and its correspondent emotion. This
simulation is then used as the representation of meaning
on which an interpretation or judgment is based.
In social psychology, such a view partly involves themarriage of facial feedback theory and affect as information theory. The first holds that facial musculature produces afferent feedback that alters subjective state(McIntosh 1996; Zajonc et al. 1989). The latter holds
that when individuals believe that their affective state
was caused by the current object of perception, they usethat state to evaluate affective features of the object

Niedenthal et al.: The Simulation of Smiles (SIMS) model(Clore & Storbeck 2006). In neuroscience, embodied
simulation has been closely linked to the construct of
mirror neurons and mirror systems, and the notion thatbrains resonate with the motor and affective states of perceptual objects with appropriate biological similarity (e.g.,
Gallese 2007; Keysers & Gazzola 2007).
The rest of this article is about the role of embodied
simulation in representing the meaning of the smile.
After providing theoretical justification for our accountof smiles, we describe the simulation components fordifferent types of smiles and describe the conditions
under which the meaning of a smile is represented bythis process. The integration of these ideas is called theSimulation of Smiles Model (SIMS).
2. What is a smile?
The smile is characterized by the upward turn of the cornersof the lips, which is produced by the contraction of thezygomaticus major muscle (Ekman & Friesen 1978). The
zygomaticus major, like other muscles involved in the production of facial expression, is activated by the seventh
cranial nerve, or the facial nerve (Rinn 1991). The facial
nerve can be innervated by one of two motor systems. The
subcortical motor system, also known as the extrapyramidal
circuit, supports non-voluntary, facial expression. The cortical motor system, also known as the pyramidal circuit, supports learned, voluntarily facial expression, which may varyacross cultures and be produced and inhibited intentionally.
Early research on the smile revealed that the frequency,
intensity, and duration of the zygomaticus major muscle
activity positively predicted self-reported happiness ofthe smiler (Ekman et al. 1980; Cacioppo et al. 1986). Zygomaticus major contraction, however, is observed not only
when positive emotions are experienced, but may also be
observed when individuals report feeling negative
emotions such as disgust (Ekman et al. 1980), disappointment (Kraut & Johnston 1979), sadness and uncertainty(Klineberg 1940), and general discomfort (see Ekman
et al. 1990, for a review).
2.1. True and false smiles

Due to the failure to observe a clear correspondence
between activation of the zygomaticus major and positive
feelings, several theorists have suggested that the smile
should not be treated as a single category of facial expression(e.g., Ekman & Friesen 1982). In the proposed distinctions,“true” or “sincere” smiles were defined as involuntary displays of positive affect, whereas ”false” or “insincere”
smiles were defined as smiles voluntarily used to communicate that a positive emotion was felt when it was not (andwhen, in fact, it served to mask negative feelings).
Several morphological and dynamic markers have been
proposed to distinguish these two types of smiles. The
most frequently cited morphological indicator of a true
smile is the Duchenne marker, thanks to Duchenne’s
empirical work on smiles and other facial expressions(Duchenne 1862). The Duchenne marker involves the contraction of the muscle around the eye, the orbicularis oculi,pars lateralis. The orbicularis oculi causes a lifting of thecheeks, a narrowing of the eye opening, and wrinkles
around the eyes. The combination of zygomaticus major
contraction, along with orbicularis oculi contraction, issometimes indicative of positive emotion (Frank et al.
1993; Soussignan 2002). The perceiver of the “Duchenne”
smile also interprets it as expressive of positive emotion(Miles & Johnston 2007) and may respond to it with positive
affect (Surakka & Hietanen 1998). Smiles lacking themarker have been referred to as “false,” “masking,” and“non-Duchenne” smiles (see Figure 1; École Nationale
Supérieure des Beaux-Arts 1999). Other research showsthat dynamic features of smiles, such as their symmetry,smoothness, duration, and synchrony, may distinguish trueand false smiles as well (Cacioppo et al. 1986; Ekman
et al. 1980, reviewed in Frank 2002; Hess & Kleck 1990).
Recent research, however, has shown that the utility of
the Duchenne versus non-Duchenne distinction is limited(e.g., see Abe et al. 2002, for a review). For instance, somestudies have demonstrated that dynamic characteristics ofthe smile can override the Duchenne marker’s importancein determining judgments of how true (or sincere) thesmile is (Hess & Kleck 1990; Krumhuber et al. 2007).
Others have shown that non-Duchenne smiles may be
associated with self-reported happiness in adults (e.g.,
Hecht & LaFrance 1998; Hess et al. 1995; Jakobs et al.
1999), and, conversely, that Duchenne smiles may be displayed in situations in which false smiles would be predicted (Bourgeois & Hess 2008). Finally, there is someevidence that the importance of the Duchenne marker
varies with culture (Thibault et al. 2008).
2.2. Functional smiles

The distinction between true and false smiles itself may belargely superceded by another more useful distinction,
namely, distinctions based on smile function (e.g.,
Barrett 2002). We describe three types of smiles that we
believe have important and discrete functions, andwhich may map onto identifiable brain systems that represent different meanings.
As we have already implied, and consistent with experience, many smiles are simply readouts of positive internal
states such as happiness (Buck 1984). The “play-face” in primates, such as chimpanzees, held to be a homologue oflaughter in humans, corresponds in musculature to thehuman smile (e.g., Parr & Waller 2006; see our Figure 2,
left panel). Humans and some primates smile spontaneously during experiences of pleasure, including visual,auditory, gustatory, and tactile stimulation (Ekman &Friesen 1982). Smiles that are readouts of happy feelings
reinforce the behaviors that elicited them in the firstplace. Thus, the communication of positive emotion
through the smile is essential, among other things, for learning in infants, when mothers smile at babies to encouragedesired behaviors (Klinnert et al. 1983). We will refer tosmiles that express happiness as enjoyment smiles.
Second, smiles can be readouts of positive social intentions that are essential for the creation and maintenance ofsocial bonds, without necessarily being about personalenjoyment (Cashdan 2004; Fridlund 1991; 2002). Suchsmiles may include the “greeting” smile (Eibl-Eibesfeld1972), as well as those of appeasement, and perhaps thesmile component of embarrassment (Keltner 1995; Hesset al. 2002). In most primates, the silent bared-teeth
display (right panel of Figure 2) serves to communicatethat the smiler intends no harm and that there is no
threat of aggression (van Hooff 1976; Waller & Dunbar
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

419

Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 1. The top two photographs show the Duchenne (left) and non-Duchenne (right) smiles as elicited by Guillaume-Benjamin
Duchenne de Boulogne himself, using electrical impulses to manipulate relevant facial muscles. The bottom two photographs show
more recent posed versions of the same.

Figure 2. Two chimpanzee facial expressions related to the human smile. The left panel shows a play face believed to be a homologueof laughter and sharing morphological features with the human enjoyment smile (Parr & Waller 2006). The right panel shows a silent
bared-teeth display, used in affiliative and appeasement contexts, believed to be homologous with the human affiliative smile andsharing similar musculature (Parr & Waller 2006). Photos courtesy of Dr. Lisa Parr, National Primate Research Center, Emory
University, Atlanta, GA. Used with permission.

420

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 3. Tony Blair, among other world leaders, has been said to be a “skilled proponent of the dominant smile” (Senior et al. 1999), a
fact that has not been ignored by caricature artists. Left: Photo # Crown Copyright. Right: Caricature by Paul Baker, courtesy of Paul
Baker.

2005). In primates with relatively egalitarian social
systems, including some macaques, mandrills, Geladababoons, and chimpanzees, the silent bare-teeth display
is also seen in grooming, sexual solicitation, and reconciliations (Preuschoft & van Hooff 1997; Waller & Dunbar2005). Smiles that express positive social motives will becalled affiliative smiles.
Finally, what we will call dominance smiles are
expressions that reflect social status or control, and may
include displays that have been called “scheming smiles”(e.g., Öhman et al. 2001; Tipples et al. 2002), “criticalsmiles,” and perhaps the smile component of theexpression of pride (Tracy & Robins 2004; 2008).
Darwin also referred to derisive or sardonic smiles in his
discussion of sneering and defiance (Darwin 1872,
p. 251). Recent analyses of the signals of leadership anddominance in human societies refer to this type of smile.
Senior et al. (1999) note that (former Prime Minister ofBritain) Tony Blair and (former American President) Bill
Clinton have been called “skilled proponents of the dominant smile” (p. 344).
Whether a smile is voluntary on the part of the smiler isprobably not a factor that best distinguishes enjoyment,
affiliative, and dominance smiles. That is, the involvementof the Duchenne marker will probably not turn out to bediagnostic of one type of smile. Specific instances of the proposed functional types may however be associated withspecific postural or other facial features (e.g., the greetingsmiles contains an “eyebrow flash,” and the embarrassed
smile is part of a larger gesture). Thus, it might in principle
be possible to construct a visual description of the fullbodily and facial characteristics of enjoyment, affiliative,and dominance smiles. But the present issue is: How doperceivers arrive at those meanings? We propose that perceivers use a set of neural and behavioral processes toextract a smile’s meaning that allows them to distinguishbetween the three functional smile categories in terms of
the feelings they generate in the perceiver.

3. What is simulated in the simulation of a smile?
In this section, we explore the neural structures that could
play a central role in the embodied simulations that represent different smile meanings. Specifically, we address
how the basal ganglia, prefrontal cortex, amygdala, motor
system, and somatosensory system all contribute to theexperienced meaning of a perceived smile on anotherperson’s face. Later sections will discuss when and how
these simulations are triggered, and the implications foremotional experience. Although the focus is on specificneural structures, this does not mean that we endorse a
localization approach. It will remain for neuroscientists tofully establish the larger distributed circuits that support
the functions we suggest that these structures play.
3.1. Subcortical and cortical affective systems
3.1.1. Basal ganglia. As people perceive smiles, thereward system in their brain may simulate the experienceof reward. Research shows that the smile can function as asocial reward for both adult and infant humans (Trevarthen 1974; Tronick et al. 1978).
This view is consistent with evolutionary and ecologicaltreatments of the smile (e.g., Buck 1991; Burgoon et al.
1996; Haith 1972; McArthur & Baron 1983). Researchon both primates and rats implicates the striatum andthe ventral tegmental area (VTA) in reward processing(Kawagoe et al. 1998; Parkinson et al. 2000; Schultz
et al. 2000). Further studies indicate that these regions
are similarly related to reward processing in humans(Damasio et al. 2000; Davidson & Irwin 1999; Laneet al. 1997; 1999; Mobbs et al. 2003; O’Doherty et al.
2003; Rauch et al. 1999; Redoute et al. 2000). Recent
research has also linked the reward areas of the basal
ganglia with the perception of smiling faces (Chakrabarti
et al. 2006; Lawrence et al. 2004; Lee et al. 2006; Morris
et al. 1996; 1998; Okun et al. 2004; Phillips et al. 1998;
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

421

Niedenthal et al.: The Simulation of Smiles (SIMS) model
Whalen et al. 1998). When people perceive smiles, theyexperience them as rewarding.
3.1.2. Prefrontal cortex. Simulations of reward as people

perceive smiles may also reflect contributions from prefrontal cortex. The reward circuitry just described in thebasal ganglia is associated with what Davidson (1993)
has termed pre-goal attainment positive affect. In his
view, activation in subcortical reward centers supports
an organism’s approach toward an appetitive goal. Davidson contrasts the approach state with post-goal attainment
positive affect, which represents a functionally different
type of positive emotion, characterized by feelings ofaffiliation and attachment (biologists refer to the former
as incentive value and the latter as reward outcome).
Post-goal attainment positive affect has been linked in a
number of studies with activation in the orbitofrontal
cortex (OFC) (e.g., Anderson et al. 2003; Elliott et al.
2000; O’Doherty et al. 2001; Rolls 2000). Findings by
Davidson and colleagues, for example, demonstrate thatsmiles displayed by offspring are related to OFC activationin mothers who perceive these smiles (Nitschke et al. 2004).
Specifically, the OFC differentiates the sight of one’s own
smiling baby from the sight of an unknown smiling baby
(who is nevertheless cute and positive in impact). Thisfinding has recently been replicated and extended (e.g.,
Minagawa-Kawai et al. 2009). The reward outcome systemin the brain probably extends to medial orbital and medial
prefrontal (dorsal to orbital) cortices, as well as to anterior
cingulate cortex (O’Doherty et al. 2003). This system may
also be involved in reward learning. In addition, and very
importantly, the system is implicated in processing status
relations (Zink et al. 2008), and thus may contribute to distinguishing dominant versus submissive smile meanings.
Although the specific terms, pre-goal and post-goalaffect, are not necessary for our theorizing, the distinction
is relevant for differentiating the perception of enjoyment
and affiliative smiles. Because the OFC and contiguousareas process learned emotional responses (Rolls 2004),
OFC activation may distinguish the basic rewarding properties of smiles from the reward of experiencing smiles
made by people with whom an individual has a significant,previously established emotional, relationship (e.g., family
members, in-group members). Indeed, other evidence
and reasoning suggests that reward systems in prefrontal
cortex play a distinct role in responding to affiliative
smiles, reflecting their association with attachment information (e.g., Schore 2001).
3.1.3. Amygdala. As people perceive smiles, the amygdala

may produce states that further contribute to simulations
that underlie how these smiles are interpreted. The amygdala’s involvement in perceiving fear expressions has been
considered at length (e.g., Adolphs 2008; Atkinson 2007;
Heberlein & Atkinson 2009; LeDoux 2007). Lesion andneuroimaging studies initially indicated that the amygdala
was not only vital for recognizing the emotional expressionof fear, but that it further supported the full experience offear and its behavioral implications (e.g., Calder et al.
2001). These studies did not demonstrate any role for the
amygdala in smile processing (e.g., Whalen et al. 1998),
and patients with amygdala lesions had not been reported
as showing deficits in recognizing happy faces (Adolphs
et al. 1994; Hamann et al. 1996; Calder et al. 1996).
422

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Yet, when measuring amygdala activity in human volunteers during rapid visual presentations of fearful, smiling,
and neutral faces, Breiter et al. (1996) found that theamygdala also responded preferentially to smiles versus
neutral faces (see also a meta-analysis by Fusar-Poliet al. 2009). Other studies supported the conclusion that
all emotional expressions can activate the amygdala. For
example, a functional magnetic resonance imaging(fMRI) study by Winston et al. (2003) used emotion perception tasks that were either implicit (ratings of maleness)or explicit (deciding whether a face was more or lessemotional). Both tasks activated broad cortical and subcortical regions for disgusted, fearful, happy, and sad
expressions of either low or high emotion intensity. Most
importantly, all expressions activated the amygdala, afinding further supported by Fitzgerald et al. (2006).
Additional results demonstrate higher activation of theamygdala for high versus low intensity emotions for explicit and implicit tasks (cf. Surguladze et al. 2003).
Taken together, these results suggest that the amygdala
responds to stimuli of motivational significance independently of emotion and processing goals. Indeed, more
recent evidence favors an even broader account of amygdalafunction, namely, that it responds to everything of uncertain
meaning to the organism (e.g., Murphy et al. 2003; Sander
et al. 2003; Whalen et al. 2001). Applying this hypothesis
to the smile, Yang and colleagues proposed that individuals
may be uncertain about the meaning of a smile, not merely
because of its perceptual features, but also because of its
meaning as reflected in the current social or experimental
context (Yang et al. 2002). These authors further note thatwhen the meaning of a smile is unclear (such as a smile displayed by an enemy), additional information is needed,similar to our proposal here (see also Lee et al. 2006).
3.2. Facial mimicry in the motor system and sharedneural substrates

As people perceive smiles, the motor system may simulate
the experience of performing the perceived action, further
contributing to how the perceived smile is understood.
3.2.1. Facial mimicry. Facial mimicry is defined as thevisible or non-visible use of facial musculature by an observer to match the facial gestures in another person’s facial
expression. Perceivers of smiles often automatically
mimic these smiles. For instance, electromyographic(EMG) recordings reveal that when individuals view asmile, their zygomaticus major muscle contracts, usually
within 500 milliseconds after the onset of the stimulus(Dimberg & Thunberg 1998). Mojzisch et al. (2006) similarly demonstrated that observers automatically mimic
smiles expressed by virtual characters in dynamic animations, as did Hess and Bourgeois (in press) in an interactive
live setting (see also a review in Hess et al. 1999). Automatically mimicking a smile interferes significantly withsimultaneously production of an incongruent facial
expression, such as anger (Lee et al. 2007).
3.2.2. Link to corresponding emotions. As anticipated byfacial feedback theory, facial mimicry may be accompanied
by self-reports of a corresponding emotion, sometimes
called emotional contagion (Hatfield et al. 1992; 1993;
Laird et al. 1994; Strayer 1993; Wild et al. 2001; Soussignan

Niedenthal et al.: The Simulation of Smiles (SIMS) model
2002). For afferent feedback to contribute to an embodied
simulation of a perceived smile, however, the perceiver
does not necessarily have to experience a conscious change
in emotional state – such simulations often appear to haveunconscious impact. Findings that mimicry produces
emotional effects implicitly are therefore also important.
In an innovative study that assessed the causal relationship between facial mimicry and implicit emotion, Botulinum Toxin (BOTOX) was used to block facial mimicry forexpressions of anger (Hennenlotter et al. 2009). Participants
were directed to mimic angry and sad facial expressions instill photographs. During the anger (though not sadness)mimicry task, participants whose brows had received
BOTOX injections exhibited significantly less activation in
the limbic system compared to control individuals who
received no injection. The result for anger causally links
facial mimicry to emotion, given that disabling the facial
musculature decreased emotion activation. When themuscle pattern associated with anger is blocked, part ofthe embodied meaning associated with anger is lost, suchthat the emotion is experienced less intensely. Consistent
with this experimental finding, and specifically relevant tosmiles, is a correlational finding by Lee et al. (2006). In
that study, participants mimicked faces expressing smiles,
as well as other nonemotional facial movements andexpressions. The more participants mimicked the observed
smiles, the greater the activations in their striatum andamygdala (see also Schilbach et al. 2006).
A recent paper by Schilbach et al. (2008), who collected
fMRI and EMG data simultaneously, is also noteworthy
here. Their results showed that spontaneous mimicryoccurred during the perception of smiles, which was
accompanied by neural activity in the motor system,
specifically in the inferior left precentral gyrus. Consistent
with previous findings, this study also implicated the
medial temporal lobe in the spontaneous mimicry ofobserved emotional expressions. Finally, the dorsal midbrain was also active, which can be interpreted as signaling
arousal induced by direct eye gaze.
In sum, there is evidence that mimicry has a role incausing emotion. Our interest here, however, is not withchanges in consciously reported emotional state. Instead,
we focus on the fundamental role of mimicry in creating
the embodied feeling of a smile, which becomes part ofits meaning. In a study motivated by a similar idea,
Zajonc et al. (1987) showed that the appearance of
spouses who had lived together for at least 25 years andhad strong marriages grew more similar in facial appearance over time. Their interpretation was that mimicry supports empathy through afferent feedback, with theincidental effect of producing more similar facial musculature (not through more similar emotion). This preliminaryinterpretation requires further empirical tests (see Bastiaansen et al. 2009 for further discussion and insight).
3.2.3. Shared neural substrates. Facial mimicry receives
considerable attention in the literature on so-called
mirror neurons and the notion of emotional resonance.
Mirror neurons were first observed in the brains ofmonkeys in response to limb actions (Gallese et al. 1996;
Rizzolatti & Craighero 2004; Rizzolatti et al. 1996).
Recordings of mirror neurons in the monkey motor
cortex, particularly area F5, associated with the productionof hand and mouth movements, were interpreted as

constituting a mechanism that could support the implicit
comprehension of an action, specifically, the goal of anaction (Gallese et al. 1996; Rizzolatti et al. 1996). Morerecently, a similar mirror system for perceiving and performing action has been described in humans (Fadiga
et al. 1995; Gallese et al. 1996; Iacoboni et al. 1999; Rizzolatti et al. 1996). The work on humans points to the premotor and parietal areas as the most likely humanhomologue of monkey area F5 (e.g., Decety & Grezes
1999).
Inspired by this approach, Carr et al. (2003) found thatthe passive perception and the intentional mimicry of
emotional facial expression activate overlapping sets ofbrain structures, including the ventral premotor cortex,
the insula, and the amygdala (see also Wicker et al.
2003). Mimicry relative to perception, however, wasaccompanied by greater activation in regions related toemotion processing, in particular, the amygdala andanterior insula, and also the primary motor cortex. Similarly, Hennenlotter et al. (2005) found that voluntary production and perception of smiles activated both the rightpremotor cortex and the inferior frontal cortex, as well as
the right parietal operculum and the left anterior insula.
These findings were replicated in a careful study by van
der Gaag et al. (2007). Although the reviewed findings
are compelling, and do suggest that emotion processing
is affected by mimicry, we note that the mirror neuron
construct and its roles in human emotion recognition
have received considerable criticism (e.g., Turella et al.
2009; Dinstein et al. 2008; Jacob 2008; 2009). A clearer
understanding of this construct awaits further research.
The robust activation of premotor areas during theobservation of facial expressions (unaccompanied by
mimicry) also requires discussion. This finding has beeninterpreted as demonstrating the presence of an as-if
loop, meaning that perception activates the programs for
producing facial expressions (e.g., Leslie et al. 2004).
The processing of as-if loops has also been called off-line
simulation (e.g., Atkinson 2007). One interpretation ofthese findings is that as-if simulations, when they occur,
contribute to smile meaning. In support of this view,
research has found that perception-plus-mimicry tends
to produce stronger affective responses to smiles than perception alone (e.g., Carr et al. 2003). Motor mimicry seems
to play an important role in how smiles and their meanings
are simulated.
3.2.4. Roles of mimicry in processing perceived facial
expression. If facial mimicry is important in constructing

embodied simulations for smiles, then mimicry should
affect performance on tasks that measure recognition
and access to meaning (Adolphs 2002; 2003; Heberlein& Atkinson 2009; McIntosh 2006). In their review offacial mimicry, however, Hess and colleagues (Hesset al. 1999) did not find evidence that mimicry was causally
related to the simple recognition of emotional facial
expressions, either directly or as mediated by changes inself-reported emotional state (e.g., Blairy et al. 1999).
Hess and Blairy (2001) considered the possibility that
failure to support a causal path from facial mimicry toemotion recognition may have resulted from the use ofvery prototypical facial expressions. When they used naturalistic dynamic stimuli to remedy this problem, theyobserved facial mimicry, along with a relationship
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

423

Niedenthal et al.: The Simulation of Smiles (SIMS) model
between the perceived expression and self-reported
emotional responses (e.g., happiness when viewing
smiles). Nevertheless, they found no evidence of a linkfrom motor mimicry to recognition accuracy, either
directly or through changes in emotional responding.
Many additional findings further demonstrate thatmimicry does not always play a central role in emotion recognition tasks. As noted previously, recognition tasks on
prototypic expressions can be accomplished by perceptual
analysis alone, without motor mimicry (e.g., Adolphs
2002). Indeed, high-functioning autistic individuals, who
do not spontaneously mimic others’ facial expressions(e.g., McIntosh et al. 2006), perform as well as controls
when categorizing facial expressions of emotion (e.g.,
Spezio et al. 2007a). Similarly, individuals with facial
paralysis perform quite normally on various recognition
tasks (e.g., Calder et al. 2000a; 2000b; Keilloret al.
2002). Finally, actively keeping individuals from mimicking, for example, by asking them to turn the corners of
their mouth down while seeing a smile, does not hinder
emotion recognition (Blairy et al. 1999).
As all of these findings indicate, facial mimicry is notalways required to recognize emotional expressions in
simple recognition tasks. In some cases, however,mimicry does facilitate recognition. Niedenthal et al.
(2001), for example, observed effects of mimicry whenparticipants had to detect the boundary of facial
expression between happiness and sadness. In a more
recent study, Stel and van Knippenberg (2008) foundthat blocking mimicry affected the speed, but not the accuracy, of categorizing facial expressions as positive or negative. Additionally, individuals showing strong automatic
facial mimicry tend to have high levels of empathy(Sonnby-Borgström 2002; Zajonc et al. 1987). These findings point to the possibility that simulation does become
important in recognition tasks when they require fine distinctions in smile meaning, such as the processing ofdifferent smile types.
3.2.5. Beyond facial mimicry. As we noted earlier, pos-tural and other non-facial gestures are also important components of the meaning of a smile. We have discussed
facial mimicry so far, but representation of the fullmeaning of a smile will not be independent of the entire
bodily representation of meaning. The sign of appeasement involves a smile, usually a non-Duchenne smile infact, but also a number of other head and hand gestures(Keltner 1995). Tracy and Robins (2008) have describedthe expression of pride, which involves a small smile but
also a backward tilt of the head, resulting in a lifted chin,
as well as a typical posture. We expect the embodimentof the entire expression to be useful in the interpretationof meaning.
3.3. Simulating embodied experience: Somatosensory
cortices

As people perceive smiles, the somatosensory system may
simulate the embodied experience of how the perceived
smiles feel, further contributing to representing their
meaning. One account of how facial expression could be
simulated in an embodied manner is as the output of asimulator (e.g., Barsalou 1999). From this perspective,
somatosensory cortices may be involved in simulating
424

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

the feeling of a perceived smile while processing its
meaning (Adolphs 2002; Gallese & Goldman 1999;
Keyers et al. 2004).
Right-hemisphere cortices are likely to be involved insimulating emotional expressions (Adolphs et al. 1996;
Bowers et al. 1985). For example, patients with lesions
in the right somatosensory cortex are poorer at recognizing
facial expressions than individuals without such lesions.
Specifically, Adolphs et al. (2000) assessed 108 subjects
with focal brain lesions and found that the right somatosensory cortex was central for recognizing the facial
expressions associated with the six basic emotions. Consequently, these researchers concluded that the right somatosensory cortex generates a representation or an “image”
of the felt state, which feeds into the recognition system as
a diagnostic cue (e.g., Adolphs 2002; Atkinson 2007).
Following Adolphs et al. (2000), Pourtois and colleagues
used transcranial magnetic stimulation (TMS) to selectively interfere with right somatosensory cortex function
while participants performed a same/different facial
expression-matching task (see Pourtois et al. 2004). This
selective interference disrupted task performance. Thus,
both lesion and TMS studies implicate somatosensory
simulations in the recognition of perceived facial
expressions.
Although these previous studies focused on the simple
recognition of facial expressions, it is likely that somatosensory simulations also support more subtle interpretation of
facial expressions. Not only do somatosensory simulations
facilitate recognition, they probably contribute to how perceivers experience the meaning of these facial expressions,specifically, as a felt emotion. An important issue for future
research is to assess whether somatosensory simulationsindeed play this additional role.
3.4. What do smiles feel like for the perceiver?

In the previous sections we suggest that individuals knowwhether a smile means that the smiler is expressing enjoyment, affiliation, or dominance, because those smiles feeldifferently in terms of reward, action, and somatosensory
experience. Next, we explore how these different experiences help distinguish enjoyment, affiliative, and dominance smiles. From our perspective, we propose that an
enjoyment smile involves a basic rewarding feeling of positive affect and that an affiliative smile involves a positive
feeling of attachment and intimacy (where the positive
feelings of enjoyment vs. attachment/intimacy are distinct). Our analysis of dominance smiles relies on several
further assumptions. In hierarchical primate societies
such as ours, highly dominant alpha individuals pose acertain threat insofar as they can claim territory or possessions (e.g., food) from lower status group members(Menzel 1973; 1974). Hence, the perceived presence ofa dominant other should lead to increased vigilance andpreparedness for withdrawal (Coussi-Korbel 1994).
Although speculative, we suggest that dominance smiles,
like smiles hiding negative intentions or feelings, areassociated with the experience of negative rather thanpositive affect, as indicated by right-lateralized activation(Davidson et al. 1990; Boksem et al. 2009). Thus, themeaning of dominance smiles should not involve theforms of positive emotion associated with the other twosmiles.

Niedenthal et al.: The Simulation of Smiles (SIMS) model
In summary, we have documented possible neuralsystems that could contribute to the embodied simulations
that occur while perceiving emotional facial expressions,
thereby contributing to their interpreted meaning. A fullaccount of embodied simulation, however, requires theconstruct of a trigger that initiates embodied simulation
as facial expressions are perceived. Social and developmental considerations suggest that eye contact with theexpresser of an emotion launches embodied simulations
automatically.
4. Triggering embodied simulation
For eyes can speak and eyes can understand.
— Chapman

In the previous sections, we saw that embodied simulation
is not always implicated in the recognition of emotional
facial expressions, indicating that simulation is not always
required (for analogous findings and accounts, see Barsalou
et al. 2008; Hess & Blairy 2001; Kosslyn 1976; Niedenthal
et al. 2009; Solomon & Barsalou 2004; for related situated-simulation assumptions, see Semin & Cacioppo
2008; Smith & Semin 2007; Strack & Deutsch 2004). In
this section, we review recent research suggesting that eye
contact modulates the presence versus absence of embodied simulation as people perceive smiles.
4.1. Deﬁnitions related to eye gaze

Eye gaze is the direction of one’s gaze at another’s eyes,presumably during the search for information useful toattributing the cause of the other’s behavior (vonCranach 1971). Mutual gaze refers to two people gazingat each other’s faces. Eye contact involves two people
gazing at each other’s eyes. All such behavior has
become of great interest in scientific research lately,usually under the theoretical rubric of “social relevance”(e.g., Adams et al. 2003; Hess et al. 2007; Klucharev &Sams 2004; Mojzisch et al. 2006; Richeson et al. 2008).
Our primary interest here is in eye contact. Specifically,
we argue that eye contact counts as a sufficient, but nota necessary, trigger of embodied simulation as observers
perceive smiles.
4.2. Gaze and simulation of meaning
4.2.1. Eye contact and intimacy. Several sources of evi-

dence suggest that eye contact triggers embodied simulation. In part, the roots of this prediction lie in researchon intimacy. Argyle (1972) suggested that increased eye
contact during social interaction indicates an increase in
intimacy, which is also consistent with Patterson’s (1982;
1983) functional analysis. Additional findings support
these proposals: Increased eye contact is associated withincreased maternal sensitivity (Lohaus et al. 2001). Individuals make eye contact with people with whom they
have close relationships more frequently and for longer
durations than with strangers (Russo 1975). Men show
more approach behavior after repeated eye contact(Walsh & Hewitt 1985). Dating couples of both sexes
tend to look in the eyes of their partners more often
than unacquainted couples (Iizuka 1992).

Importantly, the pupils themselves impart no actual
descriptive information (beyond dilation and constriction,
which inform arousal). Thus, these findings suggest thateye contact triggers something beyond itself that increases
intimacy. We suggest that eye contact triggers an embodied simulation of the perceived facial expression and its
correspondent feeling for use in interpretation. A classic
study by Bavelas et al. (1986) supports this proposal. In
their study, a confederate faked experiencing an injury
and expressed apparent pain facially. When participants
viewed the pained facial expression, they inadvertently
mimicked it. The critical manipulation in the experiment
was whether the victim of the painful stimulus made eye
contact with participants. Analyses revealed that eye
contact significantly affected the pattern and timing of participants’ mimicry. Specifically, participants mimicked theperceived expressions of pain most clearly when eye
contact was made.
Consistent with this initial finding, Schrammel and colleagues (2009) demonstrated that, when viewing emotional
expressions, participants’ zygomaticus major (i.e., smile)
activity was higher for happy than for angry or neutral
faces, and, most importantly, that this effect was stronger
under conditions of eye contact. Furthermore, angry faces
elicited more negative affect following eye contact, andhappy faces elicited more positive affect, relative to theno-eye contact condition. For both emotions, eye contact
modulated the intensity of the experienced emotion.
Although both studies just described found that participantsin the no-eye contact conditions recognized smiles accurately, the eye contact effects suggest that these participants
may not have produced embodied simulations.
It is important to note, however, that drawing a strong
conclusion about eye contact’s role as a trigger for the
mimicry component of embodied simulation, is difficult.
First, as we have seen, some smiles are rewarding stimuli(e.g., the enjoyment smiles used in most research). Therefore, a perceiver of a smile could smile, even without
making eye contact with the smiler, simply as a resulted ofthe positive emotion associated with the perceived smile(Mojzisch et al. 2006). In this case, the elicited smile
would not count as mimicry, but rather as the readout ofthe perceiver’s positive emotional response to the smile.
This interpretation implies that much past research on
facial mimicry is not definitive because it does not distinguish mimicking a perceived smile from responding
with positive emotion to the sight of a smile. Nevertheless,because a smile can have functionally different meanings,
it is an ideal expression for distinguishing mimicry versus
emotional responding. Specifically, researchers could determine when perceivers make correct versus incorrectinterpretations of different smile types as a function of
simply responding with enjoyment to a smile or whether
they mimic it. As described next for the SIMS model, we
present conditions under which smiles simply produce
reward responses versus embodied simulations of the perceived smile and its associated emotion. If the model iscorrect, then embodied simulation should underlie thecorrect interpretations of smiles that have specific meanings.
4.2.2. Developmental considerations. The developmen-

tal literature offers further support for a relationshipbetween eye contact and embodied simulation of emotional states. In general, eye contact is very important for
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

425

Niedenthal et al.: The Simulation of Smiles (SIMS) model
infants. For instance, Farroni et al. (2002) demonstrated
that three-day-old infants looked longer at faces withdirect gaze, as opposed to simultaneously presented
faces with averted gaze. Thus, infants prefer faces that
establish eye contact. Furthermore, eye contact is associated with stronger neural processing, as demonstrated
by analyses of the infant N170 (Farroni et al. 2002;
Farroni et al. 2004). Infants also show enhanced neuralprocessing of angry expressions when these expressions
are accompanied by direct eye gaze (Striano et al. 2006).
Eye contact has thus been called the “main mode of establishing a communicative context between humans”(Farroni et al. 2002, p. 9602).
Infants’ interest in eyes has been interpreted as an interest in the perceptual features of the eyes per se, and as aninterest in direct gaze. Because infants benefit when adults
understand their internal states and needs, however, andbecause they have limited means to express those needs,
their interest in eye contact could instead reflect an evolutionary-based mechanism for triggering embodied simulation in caretakers. Such an interpretation receives
additional support from research showing that infants
use behaviors such as smiling, gurgling, and flirting toachieve eye contact with adults who are not looking atthem (Blass & Camp 2001). Infants also engage longer
and smile more often at individuals who make eye
contact (Hains & Muir 1996; Symons et al. 1998).
4.2.3. Role of the amygdala. An increasingly clear con-nection between eye contact and amygdala activation
further supports this account (Dalton et al. 2005; Pourtois& Vuilleumier 2006). Complete amygdala lesions result ina severe reduction in eye contact during actual conversations (Spezio et al. 2007b). Conversely, as neural activity
in the amygdala increases, monitoring for emotional gaze
events in others increases (Hooker et al. 2003).
A recent study that recorded electroencephalogram(EEG) and EMG while participants viewed smiles andsad expressions showed a similar effect (Achaibou et al.
2008). Peak EEG activation around 100 msec (P1) was
associated with greater correspondent EMG activity slightly
later, at around 300 msec. Achaibou et al. interpreted theincreased P1 amplitude as a signature of the participant’s
attentional state, suggesting that deeper visual processingof the facial expression was associated with enhanced
mimicry, as indexed by EMG. Although amygdala activation was not measured, all the findings reviewed here
on functions of the amygdala suggest that, when the amygdala becomes active, it directs attention to the eyes, and that
resulting eye contact elicits greater or more correspondent
facial mimicry. The link between amygdala activation andmimicry further follows from involvement of the dorsal
midbrain, which has been linked to increases in arousal produced by direct gaze (Donovan & Leavitt 1980). Although
we have assumed conservatively here that amydala activation increases motor mimicry in smiling, it is also possiblethat it increases embodied simulation as well, thereby producing stronger emotional responses.
4.3. Perceptual information from the eye region versus
eye contact

A potential confound exists between looking at the eyes asa way to extract perceptual information and making eye
426

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

contact as a possible trigger to simulation. Adophs and colleagues, for example, reported extensive study of patient
S.M., who had a bilateral brain lesion that encompassed
all nuclei of the amygdala (Adolphs et al. 2005). In initialassessments, S.M. showed a compromised ability to recognize fearful expressions. In subsequent assessments, thisfailure was attributed to not using information from theeyes. Failure to look at the eye region during smiles may
increase compensatory looking at the mouth region.
Because S.M. used information from the mouth normally,
she was able to correctly distinguish smiles from fear
expressions. More recent findings from the same laboratory found that high-functioning autistic children also
avoid gazing at the eye region when performing a facial
expressions recognition task (Spezio et al. 2007a). Here,too, the autistic group performed as well as the control
group in recognition accuracy (based on impoverishedinformation provided in the Bubbles technique), perhaps
reflecting the use of other diagnostic perceptual cues,
such as the mouth.
The specific ability to distinguish Duchenne from nonDuchenne smiles has been shown to be compromised in
autistic adults, who failed to look at the eyes while
making judgments (Boraston et al. 2008). More specifically, this result was interpreted as reflecting failure touse information in the perceived expression provided bycontraction of the orbicularis oculi muscle in the vicinityof the eyes. Although these findings have typically beeninterpreted as showing that information from the eyeregion carries specific perceptual information, they canbe also interpreted as demonstrating the importance ofthe eye region for triggering embodied simulations. As
described in the SIMS, we propose that these findings
actually demonstrate that eye contact has evolved as a
trigger for embodied simulations, which endow smiles
with their different functional meanings.
5. Conceptual knowledge about facial expression
We have already mentioned that individuals possess anduse stereotyped knowledge and expectations about themeanings of smiles (Halberstadt et al. 2009; Kirouac &Hess 1999). A challenge for the present approach is tosuggest how this knowledge is represented and also how
it relates to embodied simulations of perceived smiles.
As will be seen in later sections, conceptual knowledgeof smiles plays a central role in the SIMS model of how
people interpret their meanings.
The classic view of emotion concepts relies on generalmodels of representation in the cognitive sciences thatview concepts as redescriptions of the input frommodality-specific systems into an abstract representation
language (e.g., Bower 1981; Johnson-Laird & Oatley1989; Ortony et al. 1987). According to these accounts,
higher-order mental content is represented in an abstract,
language-like code that has an amodal character (e.g.,
Fodor 1975). Notably, an amodal representation does
not preserve anything analogical about the perceptual
experience of the object, event, or state. Instead it isabstracted away from this experience and transduced
into some sort of representation language that is abstract
and amodal in format. Whether the resulting representation takes the form of something like a logical structure,

Niedenthal et al.: The Simulation of Smiles (SIMS) modela feature list, or vectors on which different values can bepositioned, the assumption is that the multi-modal experience of something and its conceptual representation do
not take place in the same system (Barsalou et al. 2003;
Niedenthal et al. 2005b).
The social psychology and emotion literatures are filledwith evidence, now supported by findings from neuroimaging studies, that is consistent with an embodied or
grounded account of conceptual processing (Barsalou
1999; 2005; 2008; Gallese & Lakoff 2005; Niedenthal
2008). From this perspective, the modality-specific statesthat represent perception, action, and introspection
when one interacts with a particular entity or has a particular subjective experience also represent these same stimuli
and events when processed in their absence. For example,
retrieving the memory of a landscape involves reactivating(simulating) parts of the visual states that were active while
perceiving it. From this perspective, then, having a
concept is having the ability to reenact experiencing an
instance of the respective category.
As can be seen from these two accounts of concepts, theembodied account provides a natural way to link conceptual knowledge about smiles and the related social situation to the actual perception of smiles. Rather thanassuming that a smile activates an amodal knowledge
structure to represent its meaning, the embodied simulation account proposes instead that a smile triggers asimulation of a smile experience that includes emotion,
motor activity, and somatosensory experience (Niedenthal
et al. 2005a). Once this simulation becomes active, it provides a conceptual interpretation of the perceived smile,
going beyond the information given to place it in acontext. Clearly, much about this account remains to be
developed, especially the relevant neural systems andtheir role in social processing.
The grounded cognition position is important in presenting the SIMS model for a specific reason, namely,
because an embodied simulation is not always a response
to a currently perceived smile. Additionally, an embodied
simulation can also be triggered by the activation ofrelated knowledge or expectations (e.g., stereotypes). For
instance, when someone activates conceptual knowledge
about infants, this could produce an embodied simulationof an infant. Significantly for the SIMS account to follow,
these conceptually triggered simulations may not always
be relevant or correct in the current situation, when adifferent embodied simulation triggered by the actual situation and facial expression offer a more correct interpretation. Rather than mimicking and feeling the reward valueof an enjoyment smile present in the current situation,for example, an individual who expects to see a dominance smile may experience negative feelings and interpret the smile as expressing superiority, as the result of
unjustified conceptual processing (Halberstadt et al.
2009).
6. The simulation of smiles (SIMS) model
We now present a model that seeks to integrate brain andbehavior in accounting for the representation of smile
meaning. We focus on the possible meanings of a smile
as expressing enjoyment, affiliation, or dominance. These
judgments can be seen as having correct answers only inso far as smiles with certain identity, postural, morphological, and dynamic markers have been selected or developed
as stimuli. In addition, stimulus development would
require validation by reference to the intentions and feelings of the person who is smiling, as well as the demonstration of reasonable consensus in interpretation by
perceivers. For example, a set of empirically derived
enjoyment smiles would be static or dynamic facial displays of people who were experiencing joy, as indicatedby self-report, at the time of the smile.
First, we present the core SIMS model, which establishes how grounding smiles differentially in neuralsystems and behavior causes them to be interpreted differently as meaning enjoyment, affiliation, or dominance (seeFigure 4 in sect. 6.1.1). Then, we propose conditions
under which perceptual cues, experiential cues, and conceptual knowledge are used to interpret smiles in thesedifferent ways (Figures 5 –7; Adolphs 2002; Kirouac &Hess 1999). The SIMS model does not attempt toaccount for the entire neural circuitry that underlies theprocessing of emotional facial expressions (as discussed,
e.g., in Adolphs 2002; 2006; Heberlein & Atkinson2009), nor does the SIMS model attempt to account forthe details of real-time neural time courses. Instead, thefocus is on the conditions under which embodied simulation and other cues are used to arrive at judgments ofthe three smile meanings. Thus, our account aims to represent cognitive-behavioral function, but not the timing ofneural activity.
The SIMS model has been largely developed using datacollected in Western countries. Nevertheless, it is essential
to note that cultural differences may modulate our
account. Some clear predictions across cultures can be
imagined and have been articulated elsewhere (Niedenthal et al., in preparation).
6.1. Details of the SIMS model
6.1.1. Core SIMS model. We begin by considering how

smiles are interpreted in the most ecologically validsituation, that is, in which the smile has an uncertain
meaning (in the sense of being unexpected in thecontext). First, consider the empirically derived enjoyment
smile illustrated in the top panel of Figure 4. As the resultof eye contact, a reward experience is generated in thebasal ganglia, and motor mimicry is generated in the cortical motor regions (in particular, those described by Schilbach et al. 2008). In turn, these two activations produce
corresponding bodily experience in the somatosensory
cortex. The top panel of Figure 4 represents this overall
pattern. Thus, the most straightforward case of processing
a smile’s meaning begins with the detection of uncertainty(producing amygdalar activation), which, in turn, directs
gaze toward the eyes (eye contact), followed by the generation of reward, motor mimicry, and corresponding somatosensory experience. As this initial example illustrates, anembodied simulation of enjoyment grounds visual perception of the perceived smile, leading to the interpretationthat the person is “happy.”
The middle panel of Figure 4 illustrates the sameprocess for the judgment that a smile is an affiliative
smile (e.g., that the smile is understood as “friendly” or“conciliatory”). Many of the relevant brain regions arevery similar to those just described for the judgment of
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

427

Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 4. The top panel illustrates the case of an enjoyment smile (A) presented such that the meaning is initially uncertain. The
perception of the smile is accompanied by activation in the amygdala (B). Research suggests that amygdala activation enhances theprobability that eye contact with the smiling person will be made (C). In the SIMS model, eye contact has evolved as a trigger for
embodied simulation. Eye contact thus produces increased activation in the reward centers of the basal ganglia (D1) and in motor
regions, described by Schilbach et al. (2008) (D2), that support motor mimicry (E). These motor and limbic processes thenproduce bodily sensations in somatosensory cortex (F). On the basis of these neural activations and behaviors, the smile is judged asindicating that the smiling individual feels happy (G). The middle panel illustrates the process that results in the judgment of asmile as affiliative. The only difference between the content of the two panels (A0 – G0 ) is the additional OFC (D30 ) activation,
which in theory supports distinctive positive affect related to attachment. The bottom panel shows the processing of a dominancesmile (A00 ). Amygdala activation would again be expected (B00 ) and eye contact would be predicted to occur (C00 ). Dominance smiles
may be associated with a pattern of asymmetrical neural activation related to withdrawal-related negative affect (e.g., Davidson
et al. 1990; D100 ). Activation in relevant motor regions (D200 ) would be expected and output resulting in mimicry (E00 ). Because of
the role of prefrontal cortices in processing status, OFC or contiguous regions may also be involved (D300 ). Implications of these
supported by somatosensory cortices (F100 ) will ground a judgment of a smile as a smile of superiority of some type (G00 ).

428

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 5. The top panel shows processing of an enjoyment smile (A) when eye contact is not achieved for experimental reasons (B), as
in a blocked design (in which uncertainty is low). As a result, processing primarily focuses on visual features of the smile. The matchingof visual input for the perceived smile to stored perceptual representations of previously experienced smiles is indicated by theactivation of occipito-temporal cortices (C) (Adolphs 2002; Calder & Young 2005). Semantic associations to the perceptual
representation requiring involvement of prefrontal cortex, such as OFC (D), could be relied on for producing a judgment (E). The
bottom panel depicts a situation in which motor mimicry to an enjoyment smile (A0 ) is inhibited for experimental reasons, as
through the blocking of facial musculature (e.g., Oberman et al. 2007). Perception of the smile will be associated with amygdala
activation (B0 ), and eye contact will be made (C0 ). Because motor mimicry is inhibited, activation in motor systems and emotion
systems will be absent or reduced. Matching of visual input to stored perceptual representations still occurs in occipito-temporal
cortices (D10 ), and premotor regions may be weakly active (D20 ), reflecting the processing of an as-if motor loop. Again, semantic
associations, requiring involvement of prefrontal cortex (E0 ), would be necessary for a specific judgment of meaning (F0 ).

an enjoyment smile. For an affiliative smile, however, processing would also involve OFC activation and perhaps theinvolvement of closely related prefrontal regions. As discussed previously, these areas may selectively support
the distinctive positive feeling of seeing an individual
smile with whom one has a close relationship, as in thefinding mentioned earlier when OFC differentiates thesight of one’s own smiling baby from the sight of an
unknown smiling baby (e.g., Minagawa-Kawai et al.
2009; Nitschke et al. 2004).
The bottom panel of Figure 4 illustrates the processingof a dominance smile. Again, amygdala activation would be
expected to signal the expression’s uncertainty and its
potential significance. Here, however, negative rather
than rewarding affect may be experienced. Although
speculative, we have suggested that dominance smiles,
like smiles hiding negative intentions or feelings, areassociated with the experience of negative rather thanpositive affect, as indicated by right-lateralized activation(Boksem et al. 2009; Davidson et al. 1990). Because

OFC and contiguous regions are central to processing
social status (Zink et al. 2008), ventral frontal cortex may
also be involved. As before, activation of cortical motor
regions and correspondent mimicry also occur andproduce corresponding somatosensory experience associated with the feeling of being dominated (e.g., that thesmile is experienced as “superior” or “condescending”).
6.1.2. Behavioral constraints in SIMS. Altering the core

SIMS model in principled ways facilitates further understanding of smile processing.
6.1.2.1. Experimental inhibition of behavior. In contrast to

the three cases in Figure 4, the two cases in Figure 5
assume that embodied simulation is not likely to occur.
An enjoyment smile is used as the example stimulus
event. As the top panel of Figure 5 illustrates, no eye
contact is achieved with the smiler as was the case for
Figure 4. This may occur experimentally when the smile
has low uncertainty, as in a blocked design when many
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

429

Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 6. All three panels illustrate situations in which eye contact does not occur, analogous to the three cases in Figure 4 when eye
contact does occur. Although the neural activations are very similar, the suppressed social behavior in Figure 6 detaches any emotional
processing that occurs from the perceived smile and renders it knowledge-based. Thus, the emotional content is determined bysimulation of conceptual knowledge in memory, rather than being driven by the experience of eye contact (Niedenthal 2008). The
top panel illustrates the case in which a smile is believed to be a smile of enjoyment (A). Perception of the smile is accompanied by
activation in the amygdala (B). We have defined this as a situation in which eye contact is avoided (C). Nevertheless, because thesmile is believed for other reasons to be an enjoyment smile, activation of the reward centers of the basal ganglia (D1) occurs, andalso the motor brain regions described by Schilbach et al. (2008) (D2). Correspondent smiling (E) occurs, but is determined bysimulation of conceptual knowledge and does not count as mimicry (as in Halberstadt et al. 2009). Conceptual implications are
represented by somatosensory cortex (F) and confirmation of the smile as an enjoyment smile (G) is made on this basis. Note, ofcourse, that the judgment could be wrong with regard to smiler intention. The middle panel illustrates the same process that
results in the judgment that a smile is an affiliative smile (A0 – G0 ), again without eye contact. The only difference with the top
panes is the again additional robust OFC activation (D30 ), which in theory supports positive affect related to attachment. The
bottom panel shows the processing of a smile believed to be a smile of dominance (A00 ), where again amygdala activation simply
reflects visual cues from the mouth (B00 ) and eye contact does not occur (C00 ). Withdrawal-related negative affect (e.g., Davidson
et al. 1990; D100 ) and activation in relevant motor regions (D200 ) that support smile production is again expected (E00 ), as describedearlier for Figure 4. Because of the role of prefrontal cortices in processing social status (Zink et al. 2008), OFC or contiguous
regions may also be involved (D300 ). The representation of these inputs in somatosensory cortices (F00 ) will serve to confirm theinterpretation of the smile as an expression of dominance (G00 ).

430

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 7. The figure illustrates the judgment that a smile communicates enjoyment when mimicry is inhibited. Perception of the smile
is associated with amygdala activation (B) and eye contact is made (C). Because motor mimicry is inhibited, activation in motor systems
and emotion systems is absent or reduced. Premotor regions may be active (D1), reflecting the processing of an as-if motor loop, driven
by semantics in the temporal and prefrontal cortex (D2) to grounding interpretation (E).

smiles are presented one after the other. As a result, therich behavior and brain activity associated with eye
contact is absent. Instead, recognition of the expression
as a smile only reflects processing visual features of themouth. The matching of this visual input to perceptual
representations of stored smiles is indicated in the top
panel of Figure 5 by activation in occipito-temporal cortices (Adolphs 2002; Calder & Young 2005). Further
interpretation of the smile’s meaning beyond simple recognition could occur if conceptual knowledge produces
expectations about smiling and possibly a simulated
smile (as described in further detail later). Because eye
contact is not available to trigger embodied simulation,some other source of information must be consulted toprovide meaning, should additional meaning be relevant.
The bottom panel depicts a second case in which facial
mimicry to a smile is inhibited. Inhibition can occur experimentally when spontaneous expression is blocked by mechanical means (e.g., Oberman et al. 2007). In these situations(when uncertainty is again high), perception of the smile
will be associated with amygdala activation. Because
motor mimicry in inhibited, however, activation in motor
systems and emotion systems will be absent or reduced,
although premotor regions may be partially active, reflecting the as-if motor loop that simulates motor activity at a
relatively abstract level. Because actual motor activity
does not occur however, the smile is not felt in somatosensory areas, and recognition occurs on the basis of matchingvisual input about the perceived face to stored perceptual
representations. Again, interpretation of the smile’s specificmeaning based on embodied simulation could occur, butwould require the use of conceptual knowledge or expectations, given the absence of embodied simulation that
results from eye contact.
6.1.2.2. Social inhibition of behavior. The experimental situ-

ations represented in Figure 5 also occur in natural social
situations, but for far more interesting reasons. Eye contact
and mimicry will sometimes be suppressed when a smile isdisplayed by an individual with whom exchanging eye

contact and mimicry is too intimate, risky, or aversive (Leeet al. 2006; Likowskiet al. 2008; Mondillon et al. 2007).
Unlike in the artificial experimental context, however, it islikely that embodied simulations will become active and contribute to judging a smile’s meaning. Such simulations,however, will again not be responses to the perceived
smile, but will result from activating conceptual knowledge.
Emotions caused by conceptual knowledge can indeed
affect the interpretation of a perceived facial expression
via embodied emotion simulations. Niedenthal et al.
(2000), for example, induced states of happiness,sadness, and neutral emotion in participants, triggeredby conceptual knowledge. As participants then viewed
movies of faces whose expressions changed from happiness or sadness to neutral emotion, their task was to ascertain when the initial emotional expression on the face
ended. The induced emotional state of the participants
affected their perception of that offset point. In otherwords, an emotion-congruence effect occurred between
a conceptually activated embodied simulation and a perceptually processed face. Participants in a happy state
saw smiles persist longer into the ambiguous neutralzone of the expressions, whereas sad participants saw
sadness persist somewhat longer. Emotional traits andstyles similarly affect performance on this task, further
indicating that conceptual knowledge contributes to theinterpreted meaning of a perceived facial expression via
embodied simulations of emotion (Feldman-Barrett &Niedenthal 2004; Fraley et al. 2006). Again, conceptual
knowledge about the smiler (or about the context inwhich the smile was encountered) triggered embodied
emotion simulations that affected judgments of perceived
smiles (e.g., Halberstadt et al. 2009).
Figures 6 and 7 illustrate socially inhibited eye contact
and mimicry, respectively. For the former, the components are similar to the analogous ones in Figure 4.
Here, however, the suppressed social behavior detaches
processing from the stimulus and renders it knowledgebased, with embodied simulation and emotional content
being provided by conceptual knowledge in memory for
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

431

Niedenthal et al.: The Simulation of Smiles (SIMS) model
the respective facial expressions (indicated in portion A ofeach panel; Niedenthal 2008). Facial expression may
occur, but is not necessarily related to the perceived
smile, therefore not counting as mimicry. Instead, suchfacial expressions reflect simulation of emotions associatedwith conceptual knowledge. As a result, the observer’s
smile may not constitute mimicry (this may have occurredin Mojzisch et al. 2006), and may obscure the smile’s functional meaning. An interesting question for future
research is to examine the neural basis of using one’s
own knowledge to generate an embodied emotion simulation versus using facial cues in perceived expressions,such as eye contact (e.g., Decety & Grèzes 2006). In
Figure 7, facial mimicry is the behavior that is inhibited.
As discussed earlier (sect. 3.2.2.), inhibited mimicry
reduces emotional processing. Even when a prior expectation for a smile meaning exists (such as the expectationthat the person is expressing an enjoyment smile), emotional processes may still be inhibited because the motor systemcannot become engaged as usual. Under these conditions,conceptual knowledge about emotion in memory may be
used instead to establish the meaning of the smile, rather
than embodied simulation triggered by eye gaze. Unlike
the process illustrated in Figure 6, however, perceivers
here may rely more on “disembodied” linguistically based
associations and social concepts. Brain regions involved in
the use of semantic associations depend on the task andthe way in which the semantic knowledge is called for(e.g., Barsalou et al. 2008).
6.2. Judgments of smiles as true versus false

One particular distinction between types of smiles hasreceived considerable attention: whether a smile is trueor false (e.g., Miles & Johnstone 2007). Asking this question already implies a culture-laden value that smiles canbe false per se (Fridlund 2002). Some cultures and someindividuals seem to mean by false that there is a desire
to deceive or manipulate (Frank 2002). This would beakin to saying that a smile put on to hide a negative
intent is a false one. Alternatively, individuals sometimes
mean that the smile was produced voluntarily and that
this act is itself inauthentic or false (Fridlund 2002). This
would be akin to saying that any intentional smile is, bydefinition, false. Although the definitions of true andfalse smiles will ultimately have to be bootstrapped, atleast by culture, we can propose a working definition
here: Judging a smile as a true smile is the normative or
default judgment, and it means that the smile signifies
positive feelings of some kind (e.g., pleasure or joy) or
positive intentions. Conversely, judging a smile as false
means that the perceiver believes that the smile was motivated by a desire to hide, moderate, or justify something
negative (e.g., a lie, a criticism, a feeling of superiority orcontempt, a manipulation, an inappropriate affect).
Rather than a clear dichotomy, the true-false distinction
is a continuum that is most often represented in the
English language by a value on a scale of “genuineness,”“authenticity,” or “sincerity” (Hess & Kleck 1990). Different measurable features of smiles have been claimed to be
associated with these judgments. Indeed, Duchenne
smiles are often judged as more genuine than non-Duchenne smiles. Individuals also judge smiles with a slow
onset, a slow offset, and a shorter apex as more genuine
432

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

than smiles that have a sudden onset, a sudden offset,
and a long apex (Krumhuber & Kappas 2005). The
former smiles are further associated with more positive
personality judgments (e.g., Krumhuber et al. 2007).
Notably, however, these features do not in any way map
isomorphically to the true/false judgment. For instance,
as already mentioned, the importance of Duchenne
markers for judging smiles in still photos varies with
culture (Thibault et al., submitted). In addition, the Duchenne marker is far less important in dynamic as compared
to still smile stimuli (e.g., Krumhuber et al. 2007).
The SIMS model predicts when a smile will be judged
as more or less genuine and on what basis. For example,
enjoyment and affiliative smiles with initially uncertain
meaning should always be judged as true smiles. This isbecause the default judgment of a smile being genuine isbased on the presence of positive affect and motor
mimicry in an embodied simulation associated with it. In
contrast, dominance smiles should be more likely to be
judged as false smiles, due to the greater presence of negative affect and withdrawal in their associated simulations(e.g., Davidson et al. 1990).
In contrast, when eye contact is not made, or whenmimicry is inhibited by social factors, true/false judgments
should be guided by cultural beliefs and stereotypes stored
in conceptual knowledge. As a result, true/false judgments
may be much less systematic than when eye contact andmimicry both support the embodied simulation.
A recent study by Maringer et al. (in press) supports this
proposal. Participants were given empirically derived“true” dynamic smiles or “false” dynamic smiles to see
and asked to rate them on scales of genuineness (Krumhuber et al. 2007). Orthogonally crossed with the true-false
manipulation, half of the participants were able to freely
mimic the smiles, and the remaining half held a pencil
in their mouths so as to block facial mimicry. The results
were clear: In the mimicry condition, participants who
saw true smiles rated them as more genuine than participants who saw false smiles. In the mimicry-blocked
condition, participants did not distinguish between the
two types of smiles. Thus, the ability to mimic facial
expressions was essential for distinguishing true fromfalse smiles, implicating embodied simulation in performing these judgments accurately.
In a second study, participants viewed true smiles andwere told either that these smiles occurred in a context inwhich a true smile would be expected by cultural stereotypes or that these smiles occurred in a context in which a
false smile would be expected. Mimicry was blocked inhalf of the participants, and all participants rated thesmiles on genuineness. Results showed that whenmimicry could occur, cultural beliefs and stereotypes did
not affect the perceived genuineness of the smiles, butwhen mimicry was blocked, smiles that occurred in contexts
associated with true smiles were judged as more genuine
than those that occurred in contexts in which false smiles
could also occur. Again, embodied simulation appears to
be a critical cue for establishing that a smile is genuine.
7. Beyond smiles
The present article and model focus on the smile because
it is probably the most complex of all emotional

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
expressions. Not only are smiles highly diverse in the conditions under which they occur, they are also highly
diverse in their possible meanings. Significantly,
however, similar embodied processes should support
interpreting the expression of other basic emotions (e.g.,anger, sadness), as well as the expression of secondary
emotions (e.g., shame, embarrassment). In general, theperception of any facial expression should trigger affective,
motor, and somatosensory experience that gives theexpression meaning. Thus, the structure, but not thecontent, of the SIMS model can be used analogically toinvestigate the interpretation of facial expressions
besides the smile.
As an illustrative example, consider anger. Expressionsof anger are used to establish and maintain social hierarchies, among other social functions (Keltner & Haidt
1999). Anger expressions are slightly different than thoseof contempt (Ekman & Heider 1988), but may occur insimilar situations. Indeed a family of anger-related
emotions exists, ranging from frustration to outrage.
Given their different social functions, it would be essential
for socially competent individuals to correctly experience
their subtle meanings in the respective situations (e.g.,
Fischer & Roseman 2007).
A functional analysis of anger expressions is likely tosuggest different social determinants of eye contact andmimicry than those for smiling. For instance, there is evidence that anger expressions are mimicked less to thedegree that a situation is social, unless anger is expressedwith averted gaze indicating that the source of the anger
is not the perceiver (Bourgeois & Hess 2008; Hess et al.
2009a). Furthermore, because anger is statistically less frequent in most environments than the smile, and has different implications for the observer, some neural circuits,
such as those involving the amygdala may be activatedmore often and be less sensitive to environmental determinants of uncertainty. Perceiving and mimicking an anger
expression may also have different somatosensory experiences than perceiving and mimicking related emotions,such as frustration, contempt, and outrage. Finally,
anger has been linked to neural structures that do notappear relevant for interpreting smiles, including theventral striatum (Calder et al. 2004) and dopamine receptors (Lawrence et al. 2002).
This example is not intended to be comprehensive.
Rather, it simply suggests that the present model can beextended to other emotions, but cannot be extended effectively without due consideration of the specific circumstances, together with the accompanying behavioral andneural responses. We believe, however, that the basic
structure of the model, with its emphasis on integrating
specific social behaviors with related forms of embodied
simulation, is viable in modeling the meaning of all
emotional facial expressions.
8. Conclusions
The impact of Darwin’s (1872) writing on facial expression
and its centrality in more recent theories of emotion (e.g.,
Ekman 1989) has pushed the study of emotion expression
toward a dictionary of facial muscles and their combinations as signs to internal states. The notion that a physical description has access to disembodied amodal

knowledge, which can, in turn, be used to interpret theperceptual input, has been a natural way to think about
the processing of emotional facial expressions in thecontext of the Cognitive Revolution. Given more recent
developments in grounded cognition and neuroscience,
however, it seems increasingly likely that a construct
along the lines of embodied simulation is necessary for a
full account of how people establish the meaning offacial expressions. Increasingly, new empirical techniques
provide additional ways of measuring embodied simulation and establishing causal evidence for its roles insocial processing (e.g., the use of transcranial magnetic
stimulation [TMS] in Pitcher et al. 2008). The SIMS
model of the smile is an attempt to show how behavioral
and neuroimaging findings can be integrated to generate
new and productive process models of facial expression
more generally. Solving the riddle of the smile will, we
believe, provide important groundwork for understanding
the full array of emotional facial expressions.
ACKNOWLEDGMENTS
The authors thank Ralph Adolphs, Anthony Atkinson, Markus
Brauer, Julie Grèzes, and Piotr Winkielman for comments onan earlier draft of the paper. We extend a particularly grateful
thanks to Lawrence Barsalou for providing very specific anddetailed feedback on multiple drafts. The work of Paula
M. Niedenthal and Martial Mermillod was supported by agrant (FaceExpress – Blanc CSD9 2006) from the Agence
National de Recherche (ANR), France; the work of Ursula
Hess by a grant from the National Science Foundation (NSF)
and the Social Sciences and Humanities Research Council of
Canada.

Open Peer Commentary
Mimicry and simulation in gesturecomprehension
doi:10.1017/S0140525X10001445
Martha W. Alibalia and Autumn B. Hostetterba

Department of Psychology, University of Wisconsin–Madison, Madison,
WI 53706; bDepartment of Psychology, Kalamazoo College, Kalamazoo,
MI 49006.
mwalibali@wisc.edu
Autumn.Hostetter@kzoo.edu
http://psych.wisc.edu/alibali/home.html

Abstract: According to the SIMS model, mimicry and simulation
contribute to perceivers’ understanding of smiles. We argue that similar
mechanisms are involved in comprehending the hand gestures thatpeople produce when speaking. Viewing gestures may elicit overt
mimicry, or may evoke corresponding simulations in the minds ofaddressees. These real or simulated actions contribute to addressees’
comprehension of speakers’ gestures.

In their target article, Niedenthal et al. present the SIMS model,
which provides a theoretical account of how embodied simulation is involved in interpreting smiles. We suggest that mechanisms like those proposed in the SIMS model are also involved
in interpreting the spontaneous hand gestures that people
produce when speaking. In past work, we have argued thatgesture production is based on simulated actions and perceptions(Hostetter & Alibali 2008). In this commentary, we make the
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

433

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) modelcase that embodied simulation is also involved in gesturecomprehension.
The Gesture as Simulated Action (GSA) framework (Hostetter& Alibali 2008; 2010) explains how representational gestures
might arise from an embodied cognitive system. Aligned withtheories of embodied cognition (e.g., Barsalou 1999; Glenberg
1997), the GSA framework holds that language processing activates perceptual and motor states. Speakers form mental
images of information they are expressing; these mental images
activate the same perceptual and motor systems that are involvedin perceiving and interacting with physical objects in the world.
This activation of the motor system is sometimes overtly
expressed as representational gestures.
Here, we consider whether simulated actions could also be
involved in gesture comprehension. Niedenthal et al. argue thatwhen people perceive smiles, they engage their motor systems,
either to mimic the perceived smiles or to simulate the experience of performing the perceived smiles. This mimicry or simulation contributes to perceivers’ understanding of the perceived
smiles.
We suggest that the same mechanisms – mimicry and simulation – might also be involved in comprehending representational gestures. Imagine a speaker describing how he makes
pizza, who uses gestures to depict how he uses his fingertips
and his knuckles to flatten the dough. Imagine further that the
speaker’s addressee has never made pizza crust before. In comprehending the speaker’s description and gleaning informationfrom his gestures, the addressee might overtly mimic the speaker’s actions, or she might mentally simulate such actions.
Do speakers overtly mimic other speakers’ gestures? A
growing body of evidence indicates that they do (e.g., Kimbara
2006; 2008; McNeill, in press). Mimicry does not arise as asimple coincidence of people using similar gestures when theytalk about similar things. Instead, similarity of gesture form ismore likely when interlocutors can see one another than when
they cannot, suggesting that mimicry is purposeful (Kimbara,2008). McNeill (in press) observes that researchers who code
gestures often use mimicry as part of their effort to make senseof a speaker’s gestures. He argues that addressees use mimicry
to understand gestures that may not be readily interpretable
otherwise.
Although overt mimicry of gestures does occur, it is not as
widespread as might be expected if it were the primary mechanism of gesture comprehension. Recall that Niedenthal et al.
emphasize that smile perceivers sometimes simulate the experience of the perceived smile, without overtly mimicking it.
Along similar lines, it seems likely that gesture comprehenders
sometimes simulate the gestures they perceive, without overtly
producing them. These embodied simulations of gesturalactions may contribute to comprehenders’ interpretation of
the perceived gestures. Support for this mechanism comesfrom studies that have shown activation of premotor areas
when people observe actions (for reviews, see Jeannerod
2001; Rizzolatti et al. 2001). Premotor cortex is also activatedwhen people observe gestures (e.g., Montgomery et al. 2007),and this activation is modulated by semantic information fromthe accompanying speech (Willems et al. 2007). It has beenargued that, when addressees perceive gestures, they interpretthem, at least in part, by activating the cortical networks
involved in producing gestures, via an “observation-execution
matching system” (e.g., Holle et al. 2008; see Dick et al.
2009, for discussion). This idea is similar to the one weadvance here.
The SIMS model holds that, in the case of smiles, eyecontact triggers embodied simulation. It is possible that eye
contact may also initiate embodied simulation of the co-occurring gestures; however, no studies to date have specificallyexamined the relation between eye contact and gesture comprehension. Instead, the little research that has explored therole of eye gaze in gesture comprehension suggests that434

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

addressees are most likely to attend to speakers’ gestures
when speakers direct their gaze to the gestures (e.g., Gullberg& Holmqvist 1999). Further, addressees are more likely toincorporate information from the gesture into their comprehension when speakers have fixated on the gestures (Gullberg& Kita 2009).
One explanation for these findings is that speakers’ gaze totheir own gestures signals that the gestures are important;addressees then directly attend to those gestures and run “asif” simulations of the gestures that facilitate comprehension.
Speakers’ shift in eye gaze may be particularly important for initiating addressees’ simulations of speech-accompanying gestures,
because addressees typically focus on speakers’ faces, making it
difficult to attend to detailed features of gestures produced inperipheral space.
Could mimicry and simulation account for individual andsituational differences in addressees’ reliance on gesture for comprehension? Research has shown that gestures make a greater
contribution to comprehension when the material to be comprehended is complex, ambiguous, or challenging relative to theaddressee’s skills (Graham & Heywood 1976; McNeil et al.
2000). It also seems likely that individuals differ in their relianceon gesture for comprehension, although there is little data
directly addressing this point. We suggest that addressees varyin how likely they are to engage in overt mimicry and/or simulation. Across contexts, individuals may be more likely toengage in mimicry and/or simulation in situations wherelanguage comprehension is more difficult, as suggested by
McNeill (in press).
In sum, building on our past work about simulated action as an
explanation for why speakers produce gestures, we suggest thatsimilar mechanisms may be involved in comprehension of gestures. Viewing a speaker’s overt simulation of an action or
event may elicit overt mimicry, or it may evoke a corresponding
simulation in the addressee’s mind. In the SIMS model, mimicryand simulation contribute to perceivers’ understanding of smiles.
We suggest that mimicry and simulation also contribute toaddressees’ comprehension of speakers’ gestures.

Emotion simulation and expression
understanding: A case for time
doi:10.1017/S0140525X10001457
Marian Stewart Bartlett
Institute for Neural Computation, University of California, San Diego,
92093-0440.
marni@salk.edu
http://mplab.ucsd.edu/ marni

Abstract: Niedenthal et al. present a model for embodied emotion
simulation and expression understanding that spans multiple brain
systems. This commentary addresses the potential role of time in thismodel, and its implications for understanding social dysfunction.

Niedenthal et al. diverge from an increasing trend for localizationalism in cognitive neuroscience, and present a model of
emotional expression and understanding that spans the somatosensory system, motor system, reward system, amydgdala, basal
ganglia, and prefrontal cortex, as well as systems involved ineye gaze and body posture. This model enfolds ideas relating tomirror neurons and the link between perception and production,
but as a subset of a larger system for understanding emotions inothers.
Not only is this model of emotion unlocalized in space, it isalso not localized in time. The systems involved must communicate and pass activation through time in order to fully engage
this emotion system, akin to a closed-loop dynamic process.
It may be crucial that the timing of the interactions of theseCommentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
structures be right. Disorders of temporal processing at any
level of this system, whether in the perception or the production of smiles, or in passing activation between brain structures, could disrupt this system and hence disrupt emotion
understanding.
Consideration of the temporal dimension of such a model
may lead to a better understanding of social deficits, such as
those in autism spectrum disorders as well as those reported
in association with attention deficit disorder. Indeed, Gepnerand Feron (2009) describe a theory of temporal processing
deficits that may underlie a range of deficits in autism. Is
this theory at odds with the mirror neuron hypothesis (e.g.,
Williams et al. 2001), that a disruption of the motor neuron
response to the perception of movement in others underlies
social processing dysfunction in autism? Probably not. Bothmay be components of a larger emotion understanding systemthat involves multiple structures and their interactions intime. In support of this perspective, Oberman et al. (2009)
showed that individuals with autism do indeed show spontaneous mimicry of facial expressions, but that the response isdelayed.
Such models inform the development of interventions to
help people with social dysfunction. The Niedenthal et al.
model, together with theories such as the one proposed by
Gepner and Feron (2009), suggests that social processing
interventions should tap multiple processes, not individually,
but together and at the right temporal intervals. Recent technology for automatically recognizing and responding to facial
expression, head pose, and eye gaze in real-time opens up
new possibilities for intervention systems that link perception
and production on timescales related to social responding(Bartlett & Whitehill, in press; Cockburn et al. 2008). Such technology contributes not only to clinical research, but also to thestudy of learning and plasticity in perception and production
systems, and to understanding the cognitive neuroscience ofemotion.

“Smile down the phone”: Extending
the effects of smiles to vocal social
interactions
doi:10.1017/S0140525X10001469
Frédéric Bassoa and Olivier OullierbaUniversity of Rennes 1, CREM CNRS UMR 6211, IGR-IAE de Rennes 1,
35000 Rennes, France; bLaboratoire de Psychologie Cognitive (UMR 6146),
Université de Provence & CNRS, Aix-Marseille Université, 13331 Marseille
Cedex3, France.
frederic.basso@univ-rennes1.fr
olivier@oullier.fr
http://www.igr.univ-rennes1.fr/personnes/detail_fr_174.htm
http://www.oullier.fr

Abstract: The SIMS model offers an embodied perspective to cognition
and behaviour that can be applied to organizational studies. This model
enriches behavioural and brain research conducted by social scientists
on emotional work (also known as emotional labour) by including the
key role played by body-related aspects in interpersonal exchanges.
Nevertheless, one could also study a more vocal aspect to smiling as
illustrated by the development of “smile down the phone” strategies inorganizations. We propose to gather face-to-face and voice-to-voiceinteractions in an embodied perspective taking into account Lakoff andJohnson’s (1980) theory of conceptual metaphors.

Emotional work (also known as emotional labour) has been originally defined by Hochschild (1979, p.561) as “the act of trying tochange in degree or quality an emotion or a feeling.” It is noteworthy that research in that perspective started some thirty
years ago, studying smiles of flight attendants, in a face-to-facesocial setting. But, following Hochschild’s early suggestion, one

should also consider how the effects of smiles can be effective
during phone conversations, in a voice-to-voice setting when no
visual information is shared (e.g., Sutton 1991).
Thanks to a substantial amount of field studies, data along thatapproach became available to support the presence of emotional
work within organizations (Fineman 2000), whether for profit(e.g., between a sales representative and a customer; Ashforth& Humphrey 1993; Rafaeli 1989) or not (e.g., between a nurse
and a patient; Froggatt 1998). By focusing on interactions andbodily cues such as facial expressions, eye contact, posture, andgestures, the SIMS model encapsulates the different aspects(and roles) a smile can play in all those contexts.
Nevertheless, the visual modality is not the only one throughwhich smiles can be expressed. There is indeed a more vocal
aspect to smiling as illustrated by the development of “smile
down the phone” strategies in organizations (e.g., call centres).
Because of the absence of face-to-face interactions duringphone conversations, those “vocal smiles,” including the tone ofvoice, constitute one of the keys to understanding emotional
work in call centres (Belt et al. 2002; Taylor & Bain 1999).
These centres are the illustration of the marketing logic known
as Customer Relationship Management, aiming at developing
long-term relationships between companies and their customers(Gans et al. 2003).
This managerial framework clearly constitutes an expression ofemotional work centered around smiles down the phone tocreate empathy between the client and the sales representative(Richardson & Howcroft 2006). It appears that this vocal
aspect to smiling is not considered in Niedenthal et al.’s SIMS.
For instance, they focus on face-to-face interactions and do nottake into account, so far, how efficient smiles can be in voiceto-voice ones.
Previous studies on emotional work complement the perspective offered by SIMS, especially concerning affiliative smiles (seesects. 2.2 and 6.1.1 of the target article). Hence, we suggest gathering face-to-face and voice-to-voice interactions in an embodied
perspective – that is, still in opposition to an “amodal” view ofknowledge (Lakoff & Johnson 1999). In our view, Barsalou’s(1999) simulator would be enhanced by Lakoff and Johnson’s
theory of conceptual metaphors for it would ground SIMS innatural language – although this theory is not limited to thestudy of words.
Emotional work sheds new light on the metaphors used byactors in organizations. According to Froggatt (1998, p.332),
the metaphorical language employed by nurses (e.g., draining
and burden) reflects the emotional aspect of their work. Alongthe same line, Rees et al. (2007) argue for the existence of metaphors in describing the relationship between patients and physicians. Some therefore consider the emotional work of thephysician towards the patient as a “metaphoric framework ofclinical empathy” (Larson & Yao 2005, p. 1104). Here too, thechallenge of creating empathy with others can be satisfied byusing metaphors given that empathic processes at stake are at
the core of the emotional work of the physician (Larson & Yao2005).
Hence, metaphors are a way to access the emotional workof the actors in that the expression of emotions is metaphorical by essence. In line with Lakoff and Johnson (1980), Hochschild (2005, p. 344) stresses how metaphors “guide feeling,
and, of course, feelings also guide metaphors.” Similarly, whenaddressing emotional concepts, Lakoff (1987, p. 377) points
that: “When we act on our emotions, we act not only on thebasis of feeling but also on the basis of that understanding.
Emotional concepts are thus very clear examples of concepts
that are abstract and yet have an obvious basis in bodily
experience.”
Overall, it is noteworthy in this context that the three key
elements used in SIMS to decrypt what lies beneath smilingface-to-face (perceptual cues, experiential cues, and conceptual
knowledge; see sect. 6) can be found in voice-to-voice contexts.
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

435

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) modelIn our view, perceptual cues can be embedded in tone of voice(see, e.g., Morris & Keltner [2000] about aggressive behaviours
in negotiation) and experiential ones in the words employed(e.g., Rafaeli & Sutton 1990). Despite the absence of eye
contact, other components of emotional labour (greeting, thanking, and smiling; Sutton & Rafaeli 1988) are clearly observable invoice-to-voice interactions.
Conceptual metaphors being essential to express emotions(Kövecses 1990; 2008; Lakoff 1987; 1993), a conceptual knowledge of emotions is metaphorical. Of course, a conceptual metonymy such as FACE STANDS FOR FEELINGS (Yu 2008, p. 251; seealso, Lakoff 1993, pp. 34 – 35) illustrates the importance of faceto-face interactions in emotions. But smiles participate to thebodily experience of emotional concepts (e.g., smile and happiness, Lakoff & Johnson 1980, p. 18; see also, Peña Cervel 2001,
p. 258). To a larger extent, THE BODY IS A CONTAINER FOR THE
EMOTIONS (Lakoff 1987, p. 383).
Conceptually, the link between the metaphorical approach toemotions and emotional work becomes tighter when one considers the role of relationship metaphors in voice-to-voice interactions. Although one has to be careful not to conclude that theemotion and the human relationship domains can be confounded, human relations are conceptually close to that ofemotions (Kövecses 2008, p. 387).
Hence, regardless the modality (visual or vocal), the perception and the understanding of a smile remain embodied. Moreover, the emotional work perspective provides another way toidentify “whether a smile is true or false” (sect. 6.2).
For if the issue of emotional work is to make the sales representative’s smile sincere to satisfy the consumer (Grandey et al.
2005), this smile is not always perceived as such. The employee
is not necessarily experiencing the emotion conveyed by theexpressed smile (Pugh 2001) as it sometimes seems insincere(and therefore false; sect. 2.1) in the customer’s eyes. Over thephone, an insincere tone of voice can even lead to a “phonerage” of the customers, i.e., “people losing their temper on thetelephone” (see Deery & Kinnie 2002, p. 8, reporting a study
conducted in 1997 by Reed Employment Services on morethan 500 organizations in the United Kingdom).
In summary, emotional work permits one to introduce concepts like sentimental work (how people use their own emotions
to influence someone else’s emotions; Zapf 2002) and emotional
dissonance (how people experience the gap between emotions
that are felt and those which are displayed in a given situation;
Hochschild 1983, p. 90). Together with SIMS, emotional work
therefore leads to a more complete approach to smiling that
encapsulates the vocal modality.

Expressive smiles or leucosignals?
doi:10.1017/S0140525X10001627
Paul Bouissac
French Linguistics, University of Toronto, Toronto, Ontario M5S 1K7, Canada.
paul.bouissac@gmail.com

Abstract: The assumption that a complex and fuzzy notion like smile canbe the basis of a scientific, rather than semantic, inquiry can only lead toconfused and inconclusive results. It would be more productive to startwith the well-defined and measurable patterns of the clearly visible
contrasts that are produced on the human face by various muscular
contractions around the white patches formed by the sclera and theteeth. These features are universal, whereas a common word, inwhatever language, is necessarily ambiguous, culture-dependent, andhistorically rather than biologically determined.

To investigate facial signaling, the best strategy is to focus on thevisual features that are obviously adaptive in a species in which
territoriality and ranking are prominent. What is adaptive in

436

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

facial signaling is the correct reading of conspecifics’ intentions
at a safe distance. The signals must be robust if they are to beadaptive for both the emitter and the receiver. One cannot
afford to hesitate whether an approaching outsider or a groupmember is friend or foe. But jumping too fast to the defensivein the presence of ambiguity can be equally ill-adaptive in
species depending on cooperation which must have evolved
unambiguous peace-making signals. Natural selection has
favored signals based on the chromatic opposition: white versus
any color of the spectrum. White ensures optimal reflectanceeven in reduced luminosity; the color upon which white patterns
are produced by muscular contractions can vary indefinitely
under other evolutionary forces, from camouflage to ostentatious
handicap. Chemical and acoustic signaling is efficient in someenvironments, but, for open space species, chromatic signalingis the medium of choice, as it covers at the speed of light the distance required for fight or flight decisions. Tigers sport tuffs ofwhite hairs on the back of their ears, which they twitch to
produce white flashes toward approaching conspecifics to warn
them of their alertness to the situation. Similar contrasts are
exploited in the baring of teeth in canines and in the permanent
status display of the tusks in elephants. The rich muscular systemof the face in the primates makes it possible to vary the chromatic
patterns, thus generating a range of leucosignals (from Greek
leukos ¼ brilliant white).
The human face makes constant use of the contrasts between
the white patches of the sclera and the teeth, and the color of the
facial skin (Bouissac 2001; 2005). These signals are well-defined,measurable, and subject to experimental manipulations. Muscular contractions create typical configurations of highly visible
white patterns, which combine information pertaining to thegaze with information on the status of the jaws (relaxed vs.
tense or clenched). The foldings of the skin can be relevant indicators, but can be only visible at a much closer range than thesalient leucosignals and their well-defined morphologies. The
range of data processed by Niedenthal et al. presupposes close
range perception, while an inquiry into smiles as leucosignals is
congruent with interaction at greater distances. The primate
brain’s face detectors are fast processors that must lead to
instant decisions based on unambiguous signals (Tsao & Livingstone 2008).
It can be predicted, as Niedenthal et al. show, that leucosignals
involve the amygdala, since previous evidence indicates that its
activation correlates with processing social signals such as facial
fear-generating information, including threats linked to social
ranking (Buchanan et al. 2009, pp. 289– 318). Neurons in the
amygdala are indeed responsive to faces, particularly in a hierarchic group (Leonard et al. 1985; Whalen et al. 2009), in
which the white of the eyes and the teeth form distinct visual parameters (compare the ratio holding between sclera, iris, andeyelids in the fear face with the same ratio in the angry face).
In all social species, the most dangerous enemies are conspecifics. Other predators of humans can be more easily manipulated
or controlled, because their semiotic mismatch offers more
windows of opportunity once it is deciphered. But in thehuman primates, peace-making is equally vital. It could thus
also be predicted that the same neuronal systems would instantly
switch from high alert to pleasure arousal or, at least, be processed by contiguous neuronal systems. Indeed, nothing ismore threatening for a human than a new face on which intentions cannot be read. The production of friendly leucosignals
may suddenly defuse the tense state of fight readiness andtrigger the dopamine flow that stimulates the reward centers(Schultz 2007). All this is consistent with the data that are marshalled in Niedenthal et al.’s article without having to hypothetically construct the cognitive and emotional embodiment detailed
by Niedenthal (2007). This is not to deny that such embodiment
may occur as a secondary effect with some adaptive consequences, but simply to point out that it is too costly in time andenergy to be the primary source of vital decisions such as toCommentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
trust, fight, or flee. Leucosignals can provide the fast and frugal
information that triggers adaptive behavior without having tobe emotionally processed.
Building from the basis of leucosignals, a scientific inquiry of
the many facial configurations which are covered in English by
the term smile would avoid getting conceptually entangled
in the quagmire of semantic and literary interpretations,
whether embodied or not. Niedenthal et al. rightly point out
that Ekman (2001) suggests that there might be as many as
fifty kinds of smiles. The enigmatic or ironic smile (e.g., Mona
Lisa with leucosignals reduced to the very minimum), the socalled Duchesne smile, the seductive smile (e.g., Barack
Obama showing only his upper teeth), and the provocative triumphal smile in the face of adversity (e.g., Tony Blair showing both
upper and lower teeth), to name only a few examples from contemporary icons, are most likely governed by neuro-behavioral
systems that have evolved separately under a variety of selective
constraints, and have in common only the fact that they end up
on the display board of the face in the form of leucosignals
whose meanings depend on the raw social context in which
they occur. Whether they should all be labelled with the same
term is highly questionable. Some are obviously gestures aimed
at influencing conspecifics; others are probably mere leakagesof a dopamine flooded inner state; and still others are likely the
results of various interferences of any of these.
However, dealing with smiles from a purely biological evolutionary point of view definitely would miss the gene –culture
coevolution, which cannot have failed to impact this mostimportant tool of sociality. This latter dimension deserves more
scrutiny than the lip service paid by Niedenthal et al. to thecross-cultural investigation of smiles.

The role of embodied change in perceiving
and processing facial expressions of others
doi:10.1017/S0140525X10001639
Pablo Briñol,a Kenneth G. DeMarree,b and K. Rachelle
Smithba

Department of Psychology, Universidad Autónoma de Madrid, 28049 Madrid,
Spain; bDepartment of Psychology, Texas Tech University, Lubbock, TX
79409.
pablo.brinnol@uam.es
http://www.uam.es/otros/persuasion/
ken.demarree@ttu.edu
http://webpages.acs.ttu.edu/kdemarre/rachelle.smith@ttu.edu

Abstract: The embodied simulation of smiles involves motor activity that
often changes the perceivers’ own emotional experience (e.g., smiling canmake us feel happy). Although Niedenthal et al. mention this possibility,
the psychological processes by which embodiment changes emotions andtheir consequences for processing other emotions are not discussed in thetarget article’s review. We argue that understanding the processes
initiated by embodiment is important for a complete understanding of
the effects of embodiment on emotion perception.

People attempt to make sense of the facial expression of emotion
they see in others. In their review, Niedenthal et al. examine therole of motor simulations (e.g., changes in perceivers’ facial
expressions) in recognizing and processing the smiles of others.
According to authors’ insightful embodiment account, people
mentally simulate, and often mimic, the smiles they see.
Although this facial mimicry is not critical for smile recognition(distinguishing whether an expression is a smile or not), it
might be more important for making other judgments, such as
the type of smile (e.g., distinguishing between joy, affiliative,and dominance smiles), and the sincerity of the smile (distinguishing true from false smiles).
The embodied simulation of smiles not only involves motor
activity, but often changes the perceivers’ own emotional (posi-tive) experience. Although Niedenthal et al. mention this possibility, the psychological processes by which embodimentchanges emotions and the consequences for processing otheremotions are not discussed in the target article. We argue that
understanding the processes initiated by embodiment is important for a complete understanding of the effects of embodimenton emotion perception. For example, smiling often makes us feelhappier and can increase our liking of everything around us. In
accord with multi-process theories of judgment, such as the elaboration likelihood model (Petty & Cacioppo 1986), smiling andother positive behaviors, such as head nodding, can influence
judgments and emotions by serving as a simple positive cue, orby affecting either the amount (less thought) or direction(biased positive thoughts) of thinking (for a review, see Briñol& Petty 2008). If smiling reduces thinking, then embodied
mimicry could influence subsequent judgments by reducing
the discrimination between true and false smiles or between
different types of smiles. Mimicking a smile might also facilitate
the processing of other smiles when it make us feel happier by
biasing our thoughts or serving as a positive cue, whereas thevery same action might increase the difficulty in processing
other, subsequent smiling faces when it operates by reducing
our amount of thinking.
As these examples illustrate, understanding the processes
underlying embodied perception and change are essential to
fully understand whether, when, and how individuals process
emotional information. Recently, we have proposed that embodiment can not only influence what people think, but can also
impact what people think about their own thoughts (i.e., a metacognitive process called self-validation; Petty et al. 2002). Metacognition refers to thoughts about thoughts (see Briñol &DeMarree, in press, for a review). The main idea behind theconcept of embodied validation is that people’s own behaviors
can impact their judgments by affecting thought confidence.
The confidence that emerges from behavior can magnify theeffect of any available mental contents (Briñol & Petty 2003).
In a recent study of embodied validation, Briñol et al. (2009)
asked participants to think about and write down their best or
worse qualities while they were sitting with their back erect
and while pushing their chest out (confident posture) or slouchedforward with their back curved (doubt posture). Then participants reported their self-esteem. In line with the self-validation
hypothesis, thoughts generated about the self only affected
self-attitudes in the relatively high confidence posture. Conceptually similar, DeMarree et al. (2010) conducted research inwhich participants were presented with different facial
expressions of emotion on a computer screen after generating
positive or negative thoughts. Consistent with the notion thatmerely perceiving facial expressions of emotion can influence
thought-confidence, people relied on their thoughts more when
exposed to facial expressions depicting emotions associatedwith confidence (e.g., happiness, anger) than when exposed tofacial expressions depicting emotions associated with doubt(e.g., sadness, surprise). Importantly, this research suggests thatsmiling can lead to negative feelings when it increases the confidence in previously generated negative thoughts. If smiling validates negative thoughts making people feel bad, it might produce
a number of different consequences for processing subsequent
smiles (e.g., introducing a negative bias, increasing attention
and therefore discrimination between different types of smiles).
Furthermore, this research suggests that the confidence construal associated with smiles can be important, because happy
and angry emotions produce similar outcomes by validating
thoughts. Importantly, although the types of smiles discussed
by Niedenthal and colleagues all have in common the presenceof a positive valence, they might differ in their associated levelof confidence, and these variations in confidence can also moderate some of the results described in the review. For example,
smiles associated with more confidence (e.g., dominancesmiles) might be perceived more easily than less confident
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

437

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
smiles (e.g., affiliative smiles) because of the different amount ofconfidence created in the perceiver by mimicking these smiles.
Finally, consider research on behavioral mimicry beyond
smiles, in which one person matches another’s behavior (e.g.,
Chartrand & Bargh 1999). Smith et al. (2010) have found thatparticipants act more confident when they are mimicked byothers. The thought-confidence that emerges from this mimicry
could potentially validate positive or negative thoughts. As a consequence of this process, mimicked individuals might produce
more or less smiles, and process other smiles more or less.
Taken together, these lines of research suggest that both agentsof any interaction can mimic the smiles they see, and also be
mimicked simultaneously, and dynamically.
Along with the target article, this commentary has focused on
cases when people imitate what they observe in others. Alternatively, however, people sometimes respond to others’ behavior incontrasting, complementary ways (e.g., dominance-submission).
Ideally, moderators of imitation versus complementarity should
be discussed with regard to smiling.
In sum, although the ability of our bodily actions to influence
our mind seems to be a well-established phenomenon, mostresearch on this topic has not focused on the psychological mechanisms by which the body affects attitudes and emotions. We
argue that understanding these processes is essential formodels of embodiment.

Beyond smiles: The impact of culture andrace in embodying and decoding facial
expressions
doi:10.1017/S0140525X10001470
Roberto Caldara
Centre for Cognitive Neuroimaging (CCNi), Institute of Neuroscience andPsychology, University of Glasgow, Glasgow, G12 8QB, United Kingdom; andDépartement de Psychologie, Université de Fribourg, 1700 Fribourg,
Switzerland.
r.caldara@psy.gla.ac.uk
http://www.psy.gla.ac.uk/∼ roberto/

Abstract: Understanding the very nature of the smile with an integrative
approach and a novel model is a fertile ground for a new theoretical visionand insights. However, from this perspective, I challenge the authors tointegrate culture and race in their model, because both factors would
impact upon the embodying and decoding of facial expressions.

The central idea of this stimulating target article is that embodied
simulation represents a critical feature to effectively decode thenature of a smile. I endorse the argument and also agree with
Niedenthal et al. that the integrative approach they propose istimely and necessary to understand the decoding of smiles (andfacial expressions). The model posited here by the authors summarizes their view, offering a novel vision and prompting manyquestions for future research. Niedenthal et al. state that:
The SIMS model has been largely developed using data collected inWestern countries. Nevertheless, it is essential to note that culturaldifferences may modulate our account. Some clear predictions acrosscultures can be imagined and have been articulated elsewhere. (sect.
6, para. 3)

The authors cite their ongoing work in this regard (Niedenthal
et al., in preparation). However, I think it is necessary that theyclarify their view regarding the impact of culture and race fromtheir theoretical position in the target article.
Human populations are marked by a variety of practices,beliefs, norms, and forms of organization. The term “culture” istypically used to describe the particular behaviors and beliefs
that characterize a social or ethnic group, often located in aparticular geographical location. Thus, by definition, culture

438

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

represents a powerful deterministic and invisible force, whichis responsible for shaping the way people think and behave.
Importantly, culture also shapes the way people express theiremotion. Westerners live in individualistic societies and tend to
express their emotions explicitly. By contrast, Easterners live incollectivist societies and tend to suppress their emotions to maintain harmony amongst the group (Markus & Kitayama 1991).
More importantly, it has recently been shown that culture
impacts not only upon visual perception, but also, critically,
upon the extraction of information from faces.
My colleagues and I (Blais et al. 2008) have recently shownthat culture shapes how people deploy eye movements tosample information from the facial input space. Western Caucasian adults employ a well-established triangular fixation patternduring face learning, recognition, and categorization by race
(Blais et al. 2008). Contrary to intuition, East Asians direct fixations to the central area of the face, around the nose, for thevery same visual categorization tasks, while reaching a comparable behavioral performance. The eye movement strategy
deployed by the East Asian observers might not straightforwardly
relate to gaze avoidance, since this cultural contrast in eye movement generalizes to other visually homogenous categories (Kelly
et al. 2010) and is abolished in constrained viewing conditions(Caldara et al. 2010). However, these observations do not ruleout the possibility that the eye movement strategy used by Easterners to process visually homogenous objects, might arise fromfacing a continuous pressure from the Eastern cultural norm promoting gaze avoidance during human face-to-face interactions(Knapp & Hall 2005).
In stark contrast with those previous findings in face recognition, East Asian observers, compared to Western Caucasianobservers, oversample information from the eye region during
the categorization of facial expressions (Jack et al. 2009), ignoring
information from the mouth region, even for the “happy”
expression. Besides showing that the eye movement strategies
deployed to decode facial expressions are culturally specific,
our data also suggest that the transmission of facial expressions
is not universal. Easterners focus uniquely on the eye region to
decode signals, as they might expect expressive diagnostic
signals coming from this facial region. Indirect evidence forthis position is provided by the emoticons. While Westerners
use a change in the mouth to convey a change from “happy” to“sad” with emoticons (i.e., :-) vs. :-( ), Easterners emphasize information from the eye region (i.e., ^_^ vs.; ^; ), with a neutral
mouth. Critically, this perceptual bias towards the eyes for the
Easterners generates confusions in facial expression decoding,particularly for the expressions of surprise, fear, anger, anddisgust. Therefore, our eye movement data on face recognition
showing the lack of direct gaze contact in Easterners, coupledwith those on expressions showing that face expressive signals
are culturally tuned, point to culturally specific decoding errors
and perhaps the necessity of a culturally specific SIMS model.
The confusion shown by East Asian observers in decoding particular facial expressions might arise from a lack of embodimentof Westerner expression signals, offering a novel interpretation to
the cross-cultural impairment in facial expression decoding.
An interesting side note on this point: East Asian observers
should potentially be better than Westerners in categorizing
fake and genuine smiles, as they concentrate (and perhaps
develop a particular expertise for) their fixations on the eyes only.
Human populations are also marked by physiognomic variations. Race is a universal, socially constructed concept used to
rapidly categorize humans originating from different geographical locations by salient physiognomic variations (i.e., skin tone,eye shape, etc.). Humans are markedly better at recognizing
same- compared to other-race faces. Our studies advocate theexistence of finely tuned mechanisms to process same-race(SR) faces, probably developed as a by-product of visual experience (e.g., Caldara & Abdi 2006; Michel et al. 2006a; 2006b),
which drive the rapid categorization of other-race faces (e.g.,Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) modelCaldara et al. 2003; 2004; Vizioli et al. 2010). But more interestingly for the present framework, recent evidence suggests thatsuch ingroup/outgroup categorization also has an impact inmany social cognition judgments. It has been demonstrated
that the neural responses to empathy of pain are modulated
and interact with ingroup/outgroup membership (Xu et al.
2009). As a consequence, one would once again predict thatthe race of the observers, coupled with culture, would impact
upon the embodying and decoding of facial expressions andfavor the existence of specifically tuned ingroup SIMS models.
Regardless of the precise role of culture and race in thenovel model put forward by Niedenthal et al., this target
article makes a significant contribution in the field andreminds us that integrative approaches are necessary toachieve an understanding of complex human social behaviors
and interactions.
ACKNOWLEDGMENT
The author is supported by The Economic and Social Research Council
and Medical Research Council (ESRC/RES-060-25-0010).

Honest smiles as a costly signal in social
exchange
doi:10.1017/S0140525X10001287
Samuele Centorrino,a Elodie Djemai,a Astrid Hopfensitz,aManfred Milinski,b and Paul SeabrightaaToulouse School of Economics, 31000 Toulouse, France; bMax Planck
Institute for Evolutionary Biology, 24306 Plön, Germany.
samuele.centorrino@hotmail.it
elodiedjemai@gmail.com
Astrid.Hopfensitz@tse-fr.eu
milinski@evolbio.mpg.de
Paul.Seabright@tse-fr.eu

Abstract: Smiling can be interpreted as a costly signal of future benefits
from cooperation between the individual smiling and the individual towhom the smile is directed. The target article by Niedenthal et al. gives
little attention to the possible mechanisms by which smiling may have
evolved. In our view, there are strong reasons to think that smiling has
the key characteristics of a costly signal.

When considering the role of smiles in social interactions, it isimportant to consider situations involving incomplete information about our partner in the interaction. In many social
interactions uncertainty typically exists about the extent ofcooperation to be expected from the interaction, and there may
also be uncertainty about the stakes of the interaction and thepossible choices available to the partner. A self-interested individual will never trust her partner if she believes that thepartner will act selfishly in a situation that calls for cooperation.
Meanwhile, mutual trust among a group of individuals might
be socially beneficial, a situation typical for public goods. Given
that trust and cooperation are widely observed in humans andother animals, we have to ask which signals might be used to
identify trustworthy partners. Smiling may be just such a signal.
The evolution of such signals is not easy to explain. If trustworthy individuals have observable characteristics that distinguish them from non-trustworthy individuals, these signals
must have an inherent cost that make them impossible or difficult
to fake. Because a non-trustworthy individual would profit frominteracting with a trustworthy individual, he has a large incentive
to fake the signal. As has been known since the work of Zahavi(1975) in biology and Spence (1973) in economics, a signal canremain reliable in the face of incentives for mimicry only if its
inherent cost is larger than this potential gain to the mimic –but it must still be a cost worth bearing for a genuine bearer ofthe signaled trait. Since trustworthiness might depend on thesituation at hand, it seems unlikely that a fixed physiologicalcharacteristic would be an appropriate signal, but it is likelythat behaviors revealing internal emotional and psychologicalstates could be effective. Smiles are likely to be a signal identifying possibly valuable interaction partners via their correlation
with psychological states that reliably indicate trustworthiness.
The information content of the signal might concern either personal characteristics of the partner (the degree of altruism orstrong reciprocity) or unobservable characteristics of the situation (possible gains to be made or the choice options available
to the partner). There is a literature on smiling as a signaling
device that has traditionally focused mainly on the former, butboth might be equally important and influential.
The idea that emotions can serve as commitment and signaling
devices has been developed in economics since Frank (1988) and
Hirshleifer (1987). In particular, negative emotions can trigger
costly punishment behavior of non-cooperative actions (Fehr &Gächter 2000). Smiling has been identified as a signal interpreted
as representing cooperative tendencies (Krumhuber et al. 2007).
When participants in an economic trust game are presented withpictures showing their partner either smiling or with a neutral
face, they are more likely to trust the smiling partner. Whether
this is an adaptive strategy is a separate question. In particular,
the reliability of the picture as a guide to the partner’s behavior
might depend on when the picture was taken. Photos taken independently from the decision situation seem to contain little information, while photos taken when the partner was in the processof the relevant decision seem to contain nonverbal cues that allow
subjects to deduce the other players’ strategy. Indeed, it is not
smiles as such, but rather smiles that are perceived as “honest”
that are conducive to trust. “Genuine” or Duchenne smiles arereportedly difficult to fake and most easy to produce when the
person is in a positive and sharing mood (Mehu et al. 2007).
Indeed, lying is known to induce a significant level of cognitive
load on the liar, which makes it difficult to react spontaneously,
and which reduces subtle movement (Vrij et al. 2006). Honest
smiling, which involves a contraction of the orbicularis oculi,
further has the obvious cost of reducing the smiler’s visual
field. Doing so might have fitness costs for the smiler, and thus
should be done only sparingly, which can explain why smilingconvincingly is cognitively costly. Genuine smiles can therefore
be taken as an honest signal of the partner being in a positive
mood, one that makes deception of the interaction partner
less likely. In a situation concerning trust, this might meanthat the trustee has already made a trustworthy decision, orthat he is aware that he has something interesting to offer tothe partner.
We can relate this idea to the smile types identified by Niedenthal et al. “Enjoyment smiles” are smiles elicited by ageneral positive atmosphere, and which are seen as honest
signals that the partner feels comfortable in the current situation.
“Affiliative smiles” are smiles that are used as a signal to thepartner. Niedenthal et al. claim that affiliative smiles will be
initially considered as “true smiles” (section 6.2) and as “false”
when: “the perceiver believes that the smile was motivated by adesire to hide, moderate, or justify something negative” (section
6.2, para. 1, emphasis ours). We believe that it will depend
partly on the specific situation of the smiler whether the smile
is perceived as false or not. Whether an affiliative smile has
characteristics that lead it to be judged true or false is therefore
influenced not only by the situational knowledge of the observer,
but also by unobservable characteristics of the smiler. Therefore,
the smile can be seen as a valuable signal of this unobservable
information.
Indeed, we have observed in a recent experimental study(Centorrino et al. 2010) that honest smiles lead to higher levelsof trust, and that a smile is rated as more convincing when thetrustee has larger stakes at his disposition. Therefore, honest
smiles can be seen as a costly signal that can only be produced
when in a situation potentially beneficial to the observer of thesmile, and can thus serve as a signal valuable to the observer
and to the smiler alike.
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

439

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model

Eyes, amygdala, and other models of faceprocessing: Questions for the SIMS model
doi:10.1017/S0140525X10001482
Bhismadev Chakrabarti
School of Psychology and Clinical Language Sciences, University of Reading,Whiteknights, Reading RG 6 6AL, United Kingdom.
b.chakrabarti@reading.ac.uk
http://sites.google.com/site/bhisma

Abstract: This commentary raises general questions about the parsimony
and generalizability of the SIMS model, before interrogating the specificroles that the amygdala and eye contact play in it. Additionally, thissituates the SIMS model alongside another model of facial expression
processing, with a view to incorporating individual differences inemotion perception.

The target article proposes a new model for processing smiles, asa specific case within the broader domain of facial expressions ofemotion. Niedenthal et al. suggest that three types of smiles –dominance, affliative, and enjoyment – are distinguished fromone another using an embodied simulation (ES) based framework. ES, according to the authors, can be driven both by eye
contact as well as by activation of conceptual stereotypes ormemory. This work extends current simulation-theory basedmodels of facial expression perception (Goldman & Sripada
2005), by suggesting a key role for eye contact in triggering ES.
This is a novel suggestion, and one that will be examined laterin this commentary.
In order to understand the SIMS model, it is important to firstevaluate the exact nature of the differences between the threekinds of smiles. The authors point out that Duchenne markers
are not sufficient to discriminate between these three smiles,
and it is unclear whether there are physical differences in facial
musculature among these expressions (e.g., as coded for by
facial action units; Ekman & Friesen 1978). It is also not clear
if this has been tested empirically, using a standard emotion recognition paradigm, where facial expressions are presented withand without social context. If it is not possible to discriminate
between the three kinds of smiles based purely on their physicalproperties (e.g., facial musculature and dynamics), then the
differences might lie in processing of other variables such as
the context (Aviezer et al. 2008), or the conceptual stereotypes
involved (e.g., race, ethnicity; Hess et al. 2000).
The three types of smiles that the authors suggest can be
regarded as examples of smiles in three among many possible
contexts. Indeed, Bharata’s treatise on the dramatic arts (Natyasastra, c. 200 A.D.) discusses at least six different kinds of smiles

and more than eight different elicitors for them (Vatsyayan 1996).
More recent work suggests at least 30 different emotion states tobe linked with smiles, at varying levels of social complexity(Golan & Baron-Cohen 2006). In view of this, one could thinkof emotion recognition as a two-component process, consistingof (a) perception of the physical properties of the face, and (b)processing of the contextual information (including conceptual
stereotypes), which together lead to recognizing an emotion
expression. These components might be processed in parallel.
The question for the proposed model in its current form, therefore, is: Does it offer the most parsimonious explanation for thistwo-component process? Or would this model require additional“modules” in order to explain processing of expressions otherthan smiles, or smiles of a type that has not been explicitly discussed in the model (e.g., a mischievous or a derisive smile)?
Keeping questions about the parsimony of the model to one
side, the current form of the model raises two questions about
the proposed role of eye contact. The first is anatomical: Is
there any support for the proposition that the amygdala promotes
eye contact? While it is known that eye contact often results in
amygdala activation in the observer, the evidence in support ofthe suggested causal link in the opposite direction appears
limited. Hooker et al. (2003) reported higher amygdala activation
for trials with no direct gaze compared to those with a 40 percent
probability of a direct gaze. It is not clear if this reflected a generalised increase in arousal for monitoring any salient event, or
whether it was specific to monitoring eye contact. Amygdala
response in people with Autism Spectrum Conditions (ASC)was found to correlate positively with fixation on the eyesregion of the stimuli, but this is possibly due to social anxiety inpeople with ASC, experienced particularly while looking at someone’s eyes. Moreover, this relation between gaze fixation timeand amygdala activation was not found for the controls in thesame study (Dalton et al. 2005). In summary, it is not clear
whether the proposed model predicts higher or lower amygdala
activation in response to eye contact, and how that relates tothe extent of ES observed.
The second question asks: How crucial is eye contact for ES tooccur, and is its role emotion-specific? ES has been shown tooccur while viewing body parts only (e.g., studies on perceptionof pain [Singer et al. 2004] and touch [Keysers et al. 2004]) in theabsence of any eye contact. This raises the question of whether
the role of eye contact in the proposed model is anything morethan a device to capture spatial attention. Indeed, it has beenshown that attention can influence ES during action perception(Bach et al. 2007; Chong et al. 2008). It is reasonable to hypothesise that the same mechanisms for ES operate during

Figure 1 (Chakrabarti). A generalized model of facial expression (Chakrabarti et al. 2006). Left of the dotted line indicates systems
involved in perception of all emotion expressions. Right of the dotted line indicates structures that may selectively respond to specificemotions. IOG ¼ Inferior Occipital Gyrus, FG ¼ Fusiform Gyrus, STS ¼ Superior Temporal Sulcus, IFG ¼ Inferior Frontal Gyrus,
PM ¼ Premotor Cortex, IPL ¼ Inferior Parietal Lobule.

440

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
perception of any body part or the face, and that the eyes function as a spatial orienting signal to the face. Arguably, themouth region might be more informative for other facial
expressions (e.g., disgust), in which case attending to themouth should provoke a greater degree of ES. This hypothesis
could be tested using Wicker et al.’s (2003) elegant paradigm
by directing attention to the mouth or the eyes of the facial
stimuli expressing disgust, and measuring the observer’s insula
response in the two conditions.
Finally, the SIMS model could be extended to other emotions
by situating it alongside other models of facial expression processing, derived from behavioural and neuroimaging evidence(Bruce & Young 1986; Chakrabarti et al. 2006; Haxby et al.
2000). Although these earlier models explicitly deal with processing of the physical properties of the facial expression, they do not
postulate a crucial role for eye contact to trigger ES. Additionally,
they do not include a module for contextual processing, as is
implicit in the SIMS model (e.g., the orbitofrontal contribution
to understanding affiliative smiles). The key element in the Chakrabarti et al. (2006) model is an “action perception” (AP) module,
situated between structures that process invariant features of aface (including the fusiform gyrus and lateral occipital cortex)
and ones involved in the experience of different emotions (e.g.,
limbic structures). (See Fig. 1.)
Regions involved in the AP module have sometimes beenreferred to as a “mirror system” (Keysers & Perrett 2004),
and overlap largely with the structures involved in ES in the
SIMS model. It would be particularly interesting to explore
the predictions of the SIMS model on individual differencesin empathy, a trait that correlates with emotion recognition
abilities (Lawrence et al. 2004). Arguably, if processing ofsocial context and conceptual stereotypes depends on the prefrontal cortex, as the SIMS model suggests, prefrontal activity
should correlate positively with the cognitive component oftrait empathy. In parallel, one might speculate that the emotional/affective component of empathy will correlate more with theneural substrates underlying ES (similar to Chakrabarti et al.
2006).

Re-thinking the causes, processes, andconsequences of simulation
doi:10.1017/S0140525X10001299
Betty Changa and Nicolas Vermeulena,ba

Université Catholique de Louvain (UCL), 1348 Louvain-la-Neuve, Belgium;
National Fund for Scientiﬁc Research (FRS-FNRS), 1000 Brussels, Belgium.
Betty.Chang@uclouvain.be
http://www.uclouvain.be/en-277800.html
Nicolas.Vermeulen@uclouvain.be
http://www.ecsa.ucl.ac.be/personnel/vermeulen/b

Abstract: We argue that the meaning of smiles is interpreted fromphysical/contextual cues, and simulation may simply reinforce theinformation derived from these cues. We suggest that, contrary to theclaim of the SIMS model, positive and negative smiles may invoke
similar simulation processes. Finally, we provide alternative explanations
for the role of eye contact in the processing of smiles.

According to the Simulation of Smiles (SIMS) model, simulation
allows one to discriminate between different types of smiles.
Some types of smiles (such as “true” and “false” smiles) areassociated with particular physical characteristics (Cacioppo
et al. 1986; Ekman et al. 1980 – reviewed in Frank 2002; seealso, Hess & Kleck 1990). Other types of smiles (e.g., affiliativevs. enjoyment smiles) cannot be discriminated from each otheron the basis of physical markers. In this case, the meaning of asmile is derived from semantic/contextual information.

Because smile types can be identified based on either physicalor contextual cues, this suggests that any information provided
by simulation simply serves to enhance the identification of thesmile’s meaning (e.g., by speeding up the recognition process),rather than being critical in recognizing the smile.
One study which suggests that simulation may be critical inrecognizing smiles was conducted by Maringer et al. (in press;
cited in the target article). This study showed that mimicry canfacilitate the discrimination of true from false dynamic smiles.
However, this advantage may be due to the fact that mimicry provides an additional representation of a smile stimulus, whichallows participants who mimic to receive more input about asmile than participants who do not mimic. Future experiments
can control for the number of stimulus representations
between the mimicry and no-mimicry conditions by adopting ayoked design. This would involve videotaping the faces of participants in the mimicry condition when they view a sequence of facestimuli, and playing these recordings to yoked participants in the
no-mimicry condition while they view the same stimulus
sequence. In this way, any differences between the two conditions can be attributed to qualitative differences in stimulus
processing, rather than to differences in the number of stimulus
representations received.
According to the SIMS model, smiles elicited when viewing
positive (i.e., enjoyment) smiles indicate mimicry. In this case,
it is the current expression and feeling of the expresser that issimulated. Niedenthal et al. argue that, by contrast, people areless likely to mimic negative (e.g., dominant) smiles, butinstead, they simulate the feeling of being dominated. In thiscase, it is the previous experience of the perceiver that is being
simulated. It is not clear why the different smiles should invoke
simulations with reference to different people- and time- perspectives. Instead, it would be more parsimonious to claim thatthe same mechanism (i.e., mimicry) occurs for both types ofsmiles, at least when the smiles are presented in the samecontext, such as when viewing smile stimuli in a laboratory
setting. In support of the idea that dominant smiles may be
mimicked, studies show that people spontaneously mimic
another dominant expression, anger, when viewing pictures ofangry faces (e.g., Cannon et al. 2009; Dimberg et al. 2000).
The question of whether mimicry occurs similarly for both positive and negative smiles may be answered by analyzing the resultsof Maringer et al. (in press) in more detail. If Niedenthal et al.
consider that people should inhibit mimicry for false smiles as
they would for dominant smiles (because, to extend theauthors’ reasoning, it is not socially appropriate to mimic false
smiles), then people should not mimic false smiles in the nomimicry condition of Maringer et al.’s study, but they should
nevertheless be able to simulate reactions that they have previously felt in response to false smiles. As such, participants inthe mimicry and no-mimicry conditions should recognize false
smiles with equal accuracy. However, if participants spontaneously mimic false smiles, then false smiles should be recognized more accurately in the mimicry than in the no-mimicry
condition.
Niedenthal et al. claim that eye contact increases emotional
intensity in response to angry and happy faces (Schrammel
et al. 2009) because it triggers the simulation of expressions
and their associated emotions. However, the relation between
eye contact and emotional intensity may be explained by othermechanisms. Eye contact may enhance the emotional experienceof the perceiver because it implies that the perceiver is the targetof the expression, which in turn increases the relevance of theexpressed emotion for the perceiver (e.g., Cary 1978). Alternatively, emotional intensity may increase under eye contact
because angry and happy faces both signal an approach motivation, which is congruent with the approach motivation signaled
by a direct gaze, but incongruent with the avoidance motivation
signaled by an averted gaze (see Argyle & Cook 1976; Davidson& Hugdahl 1995; Harmon-Jones & Segilman 2001). Thus, happy
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

441

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
faces with an averted gaze may be considered to be more ambiguous than happy faces with a direct gaze, causing them to elicit
less emotion, or ambivalent emotion. In support of this, studies
show that when the approach/avoidance motivation of the gaze
direction matches the approach/avoidance motivation of theemotion expression (e.g., when happy and angry faces are presented with a direct gaze, and sad and fearful faces are presented
with an averted gaze), the emotion is identified more quickly(Adams & Kleck 2003; 2005), and the perceived emotional intensity of the expression also increases (Adams & Kleck 2005). These
results with happy and angry faces are consistent with the SIMS
model. However, the model assumes that similar processes
should underlie the simulation of all types of expressions, andthus it cannot account for why sad and fearful faces should be
identified more quickly, and considered to be more emotionallyintense, when presented with averted rather than with direct
gazes.
The SIMS model represents an ambitious integration ofsocial, cognitive, and neuro-psychology. However, one aspectthat needs to be clarified is whether simulation operates similarly for different types of smiles, such as true versus false
smiles, and smiles associated with physical markers versus
more ambiguous smiles. It also remains to be demonstrated
whether simulation is indeed critical for the interpretation ofsmiles, or whether physical and contextual cues play a greaterrole in this process. Nevertheless, the model raises many testable
hypotheses, which provide a fruitful starting point for exploring
the contribution of simulation to the processing of facial
expressions.
ACKNOWLEDGMENT
The writing of this commentary was facilitated by a grant from the “Action
de Recherche Concertée”; No. ARC06/11-337.

Eye gaze and conscious processing inseverely brain-injured patients
doi:10.1017/S0140525X10001494
Camille Chatelle,a Steven Laureys,a Steve Majerus,b andCaroline SchnakersaaComa Science Group, Cyclotron Research Centre, University of Liège, 4000
Liège, Belgium; bDepartment of Cognitive Psychopathology, University ofLiège, 4000 Liège, Belgium.
camille.chatelle@ulg.ac.be
steven.laureys@ulg.ac.besmajerus@ulg.ac.be
C.Schnakers@ulg.ac.be
http://www.ulg.ac.bewww.coma.ulg.ac.be

Abstract: Niedenthal et al. discuss the importance of eye gaze inembodied simulation and, more globally, in the processing of emotional
visual stimulation (such as facial expression). In this commentary,
we illustrate the relationship between oriented eye movements,
consciousness, and emotion by using the case of severely brain-injured
patients recovering from coma (i.e., vegetative and minimally conscious
patients).

The vegetative state (VS) is characterized by a preserved arousal
level but also by the absence of any sign of consciousness ofthe environment and of oneself (e.g., absence of oriented
responses to environmental stimulation) (The Multi-Society
Task Force on PVS 1994). The patient in a minimally conscious
state (MCS), on the contrary, demonstrates inconsistent, basic
but reproducible signs of consciousness of the environmentand of oneself (e.g., oriented responses to environmental stimulation suggesting an interaction between the patient and his/her
surroundings) (Giacino et al. 2002). Both populations may beparticularly useful for studying the concept of consciousness via

442

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

the progressive recovery of the most primary aspects of consciousness, and the behavioral and cerebral correlates that
accompany them.
Niedenthal et al. discuss the importance of eye gaze in embodied simulation and, more globally, in the processing of emotional visual stimulation (such as facial expression). Eye gaze involves
interaction between the subject and his/her surroundings. At aneurobehavioral level, oriented eye movements are usuallyassociated with conscious processing (Giacino et al. 2002).
MCS patients are able to track a person moving in front ofthem, whereas VS patients are not. Only MCS patients cantherefore detect emotional visual stimuli present in theirsurroundings. According to a recent study, visual pursuit ismore frequently detected in response to emotional stimulation(e.g., the appearance of one’s face in a mirror) than to nonemotional stimulation (e.g., a person moving) (Vanhaudenhuyse
et al. 2008).
MCS patients also show more complex emotional responses
than VS patients. For instance, VS patients may smile or
grimace, shed tears, or make grunting or groaning sounds, but
for no discernible reason (The Multi-Society Task Force on
PVS 1994; Working Party of the Royal College of Physicians2003), similarly to what can be observed in anencephalicinfants lacking a functional neocortex (Massimelli 2007). MCS
patients can manifest appropriate smiling or crying in responseto specific linguistic or visual content of emotional stimuli,
suggesting an appropriate interaction between the patient andhis/her environment (Giacino et al. 2002). Here, consciousness
is reflected by the presence of basic but contingent social andemotional behaviors, those requiring a complex cortical activity.
Using auditory (Laureys et al. 2000) and noxious stimuli (Boylet al. 2008; Laureys et al. 2002), previous studies in functional
neuroimaging have shown that, while brain activation isolated
to primary cortices is observed in VS patients, an activation inassociative areas as well as preserved functional connectivity
between cortico-cortical areas (i.e., between primary and associative cortices) is observed in MCS patients (see our Fig. 1). MCS
patients seem, therefore, to present a more complex social andemotional pattern at a behavioral level and a more integrative
brain processing than VS patients.
Even if oriented eye movements, such as visual pursuit, areusually considered a sign of consciousness, controversies
remain concerning visual fixation. Visual fixation includes eye
gaze and can be described as the eyes changing from theirinitial fixation point and refixating on the new target location(Giacino et al. 2004). As eye gaze involves interaction between
the subject and his/her surroundings, visual fixation would
have to be considered as a sign of consciousness. However,whereas the Aspen Workgoup considers this behavior as a signof consciousness (Giacino et al. 2002), the College of Physiciansconsiders its presence as reflecting a reflexive activity (Working
Party of the Royal College of Physicians 2003). Using active paradigms (i.e., the patients were asked to mentally perform a task),
voluntary brain activity has been obtained in patients showing
mainly low-level behavioral responses such as visual fixation,suggesting it may be a sign of consciousness (Owen et al. 2006;
Schnakers et al. 2008). However, we have recently shown nodifferences in brain activity and connectivity in VS patients presenting or not presenting visual fixation, suggesting that it does
not necessarily reflect consciousness and higher-order cortical
functioning (Bruno et al. 2010). According to these results, it is
not clear whether visual fixation reflects conscious interaction
between the patient and his/her surroundings.
In conclusion, consciousness level is associated with social andemotional behaviors of different complexity. Conscious patients(i.e., MCS) show high-level behavioral responses and brain activation as compared to non-conscious patients (i.e., VS). Nevertheless, the particular case of visual fixation would have to befurther investigated in order to better define its relationship to
consciousness.

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 1 (Chatelle et al.). Cerebral activation to noxious stimulation. In the darker brain regions (which appear in red in the online
version, available at www.journals.cambridge.org/bbs) that activated more during noxious stimulation in healthy controls (top), in
vegetative state (bottom, left side) and in minimally conscious state (bottom, right side) as compared to rest (adapted from Bolyet al. 2008 and Laureys et al. 2002).
ACKNOWLEDGMENTS
Our research is funded by the Belgian Fund for Scientific Research(FNRS), European Commission, James McDonnell Foundation, Mind
Science Foundation, French Speaking Community Concerted Research
Action (ARC-06/11-340), Fondation Médicale Reine Elisabeth,
University of Liège, and the CNRS/FNRS-CGRI collaboration funds.

How does perceiving eye direction modulateemotion recognition?
doi:10.1017/S0140525X10001305
Laurence Conty,a Julie Grèzes,a and David Sanderb,caCognitive Neuroscience Laboratory, INSERM U960, Ecole Normale
Supérieure, 75005 Paris, France; bLaboratory for the Study of Emotion
Elicitation and Expression (E3 Lab), Department of Psychology, Swiss Center
for Affective Sciences, University of Geneva, CH-1205 Geneva, Switzerland;cCentre Interfacultaire en Sciences Affectives (CISA) – University of Geneva,
CH-1205 Geneva, Switzerland.
laurence.conty@ens.fr
http://www.grezes.ens.fr/people.php?id¼2
julie.grezes@ens.fr
http://www.grezes.ens.fr/people.php?id¼1
david.sander@unige.ch
http://cms.unige.ch/fapse/EmotionLab/Sander.html

Abstract: Niedenthal et al. postulate that eye contact with the expresserof an emotion automatically initiates embodied simulation. Ourcommentary explores the generality of such an eye contact effect foremotions other than happiness. Based on the appraisal theory of
emotion, we propose that embodied simulation may be reinforced bymutual or averted gaze as a function of emotional context.

We congratulate Niedenthal et al. for their outstanding contribution
to both empirical and conceptual approaches in psychology andneuroscience. As reported by the authors, smiling faces are
judged happier when gazing at the participant as compared togazing sideways (e.g., Adams & Kleck 2005). The authors advancethat such an increase in positive intensity could reflect a particularly
accurate embodied simulation during smile perception. Here, we
explore whether such a model could account for recognition of all
emotions. Alternatively, the triggering function of eye contact may
be specific to a subset of emotions that includes happiness.

Although facial expressions are efficient emotional signals, eyegaze direction is important for signaling the referent of an
expression and, therefore, the attended object of the elicitedemotion (George & Conty 2008). Converging evidence suggeststhat these two signals are integrated during the perception of
facial emotion. Further, behavioral studies have reported thatthe perception of facial expressions can be modulated by eyegaze direction. However, the effect of gaze on emotion recognition
depends on the type of expression. For example, it has been shownthat angry faces are perceived to express more anger with direct
than averted gaze, whereas fearful faces are perceived to express
more fear with averted than direct gaze (Adams & Kleck 2003;
Sander et al. 2007). These results can be explained within a selfrelevance framework (Sander et al. 2003). Indeed, it has been
proposed that angry expressions with averted gaze or fearfulexpressions with direct gaze have less relevance for oneself thanthese expressions with a direct or averted gaze, respectively. The
rationale is that the aversive dimension of anger is higher when
one is the target of it (and hence gazed at); similarly, the threatening content of fearful expressions is higher when the face is looking
away from the observer, as the object of fear for another agent canalso reflect a potential menace for the observer. So far, theseresults suggest that emotion perception is influenced by the selfrelevance of expression based on gaze direction, and this process
appears to be emotion-dependent.
Niedenthal et al. propose that the amygdala may produce statesthat further contribute to simulations that underlie how smiles areinterpreted. A recent brain imaging experiment is particularly relevant to this view and the prediction for the role of gaze contact in
the processing of happiness: Sato et al. (2010) showed that dynamic
happy faces elicit more amygdala activation when gazing at – as
opposed to away from – the observer. Yet, the amygdala does
not always show greater response to direct as compared toaverted gaze for all emotions, but rather supports its involvementin the appraisal of self-relevance (N’Diaye et al. 2009). Brainimaging and patient studies suggest that the amygdala deals with
the integration of emotional expression and gaze direction(Conty et al. 2010; Cristinzio et al. 2010), with typical differential
amygdala activity for anger versus fear processing as a functionof direct versus averted gazed (Adams et al. 2010; Hadjikhani
et al. 2008; Sato et al. 2010). Therefore, amygdala responses to a
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

443

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
given facial expression vary as a function of self-relevance, that is,on concomitant gaze direction and perceived social meaning.
Thus, one can advance that embodied simulation can be initiated
and/or reinforced either by mutual gaze (as in the case of happiness or anger) or by averted gaze (as in the case of fear).
A possible source of differential self-relevance processing as afunction of gaze and emotion is ontogenetic development.
Indeed, it has been argued that an early sensitivity to eye contact
serves as a major foundation for later development of social skills(Csibra & Gergely 2009). In line with this view, the processing relative to averted gaze perception seems to emerge several weeks after
the initial sensitivity to eye contact. Only by the age of 3 months do
infants show the ability to automatically follow another’s eye gaze
toward the surrounding space (D’Entremont et al. 1997). Furthermore, the spontaneous shift of spatial attention involved byanother’s averted gaze perception at an early age occurs when
initial eye contact happened between the infant and the caregiver(Farroni et al. 2003). It is as if the initial attentional capture triggered by eye contact (Farroni et al. 2002) was gradually extended
to other eye directions (see Reddy [2003] for similar proposal).
Of interest here, the ability to discriminate between emotions
only appears to be well established after the second half of thefirst year (Montague & Walker-Andrews 2001) and, once arising,brain processing of emotional faces does integrate the information
provided by eye direction (Hoehl & Striano 2008). Recognizing
various emotional expressions therefore emerge after the ability
to process eye direction, whether it is directed toward the self or
toward an external object.
It may well be the case that Niedenthal et al. reveal a very basic
process by proposing that eye contact increases embodied simulation when perceiving smiling faces. Indeed, the exchanges ofsmiles and eye contact between a mother/father and her/his
infant is probably the most primary form of human social relationship, and could represent the most basic bricks in the developmentof embodied simulation. However, this phenomenon must evolve
over the course of development to understand the diversity ofsocial contents. Thus, the mechanisms primarily involved in eye
contact perception may gradually be extended to other eye directions and become a fundamental determinant of emotion recognition. Still, a theoretical question remains: Must eye-gaze
direction already somehow be integrated with facial expression
before any simulation can be triggered? If integration takes place
before simulation, this would mean that one has to recognize, atleast very coarsely, the expression before simulating it. As integration certainly arises only after both gaze direction and emotion
expression have been processed independently (see Pourtois
et al. 2004), we wonder what would trigger simulation in the firstplace, before it is possibly reinforced by mutual or averted gaze.

Cultural variations on the SIMS model
doi:10.1017/S0140525X10001317
Christine M. Covas-Smith, Justin Fine, Arthur M. Glenberg,
Eric Keylor, Yexin Jessica Li, Elizabeth Marsh, Elizabeth
A. Osborne, Tamer Soliman, and Claire Yee1
Department of Psychology, Arizona State University, Tempe, AZ 85282.
ccovas01@gmail.com
jmfine@asu.edu
Glenberg@asu.edu
ekeylor@asu.edu
yjli1@asu.edu
Elizabeth.R.Marsh@asu.edu
Elizabeth.A.Osborne@asu.edu
tamer.soliman@asu.edu
ciyee@asu.edu
http://psychology.clas.asu.edu/glenberg

Abstract: Niedenthal et al. recognize that cultural differences areimportant when interpreting facial expressions. Nonetheless, many of
their core observations derive more from individualistic cultures thanfrom collectivist cultures. We discuss two examples from the latter: (1)lower rates of mutual eye contact, and (2) the ubiquity of specific444

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

“functional smiles.” These examples suggest constraints on theassumptions and applicability of the SIMS model.

Niedenthal et al. posit an embodied model for perceiving themeaning of smiles through implicit simulation. As noted in
section 6, the model is grounded on insights taken from predominately individualistic cultures. This commentary focuses on two
culturally limited aspects of the SIMS model. First, the modelemphasizes that eye contact serves as a triggering mechanism
for the activation of embodied simulations. Second, the model
explicates three principal functions that smiles and the simulationof those smiles serve in interpersonal settings. There are compelling reasons to believe that differences in social and environmental
context will modulate these particular components of the model.
Eye contact may not provide the primary route toward successfulinterpersonal simulation in all cultures. (Indeed, as noted in section
4.1, eye contact is presumed to be a sufficient but not a necessary
trigger.) Some collectivist cultures, like East Asian cultures, tend
to avoid eye contact when processing facial expressions. Children
in these cultures are taught that direct eye contact is a sign of disrespect (Argyle & Cook 1976). Such cross-cultural differencessuggest that eye contact is a socially established cue that varies
with particular cultural norms. How might such a socially refined
behavior affect the SIMS model? Assuming that eye contact isthe most frequent triggering mechanism for interpersonal simulation, East Asians would be predicted to often miss this trigger.
The result would be little direct simulation or unmediatedinterpretation of the facial expressions of others. Instead, East
Asians would need to rely on conceptual or contextual knowledge
when reading facial expressions. Although possible, this conclusion
is suspicious. Why should it be the case that any culture would
develop in such a way as to negate the occurrence of direct affective
resonance? This prediction seems especially peculiar given that“collectivist” cultures are defined by their strong tendency towardshared social intentions and responsibilities.
A reasonable alternative is that the triggering mechanism ofaffect simulation is not solely based on the activation of a focal
facial component, but rather on a gestalt of facial muscular configuration encompassing the supra- and infra-orbital, paranasal,
and peribuccal regions. The targeted relational contour could
be detected by a fixation that lands centrally at (or slightly
above) the tip of the nose, hence making it possible to peripherally process the relevant broader area at once. Evidence for suchan alternative relational account is suggested by the findings of across-cultural face-processing study (Blais et al. 2008). East Asianparticipants did not exhibit the fixation triangle of eyes and nose
adopted by their American counterparts, but they preferredinstead to fixate a narrow nasal locus.
The relational alternative could extend beyond a biological(facial or otherwise) narrow scope. There is evidence that, inmaking decisions, people in collectivist cultures forge a broad
relational model that gives primacy to social and physical cues
assimilated from the environment. Kitayama et al. (2003) demonstrated that American participants were accurate when attempting to replicate the absolute length of a line drawn in a square
frame, whereas East Asians were more accurate when replicating
relative line length, taking into account both the length of the line
and its relation to the size of the frame. Similarly, people fromindividualistic cultures are more inclined to make personcentered attributions of an event’s cause, whereas people fromcollectivist cultures are more inclined to identify a situationcentered cause (Morris & Peng 1994). This idea is consistent
with Niedenthal et al.’s suggestion in section 6.1.1 that context,
such as the strength of a personal relationship, is likely to playa role in simulation.
The functional roles of smiling might also be culture-specific.
Niedenthal et al. make the cogent suggestion that smiles are distinguished based on the part they play in facilitating interpersonal actions and reactions. Therefore, changes in social structure
should coincide with changes in the function of smiles that
operate within that structure. Accordingly, some functional

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
smiles described in Niedenthal et al. may be less common or evennonexistent in collectivist cultures. The “dominance smile,” forexample, may never have developed as a display of pride in a
culture that values equality over individual achievements.
Indeed, individuals in collectivist cultures who express toomuch pride are seen negatively and can be ostracized fromsociety (Kalat & Shiota 2006).
Given a culture with salient group identity, the utility of individual expressions to convey affect might be lessened or replacedby other environmental cues. An assessment of the social appropriateness of smiling in Chinese individuals, conducted by
Wiseman and Pan (2004), found that Chinese are overall lesslikely than Americans to smile. On one hand, the SIMS model
would predict this finding because less eye contact should leadto fewer simulations of smiles. Indeed, collectivist cultures arecharacterized as tending to appear more passive and lessemotionally expressive than typical individualistic cultures (Matsumoto et al. 2008). Does this mean that people from collectivist
cultures, because they are less likely to smile and to simulate
smiles, experience less direct interpersonal affect? If so, thenpeople from collectivist cultures would have to rely almost exclusively on conceptual judgments (based on stereotypes) wheninterpreting each other’s emotions and intentions.
This possibility seems unlikely given that affect guides action
and that the implicit, contextual communication of affect facilitates interpersonal actions. An alternative hypothesis is that thesame affect that is functionally communicated through facial
simulation in individualistic cultures is directly simulated
through other environmental media in collectivist cultures.
Cues external to any one person might provide better information about the operation of the entire group, so a lessened
importance on the individual might mean a lessened utility of
an individual’s expressions to convey meaningful affect, that is,
to guide action. Again, this analysis is consistent with the invocation of context in section 6.1.1 of the target article.
In summation, socially learned behaviors and goals may shapethe very foundations of affective resonance, thus rendering any
model that describes a particular, fixed route to this resonancetoo specific to apply across all cultures.
NOTE
1. All authors contributed equally.

Embodied simulation and the meaning offacial expression in autism
doi:10.1017/S0140525X10001500
Kris Evers,a,b,d Ilse Noens,c,d Jean Steyaert,b,d and Johan
Wagemansa,daLaboratory of Experimental Psychology, University of Leuven (K.U. Leuven),
Belgium; bChild Psychiatry, Department of Neuroscience, University of Leuven(K.U. Leuven), Belgium; cCentre for Parenting, Child Welfare and Disabilities,
University of Leuven (K.U. Leuven), Belgium; dLeuven Autism ResearchConsortium (LAuRes), University of Leuven (K.U. Leuven), Belgium.
kris.evers@psy.kuleuven.be
ilse.noens@ped.kuleuven.bejean.steyaert@med.kuleuven.bejohan.wagemans@psy.kuleuven.be
www.gestaltrevision.be

Abstract: We outline three possible shortcomings of the SIMS modeland specify these by applying the model to autism. First, the SIMS
model assigns a causal role to brain processes, thereby excluding
individual and situational factors. Second, there is no room for
subjective and high-level conceptual processes in the model. Third,
disentangling the different stages in the model is very difficult.

The Simulation of Smiles (SIMS) model holds that observing asmile leads to the extraction of meaning via different behavioral
and neural processes on in the perceiver’s part. In what theycall their core model, Niedenthal et al. propose that the same
smile is interpreted in three different ways, according to its functional meaning as enjoyment, affiliation, or dominance, by engaging different brain areas (see their Figure 4). By doing so, theSIMS model subscribes to some kind of mind-brain identity
theory, establishing a one-to-one mapping of mental states tobrain processes. Although such a philosophy of mind is implicit
in most of modern cognitive neuroscience, SIMS is very explicit
in assigning a unique, causal role to the brain processes andleaving no room for individual and situational factors which
may also affect how a particular smile is interpreted. Moreover,
the SIMS model appears to ignore the role of subjective andhigh-level conceptual processes which are crucial in emotion
processing (e.g., using one’s knowledge about the person who
is smiling or the context in which the smile takes place).
Finally, disentangling the different stages in the model is difficult,which complicates the process of deriving specific hypotheses
from it and testing them empirically. Niedenthal et al. provide
a fascinating synthesis of the huge literature on embodied simulation and emotion, both in psychology and the neurosciences,
but distinguishing between perceptual processes, motor
mimicry, and empathizing in particular cases of emotion understanding turns out to be quite a challenge. In the remainder ofthis commentary, we will make these arguments more specificby trying to apply the SIMS model to autism.
Autism is an early-onset neurodevelopmental disorder characterized by the co-occurrence of impairments in social reciprocity
and communication, accompanied by stereotyped and rigid patterns of interests and activities. Early behavioral signals ofautism, such as atypical eye contact, absence of social smiles,
and absence of gaze following, are closely related to the processing of facial expressions (including smiles). At the cognitive
level, children and adults with autism often have difficulties inascribing mental states to others, referred to as mentalizing or
Theory of Mind problems (e.g., Tager-Flusberg 2007). Thismeans that they experience problems in placing themselves in
the mental worlds of others and in empathizing with others.
Therefore, autism provides an ideal test case to evaluate the
SIMS model. Applying SIMS to autism (SIMS-A) should yieldadditional insight or at least testable hypotheses, if the model
pretends to be more than hocus-pocus with component processes
and neural circuits (cf. magic words “Sim Sala Bim”).
First, linking psychological processes to neural mechanisms isdefinitely a way to make progress, also in research aimed at understanding the causal pathways behind a complex disorder like
autism. There is a significant genetic component to autism, andsome of the genes involved are clearly affecting the developing
brain (e.g., Bourgeron 2009). There is a huge body of literature
aimed at discovering structural and functional differences
between brains of people with and without autism (e.g., Amaral
et al. 2008). However, our understanding of the vagaries and complexities of information and emotion processing in autism cannot
be reduced to looking only inside the brain. If people with
autism have difficulties in picking up subtle cues from faces, differentiating relevant from irrelevant stimulus changes in faces andfacial expressions, integrating local details with more global
context information and bottom-up perceptual information with
top-down knowledge about the situation, and especially in doing
all of this automatically and without painstaking effort, it is clearthat an exclusive focus on neural circuits alone will not tell thewhole story. At least not at the present stage of our scientificendeavor, when linking functional explanations at the psychological level to possible neural correlates is still a tricky business as far
as complex, multi-factorial phenomena are concerned.
Second, autism researchers often take individual and situational factors into account when trying to explain the inconsistencies between studies finding anomalies in emotion processing inchildren and adults with autism and studies that do not. Clinicalsamples differ in factors such as age, language skills, general
intelligence, subset and severity of symptoms, etc. Moreover,
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

445

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
researchers have used a huge variety of stimuli (e.g., faces in isolation vs. embedded in complex scenes and events, static vs.
dynamic facial expressions, uni- vs. multi-modal, brief vs. unlimited presentations), tasks (e.g., explicit vs. implicit, focused vs.
distributed attention, sorting, matching, labeling), and measurements (e.g., performance levels, response times, eye movements). All of these factors have shown to influence theoutcome of studies, but none of them is currently incorporated
into the SIMS model.
Third, the SIMS model claims to be able to distinguish the roleof processes like establishing eye contact, facial mimicry, simulating embodied experience, using conceptual knowledge, social
inhibition, and so forth; but we are not convinced that it succeeds
in doing so. For instance, can SIMS distinguish, functionally orneurally, cases of embodied emotion simulation triggered bybottom-up facial cues or by top-down knowledge? How do we
differentiate the SIMS model from the many different models
making use of components from the so-called mirror neuron
system (e.g., Rizzolatti & Fabbri-Destro 2008)? If all of these
crucial yet relatively basic distinctions appear to be difficult,what about all the subtleties which appear to be involved in theliterature on autism? Will it be possible to specify the SIMS
model sufficiently so that testable hypotheses can be derived
from it? We hope so, but there is clearly still a very long way to go.
Addressing these three general issues regarding the SIMS modeland their specific elaboration regarding autism, will clarify whatkind of model SIMS is, and what it can and cannot offer.

Le bon dieu est dans le detail: Is smiling therecognition of happiness?
doi:10.1017/S0140525X10001512
José-Miguel Fernández-Dols and Pilar Carrera
Facultad de Psicologı́a, Universidad Autónoma de Madrid, 28049 Madrid,
Spain.
jose.dols@uam.espilar.carrera@uam.es

Abstract: We question two conceptual assumptions made by Niedenthal
et al.: the dichotomy between true and false smiles and the close tie
between recognition and experience of emotion. An excessive dependence
on everyday language suggests overly parsimonious accounts of a complex
set of relations between smile, experience, and context.

Recognition is a fundamental topic in the study of emotion,
central to the claim of universality (Darwin 1872/1965), and is
the subject of great controversy. For some theorists (e.g.,
Russell 1994), recognition is thought of as the attribution of asemantic label to another’s state, whereas for basic emotion theorists (e.g., Ekman 1994), recognition is thought of as part of anemotion signaling system grounded in a specific brain region.
Niedenthal et al. advance the study of how emotion is recognized in others. First, they offer new data on the brain regions
involved in recognition of emotion from facial expression.
Second, they go beyond the isolated, still, photographed facedevoid of context to the facial expression as dynamic muscular
movements embedded in a context of interaction between thesender and the perceiver (see Fernández-Dols & Carroll 1997;
Fernández-Dols et al. 2008). Third, they propose that recognition is a form of emotion knowledge based on embodied simulation of the sender’s emotional state.
Nevertheless, we see two main problems: (1) We do not agreewith Niedenthal et al.’s distinction between true and false smiles;and (2) we question the assumption, common to basic emotion
theory and Niedenthal et al.’s model, that recognition ofemotion is closely tied to the experience of emotion.
True and false smiles. Niedenthal et al.’s model is focused on
the detection of true smiles of enjoyment, dominance, or

446

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

affiliation. We are glad to see that Niedenthal et al. have acknowledged that the Duchenne smile (Ekman et al. 1990) is not themarker of a true smile. Duchenne smiles can be effortlessly
posed, do not predict senders’ affective state (Krumhuber &Manstead 2009), and can be caused by negative experiences(e.g., losing a game: Schneider & Josephs 1991; pain: Kunz
et al. 2009). Nevertheless, Niedenthal et al. maintain the original
theoretical assumption behind the Duchenne smile, namely, thetrue versus false dichotomy. According to Niedenthal et al., true
smiles are honest displays that reveal positive feelings, whereas
false smiles are manipulative displays that hide negative feelings.
In our view, any model aimed at detecting the “truth” or“honesty” of an expression is, ultimately, scientifically unsound.
Smiles are not essentially true or false. Indeed, we doubt thatany dichotomous classification of smiles will survive scientificscrutiny for long. Smiles are too morphologically diverse for adichotomy: they can be located on a continuum based on theirtempo, intensity, asymmetry, and concomitant facial movements(Krumhuber & Kappas 2005; Krumhuber & Manstead 2009).
This dynamic diversity has consequences for the receiver’s judgments (e.g., perception of trustworthiness: Krumhuber et al.
2007) and is connected to the sender’s relational strategies(e.g., Vigil 2009). Thus, each particular facial expression is toocomplex in its meaning to be either true or false. A consciously
exaggerated “false” smile is “true” – and extremely functional –in the context of baby talk (Chong et al. 2003). The equally functional spontaneous “true” smile of a two-month-old baby isclearly “false” in that it lacks any of the “honest” emotional or cognitive correlates of a “true” adult smile.
Simulation as emotional experience. Niedenthal et al.’s simulation-as emotional-experience theory sees emotions as “natural
kinds” in which experience and expression belong to a tightcluster of properties caused by the same neural structures(Barrett 2006; Fernández-Dols & Russell 2003). On this view,
emotion knowledge is expression (mimicry), experience, and recognition of basic emotion. There are two apparent sources of evidence in support of this view: experimental and clinical.
We question the experimental support (e.g., Halberstadt et al.
2009; Niedenthal et al. 2009). Electromyography shows a weak oreven contradictory relation between emotion knowledge andfacial mimicry (e.g., activity of the levator labii when processing
joy or nonspecific activity of the corrugator when processing
disgust; Niedenthal et al. 2009). The claim that such mimicrycauses the experience of the corresponding basic emotion is
not scientifically justified on the basis of current evidence. We
see no reason to assume that the link between emotion knowledge and facial mimicry is different from the observed link
between nodding your head and emotion knowledge (Niedenthal
2007) – that is, culture-specific and learned. No evidence forces
us to the additional assumption of an innate connection between
mimicry, experience, and recognition of emotion. Mimicry could
be the outcome of several parallel processes (Lishner et al. 2008;
Yabar et al. 2006).
We also question the clinical support for the recognitionas-experience view. Case studies show paired deficits in bothexperience and recognition of emotions (e.g., fear or disgust)apparently caused by damage in one specific brain region (e.g.,
the amygdala or the insula and basal ganglia; for a review, seeGoldman 2006). Nevertheless, further research on the sharedneural substrate of such paired deficits has raised doubts as towhether these regions are organized for experiencing and recognizing basic emotions. The functions of these regions are broader(e.g., Heberlein et al. 2008), more diverse (e.g., Heberlein &Atkinson 2009), and more complex than originally thought(e.g., Chen et al. 2009; Straube et al. 2010). Better experimental
procedures and measures challenge the original interpretation of
the paired deficits (Milders et al. 2003; Rapcsak et al. 2000).
Understanding recognition requires a more conceptually soundanalysis of the dynamics and functions of specific brain regions(see Kagan 2007).

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Conclusion. The study of the recognition of emotion mustmove beyond everyday language. The assumption that words
such as “enjoyment” or “smile” carve nature at its joints
prompts us to unhelpful dichotomies, such as “false versus
true” smile or “basic versus non-basic” emotion. Moving
beyond overly parsimonious assumptions would open the way
to more complex but realistic models in which the perceptionof an expression is a consequence of not only affect or emotion,
but also of diverse variables, such as, for example, stimulus
acuity (e.g., Bohrn et al. 2010), sender’s gender (e.g., Vazire
et al. 2009), facial structure (e.g., Zebrowitz et al. 2010), andculture (e.g.,Elfenbein & Ambady 2002).
ACKNOWLEDGMENT
The authors are supported by grant PSI2008-0849 from the Spanish
Government.

Does motor mimicry contribute to emotion
recognition?
doi:10.1017/S0140525X10001524
Cindy Hamon-Hill and John Barresi
Department of Psychology, Dalhousie University, Halifax, Nova Scotia B3H
4J1, Canada.
cindy.hamon-hill@dal.ca
jbarresi@dal.ca
http://jbarresi.psychology.dal.ca/

Abstract: We focus on the role that motor mimicry plays in the SIMS
model when interpreting whether a facial emotional expression isappropriate to an eliciting context. Based on our research, we findgeneral support for the SIMS model in these situations, but with somequalifications on how disruption of motor mimicry as a process relates
to speed and accuracy in judgments.

We applaud the efforts of Niedenthal et al. to integrate multiple
processes underlying recognition of smiles in facial expression
into a model that can generalize to the recognition of otheremotions. However, we detect a need for greater clarity regarding the role of motor mimicry in a complex judgment task. In ourparadigm for recognizing emotion in a naturalistic facial
expression, a static contextual image eliciting one of eight possible emotions is presented simultaneously with a five-second
facial expression that is either static or dynamic. Participants
judge when the facial expression is the genuine response to thepaired contextual image (“match”) and when it is not (“mismatch”) (Hamon-Hill & Barresi 2008a; 2008b; 2009; Ouelletteet al. 2010). To test for reliance on motor mimicry during thistask, participants hold a pen horizontally in their mouth throughout alternating blocks of trials. In agreement with the SIMS
model, we recognize embodied simulation as a key component
in the underlying processes activated during this task (Barresi& Moore 1996; 2008). We propose the observers, while perceiving the paired stimuli, generate a first-person embodied responseto the contextual image, which includes autonomic, somatic, andsemantic aspects. The observers also generate a third-person representation of the target facial expression and search for a match
between their own embodied response to the contextual image
and their understanding of the target facial expression. For
instance, observation of an elderly man holding the hand of his
dying wife may elicit a response of “feeling sad,” prompting theobservers to search for signs of sadness in the target facial
expression.
Earlier simulation models for emotion recognition propose
facial mimicry contributes to recognition of facial expressions(Goldman & Sripada 2005); however, there is limited support
for this claim (Blairy et al. 1999; Hess & Blairy 2001), andimaging studies measuring motor activity seldom test for

accuracy in judgments (Carr et al. 2003). Using our paradigm,
we tested for an effect of disruption to motor mimicry on judgments about a dynamic facial response as genuine to a given
context. We tested the effect of two experimental manipulations,
one to inhibit mimicry (Niedenthal et al. 2001) and the other tointerfere with mimicry (Oberman et al. 2007). These two manipulations, which require holding a pen in one’s mouth, were cited
indiscriminately in the target article, suggesting both inhibit
motor mimicry. However, Oberman et al. (2007) established a
difference between the two: Whereas holding a pen between
the lips and teeth (Niedenthal et al. 2001) holds the muscles ina neutral state, thereby “inhibiting” a motor response, “biting”
down on a pen held horizontally between one’s teeth, without
the lips touching the pen, “interferes” with spontaneous motor
mimicry due to “noise” generated by consistent activation of multiple facial muscles (Oberman et al. 2007). During this manipulation, activation in the motor systems is neither absent nor
reduced as suggested in the target article.
Like Oberman et al. (2007), we saw no effect of disruption to
mimicry on accuracy resulting from the inhibition manipulation.
However, observers in the inhibition condition and the no-pen
control condition both showed a significant response bias tojudge the target face as a match to the paired context image. Surprisingly, the response bias was absent in the interference condition, where participants correctly discriminated when faces
were not a genuine response to the paired contextual image.
When we repeated the experiment applying only the interference
manipulation to test for a differential effect on processing
dynamic or static facial expressions, once again accuracy was
not altered, but processing was affected. When mimicry was disrupted, observers were slower to process static, positive faces, yetwere faster to process dynamic, positive faces.
The SIMS model offers an explanation for a response bias tofacial expressions based on conceptual knowledge. According to
the model, an embodied response to a context stimulus shouldprime the observer’s judgment about the emotion perceived in atarget facial expression. Additionally, disruption to motor
mimicry should increase the priming effect due to greater relianceon this embodiment of conceptual knowledge. Our paradigm
emphasizes the incorporation of embodied responses to context
into judgments made about facial expression. However, theresponse bias was strongest when participants were free tomimic and while mimicry was “inhibited”, yet it was significantly
diminished during “interference” with mimicry. Niedenthal
et al.’s account of processing facial expression when motor
mimicry is disrupted does not adequately explain what we haveobserved. We suggest the interference generated by the bitemanipulation causes a mismatch state in the self with respect tothe embodied response to the context image, and, consequently,
the observer searches for a “mismatch” in the other. This leads
to a reduction in the bias to see a match and increases the likelihood of improved performance on mismatch trials.
Another limitation to the model is the lack of adequate discussion regarding the quality of stimulus, in particular the difference
due to static and dynamic emotional expressions. Neuroimaging
studies indicate greater motor activity during observation of
static compared to dynamic faces (Enticott et al. 2008; Kilts
et al. 2003). Moreover, we would expect less reliance on embodied
simulation involving motor mimicry during perception of dynamic
faces because of the rich quality of information available in a
dynamic facial expression (Ambadar et al. 2005). Dynamic facial
displays can reveal blended emotions that change over time, shifting in degrees of expressivity. Perception of static faces, which typically present the apex of the expression, may require the observer
to enrich the information by simulating the full expression. One
consequence of this difference between static and dynamic
expressions is that we would expect greater interference with processing static faces when motor mimicry is disrupted. That isindeed what we observed. Disruption to mimicry resulted inslower processing of static but not dynamic faces.
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

447

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
In sum, motor mimicry, while a contributor to a fully embodied form of emotion recognition, may not be an essential component when interpreting rich facial expressions in naturalistic
contexts, but rather may primarily serve social functions suchas indicating sympathy and understanding.

No mirrors for the powerful: Why dominantsmiles are not processed using embodied
simulation
doi:10.1017/S0140525X10001536
Li Huang and Adam D. Galinsky
Department of Management and Organizations, Northwestern University,
Evanston, IL 60208.
l-huang@kellogg.northwestern.edu
agalinsky@kellogg.northwestern.edu
http://www.kellogg.northwestern.edu/faculty/huang
http://www.kellogg.northwestern.edu/faculty/bio/galinsky.htm

Abstract: A complete model of smile interpretation needs to incorporate
its social context. We argue that embodied simulation is an unlikely routefor understanding dominance smiles, which typically occur in the context
of power. We support this argument by discussing the lack of eye contact
with dominant faces and the facial and postural complementarity, rather
than mimicry, that pervades hierarchical relationships.

Not all smiles are equal. As indicated by Niedenthal et al., somesmiles express joy and affiliation, whereas others express dominance. A complete model of smile interpretation needs toexamine the different aspects of social relationships that eachtype of smile facilitates. Armed with this understanding, one
can determine how likely it is that embodied simulation would
occur in any given context. This commentary proposes that dominance smiles are categorically different from enjoyment andaffiliative smiles in terms of their social contexts and implications
and are therefore less likely to be processed through embodied
simulation.
Social relations vary in both the “horizontal” and “vertical”
dimensions (Fiske 1992). The “horizontal” dimension describes
the affective or socioemotional dimension, encompassing thevalence of feelings and emotional closeness of interpersonal
relations (Berger 1994; Osgood et al. 1957). Enjoyment andaffiliative smiles, which communicate positive emotions andcooperative social motives, tend to occur when social relations
are characterized horizontally.
The “vertical” dimension, in contrast, describes the fact thatinterpersonal relations can be organized along a low-to-high continuum of dominance, power, status, or hierarchy (Hall et al.
2005). Dominance smiles, which express status and control,
facilitate social relations that take place in the vertical dimensionof sociality. In this dimension, individuals seek to identify and/or
establish their rank in a hierarchy rather than to build social
bonds. As a result, eye contact tends to be suppressed, andcomplementarity of facial and postural expressions between
dominant and non-dominant individuals is prevalent. These processes should render embodied simulation a less frequently usedmethod for understanding a dominance smile.
With regard to eye contact, Niedenthal et al. argue that it is a
key trigger for embodied simulation. Eye contact is a nonverbal
behavior made possible only when two people are gazing into
each other’s eyes. However, there is significant difference inthe frequency of eye contact within vertical and horizontal
relationships. Horizontal relationships between mutually dependent individuals facilitate eye contact, which helps communicate
positive emotions, seek reconciliation, solicit positive attitudes,and establish intimacy (Edinger & Patterson 1983; Mehrabian1969). On the other hand, because dominant individuals are

448

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

less dependent, they tend not to reciprocate the gaze, making
it less possible for eye contact to occur.
Niedenthal et al. also suggest that risk or aversiveness is one of
the reasons individuals may suppress eye contact. Numerous
findings suggest that exchanging eye contact with individuals
who display dominance smiles may be deemed too risky andthat subordinate individuals may avert their gaze to prevent inappropriately challenging their dominant counterparts. Dominancefaces are more similar to threatening or angry faces than they are
to happy or sad faces (Lipp et al. 2009; Öhman et al. 2001).
Because individuals avoid eye contact with those who display
aggression (Ellsworth & Carlsmith 1973), they would presumably
avoid eye contact with those who display dominance as well.
Similarly, a variety of primate species respond to threat fromdominant animals by averting eye contact and even the face all
together (Bertrand 1969; de Waal 1989; Van Hooff 1967). Avoiding eye contact removes the threatening stimulation, creating a“cut-off” to reduce tension in the recipient of the threat while
sending out a submissive signal (Chance 1962; Redican 1975).
Thus, both the individuals producing dominance smiles and theperceivers of those smiles tend to avoid eye contact.
With regard to mimicry and complementarity, we propose thatwhen individuals do smile in response to a dominance smile, they
are likely to produce a knowledge-based “complimentary or submissive” smile, instead of an “exact replica” of the dominancesmile. Niedenthal et al. state that embodied simulation may
still occur when eye contact is suppressed. They argue that individuals can use conceptual knowledge to create facial mimicry,which then activates an off-line simulation, or an as-if loop.
However, there is ample reason to believe that facial mimicrydoes not typically occur in the presence of a dominance smile.
Our argument is bolstered by the frequently observed complementarity, as opposed to mirroring, of bodily expressions in vertical
relationships between humans or primates. Humans use smiles
not only to express enjoyment, affiliation, or dominance, but also
to express submissiveness (Whalen & Kleck 2008). Submissive
smiles appease and show deference. Darwin misinterpreted thisexpression as a true smile, but it is arguably the “descendant” offear grimace seen in monkeys – subordinate monkeys offer
grimace toward dominant monkeys as a signal of submission.
Similar findings are found in humans with regard to posture –dominance postures tend to induce submissive postures in others,
a process called dominance complementarity (Tiedens & Fragale2003). This research suggests that, when individuals smile inresponse to a dominance smile, they will produce a submissive
smile. To the extent that the entire embodied simulation processrelies on facial mimicry, a lack of facial mimicry suggests theabsence or partial absence of embodied simulation. The absenceof facial mimicry in vertical relationships implies that dominancesmiles are less likely to be processed through embodied simulation
We do, however, acknowledge that in certain contexts, presumably what Niedenthal et al. have called high-uncertainty situations, dominance smiles may be processed through embodied
simulation. When the perceiver is competing for a higher rank,
instead of adhering to an established social hierarchy, eyecontact and facial mimicry may be observed, and embodied simulation of dominance smiles may occur.
In conclusion, embodied simulation is less likely to be used toprocess dominance smiles because they typically occur in a social
context where eye contact and motor mimicry are minimal.
Future research should explore the precise processes used tounderstand dominance smiles. More broadly, future researchon the SIMS model should investigate the role of social context
in the use of embodied simulation. It should also discuss
whether social contexts affect the types of smiles perceivers
produce. Given that hierarchy is the dominant form of socialorganization (Leavitt 2005; Magee & Galinsky 2008), considering
this context will not only help expand an understanding of whenembodied simulation is used to understand facial expressions,but will also produce a more complete taxonomy of smiles.

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model

Could embodied simulation be a by-product ofemotion perception?
doi:10.1017/S0140525X10001548
Julian Kiversteina and Edoardo ZamunerbaSchool of Philosophy, Psychology and Language Sciences, University of
Edinburgh, Edinburgh, EH8 7PU, Scotland, United Kingdom; bDepartment of
Philosophy, LaTrobe University, Victoria 3086, Australia.
j.kiverstein@ed.ac.uk
e.zamuner@latrobe.edu.au
http://www.philosophy.ed.ac.uk/people/research-staff/
julian-kiverstein.html
http://www.latrobe.edu.au/philosophy/staff/edo.html

Abstract: The SIMS model claims that it is by means of an embodied
simulation that we determine the meaning of an observed smile. Thissuggests that crucial interpretative work is done in the mapping thattakes us from a perceived smile to the activation of one’s own facial
musculature. How is this mapping achieved? Might it depend upon aprior interpretation arrived at on the basis of perceptual and contextual
information?

Smiles can be used to communicate a range of different psychological phenomena, including positive emotions, complex social
intentions, and even a person’s social status. Given this diversityof possible meanings, how do we succeed on a given occasion in
working out what a particular smile means? To do so, we must
single out which of these many nuanced psychological phenomena the smile expresses. Niedenthal et al. say we solve thisproblem in part by “simulating” the nuanced states that we
observe in others. We agree that embodied simulation may
make an important contribution to the type of understanding
we have of a smile. However, the exact nature of this contribution seems to us to remain an open question. In the spiritof friendly critics, it is this question we will take up in ourcommentary.
The SIMS model claims that we work out what a smile meansin three interrelated stages. Given the work that motor mimicry
and its effects is being asked to do in the determination of asmile’s meaning, something important is clearly happening in
the transition from perception at Stage 1 to motor mimicry or
action at Stage 2. There are a number of possible affective
states that the smile you are producing might be expressing;but when I copy your facial expression, this results in me
expressing the very same affective state. Thus, the processthat allows me to map the perception of your behaviour onto
the activation of my own facial musculature must somehow besingling out the meaning of the smile I am seeing. We don’t
wish to deny that this might be possible, but the SIMS
model, so far as we can tell, doesn’t tell us how this mapping
is supposed to be effected.
The authors appeal to eye contact to explain how an embodied
simulation gets triggered. Hence, they clearly think that at least
sometimes the embodied simulation is sufficient for us to arriveat an interpretation of a smile. Supposing this is so, this makes
it all the more urgent to know how the problem of determining
what a smile means is solved by producing an embodied simulation. In order for me to mimic a smile that is affiliative,
mustn’t I have already somehow worked out that the smile isaffiliative? If so, how?
Consider, now, the studies in which facial mimicry is blocked
or socially inhibited (see sects. 6.1.2.1. and 6.1.2.2. of the target
article). The absence of motor mimicry in these studies has theconsequence that “activation of motor systems and emotion
systems will be absent” (sect. 6.1.2.1, para. 2). Hence, if recognition is achieved, it must be by some means other than embodied simulation. Niedenthal et al. suggest this could beachieved by matching visual input to a stored perceptual representation. If we sometimes have recourse to this strategy,
why don’t we always use this strategy? Niedenthal et al. go on
to allow that embodied simulation could still occur in this

scenario, but it would have to be triggered by the use of conceptual knowledge since it does not arise from eye contact. However,
if an interpretation of a smile has already somehow beenachieved by matching visual input to a perceptual representation,
what work is left for the embodied simulation to do?
Furthermore, how is the perceptual representation selectedthat is used to give meaning to the visual input? Niedenthal
et al. have endorsed an embodied or grounded account of perceptual processing. Therefore, when they talk about conceptual
knowledge triggering an embodied simulation, they must meanthat some reactivated multi-modal representation is what triggers
an embodied simulation. However, they don’t explain how visual
input leads to the reactivation of the specific multi-modal representations that provide us with the interpretation of a smile.
Once again, an appeal is made to a mapping from visual input
but this time to a multi-modal representation, and it is bymeans of this mapping that we come to assign a meaning to asmile. However, there is no account given of the mechanismsthat might bring about such a mapping.
Could it be that the problem of interpreting the smile is alreadysettled at Stage 1 in perception, and this perceptual interpretation
is what subsequently causes motor mimicry and its associatedeffects? Consider a parallel problem of determining the goal of
an instrumental action. Csibra (2007) has argued that prior to anembodied simulation of an instrumental action is a stage of processing in which a visual analysis is formed of an action and thecontext in which the action is taking place. He hypothesises that
it is on the basis of this visual analysis that the goal of the action
is determined. Perhaps a comparable visual analysis takes placein the case of smiles in which contextual information is combinedwith information gathered from complex visual cues to arrive at aninterpretation of a smile’s meaning.
This is not to say that embodied simulation makes no contribution to our understanding of expressive behaviour. It mightmake it possible for us to respond to a smile warmly and sharein the emotion the smile expresses. In the absence of an embodied simulation, our response to and understanding of anemotion is by comparison “pale, colourless and destitute” to paraphrase William James. The Hennenlotter et al. (2005) study(cited by Niedenthal et al. in sect. 3.2.2) would seem toprovide some support for this suggestion. Subjects prevented
from mimicking expressions of sadness and anger by means ofa BOTOX injection exhibited less limbic system activation thancontrols. Thus, mimicry certainly plays a causal role in generating
emotion; and that may, in turn, affect the character of a person’s
affective understanding. What remains unclear, however, iswhether we understand smiles by sharing another’s feelings, orwhether we can share in another’s feeling only because wehave already understood the meaning behind their smile.

The dynamic interaction of conceptual andembodied knowledge
doi:10.1017/S0140525X10001329
Daniël Lakensa and Kirsten I. Ruysb
aFaculty of Industrial Engineering & Innovation Sciences, Eindhoven Universityof Technology, 5600 MB Eindhoven, The Netherlands; bPsychology
Department, Utrecht University, 3508 TC Utrecht, The Netherlands.
D.Lakens@tue.nl
K.I.Ruys@uu.nl
web.tue.nl/staff/nl/d.lakens
http://www.goallab.nl

Abstract: We propose the SIMS model can be strengthened by detailingthe dynamic interaction between sensorimotor activation and contextual
conceptual information. Rapidly activated evaluations and contextual
knowledge can guide and constrain embodied simulations. In addition,
we stress the potential importance of extending the SIMS model todynamic social interactions that go beyond the passive observer.
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

449

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
In describing the processes that contribute to the meaning of a perceived smile, Niedenthal et al. astutely capture the complex natureof emotion processing. We applaud the approach taken by theauthors in the current article: Embodied perspectives on cognition
are in need of exactly such a detailed investigation of the interaction
between conceptual and embodied knowledge. The question is notwhether, but how sensorimotor processes and cognition interact(e.g., Fodor 1985) The SIMS model illustrates how embodied
responses to facial emotional expressions facilitate recognition ofthese expressions. We believe, however, that a SIMS model that
explicitly addresses the dynamic interaction between conceptualand embodied knowledge would provide an even more useful framework to develop and test specific predictions.
Given that an embodied response to smiles (e.g., motor activation in the facial muscles) emerges around 500– 1000 msec
after observation of a smile (Dimberg & Petterson 2000;
Dimberg et al. 2000), it seems important to integrate more
rapid cognitive processes in the SIMS model. First, before asmile is even observed, contextual information might play anessential role in determining which brain areas are activatedwhen eye contact is established (e.g., OFC activation whentalking to a loved one), and therefore, the type of smile that is
expected. Second, the evaluative connotation of a smile andmore specific conceptual information are activated rapidly at
the moment that a smile is observed (within 120 msec, Ruys &Stapel 2008b; see also Palermo & Rhodes 2007).
Considering that information processing occurs as efficiently
as possible, it seems unlikely for embodied simulations toemerge for the purpose of recognition when associative knowledge structures rapidly enable the recognition of functionalsmiles. Although Niedenthal et al. acknowledge that contextually
driven, rapid processing occurs, they also argue that in many circumstances embodied simulations allow observers to infer thecorrect meaning of ambiguous smiles, when conceptual orstereotypical knowledge would lead to errors of interpretation.
Two possible mechanisms may cause activation to occur of thedifferent brain areas that allow people to distinguish between
affiliative, enjoyment, and dominance smiles. The first possibility
is that these three types of smiles have unique and distinguishable motor patterns. Assuming that an observer correctly simulates the observed smile, different patterns of brain activation
lead to somatosensory activation that matches the observed
smile, which subsequently facilitates the correct recognition of
the smile’s meaning. However, the possibility of unique motor
patterns for these three types of smiles is not what Niedenthal
et al. seem to advocate, and this idea lacks empirical support.
The second possibility is that rapidly activated contextual andconceptual information guides and constrains the embodied
responses that emerge when observing a smile. Indeed,“happy” or “angry” conceptual information has been shown tolead to the activation of different facial muscles when observing
identical faces with ambiguous facial expressions (Halberstadt
et al. 2009). When a baby produces a smile with an unclear
meaning, observers are unlikely to interpret the smile as a dominance smile, given that conceptual knowledge informs us thatbabies do not express dominance smiles. Niedenthal et al.
might argue that in such a situation, where the exact meaningof the smile is ambiguous, conceptual knowledge of smilingbabies will provide the possible meaning of the smile. It is
exactly this interplay between conceptual and embodied information in determining the meaning of smiles that we propose
deserves attention in the SIMS model.
As a further example, take the orbitofrontal cortex (OFC) activation which occurs when perceiving smiles from one’s own offspring (compared to seeing other people’s smiling offspring).
OFC activation may help to distinguish between our own andother people’s offspring and to distinguish between affiliativeand enjoyment smiles. For OFC activation to be able to distinguish between an affilitative and enjoyment smile from ourown children, the OFC needs to be differentially activated450

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

when we see our own children smile different functionalsmiles. Although this is an empirical question, the current literature suggests the OFC is activated when seeing our offspring
display any emotional reaction, such as smiling, but also crying(Noriuchi et al. 2008). We believe the SIMS model needs toincorporate the interaction between contextual conceptual information and embodied simulation to account for how people
differentiate between the different meanings of functional smiles.
What is the function of the embodied simulation of an affiliative smile, when contextual conceptual information and rapid
evaluative processing can often determine the meaning of sucha smile much more efficiently? We believe that a SIMS modelthat goes beyond the passive observer can provide an answer
by examining the role embodied simulations play in the social
dynamics in which smiling behavior typically occurs. The simulation of smiles is not only a motor process that contributes tounderstanding, but also a process that serves a communicative
function (Ruys & Stapel 2008a) involving social cognitive mechanisms (Schilbach et al. 2008).
As described by Niedenthal et al., social contextual variables
can sometimes inhibit the simulation of an observed smile.
Indeed, the automatic imitation or representation of the actionsof others has been shown to be moderated by social information(e.g., Longo & Bertenthal 2009; Ruys & Aarts, in press; vanBaaren et al. 2003). Although speculative, OFC activation andrapid processing of the evaluative connotation of a smile may
together determine whether social inhibition of the motor simulation occurs or not. By determining whether corresponding
somatosensory experiences are produced on a dyadic level,
embodied simulations can contribute to the meaningfulness ofsocial interactions (e.g., Parkinson 1996).
Understanding the meaning of smiles can be further determined by a range of bodily and cognitive factors, such as the observer’s somatosensory experience at the time a smile is observed, or
the observer’s expectations and goals. We believe that understanding is an emergent process, which is guided and constrained by the
dynamic interaction between embodied and conceptual information (see also, Mesquita et al. 2010). Specifying this interaction
should be the focus of future theoretical and empirical work.

Show your teeth or not: The role of the mouthand eyes in smiles and its cross-culturalvariations
doi:10.1017/S0140525X10001263
Chao Liu,a,b Yue Ge,a Wen-Bo Luo,a and Yue-Jia LuoaaState Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal
University, Beijing, 100875, China; bDepartment of Psychology, University ofMichigan, Ann Arbor, MI 48109-1109.
Liuchao@umich.edu
http://www-personal.umich.edu/∼ liuchao/index.html
geyue818@163.com
wenbo9390@sina.com
Luoyj@bnu.edu.cn
http://psychbrain.bnu.edu.cn/teachcms/luoyuejia.htm

Abstract: Previous studies with Westerners have found that both themouth and eyes are crucial in identifying and interpreting smiles. We
proposed that Easterners (e.g., Chinese and Japanese) evaluate the roleof the mouth and eyes in smiles differently from Westerners.
Individuals in collectivistic Eastern society heavily rely on informationfrom the eyes to identify and interpret the meaning of smiles.

A major potential limitation of the SIMS model is, as authors Niedenthal et al. have explicitly stated, “The SIMS model has been
largely developed using data collected in Western countries.
Nevertheless, it is essential to note that cultural differences may
modulate our account” (sect. 6, para. 3, emphasis ours). Herewe echo this proposal and discuss how people in Eastern cultures

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 1 (Liu et al.). Top: Ancient Chinese women used fake dimples to embellish their closed-mouth smiles (Xinhuanet 2008).
Bottom: Chinese participants who focused on the eyes in a real versus fake smiles identification task showed a negative correlation
between accuracy and individualism scores, but a positive correlation for collectivism scores.
differently evaluate the role of the mouth and eyes in smiles, andhow such variations might modulate the SIMS model.
In the target article, the authors concentrate on the eyes and eye
contact; however, in Western cultures, the mouth is as crucial as
the eyes in identifying and interpreting facial expressions,especially happiness. The most unequivocal evidence comes frompatient S.M., who has severely impaired detection of anger andfear after losing the bilateral amygdala, but still maintains anentirely normal identification of happiness (Adolphs & Tranel
2000; Adolphs et al. 1994). This dissociation is due to his inability
to use information from the eyes but an intact ability to use information from the mouth (Adolphs et al. 2005). It therefore provides
direct evidence that the mouth alone could supply sufficient information for the recognition of happiness and smiles, which has also
been confirmed by the computational analysis of FACS-coded
faces (Smith et al. 2005). The second vein of evidence derives
from facial mimicry and the mirror neuron system. Facial
mimicry is critical in constructing embodied simulations forsmiles, and thus plays an important role in the SIMS model, yet
the majority of studies on facial mimicry manipulate only themouth and related zygomatic muscles (Blairy et al. 1999; Maringer
et al., in press). In addition, mimicry and embodied simulation havebeen closely linked to the mirror neuron systems (Keysers &Gazzola 2007), whereas studies of the mirror neuron system areessentially focused on actions from the mouth and hands, such as
drinking, eating, and grabbing (Iacoboni et al. 1999; 2005). Thus,
it is not surprising that in the SIMS model, the mouth is the apt
alternative of the eyes for observers in the social inhibition condition (see Figures 5 and 6 of the target article).
However, so far most data have been collected from Western cultures. There is evidence that Easterners evaluate the role of themouth and eyes differently from Westerners. For example, “doNOT show ones’ teeth when smiling” was a strict rule of discipline
for ancient Chinese women, who even used adornments (e.g., fake
dimples) to make up for the scarcity of emotional information conveyed by the mouth during their closed-mouth smiles (see ourFig.1, Top). Another example is the smile emoticons used by Easterners and Westerners. While Westerner use emoticons exaggerating the mouth with a crimped line and simplifying the eyes as two
dots, e.g., :-) or :), Japanese use emoticons with simplified mouth
but crimped eyes, e.g., (^.^) or (^_^) (Jack et al. 2009; Pollack
1996; Yuki et al. 2007). Chinese, especially females, go evenfurther by evoking the ancient tradition of attaching fake dimples,for example, ( ^_^ ) (Marshall 2003).
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

451

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Such a cross-cultural difference has been replicated in experimental research. In two studies using emotional expressions inemoticons or computer-edited photographs of real faces, Yuki
et al (2007) compared the difference between Japanese and Americans in weighing facial cues when interpreting emotional
expressions. Results showed that Americans weighed expression
cues displayed in the mouth more when judging emotions,whereas Japanese tended to weigh expression cues in the eyesheavily. In a recent eye-tracking study investigating the decodingof facial expression signals in a facial expression categorizationtask, Jack et al (2009) found that Easterners and Westerners
adopt different decoding strategies when reading others’ facial
expressions. Westerners distributed their fixations evenly acrossthe face, whereas Easterners systematically biased their fixations
toward the eyes and ignored the mouth. In our study (Liu et al.,under review), we found that when asking Chinese speakers tojudge the Duchenne and non-Duchenne smiles as either real or
fake (Bernstein et al. 2008), those who voluntarily stated theeyes to be the most useful source of information are more accurate(mean ¼ 71.11 + 12.31%) than those who preferred the mouth(mean ¼ 62.89 + 11.34%), p , .05. More interestingly, the accuracy of participants preferring the eyes is negatively correlatedwith individualism scores, but positively correlated with collectivism scores (Fig.1, Bottom), indicating that individuals in a collectivist society heavily rely on information from the eyes to identify
and interpret others’ facial expressions and social intentions.
How could these Eastern culture data modulate the SIMS
model? We propose two possible modifications. First, how do
Easterners, as highly dependent on the information from theeyes, cope with the social inhibition situation when eye contact
is not available? One interpretation for these cross-cultural differences is that Easterners are good at regulating facial expressions
because of the restriction of expressing individual emotion in
public, especially for those expressions originating from themouth, because it is much easier to control than the eyes. Consequently, Easterners usually do not trust what they read fromothers’ mouths (Yuki et al. 2007). If this were true, we might
expect to find that Easterners could search for supplemental information from other relatively uncontrollable sources (e.g., bodylanguage) when the eyes are unavailable, rather than still relying
on those controllable sources (e.g., mouth) as Westerners do,
according to the SIMS model. Second, how could the systematic
underrating of the mouth in reading and decoding others’ facial
expressions influence Easterners’ facial mimicry ability andmirror neuron system? Would an Eastern “patient S.M.” lose
the ability to recognize happiness because of an inability to
decode the eyes? Future cross-cultural neuroimaging studies ofthe differences in positive emotions thus are very promising.
ACKNOWLEDGMENTS
This work was supported by the NSF China (30930031); the Ministry of
Education, China (PCSIRT, IRT0710); National Key Technologies R&D
Program (2009BAI77B01); and by the Global Research Initiative
Program, NIH, USA (1R01TW007897) to Yue-Jia Luo.

Motivational aspects of recognizing a smile
doi:10.1017/S0140525X10001330
Janek S. Lobmaiera and Martin H. FischerbaDepartment of Psychology, University of Bern, CH-3012 Bern, Switzerland;
School of Psychology, University of Dundee, Dundee, DD1 4HN, Scotland,
United Kingdom.
janek.lobmaier@psy.unibe.chm.h.fischer@dundee.ac.uk
http://www.kog.psy.unibe.ch/content/ueber_uns/lobmaier/index_ger.html
http://www.dundee.ac.uk/psychology/people/academics/mhfischer/b

452

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Abstract: What are the underlying processes that enable human beings
to recognize a happy face? Clearly, featural and configural cues will
help to identify the distinctive smile. In addition, the motivational stateof the observer will influence the interpretation of emotional
expressions. Therefore, a model accounting for emotion recognition is
only complete if bottom-up and top-down aspects are integrated.

Most of the research on facial expression has focused on negative
emotions, such as anger, fear, or disgust (e.g., Adams et al. 2003;
Mathews et al. 2003; Putman et al. 2006), while positive emotions
have received relatively less attention. The attempt to examine
the nature of smiles by Niedenthal et al. is therefore highly
welcome. To assist with further development of SIMS, we highlight its omission of observer bias and the importance of measuring the time course of motor resonance.
Niedenthal et al.’s proposal as to how a smile may be processedby an observer is rather one-sided. Importantly, they do not take
into account the observer’s motives and suppositions. As theauthors point out, analysis of the facial features may not always
be sufficient to classify realistic emotional expressions, and thenthe perceiver must call on alternative sources of information,such as the social situation in which the expression occurs.
More than that, the perceiver will evaluate and interpret theexpression on the basis of his or her emotional state anddesires. Specifically, if an observer is in a happy mood, then an
emotion expressed by another person will be interpreted differently compared to when the observer is sad or angry.
According to the affect-as-information theory (e.g., Schwarz1990), mood is used to make social judgments. When a person
is in a certain mood, he or she may inadvertently attribute thatmood state to other people, which in turn will influence theinterpretation of an observed facial expression. In fact, Niedenthal herself (Niedenthal et al. 2000) found an effect ofemotional state in a task where participants had to detect at
which frame of a movie a facial expression (e.g., happiness)became neutral. When the participant’s mood was congruent
with the initially displayed emotion, this expression was perceived to persist longer than when participant mood and initial
stimulus expression were incongruent. Hence, a happy person
will more readily interpret emotion is not too extreme).a given
emotional display as happy (at least as long as the displayed
In the 1950’s psychologists argued that what we perceive is influenced by our motivations or “set” (the so-called “New Look”). For
example, the size of more valuable coins was overestimated, andhungry participants were quicker than satiated participants to
identify food-related words (for critical discussion, see Erdelyi1974). Applying this insight to our point about observer bias,
basic motivational tendencies, such as what we wish to see,might also influence the recognition of smiles. Supporting thisclaim, we recently found a self-referential positivity bias when
judging where another person is looking (Lobmaier & Perrett, inpress; Lobmaier et al. 2008). People expressing a happy emotion
were much more likely to be interpreted as attending to the observer than were people with an angry or sad expression. It is beneficial for our self-esteem to assume that we are the reason forsomebody else’s happiness. Thus, in the context of the SIMS
model, expressions may be interpreted as more positive, justbecause this is what we wish the expression to be: not only will
we perceive a smile as directed to us, we might also interpret a
facial expression to be more positive than it really is.
The SIMS model explains recognition of smiles as a mainly
stimulus-driven process. This approach is in contrast withvarious current models of emotion recognition. For example,
the appraisal model of emotion (e.g., Scherer 2001) postulates
that stimuli undergo evaluation checks on the basis of various
dimensions: novelty (is a stimulus novel, predictable?), intrinsic
pleasantness (positive or negative evaluation), goal significance(is it relevant to pursue my goal?), coping potential (can I deal
with it?), and compatibility with social or personal standards (isit acceptable for me and others?). In the context of happy
emotions, this means that (unconscious) cognitive evaluation ofCommentary/Niedenthal et al.: The Simulation of Smiles (SIMS) modela seen smile will accompany any simulation of smiles. Such processes are in stark contrast with Niedenthal et al.’s bottom-up
view of emotional resonance.
Nevertheless, the SIMS model might be a suitable frameworkfor further exploring the interplay of top-down and bottom-up
factors in the categorization of emotions because it raises theissue of strategic influences and their timing. The fact that it
takes half a second (Dimberg & Thunberg 1998; cited in sect.
3.2.1 of the target article) or even longer (Bavelas et al. 1986;
cited in sect. 4.2.1) before the postulated motor resonance mechanism leads to facial mimicry is noteworthy in the light of much
faster motor resonance effects in language comprehension (for a
recent review, see Fischer & Zwaan 2008). Contrary to Niedenthal et al.’s claim (in sect. 4), the embodied or simulation
view of language comprehension postulates an obligatory simulation process, such that even abstract concepts are understood
through metaphorical grounding (Barsalou 2008). Further
chronometric studies of comprehension processes that combine
methods from different fields are needed to further refine theproposed resonance mechanism.
We argue that a model attempting to account for recognition ofsmiles is only complete if stimulus-driven bottom-up processes areintegrated with endogenous top-down processes. Indeed, thehappy expression will be simulated, but we might expect that both
the rapid simulation and the subsequent interpretation of thesmile will be monitored by more frontal regions such as the medial
prefrontal cortex (cf. Keysers & Gazzola 2006). To classify a smile
only by simulation would assume that a certain smile will mean thesame thing in every social situation. This is not the case. In some situations, we, as observers, will reflect on the meaning of the smile justas we reflect on our own state and will consequently interpret themeaning of the smile based on more than just simulation. Thus, interpreting a smile goes far beyond using simulation alone.

Grounding the meaning of non-prototypicalsmiles on motor behavior
doi:10.1017/S0140525X1000155X
Timothy A. Mann and Yoonsuck Choe
Department of Computer Science and Engineering, Texas A&M University,
College Station, TX 77843-3112.
mann23@tamu.edu
choe@tamu.edu
http://faculty.cse.tamu.edu/choe/

Abstract: We address how the motor system can contribute to the“meaning” component of smile perception. A smile perceiver canground the meaning of non-prototypical smiles by interacting with thepresenter to maintain the presenter’s type of smile. In this case, themeaning of that smile is congruent with the motor behavior that elicits
that smile (such as a funny gesture).

Niedenthal et al. define embodied simulation as “a simulation of astate in the motor, somatosensory, affective, and reward systems
that represents the meaning of the expression to the perceiver”(sect. 1, para. 4, italics in the original). In this sense, the state of
the sensory-motor system classifies the smile (“represents themeaning”), but the meaning of the classification to the perceivermust also be grounded. Although prototypical facial expressions
may be grounded in intrinsic reward systems, the subtleties of
non-prototypical smiles likely must be learned and, therefore,
may be grounded in the socio-behavioral context surrounding
the smile: Smiles are not presented or perceived in isolation;
they are surrounded by rich socio-behavioral context. For
example, it is difficult to consider the meaning of dominancesmiles without reference to more than facial expressions.
Like other forms of communication, smiles can have profoundindirect consequences. For example, Sharlemann et al. (2001)

investigated how smiles can be used to negotiate cooperation
between humans. Smiles (and, more generally, facial expressions)provide clues about the values and objectives of other agents.
Because smiles have only negligible direct physical impact, themeaning of a smile cannot be fully realized by considering only
facial sensory-motor systems. Instead, we propose that an important component of the meaning of a smile must be grounded inhigh-level behaviors. But this raises a question: How can themeanings of smiles be grounded in high-level behaviors?
Choe and Smith (2006) present a framework that explains theprocess of learning the meanings of internal sensory states bygrounding them in terms of specific patterns of motor action,actions that maintain invariance in the internal sensory state.
Our Figure 1 shows the framework, and Figure 2 shows how
invariance in internal state can link up with motor-based
meaning. The action that maintains invariance in the agent’s
internal sensory state (e.g., 458 movement in Figure 2) exactly corresponds to the meaning conveyed by the encoded internal state.
In Choe and Smith (2006), a visuomotor learning system waspresented with the problem of grounding a set of visual stateson its motor system, when the meaning of the visual states andmotor primitives were initially unknown. A useful criterion is
maintaining sensory invariance while acting. The sensory invariance criterion defines a probabilistic mapping from sensory
states to motor primitives. In this experiment, some motor primitives maintained sensory invariance (e.g., by moving the agent
parallel with the current sensory pattern), while other motor primitives violated sensory invariance (e.g., by moving perpendicular
to the current sensory pattern). The agent learned to maintainsensory invariance while viewing a scene (see Fig. 2), which
resulted in actions that moved the agent’s visual field parallel tothe current visual stimulus. The learned meaning of eachsensory state was the set of motor primitives that resulted in maintaining that sensory state. Learning was accomplished with a standard reinforcement learning algorithm, and produced a mappingfrom visual states to congruent motor primitives (see Fig. 2). Thispairing between sensory and motor primitives grounds sensory
representations using action. The meaning of a sensory state isgiven by the actions that maintain it, and this meaning can belearned through interactions with the environment.
This approach can be extended to external sensory stimuli,
such as the smiling face of a conspecific. When viewing a nonprototypical smile, a perceiver (visuomotor learner) can interact
with the smile presenter, with the objective of maintaining thetype of smile on the smile presenter’s face. For example, whena perceiver sees an enjoyment smile, the objective is to maintainthe enjoyment smile (i.e., sensory invariance). By interacting withthe presenter (e.g., telling a joke, duplicating the perceived smile,or frowning), the perceiver can learn a mapping from the particular smile and his/her own motor behavior that are likely to maintain the smile type. Like the mapping shown in Figure 2, themeaning of a non-prototypical smile can be grounded by theactions (of the perceiver) that maintain it (in the presenter).
Unlike the computational experiment from Choe and Smith(2006), where the initial mapping is randomly generated, somemeaningful structure for smile interpretation may already be
innate or learned from demonstrations of other people interacting
in response to smiles. For example, prototypical smiles may beinnately grounded, increasing likelihood of certain interactions
when those smiles are observed. This prior knowledge about therange of interactions with a smile presenter could help reduce thesearch space needed to learn the meaning of non-prototypical smiles.
While the target article explains that the meanings of smiles
are grounded in embodied cognition, the authors are not specificabout which sensory-motor systems ground the meanings ofsmiles. We propose that, although some meaning can be captured by the facial sensory-motor system, important aspects ofsmile meaning cannot be captured without grounding smiles
with respect to high-level actions making up the behavioral
context. We have also described how a simple criterion (i.e.,
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

453

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 1 (Mann and Choe).

Visuomotor agent – its internal sensory state and motor primitives. Adapted from Choe and Smith (2006).

Figure 2 (Mann and Choe). Relationship between encoded sensory property and motor primitive giving internal state invariance.
Adapted from Choe et al. (2007).

maintaining sensory invariance) can be used to ground the meanings of smiles on high-level actions.

The proximate mechanisms and ultimate
functions of smiles
doi:10.1017/S0140525X10001561
Marc Mehua and Karim N’Diayeba
Swiss Center for Affective Sciences, University of Geneva, CH-1205 Geneva,
Switzerland; bResearch Center of the Brain and Spine Institute, Université
Pierre et Marie Curie (UPMC), CNRS UMR 7225, INSERM UMR S 975, 75006
Paris, France.
marc.mehu@unige.ch
http://www.affective-sciences.org
karim.ndiaye@upmc.fr

454

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Abstract: Niedenthal et al’s classification of smiles erroneously conflates
psychological mechanisms and adaptive functions. This confusion
weakens the rationale behind the types of smiles they chose to
individuate, and it obfuscates the distinction between thecommunicative versus denotative nature of smiles and the role ofperceived-gaze direction in emotion recognition.

Although we commend Niedenthal et al. for their effort to bring
together the ethological functions of smiling and the neurocognitive mechanisms involved in its perception, we remain sceptical
about their success, for the model they propose rests on a classification of smiles that is inconsistent with primate ethology andevolutionary theory.
Our reading of the ethological literature leads us to question
the three types of smiles that Niedenthal et al. individuate (i.e.,enjoyment, affiliative, and dominance smiles). In particular, thecase made for the dominance smile category is based on anecdotal evidence and post hoc rationalization of disparate research.

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
A better coverage of the abundant literature on the relationshipbetween smiling and social status would have made a stronger
argument in favor of, or against, a “dominance” smile category.
Although dominant individuals have also been observed smiling
during interactions with subordinates (LaFrance & Hecht
1999), smiling has often been associated with submission (Hall2002; Schmid-Mast & Hall 2004). It is surprising to see in thistarget article a category of dominance smiles while the psychological and ethological literature could equally well support arguments in favor of a submission smile category (e.g., Mehu &Dunbar 2008), which would also be distinct from the affiliative
smile that Niedenthal et al. define. Finally, we doubt thatsmiling functions to establish dominance. The ethological literature rather suggests that smiling mitigates the deleterious effectdominance may have on the maintenance of egalitarian relationships (Flack & de Waal 2007; Preuschoft & van Hooff 1997).
A major asset of the target article is to propose a classificationof smiles based on their function, rather than one based on theirmorphological features (sect. 2.2). However, from an evolutionary viewpoint, this functional typology is problematic because it
mixes proximate and ultimate explanations of behavior (Tinbergen 1963). For example, the enjoyment smile is a category
based on one of the proximate factors involved in smile production (i.e., positive emotion), whereas the “affiliative” and“dominance” smile categories reflect the ultimate function ofsmiles. As these criteria pertain to two different levels of explanation, they are not exclusive of each other. Indeed, research
has shown that Duchenne smiles (also called enjoyment smiles)
could be reliable signals of cooperative (and therefore affiliative)
intentions; the honesty of the signal would be ensured by theemotional component that is difficult to strategically manipulate(Brown et al. 2003, Mehu et al. 2007). The alternative view thatenjoyment smiles are merely cues, that is, symptoms of positive
emotional experience without communicative function (Ekman
1997), could not be invoked to justify the distinction between
enjoyment smiles and affiliative smiles for the reason that cues
are not functionally related to a receiver’s response (Hasson
1994) and are, therefore, difficult to integrate within thesender-receiver process that Niedenthal et al. describe.
The confusion of proximal and ultimate factors leads to misunderstanding the crucial issue about the nature of smiles as communicative (smiles as social signals) versus denotative (smiles as
symptoms of emotions) facial expressions. This is also illustrated
in the case of dominance smiles in which Niedenthal et al. conflate the bodily expression of a positive feeling of superiority
and the communicative intentions directed at social challengers.
This conflation bears direct consequences on the embodied
simulation processes that Niedenthal et al. claim to be central
for interpreting the meaning of smiles. For example, theirmodel could with difficulty explain how the perceiver of a dominance smile experiences negative feelings that have no obvious
correspondence with the feelings actually experienced by thesmiler, whereas alternative accounts regarding the functionalrole of such smiles (e.g., defusing social conflict) escape thiscontradiction.
We believe that an appropriate classification of smiles should
be based on EITHER proximate causes (smiles with different
motivational basis) OR ultimate causes (smiles with different
adaptive functions). In the latter case, the classification is morelaborious because the function of smiling is poorly documented
in the literature (a situation that may explain Niedenthal et al.’sdifficulty in achieving a satisfactory functional classification ofsmiles). Alternatively, a classification based on morphologicaland dynamic aspects of smiles may be useful (e.g., Duchenne
vs. non-Duchenne). Such a categorization would be more relevant to the SIMS model, since the processing of a smile’s physical features is central to embodied simulation (see sect. 3.2 of thetarget article).
Because gaze represents an important vector of communicative intentions, the overlook of communicative/denotative

distinction also weakens the theory regarding the role of eye
contact. According to Niedenthal et al., eye contact would be a
sufficient, but not necessary, trigger of the embodied-simulation
process: When eye contact occurs, it triggers simulation of thecurrent expression leading to embodied simulation and a better
understanding of smiles’ meaning. On the other hand, wheneye contact is obstructed, the simulation is based on conceptual
knowledge and is more prone to error (sect. 5, para. 5 and sect.
6.2, para. 4). This implies that a simulation triggered by eye
contact is more likely to result in accurate interpretations ofsmiles, making eye contact slightly more critical (if not necessary)
than presented by the authors.
We are therefore surprised that Niedenthal et al. make little
reference to the rich literature on eye contact and gaze direction
on emotion recognition. Various studies on the effect of gaze
direction on emotion recognition have shown that eye contact
may indeed increase perceived emotion (relatively to avertedgaze), but it may also decrease it, depending on the nature of
the emotional expression (see Bindemann et al. [2008] forrecent research and references therein). These data hardly fit
with a general enhancing role of eye contact in emotion
perception.
We propose that Niedenthal et al. consider an alternative
account based on the functional role of eye contact according
to which the effect of eye direction on emotion perception
reflects the congruency of the sender’s gaze direction with therelevance of the emotional signal for the receiver: Angry (andhappy) faces are more relevant to the receiver when she isgazed at (i.e., direct gaze), while fearful faces with averted gazemay signal a relevant threat for the receiver (Sander et al.
2007). This view implies that the role of eye contact occurs at alater stage of facial expression recognition than that posited byNiedenthal et al. Instead of having a ubiquitous enhancing
effect, eye contact would thus increase or decrease the perceived
intensity of the emotional display, depending on its relevance forthe observer. Therefore, the involvement of brain structures suchas the amygdala need not reflect the uncertainty associated withsmiles (as Niedenthal et al. presuppose; see sect. 6.1.1, para. 1)
but, instead, the self-relevance of emotional expressions intheir context of occurrence (N’Diaye 2009).

Conceptual knowledge: Grounded insensorimotor states, or a disembodied deus
ex machina?
doi:10.1017/S0140525X10001573
Ezequiel Morsella,a,c Carlos Montemayor,b Jason Hubbard,aand Pareezad ZaroliaaaDepartment of Psychology; San Francisco State University, San Francisco,
CA 94132-4168; bDepartment of Philosophy, San Francisco State University,
San Francisco, CA 94132-4168; cDepartment of Neurology, University ofCalifornia, San Francisco, CA 94143.
morsella@sfsu.edu
cmontema@sfsu.edu
jashubbard@gmail.com
pzarolia@gmail.com
http://bss.sfsu.edu/emorsella/

Abstract: If embodied models no longer address the symbol grounding
problem and a “disembodied” conceptual system can step in andresolve categorizations when embodied simulations fail, then perhaps
the next step in theory-building is to isolate the unique contributions ofembodied simulation. What is a disembodied conceptual system
incapable of doing with respect to semantic processing or the
categorization of smiles?

One challenge encountered when reverse engineering the brain
is that the same function (e.g., categorizing an enjoyment smile)
can be carried out by vastly different mechanisms (e.g., visual
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6455

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) modelfeature analysis, conceptual processing, or embodied simulation).
This gives the brain tremendous flexibility and the ability to compensate for failed mechanisms. To the detriment of the scientist,it also makes it difficult to experimentally isolate the mechanism(s) underlying a given function (e.g., the categorization ofsmiles) and identify the conditions under which a given function
will be carried out by one mechanism versus another. With this inmind, Niedenthal et al. should be commended for their trailblazing contribution that specifies both the conditions under which
embodied simulation is likely to occur (e.g., following eye
contact) and the neural-physiological components of this intricate and counterintuitive mechanism.
When appreciating such an advancement, it is worth keeping
in mind that the primary explanatory value of embodied
approaches to knowledge representation (e.g., Barsalou 1999;
Glenberg 1997) rested in their ability to address the symbol
grounding problem (Harnad 1990) faced by “propositional”(aka “amodal”) accounts of mental representation (e.g., Landauer& Dumais 1997), in which meaning is represented by arbitrary
symbols (e.g., binary digits) that do not retain any of the properties of the sensorimotor states that gave rise to them (see review
in Markman & Dietrich 2000). Similar to “analogical” and“modal” accounts, embodied models propose that meaning isconstituted by (or “grounded in”) sensorimotor information,such as that furnished by facial expressions. Since their advent,embodied models have been challenged on empirical grounds.
Regarding emotion, for example, an extensive study revealed
that people with bilateral facial paralysis stemming from
Moebius syndrome can categorize the facial expressions ofothers without difficulty (Bogart & Matsumoto 2010). Moreover,
the ability of subjects to express facial expressions does not influence accuracy. Similarly, in semantics, it remains unclear
whether lesions of action-related regions of the brain lead to
impairments regarding the semantic-conceptual knowledge ofaction: Patients suffering from deficits due to brain lesions canshow impairments in the way they use certain objects but, revealing the abilities of a “disembodied” conceptual system, can nonetheless name those objects, or even recognize the pantomimes
associated with them (Mahon & Caramazza 2008).
In response to such challenges, embodied approaches now
propose that sensorimotor information plays a more humble
role: In the target article, Niedenthal et al. portray sensorimotor
information not as that which constitutes meaning, but merely as
auxiliary information that is “referred to” when semantic processing is challenged, as when categorizations are difficult to make.
In these accounts, sensorimotor information is demoted to a kindof supplementary information that, just like any kind of information, can at times facilitate or interfere with semantic processing. When the strategy of embodied simulation fails, it is theconceptual system that arrives at the scene to solve the categorization, much like a deus ex machina.
The question now is whether the conceptual system is always
necessary for categorization. From one standpoint, the answer
is yes: Until a token facial expression is typed as a member of agiven category (an inherently conceptual process), the organism
cannot know (consciously or unconsciously) the meaning of theobserved expression. If the conceptual system must always beat the scene, then what is its nature and how does it solve thesymbol grounding problem? Embodied approaches were
expected to answer this by demonstrating, for example, that disrupting the sensorimotor states in which a conceptual process isgrounded will not just modulate the process, but also eradicate it.
It seems that, apart from modulations that could be construed aspriming or its opposite (i.e., interference), there is no evidence ofconceptual processing requiring sensorimotor states. Apart fromthe aforementioned neurological evidence, this is evident withrespect to abstract concepts (e.g., that of if and justice) and everyday phenomena such as imageless thought (e.g., a state of uncertainty or readiness; cf. Hilgard 1987). Regarding imagelessthought, Woodworth (1915) concluded, “I know of no reason in

456

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

neurology or psychology for supposing that the elements of conscious content are contributed solely by the sensory receiving
centers” (p. 137).
A way forward? Identifying the limitations of “disembodied”
processing. Perhaps a critical next step in theory-building for

embodied approaches is to identify the unique contribution(s)of embodied simulation. What is it that a disembodied conceptual system is incapable of doing with respect to semantic processing or the categorization of smiles? Emotion research has
broached this topic by revealing some limits of language, aform of propositional processing. Evidence suggests that linguistic processing does not lead to the same kinds of emotional effects
and forms of affective learning as modal stimuli (LeDoux 1996).
Olsson and Phelps (2004) demonstrated that, through vicarious
classical conditioning, subjects can acquire a learned fear
response toward a subliminal stimulus by perceiving someone
else receive a shock after being presented with that stimulus.
However, simply telling subjects about the contingency
between stimuli did not lead to this form of vicarious conditioning to subliminal stimuli. This suggests that the phylogeneticallyold systems mediating fear conditioning may not understandlanguage, though they can process the meaning of basic perceptual events: “classical conditioning and observational learning . . .
might be supported by an evolutionarily old system that predates
the emergence of language” (Olsson & Phelps 2007, p. 1099).
That language fails to yield the vicarious conditioning affordedby observational learning may be because language comprehension yields “faint” activations of analogical, sensorimotor-likesymbols (Boroditsky & Ramscar 2002; Zwaan 2008) or, at odds
with embodied accounts, because linguistic-semantic knowledge
is simply distinct from sensorimotor knowledge. Regarding thepioneering investigation by Niedenthal et al., the next advancement may be to isolate that which only embodied simulations
can achieve. It may be that aspects of social mimicry andempathy require embodied simulation, or that the three smilecategorization mechanisms operate differently with respect tosubliminal or supraliminal stimuli, with the conceptual deus ex
machina being capable of arriving at the scene only when thetheater lights of consciousness are on.

What’s behind the smile?
doi:10.1017/S0140525X10001585
John J. Ohala
Department of Linguistics, University of California, Berkeley, CA 94720-2650.
ohala@berkeley.edu
http://linguistics.berkeley.edu/phonlab/users/ohala/index3.html

Abstract: Many species’ non-threat facial expression involves an open
mouth and retracted lip corners – the smile. This served to make anaccompanying vocalization sound like it originated from a smaller
vocalizer. That such signals are deceptive and benefit primarily thesignaler undermines the notion that the perception of the smile
employs embodied simulation of the smiler’s state.

The nature and function of the smile – indeed, of any facial
expression – is of interest not only in psychology and the neurosciences, but also in ethology, which is the study of behavior,especially that exhibited in similar form cross-species andwhich can be explained by Darwinian natural selection (e.g.,Andrew 1963; Bolwig 1964; Eibl-Eibesfeldt 1971).
For example, Morton (1977) showed that avian and mammalian threat and non-threat (appeasement, submission) vocalizations use fundamental frequency (F0) in remarkably similar
ways: threat vocalizations have low F0, whereas non-threat vocalizations have high F0. A dog’s low-pitched threatening growl andits high-pitched submissive yelp are familiar examples. Morton

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
argued that the modification of F0 conveys to the perceiver an
estimate of the body size or maturity of the vocalizer and thus
the degree of threat because, other things being equal, bodysize and maturity are correlated with the mass of the vibrating
membrane (vocal cords or syringeal membranes). These deceptions benefit both parties by averting injury (or worse) resulting
from actual combat.
I demonstrated that this “frequency code” applies to humanspeech as well: the degree of perceived dominance is inversely
correlated with voice F0 (Ohala 1984). Furthermore, thisaccounts for well-documented common cross-linguistic pattern(some would say “universal”) that questions, uncertainty, andpleading are marked by a high F0 somewhere in the utterance(when these intentions are not indicated by lexical, morphological, or syntactic means). In addition, I hypothesized that the frequency code helps to resolves a long-standing puzzle about themouth shapes accompanying threat and non-threat displays in
species having plastic (i.e., movable) facial masks: canids and primates, including humans. An open-mouth and retracted lip
corners is found in non-threat displays, as opposed to more protruded, rounded lip shape in threat displays. Why in non-threat
should more teeth – potential weapons – be displayed in contrast to fewer teeth showing in a threat display (Izard 1971)?
The frequency code explains these dichotomous facial
expressions, too: These different lip configurations will modify
the resulting resonances of an accompanying vocalization in a
way to convey to the perceiver an impression of a vocalizer
with shorter or longer vocal tract This, I argue, is the origin of
the smile. Via the well-known process of ritualization, the smile
can be given without an accompanying vocalization or evenwith a closed mouth. This analysis meets the condition of coherence and generality in ethological explanations advocated byDarwin (1872, p.18): “whether the same principle by which
one expression can, as it appears, be explained, is applicable inother allied cases; and especially, whether the same general principles can be applied with satisfactory results, both to man andthe lower animals.” Thus, consider a case most of us are familiar
with: a submissive dog. The vocalization has a high F0, high resonances (due to mouth corner retraction), ears flattened, no bristling of the hair on the back, tail between the legs, crouching
posture – the latter elements all designed to make the animal
seem as small as possible. A dog’s threat display is just the
opposite.
That modification of the vocal anatomy is important in conveying an impression of size is reinforced by the fact that, besides theplastic modification of the vocal structures, there are implastic
modifications evident in the sexual dimorphism in the adultvocal tract. The mature human male has a longer vocal tract
than the female (by some 15 to 17%) – this is accomplished by
the familiar descent of the larynx starting at puberty – and the
male’s vocal cords become fully 50% longer – but probably
100% more massive than the female’s. The first modificationaccounts for the quantitatively different resonances of the“same” vowels as spoken by adult males and females. The lattermodification accounts for the typical one octave difference inmales’ and females’ average speaking F0. The longer vocal
cords, by the way, are bought at the cost of enlargement of thelaryngeal cartilages in the male, resulting in a prominent“Adam’s apple.”
Numerous other species – from whooping cranes, geese, andthe howler monkey – show sexually dimorphic aspects of the
vocal anatomy which testify to the importance of vocalization in
agonistic encounters, in sexual selection, and so on. Thus,
enhancement of various aspects of adult males, human and nonhuman, gives rise to such sexually dimorphic aspects as thehuman male’s beard, the male lion’s mane, and the size dimorphism between male and female elephant seals, and the like. Size
does matter.
What does all this imply for the target article by Niedenthal
et al.? Just this: If, as I have argued, a smile is intended by thesmiler to induce a reaction or response in the perceiver that isfavorable to the smiler, then the authors’ contention that “observers of smiles sometimes construct an embodied simulation of
the nuanced affective state conveyed by the smile” (introduction,
para. 3) is undermined. My view is similar to that espoused by the“functionalist” approach to the analysis of the outwardly visible
states – call them “emotions” if one will (Campos et al. 1994).
One must differentiate between (a) inner neurophysiologic
states – which, to further the interests of the one experiencing
these states, often would be best kept secret – although insome cases uncontrollable cues to such states may inadvertently“leak out,” for example., sweating, tremor, blushing, sexual
arousal; and (b) signals such as smiles, frowns, threatening
looks, and vocalizations, which are made in order to induce a targeted reaction/response in the perceiver – that is, a response
favorable to the one emitting the signal. Ethology teaches us:stereotyped behavior – and this includes signaling – is generally
done to benefit the individual or group exhibiting it. Such signals
are not, strictly speaking, cooperative.

What’s embodied in a smile?
doi:10.1017/S0140525X10001597
Disa A. Sauter and Stephen C. Levinson
Max Planck Institute for Psycholinguistics, 6500AH Nijmegen,
The Netherlands.
disa.sauter@mpi.nl
stephen.levinson@mpi.nl
http://www.mpi.nl/people/sauter-disa
http://www.mpi.nl/people/levinson-stephen

Abstract: Differentiation of the forms and functions of different smiles isneeded, but they should be based on the empirical data on distinctions
that senders and receivers make and on the physical cues that are
employed. Such data would allow for a test of whether smiles can bedifferentiated using perceptual cues alone or whether mimicry or
simulation are necessary.

The target article proposes an ambitious model of the perceptionof facial signals, specifically the interpretation of smiling
expressions of amusement, dominance, and appeasement. Niedenthal et al. rightly draw attention to the need for differentiationin the forms and functions of different types of smiles, but thedistinctions the authors propose lack support from empiricalfindings, as they themselves concede. This limitation undermines
the proposed model. We propose that firm foundations for amodel of the perception of emotional signals should be basedon the distinctions that senders and receivers make, the cuesthat are actually employed, and the correspondence between
physical cues and subjective experience.
One study has specifically investigated the perception of thethree types of smiles that Niedenthal et al. suggest (Hess et al.
2002). When asked to pair facial expressions with vignettes
describing dominance, appeasement, and amusement scenarios,
Canadians of European extraction associated the amused vignettes with strong Duchenne smiles, but no difference wasfound between the smiles that were selected for the dominanceand appeasement scenarios. For both dominance and appeasement vignettes, responses were distributed across a range of
weak- and medium-intensity smiles. These findings demonstratethat amusement is associated with a smile expression, a findingthat has recently been replicated cross-culturally and extended
to the auditory domain (Sauter, in press; Sauter et al. 2010).
However, Hess et al.’s (2002) results suggest that viewers do
not reliably differentiate between dominance and appeasement
smiles, or, alternatively, that the facial expressions associatedwith these states are not well established. Niedenthal et al.
base their model on the three functionally defined smile
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

457

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
categories – amused, dominant, and appeasement smiles – but
as we have illustrated, it is not established that viewers can differentiate between these three expressions.
There seem, however, to be other distinctions in positive
emotion expressions for which there is better empirical grounding. Recently, a growing number of studies have started to distinguish between signals of a range of positive emotions (seeSauter 2010), providing evidence for smile categories otherthan those suggested by Niedenthal et al. For example, in a production study of posed positive emotion displays, Shiota et al.
(2003) found that amusement was expressed via open-mouthedsmiles, whereas pride was associated with smiles with compressed lips. This finding suggests that displays of some positive
affective states are signalled by physically distinct smile configurations, although it did not establish whether viewers are sensitive
to these cues. A recent study investigated both the production
and perception of spontaneous amused, embarrassed, nervous,and polite smiles, by investigating physical cues and human judgments (Ambadar et al. 2009). Ambadar et al. showed that viewers
use variation in morphological and dynamic characteristics ofdifferent kinds of smiles. For example, in comparison withsmiles perceived to signal politeness, smiles that were perceived
by viewers as amused more often included open mouth, larger
smile amplitude, larger maximum onset and offset velocity, andlonger duration. This demonstrated that viewers’ judgments
were directly related to the physical cues that differentiatedbetween these expressions. Together, these studies suggest that
distinct physical cues are associated with smiles signalling different positive emotions, and that viewers are sensitive to thesecues, but the categories are not coincident with those employedin the target article.
Regardless of the specific smile types employed, participants
can typically use perceptual, conceptual, and embodiment processes in making these judgments. Additional empirical evidenceis necessary for assessing whether an embodiment model actuallyfits the data. Specifically, Niedenthal et al.’s SIMS model proposes that smiles activate neural regions that cause motor
mimicry and somatosensory experience, which form the basisof the viewers’ interpretation of the smile they see. Some ofthe data cited by Niedenthal et al in support of their model aresuggestive of emotional mimicry. However, these data are also
compatible with the possibility that motor cortex activation inemotion perception tasks reflects downstream associations,rather than playing a primary perceptual role. The proposedmodel does little to rule out the possibility that viewers may
differentiate between different smiles by using perceptual cuesalone (perhaps together with conceptual information), before(or in the absence of) any mimicry or simulation.
The authors cite evidence for a causal role of somatosensory
cortex in emotion perception from a study showing that transcranial magnetic stimulation (TMS) of this area selectively interfereswith performance in a facial expression matching task (Pourtois
et al. 2004). However, the manipulation selectively affected thereaction time of judgments of fearful expressions, but did notaffect the judgments of happy expressions. Furthermore, nointeraction was found with the gaze direction of the faces, andno effect was found on participants’ accuracy. The results ofthe study therefore do not seem to demonstrate a causal rolefor somatosensory cortex in emotion perception.
In sum, we argue that differentiations of smiles should bebased on findings establishing which distinctions senders andreceivers make, and what physical cues are actually used byviewers in their judgments. Furthermore, convincing demonstrations that motor cortex activation plays a causal role inemotion perception are lacking, and the possibility remains thatviewers can differentiate between smiling expressions from perceptual cues alone, without recourse to simulation.

Is eye contact the key to the social brain?
doi:10.1017/S0140525X10001275
Atsushi Senju and Mark H. Johnson
Centre for Brain and Cognitive Development, Birkbeck College, University of
London, London WC1E 7HX, United Kingdom.
a.senju@bbk.ac.uk
mark.johnson@bbk.ac.uk

Abstract: Eye contact plays a critical role in many aspects of faceprocessing, including the processing of smiles. We propose that this isachieved by a subcortical route, which is activated by eye contact andmodulates the cortical areas involve in social cognition, including theprocessing of facial expression. This mechanism could be impaired inindividuals with autism spectrum disorders.

Figure 1. (Senju and Johnson). An illustration of the fast-track modulator model (FTM). Perceived eye contact (upper left) is initiallydetected by a subcortical route that projects to various regions of the social brain network (thick black lines). This signal from thesubcortical route then interacts with contextual modulation based on the task demands as well as the social context (thick greylines) to modulate the response of these regions to the following input from a cortical route (thin black lines). These pathways are
based on previous analyses of cortical and subcortical face processing, as well as top-down voluntary attention. OFC ¼ orbitofrontal
cortex, PFC ¼ prefrontal cortex, STS ¼ superior temporal sulcus. Reproduced with permission from Senju and Johnson (2009b).

458

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
The Simulation of Smiles (SIMS) model proposed by Niedenthal
et al. emphasizes the core role of eye contact, which is hypothesized to trigger embodied simulation of the perceived smile.
The authors also speculated that the same mechanism may also
mediate the processing of other facial expressions. However,eye contact is known to modulate a far wider range of cognitive
processes, such as the encoding of gender, identity, and gaze
(Senju & Johnson 2009b). We recently reviewed this phenomenon, which we have termed the “eye contact effect,” and proposed the fast-track modulator (FTM) model to explain its
neural and developmental basis (Senju & Johnson 2009b). In
this commentary, we present a brief overview of the FTM
model and discuss several areas in which the FTM model complements the SIMS model, and thus would facilitate further
exploration of the neural, cognitive, and developmental mechanism underlying the effect of eye contact on face processing.
The FTM model proposes that the eye contact effect ismediated by a subcortical face detection pathway hypothesized
to include the superior colliculus, pulvinar, and amygdala. Thisroute is fast, operates on low spatial frequency visual information,and modulates cortical face processing (see our Figure 1). Evidence that the route is fast comes from event-related potential
and magnetoencephalographic studies showing that components
associated with this pathway can occur at shorter latencies thanthose usually associated with the cortical processing of faces(Bailey et al. 2005). In addition, evidence that the subcorticalroute modulates cortical processing comes from several functional imaging studies indicating that the degree of activation of
structures in the subcortical route (amygdala, superior colliculus,
and pulvinar) predicts or correlates with the activation of corticalface processing areas (George et al. 2001, Kleinhans et al. 2008).
It has also been proposed that the subcortical route is also
responsible for face preference in newborn infants (Johnson
2005) and even in adults (Tomalski et al. 2009). We hypothesizethat the combination of this subcortical pathway and contextual
modulation given by the task demands and social context directly
or indirectly modulates key structures involved in the corticalsocial brain network.
The FTM model shares several key features with the SIMS
model. However, there are several differences between these
two models, wherein the FTM model expands and broadens
the SIMS model. First, the FTM model proposes the neuralmechanism linking eye contact and facilitation of cortical faceprocessing, including the embodied simulation. The FTM
model proposes that perceived eye contact directly activates a
subcortical route, which then modulates the cortical areas
involved in different aspects of social cognitive processing.
Thus, it is possible to incorporate the SIMS model by arguingthat the subcortical route also modulates the motor cortex,
which controls mimicry. The FTM model also provides new predictions about the effect of eye contact on the processing ofsmiles: It should be fast and operate on low spatial frequency
visual information.
Second, the FTM model can also provide alternative hypotheses about the mechanism by which eye contact facilitates theprocessing of smiles. The FTM model hypothesizes that the subcortical route receiving input from perceived eye contact directly
modulates the cortical face processing areas. This contrasts with
the SIMS model which assumes that eye contact must elicit theembodied simulation first in order to facilitate the processingof smiles. As we discussed above, the FTM does not rule out
the possibility that the subcortical route mediates the embodied
simulation in response to eye contact. However, the FTM also
leads us to propose a more parsimonious hypothesis: that the subcortical route directly modulates visual cortical areas, which thenfacilitates the processing of facial expression, including smiles.
For example, the FTM model predicts that eye contact modulates the processing of smiles even when the activation of
the motor cortex is experimentally suppressed. By contrast, theSIMS model would not predict that eye contact facilitates theprocessing of smiles under this condition, because embodied
simulation is suppressed.
Third, the FTM model presents a unique perspective on thedevelopment of the eye contact effect. In the target article, Niedenthal et al. have suggested an interesting hypothesis that thepreference for eye contact in infants reflects an evolutionarybased mechanism for triggering embodied simulation, eventhough they do not discuss how such a mechanism develops.
By contrast, the FTM model assumes that infants are bornwith widespread connections between the subcortical routeand cortical structures. As a consequence, input from perceived
eye contact initially activates widespread cortical structures,
which combines with architectural bias to form specific connections between the subcortical “eye contact detector” and relevant
cortical and subcortical structures during the course of development. Interestingly, recent studies on the early development ofautism spectrum disorder (ASD), with manifest atypical patternsof eye contact behaviour, are consistent with the predictions
based on the FTM model. Even though infants and young children with autism show apparently typical eye contact behaviour(Chawarska & Shic 2009), neuroimaging studies demonstrate
more widespread and non-specific cortical activation in responseto eye contact (Elsabbagh et al. 2009), and behavioural studies
demonstrated that eye contact does not facilitate cognitive processing in children with ASD (Senju et al. 2003). These studies
suggests that infants and young children with ASD are sensitive
to eye contact, but that it fails to modulate cortical face processing in the same specialized way as typically developing children(Senju & Johnson 2009a). Future studies will need to test
whether eye contact elicits facial mimicry and affects the processing of smiles in individuals with ASD, especially since current
evidence is inconsistent as to whether individuals with ASD
show spontaneous facial mimicry (Magnée et al. 2007) or not(McIntosh et al. 2006, Oberman et al. 2009).
Eye contact plays a critical role in face-to-face communication,
and we propose it is the key to adaptively modulate many aspectsof social cognition, including the processing of smiles. We hope
the areas of overlap and contrast between the SIMS and theFTM models will generate empirical studies, and help further
understand the neural, cognitive, and developmental mechanisms underlying human social behaviour.
ACKNOWLEDGMENTS
Atsushi Senju was supported by an ESRC Research Fellowship (RES063–27– 0207), and Mark H. Johnson was supported by the UK
Medical Research Council (G0701484).

Can we really leave gender out of it? Individual
differences and the Simulation of Smiles
model
doi:10.1017/S0140525X10001342
Elizabeth Simpson and Dorothy Fragaszy
Department of Psychology, University of Georgia, Athens, GA 30602-3013.
simpsone@uga.edu
doree@uga.edu
http://simpsone.myweb.uga.edu/
http://psychology.uga.edu/people/bios/faculty/
Dorothy_M_Fragaszy.php

Abstract: Gender differences in face-based emotion recognition, notably
differential use of mimicry, may compromise the extent to which the
Simulation of Smiles (SIMS) model can be generalized to populations
besides the adult females on which it has been tested. Much work
indicates sex differences in face-based emotion recognition, including
smile recognition.

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

459

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
One problem for the Simulation of Smiles Model (SIMS) proposed by Niendenthal et al. is that there are individual differences in face-based emotion recognition (hereafter emotion
recognition), mimicry, and embodied cognition. Given thatthese are important features of the model, the model’s predictive
power may vary as a function of these features. Sex is one important individual difference that needs to be considered (Cahill
2006; Vigil 2009).
Though facial mimicry appears to improve emotion recognition (Niendenthal et al. 2001; Oberman et al. 2007; Stel &van Knippenberg 2008) – consistent with the SIMS model –the link between mimicry and emotion recognition has only
been reported in females. This could be because women, compared to men, rely more on facial feedback for emotion recognition, at least for some expressions. That is, when facial
mimicry is inhibited, women’s emotion recognition suffers,
while men’s does not (Stel & Knippenberg 2008). Whereas
women appear to rely more on this fast and efficient “hot” (embodied) simulatory route for emotion recognition, men may rely
more on one or more “cold” (rule-based, non-embodied) strategies. This may account for the finding that women are better
than men at distinguishing a number of facial expressions, including smiles (Krumhuber et al. 2007).
Behavioral and neuroimaging evidence is congruent with thenotion of sex differences in emotion recognition strategies. For
example, men and women have different face viewing patterns
during emotion recognition (Hall et al. 2010). While women
look more at the eyes, compared to men, men have longer first
fixations to the eyes and mouth, perhaps reflecting men’sslower processing speed for information contained in thoseareas (Hall et al. 2010). Additionally, there are anatomical differences between men and women in brain regions containing
mirror neurons, which underlie simulation during emotion recognition (Goldman & Sripada 2005). During empathetic faceto-face interactions, women, compared to men, experience
more activation in brain regions containing mirror neurons(Schulte-Rüther et al. 2008). Moreover, brain regions containing
mirror neurons differ in men and women, both structurally andfunctionally (Cheng et al. 2009).
Researchers examining sex differences in emotion recognition
should use caution, for the complexity of measuring emotion recognition may not be apparent. For example, stimulus materials
should be selected carefully because women are more sensitive
to dynamic, compared to static expressions, while men are not(Biele & Grabowska 2006). The gender of face stimuli also influences the perceived emotion: both men and women are better at
detecting negative expressions on male faces and positive
expressions on female faces (Becker et al. 2007). Also, males
and females are not mimicked to the same degree: females are
mimicked more often than males (e.g., Lakin et al. 2008).
To complicate things further, the likelihood of mimicking
depends on top-down influences, including one’s conscious or
unconscious goals, such as to recognize a face, to attract a potential mate, to understand a spoken language, or to affiliate (e.g.,after social rejection). These different motivations alter ourviewing patterns of faces and the feature to which we attend(e.g., Saether et al. 2009; Schyns et al. 2002). If we are attending
to a non-emotional aspect of a face – such as skin color – wemimic less (Cannon et al. 2009). Men may attend more to nonemotional aspects of faces – such as attractiveness – while
women may attend more to emotional aspects. Men’s empathy
also appears to be mediated by social-cognitive factors that do
not influence women’s empathy. For example, similar brain
regions are active when experiencing pain and when viewing
pain in another individual for both sexes (Singer 2006);
however, men no longer display the same neural pattern when
the other person is perceived as behaving unfairly (Singer
2006). This suggests that men, compared to women, may more
readily inhibit embodied emotion recognition strategies in suchcircumstances.

460

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

One encounters difficulties when trying to interpret the literature on sex differences in emotion recognition. Studies of
mimicry often: (a) use only female participants, (b) use samples
that are too small to allow comparisons of sex differences, (c)fail to test whether there are gender differences, or (d) fail toreport the proportion of males and females in the sample. If
there are only female participants, it remains untested whether
effects would likewise occur in males. For example, females’
embodiment during facial expression viewing has not beentested in male participants (Effron et al. 2006). Although others
have assumed no gender differences, the body of work reported
here suggests this assumption may be incorrect.
It would be fruitful to know whether encouraging men to
mimic – rather than relying on their natural tendency to doso – improves their emotion recognition in either speed or accuracy. Interestingly, administering testosterone reduces facial
mimicry in females (Hermans et al. 2006); however, we knowof no work which has examined whether increased testosterone
impairs emotion recognition in either sex. This also raises thequestion of whether men’s emotion recognition would be
improved if they were instructed to attend to the eyes – something that women do naturally – since the frequencies of looks
to the eyes is positively correlated with emotion recognition accuracy (Hall et al. 2010). Developmental work is crucial for identifying the emergence of sex differences in face viewing patterns,
mimicry, and emotion recognition, to determine the extent towhich such behaviors and skills are continuous or changing
throughout life.
We agree with Niendenthal et al. that embodied simulation is
an important strategy for emotion recognition. We also concurthat it is wise to examine each facial expression of emotion separately when modeling the processes underlying emotion recognition. In addition, we believe it is of utmost importance toconsider individual differences – and especially sex differences –in the use of simulatory processes for each type of expression.

Baby smile response circuits of theparental brain
doi:10.1017/S0140525X10001615
James E. Swain and S. Shaun Ho
Department of Psychiatry, University of Michigan School of Medicine, Ann
Arbor, MI 48109.
jamesswa@med.umich.edu
hosh@med.umich.edu
http://myprofile.cos.com/jameseswain

Abstract: The parent-infant dyad, characterized by contingent social
interactions that develop over the first three months postpartum, may
depend heavily on parental brain responses to the infant, including thecapacity to smile. A range of brain regions may subserve this social key
function in parents and contribute to similar capacities in normal
infants, capacities that may go awry in circumstances of reduced care.

The often positive, complex and multifaceted aspects of the smile
may be thought to begin with babies in the early postpartum.
Perhaps preceding smiling is crying, a signal believed to notify
parents to attend to safety, acquire additional resources, and honestly indicate need and vigor (Soltis 2004). Parents will theninteract with their infants in cry-care loops (Swain et al. 2004)
in order to negotiate their return to contentment. Thus, crying
and smiling may be considered as unconditioned responses inbabies, to which various forms of conditioning are introducedby the environment – the parents’ behaviors in particular – ina gradual, increasingly contingent manner.
The parental capacity to adapt to the postpartum may dependon highly conserved hypothalamic-midbrain-limbic-paralimbiccortical circuits acting in concert to support parent-response toCommentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
infants, including the emotion, attention, motivation, empathy,
and decision-making to navigate the complexities of parenting –which include baby smile detection, and appropriate responses toprepare that baby for subsequent social settings. For the parent,
who typically endures exhaustion and sleep deprivation day-in
and day-out alongside the rewarding aspects of care-giving, ababy’s smile directed toward a parent has a profound impact
on the dyad. A smile, as a signal of fulfillment, serves as areward that facilitates the interaction synchrony, promotes thesense of relief, fulfillment, and pleasure, strengthens thebonding, and reinforces actions preceding these outcomes inboth parties of the dyad. Thus, these smiles within the infantparent dyad are precursors of all kinds of smiles in adult life. It
may be that opiate systems (Panksepp et al. 1994) are also important for these hedonic and reinforcing experiences for the dyad.
Thus, a repertoire of synchronous dyadic social interactions isdeveloped by about age 3-months in babies, in which infants
learn to recognize and respond to the communicative intents ofothers, share emotions, take turns in a vocal exchange, match
the partner’s gaze directions and facial expressions, andrespond to micro-shifts in attentive states and levels of social
involvement (Feldman et al. 1999; Tronick 1989) that predicts
the nature of future smiling.
The parent-infant dyad is being investigated by several
groups – largely from the parent side – using functional neuroimaging during parent experience of baby cry and baby smile
among other stimuli, to develop models relevant for parental
brain function. This required sensory analysis focuses on brain
regions which affect corticolimbic circuits that regulate motivation, reward, and learning about their infant, and ultimatelyorganize parenting impulses, thoughts, and emotions into coordinated behaviors (Swain 2008; 2010; Swain & Lorberbaum 2008;
Swain et al. 2007). Underlining the likely importance of the orbitofrontal cortex (OFC) (raised by Niedenthal et al. in the target
article) in responding to a smiling infant (Minagawa-Kawai
et al. 2009; Nitschke et al. 2004), this region also responds toperhaps more ethologically sound stimuli of videotaped baby
stimuli (Noriuchi et al. 2008). Noriuchi et al. interpreted OFC
and related activations as part of circuits required for the
execution of well-learned movements. Responses in the OFC
may also fit with known significance in reward, punishment,
and hedonic experiences in animals and humans (Kringelbach2005), suggesting a lateral-to-medial topographical distributionof negative to positive valence, which is consistent with the aforementioned affiliation-independent OFC activation to both own
and others’ babies’ cries. Notably, Kringelbach (2005) suggested
that the OFC loci (BA47/12l) subserve the representation ofcomplex visual reward acquired through learning and memory,which is consistent with findings using own versus others’ baby
pictures. OFC activity, also reported by Swain et al. (2003) inresponse to pleasant own-baby pictures could also be interpreted
as activation in emotion regulation and habitual behavioral
response systems that are active in a range of normal and abnormal emotion-control states, including obsessive-compulsive disorder (Feygin et al. 2006; Leckman & Mayes 1999; Swain et al.
2007).
In accord with Niedenthal et al., considering that parental
response to baby smile may employ empathy and mirror
neurons, Lenzi et al. (2009) reported that mirror neuron
systems, including insula and amygdala, were more active
during emotional expressions from a mother’s own child, andthat they are correlated with maternal reflective function (ameasure of empathy). Along the same lines, peripheral oxytocin
response to infant contact at 7 months was positively correlatedwith brain activation in the hormone-regulating hypothalamus
and reward-mediating striatum as well as being significantly
higher in secure mothers (Strathearn et al. 2009). This strengthens the links between parenting, oxytocin, and reward neurocircuitry that promises to overlap with the rewards of romantic love.
Consistent with the importance of oxytocin in regulating social

bonds, other recent parent brain neuroimaging papers arelinking this hormone with brain activity in response to infant
stimuli. Swain and colleagues have found increased activity
according to higher oxytocin circumstances of vaginal versus
cesarean delivery mode (Swain et al. 2008) and breastfeeding
versus formula feeding (Kim et al., under review). Taken
together, neuroimaging of parents suggest the importance of
amygdala, insula, hypothalamus, and striatum, in addition to
OFC, for baby-smile response.
Finally, it is interesting to speculate about the long-term
impact of some impairment in baby-smile response circuits inparents. This may be part of postpartum depression and thesevere adverse effects on children (Feldman & Eidelman 2009;
Feldman et al. 2009) It would be interesting to specificallyprobe the face response circuits of depressed mothers who arestruggling with dyadic interactions. In one attempt to address
the effects of early-life events on later parenting, as elaborated
in animal models (Kaffman & Meaney 2007), brain structureand function in response to baby stimuli varied according to ameasure of perceived maternal care (PMC) (Kim et al. 2010).
Low PMC was associated with decreased gray matter in the
OFC as well as in frontal and superior temporal areas. In
addition, functional responses were reduced in nearby corticalareas. Clearly, an integrated understanding of the brain basis ofparenting, including smile regulation, has profound implications
for long-term parent and infant mental health.
ACKNOWLEDGMENTS
James E. Swain is supported by a grant from the National Alliance for
Research on Schizophrenia and Depression, and by the Klingenstein
Third Generation Foundation, NICHD IRC2MD004767-01, and theMichigan Institute for Clinical Health Research. Shaun Ho is
supported by NICHD IRC2MD004767-01 and the Michigan Institutefor Clinical Health Research.

Embodied simulation and the search formeaning are not necessary for facial
expression processing
doi:10.1017/S0140525X10001603
Jacob M. Vigil and Patrick Coulombe
Department of Psychology, University of New Mexico, Albuquerque, NM
87131-1161.
vigilJ@unm.edu
patc@unm.edu
http://www.unm.edu/∼ psych/faculty/sm_vigil.html

Abstract: Embodied simulation and the epistemic motivation to search
for the “meaning” of other people’s behaviors are not necessary forspecific and functional responding to, and hence processing of, humanfacial expressions. Rather, facial expression processing can be achieved
through lower-cognitive, heuristical perceptual processing andexpression of prototypical morphological musculature movement
patterns that communicate discrete trustworthiness and capacity cues
to conspecifics.

Niedenthal et al. emphasize a functional perspective on facial
expression processing of variant forms of smiling behaviors. We
remain skeptical, however, of their central thesis, that simulative
sensitivities modulate the autonoetic representation or epistemic“meaning” of other people’s facial expressions. The authors
provide no direct support for how simulation or mimicry of musculature movement patterns is necessary for effectively responding to, and hence processing, other people’s facial expressions.
For example, although the authors discuss how smiles are reflexively mimicked, and how facial movement plays a role in subjective emotional experiences, they present no support to show thatthe inhibition of facial mimicry modulates the accuracy of facial
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

461

Commentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model

Figure 1 (Vigil and Coulombe). The social objects are positioned according to mean capacity and trustworthiness ratings from anormative sample of adults (n ¼ 839). The two values under each social object are the percentage of subjects that then reported adesire to either affiliate with, or to avoid the object, respectively. The four values on the right side of each object, from top tobottom, are the percentage of subjects that then responded to the object with either: admiration, disgust, fear, or sympathy,
respectively. The most frequent behavioral intention (affiliate or avoid) and emotion reported by participants for each object are in bold.
judgments. Moreover, we feel that the authors overemphasize
the role of higher-order cognitive abilities such as the need toinfer meaning of self- and other-generated expressive behaviors.
Certainly, all socially-embedded species as well as human infants
are able to mimic and effectively respond to (e.g., with avert gaze)
their social surroundings via heuristical processes that do not
require epistemic-supporting mental faculties. More parsimonious conceptualizations of facial expression processes suggest
cognitive representations of meaning play only a supplementaryrole compared to perceptual and behavioral heuristics in producing and responding to discrete facial musculature movement
patterns such as smiles.
One set of models that may alternatively explain how observers
process different types of smiles is the Socio-Relational Framework of Expressive Behaviors (SRFB; Vigil 2009). According tothis paradigm, both the appraisal and the expression of nonverbal
behavior can be understood in terms of two primary dimensionsof affective responses. The first dimension is the heuristicalmotivation to either attract or dissuade interactions and relationalbonding with others, depending on whether other people signal
the potential to be a reciprocating social partner or whether they
signal interpersonal threat. The second dimension is the appraisal (in the perceiver) and the phenotypic display (in the expresser) of what has been referred to as capacity andtrustworthiness cues. Capacity cues signal one’s aptitude orability to inflict beneficial or detrimental consequences onothers and are usually associated with the impression of462

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

dominance. Trustworthiness cues, in contrast, signal one’s interpersonal intentions and motivations and are usually manifested as
appeasement or vulnerability displays that are associated withimpressions of submissiveness. Several scientists have insisted
that the perceptual processing of capacity and trustworthinesscues in others is both necessary and sufficient for predicting
interpersonal (e.g., affiliative or avoidant) dispositions and discrete emotive responses toward others (e.g., Fiske et al. 2006;
Rosenberg et al. 1968; Todorov 2008; Vigil 2009; Wojciszke2005). As shown in Figure 1, trustworthiness perceptions are parsimonious predictors of the basic desire to either affiliate with, or
to avoid, social objects (see also Wojciszke 2005). As hypothesized by the SRFB (Vigil 2009, p. 379), simultaneous perceptions of high- versus low-capacity levels can be used to predict
specific and discrete affiliative (admiration vs. sympathy) andavoidant (disgust vs. fear) reactions toward social objects (Fig. 1).
According to the SRFB, because disgust and admiration
project the impression of dominance, while fear and sympathy
project the impression of submissiveness (see Leary 1957;
Marsh et al. 2005; Montepare & Dobish 2003), these emotions
reflect the proclivities to advertise the capacity and trustworthiness components of one’s own reciprocity potential to others(Vigil 2009). Thus, it is possible to interchange socially relevant
information through heuristical perceptual and behavioral algorithms, independent of executive mental faculties. Rather, simulative and representational abilities probably evolved to
anticipate the outcomes of social situations that can result inCommentary/Niedenthal et al.: The Simulation of Smiles (SIMS) model
variable consequences such as discussion of third-party affiliates
or monitoring future patterns of others’ behaviors (Geary 2005).
Specific and functional responding to, and hence processing of,other people’s facial expressions does not require the monitoringof pluralistic consequences, but rather instant and uniform(approach/avoidant) responses to the immediate behavioral
states of others, irrespective of representational meaning.
In this sense, discrete facial expressions, including smiling,should correspond to features of the mouth and eye regionsthat signal capacity and trustworthiness cues to others (Vigil
2009). For example, people may signal capacity cues throughthe baring of teeth (e.g., Duchenne smiles that demonstrate bilateral symmetry, healthiness, and weaponry) and through direct
threat stares, which may operate to conceal eye sclera (e.g.,demonstrating the capacities to concentrate on the environment,
to endure the risk of inciting interpersonal evaluation and potential conflict, and the motivation to protect the eye). Likewise,
trustworthiness cues can be demonstrated through the concealment of teeth (e.g., closed-mouth appeasement smiles) and exaggeration of displayed sclera, which may instead operate to
explicate one’s vulnerability (e.g., from dust particles and sunglare) and allow observers to easily monitor the signaler’s attention, thereby reducing one’s threat impression to others (tears
are also produced from wide-eyed positioning). Humans
display the teeth and conceal the sclera in correspondence withdominant emotions (e.g., joy and anger), and they conceal theteeth and expose the sclera in correspondence with submissive
emotions (e.g., sadness, fear, surprise, and sympathy). Thus,
heuristical presentation and concealment of teeth and sclera
across discrete emotive gestures may have evolved in part tosignal capacity and trustworthiness cues to others, and ultimately
promoting interpersonal affiliation and avoidance. These possibilities support an alternative model to Niedenthal et al.’s proposal,
by showing that the cognizant awareness of and the epistemic
search for the meaning of emotional sensations may not benecessary for effective responding to different types of smiles
and nonverbal expressive behaviors more generally.

Embodied and disembodied processing of
emotional expressions: Insights from autism
spectrum disorders
doi:10.1017/S0140525X10001640
Piotr Winkielman
Department of Psychology, University of California, San Diego, La Jolla, CA
92093-0109; and Department of Psychology, Warsaw School of Social
Sciences and Humanities, 03-815 Warszawa, Poland.
pwinkiel@ucsd.edu
http://psy.ucsd.edu/∼ pwinkiel

Abstract: Processing of facial expressions goes beyond simple patternrecognition. To elucidate this problem, Niedenthal et al. offer a modelthat identifies multiple embodied and disembodied routes forexpression processing, and spell out conditions triggering use ofdifferent routes. I elaborate on this model by discussing recent
research on emotional recognition in individuals with autism, who canuse multiple routes of emotion processing, and consequently can show
atypical and typical patterns of embodied simulation and mimicry.

Niedenthal et al. tackle the classic problem of how the mind processes the ubiquitous and important social stimuli – facial
expressions. The problem is tricky because most expressions
carry multiple social and emotional meanings. As the authors
say, visually recognizing that an expression is a smile (asopposed to a frown) can be relatively easy. But deciphering its
social relevance is often difficult. After all, to distinguish among
an affiliative, enjoyment, and dominance smile, for example,

one needs not only perform a finer visual discrimination, butalso integrate one’s own emotional response with the information
about the intent, social status, and veracity of the expressor. To
advance research on this tricky problem, Niedenthal et al.
present an integrative model, which, in a welcome opposition
to the extant approaches, does not attempt to brush those complexities under the carpet. The authors’ model of expressions processing identifies various embodied and disembodied ways ofaccomplishing the task. The ways range from simple perceptual
processes, such as pattern-matching, to complex conceptual processes, such as mentalizing. The most generative part of themodel spells out some conditions that promote the use of embodied simulation, which relies on somatosensory and motor processes in the brain and the body. The authors point out therole of perceptual factors (e.g., difficult tasks benefit frommotor simulation) but also social ones (e.g., eye gaze).
The goal of my commentary is to extend Niedenthal et al.’s proposals by discussing the issue of recognition and mimicry of facial
expressions in individuals with autistic spectrum disorders (ASD).
Although the target article touches on this issue (sect. 3.2.4), thetreatment is necessarily brief and does not capture the full complexity of the empirical data. This is unfortunate because the ASD literature offers good support for the authors’ thesis that the engagementof embodied simulation processes is conditional, and is triggered bya mix of perceptual, conceptual, and social factors.
When do individuals with ASD mimic? Passive observation
versus task engagement. Several studies examined facial

mimicry in high-functioning individuals with autism spectrum disorder (ASD). Some found deficits in spontaneous mimicry. That is,in an unprompted setting, ASD individuals do not smile to a smileor frown to a frown. In one experiment, participants’ task was to“just watch” large pictures of happy and angry expressions presented for several seconds (McIntosh et al. 2006). Facial electromyography revealed congruent facial reactions only in control,
but not ASD, participants. Importantly, ASD participants reacted
typically when mimicry was explicitly encouraged, thus controllingfor nonspecific motor or perceptual deficits. In another experiment, participants’ task was to “just watch” a video in which a
student described his adventures in an amusement park, displaying
happy expressions (Stel et al. 2008). Analysis of experimentercoded facial expressions revealed less spontaneous mimicry in
ASD than in control participants. This effect was observed
despite no differences in the amount of time spent looking at thescreen and no differences in voluntary mimicry. In yet anotherstudy, the task was a passive observation of 2-second presentationsof several expressions (Dapretto et al. 2006). fMRI analysis showedthat ASD individuals had less activity in several areas, including thepremotor cortex (inferior frontal gyrus), amygdala, insula, and thestriatum. In short, “just watching faces” does not trigger embodied
simulation in ASD individuals. Interestingly, similar ASD impairments were found in studies that explored spontaneous mimicryof non-emotional actions (e.g., hand movements) with EEG
measures of motor cortex activation, such as mu-wave suppression.
Thus, ASD participants show a typical level of mu-wave suppression when performing an action, but less mu-wave suppression(i.e., less motor cortex activation) to observation of an action, as
compared to typical participants (Oberman et al. 2005).
But the story is more complex. Studies of non-emotional
mimicry suggest that ASD participants occasionally show “spontaneous mirroring.” For example, reduction in action imitation in
ASD is found only for actions displayed from an egocentric perspective (away from the observer), but not from an allocentric
perspective (toward the observer) (Theoret et al. 2005). In
another example, ASD individuals show little EEG mu-wave suppression when watching an action performed by a stranger, but
normal mu-wave suppression when watching an action by afamily member (Oberman et al. 2008). In the domain of emotional mimicry, one study found no impairments of spontaneous
mimicry, measured with EMG, when ASD individuals were
given the task of integrating visual and auditory cues to
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

463

Response/Niedenthal et al.: The Simulation of Smiles (SIMS) model
emotion (Magnee et al. 2007). A recent study showed that giving
ASD participants a task of recognizing expressions (rather thanjust “watching them”) led to a comparable level of mimicry as
the typical group, even though ASD mimicry was slower(Oberman et al. 2009). In short, it appears that “task engagement” is necessary to trigger mimicry in ASD individuals.
But what is “task engagement”? What factors turn on and off
embodied simulation? This is not simply the question of attention
and general motivation. After all, all studies ensured that individuals stay “on task.” One idea is that factors like “allocentric versus
egocentric perspective,” “family versus stranger,” and “emotion
identification versus observation” work similarly to “eye gaze,” discussed in the target article. Specifically, all these factors switch theprocessing strategy from the disembodied, “pattern-matching”
strategy, to one that presumably requires access to the underlying
states of the individuals modeling the expressions or actions.
Of course, this idea needs to be tested directly. Here is one
possibility. It is known that in perceiving expressions, ASD individuals rely more heavily on a “cold” rule-based strategy (Rutherford & McIntosh 2007). As a result, they are more likely thancontrols to accept as realistic the exaggerated images of
expressions (smiles that are stronger than real smiles), presumably because those expressions represent “best fits” to the rule.
However, this “rule-based” processing should diminish when
they are dealing with faces that smile and look “at them” andare produced by family members, rather than total strangers.
Importantly, the task needs to be realistically challenging – itcannot be easily solvable by a simple pattern-matching strategy(e.g., recognizing a distinct expression containing pure happiness, or a simple emoticon, like the Walmart’s smiley). Under
such conditions, the embodiment (mimicry) should contributeto emotion recognition. This is critical, because as Niedenthal
et al. note, mimicry contributes to performance of typical participants only when the perceptual task is hard, the participant cares,
and the subjective emotional response is a useful signal for therecognition task (Oberman et al. 2007).
I hope that this exchange inspires fruitful research that will
benefit not only theoretical understanding of emotion processing,but also better research on atypical social functioning (see Winkielman et al. [2009] for further discussion).

Authors’ Response
The future of SIMS: Who embodies whichsmile and when?
doi:10.1017/S0140525X10002748
Paula M. Niedenthal,a Martial Mermillod,a Marcus Maringer,band Ursula Hessc
aCentre National de la Recherché Scientiﬁque (CNRS) andClermont Université, 63037 Clermont-Ferrand, France; bDepartment of
Psychology, University of Amsterdam, 1018 WB Amsterdam,
The Netherlands; cDepartment of Psychology, Humboldt-Universität Berlin,
12489 Berlin, Germany.
niedenthal@wisc.edu
http://wwwpsy.univ-bpclermont.fr/∼ niedenthal/martial.mermillod@univ-bpclermont.fr
http://wwwpsy.univ-bpclermont.fr/∼ mermillod/
m.maringer@rug.nl
Ursula.hess@psychologie.hu-berlin.de
http://www.psychology.hu-berlin.de/staff/1683737

Abstract: The set of 30 stimulating commentaries on our target
article helps to define the areas of our initial position that should

464

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

be reiterated or else made clearer and, more importantly, theways in which moderators of and extensions to the SIMS canbe imagined. In our response, we divide the areas of discussion
into (1) a clarification of our meaning of “functional,” (2) aconsideration of our proposed categories of smiles, (3) areminder about the role of top-down processes in theinterpretation of smile meaning in SIMS, (4) an evaluation of
the role of eye contact in the interpretation of facial expressionof emotion, and (5) an assessment of the possible moderatorsof the core SIMS model. We end with an appreciation of the
proposed extensions to the model, and note that the future ofresearch on the problem of the smile appears to us to be assured.

R1. Introduction
One of the problems with researching smiles in humans isthat humans use language that communicates theircomplex causal attributions for smile behavior, and thus
for its meaning. This fact, we believe, has strongly influenced the history of the scientific study of the smile insocial psychology. This fact has also led us astray in someways. The prime example of a complex attributional judgment is that a smile is “false” (versus “true”). The spontaneous use of these labels by observers has strongly
influenced the course of social psychological research on
the smile. Yet, even though researchers can create stimuli
in the laboratory that have the properties of smiles thatare labeled, perhaps even quite universally in laboratory
experiments, as “true” versus “false,” (or more or lessgenuine) it is probable that these are not the expressions
occurring in daily life that are judged “true” and “false.”
Hence, we agree strongly with Fernández-Dols &Carrera that the “true” versus “false” distinction is not
the best way to carve up the smile landscape (see also
Hess & Kleck 1990). Bouissac clearly agrees with this point.
Indeed, when Ekman and Friesen first studied theimportance of the Duchenne marker (e.g., Ekman &Friesen 1982), they defined as “false” smiles theexpressions made by individuals who were effortfully
trying to deceive. Specifically, the individuals whosesmiles were ultimately the subject of study were trying
to cover up distress in response to an anxiety-producing
film or situation, with the goal of convincing perceivers
that they were actually experiencing positive affect.
When individuals were engaged in this deception, there
was little evidence of the Duchenne marker. Ekman
et al. (1990) further showed that EEG recordings of individuals who expressed non-Duchenne smiles revealed evidence of negative affect or of conflict (Wacker et al. 2010).
In any event, the conclusion was that smiles called “false”
were sometimes associated with the subjective experienceof negative affect in the smiler.
But there would probably be no agreement about the
attributed causes of “true” and “false” universally. In
recently collected cross-cultural data, one of our North
American female respondents reported that even if she
is having a bad day when at her job, she wants people tofeel good, and so she smiles. Is this a false smile or atrue smile? It is false in so far as it does not reflect how
she “feels in general today,” and so perhaps some individuals or cultures would call her smile “false.” But it is true inthat she has a positive social motivation. She thinks thosesmiles are “true.”
Lurking under the “true” and “false” distinction are
therefore three plausible categories of attribution uponResponse/Niedenthal et al.: The Simulation of Smiles (SIMS) model
which the judgment relies: the feeling state of the smiler(positive, neutral, negative), the intended outcome of thesmile interaction for the smiler (positive, neutral, negative1) and the intended outcome of the smile interaction
for the perceiver of the smile (positive, neutral, negative).
The relative weight of each of these dimensions in determining the judgment that a smile is “false” versus “true”
probably varies across culture. Our instinct, based on
responses to some of our open-ended questions, is thatin North American culture the attribution that the smilerintends harm to the perceiver of the smile is the dominant
information that drives the “false” judgment. However, inother cultures, perhaps in Europe, attributions based onother dimensions are also likely to be equal grounds forthe judgment of “false.”
So, all we really know about the terms “true” and “false”
as regards smiles, is that they are used. We also know that
we can construct smiles that look more or less true andfalse, and receive those of meaning judgments judgments.
But we do not know more than that, and perhaps that is
not the best way to go. Indeed, the SIMS was not developed to distinguish between the type of person or
culture that would judge a smile to be “true” or “false.”
That is what our own cross-cultural data collection isabout. The SIMS model is an attempt to define the behavior and brain processes that support certain general classes
about smiles. As regards the judgment that a smile is“false” in the core model, we argued that dominancesmiles are not likely smiles of positive affect; and so,
depending upon the culture, the simulation of a dominance smile may usually support a judgment of “false”(i.e., in particular, when the judgment scale that is used
is a scale of “authenticity” or “genuineness”). But any
attention we paid to the “true” and “false” distinction
was meant to lend coherence to the literature. As stated,
we believe that a different approach to carving up thesmiles space is more useful, and that is why we promotea functional account of smiles. A parallel semantic analysis
would yield the most convincing knowledge base, ofcourse (as suggested by Bouissac).
R2. Functional arguments
Our ethologist commentators such as Mehu & N’Diaye,
Ohala, as well as Sauter & Levinson and Centorrino,
Djemai, Hopfensitz, Milinski, & Seabright (Centorrino et al.), contest in different ways our use of “functional” language in defining the three major smile typesof interest to the SIMS. They also bemoan our lack ofdata (so do we, but we are working on it). It is therefore
our burden, first, to define “functional” in our account.
Further discussion of our typology follows in section R3.
For ethologists, functional is synonymous with “adaptive.” But it is also the case that the discussion of therelationship between function and form, among ethologists and evolutionary theorists, is not settled or consensual. The debate is, in fact, a quite lively one (e.g., Allen& Bekoff 1995; Owren & Rendall 2001). We are awarethat it is reckless to make functional (i.e., adaptive) arguments about any species that is living in a habitat inwhich it did not evolve, such as human beings living in suburban environments (Moffatt & Nelson, 1992). Hence, wehave been most influenced by emotion researchers who

use the language of functionalism in a way that is relevant
to humans and was initially motivated by responses to theidea that emotions disrupt reason and are generally
harmful psychological events (Niedenthal et al. 2006).
These researchers include Frijda, Scherer, Oatley,
Fischer, Haidt, Tracy, and Keltner, among many othermodern emotions researchers.
These researchers’ definition of function with regard toemotion and emotional expression is consistent with
Keltner and Gross (1999) or Scherer (1987), who holdthat emotions are solutions to problems and opportunities
related to physical and social survival. Those authors
further state “Functional ascriptions, therefore, refer tothe history of a behavior, trait, or system, as well as its
regular consequences that benefit the organism, or more
specifically, the system in which the trait, behavior, or
system is contained” (Keltner & Gross 1999, p. 469).
Social functionalist accounts propose specifically thattwo very important challenges for human survival
include (1) attachment to caretakers and potential mates,and (2) integration into groups. Another important challenge is the establishment and maintenance of social hierarchies. The assumptions are that human survival depends
upon group membership and that long-term group functioning requires effective leadership (Keltner & Haidt
1999). The importance of the smile as a foundationalsocial glue used in the solution of these problems is
made even more precisely by some of our own commentators. For example, as Swain & Ho note, the baby’s smile isa highly physiologically rewarding stimulus to a parent. It
helps the parent hang in there. Additional research,
reported by Swain and colleagues (e.g., Lenzi et al.
2009; Strathearn et al. 2009; Swain 2008), provides evenmore information about the circuits by which the brain
supports responding to attachment objects in particular.
A social functional account does not shun comparative
psychology or ethology, however, and we rely on it for
our account of the landscape of the human smile.
Indeed, as de Waal (2003) has suggested, Darwin was on
to something when he focused on facial expression ofemotion as a good candidate test of his evolutionary
theory. Along with de Waal, we consider the primate
and comparative psychology literatures to be useful for
motivating our distinction among smiles as conforming
to enjoyment smiles, affiliative smiles, and dominancesmiles (noting that these are names for expressions thatyield classes of human attributional judgments that canbe summarized as reflecting enjoyment, affiliation, anddominance).
R3. Our smile typology
But there are a number of conceptual problems to beresolved, especially about the proposed category of dominance smiles. Some of those issues require collaboration
with ethologists. One question has to do with the evolutionof displays of teeth in primates and the role of teeth display
in human smiles. There has been confusion in this regard
because of the initial reliance on the study of rhesus macaques, which have a strong linear, hierarchical societal
structure. Referring to Darwin, de Waal (2003) writes:
For example, he noted that the bared-teeth expression . . . by ablack Sulawesi macaque, occurs when the animal is pleased to
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

465

Response/Niedenthal et al.: The Simulation of Smiles (SIMS) model
be caressed. Retraction of the lips to expose both rows of teeth
is indeed a relaxed, friendly expression in this species as
opposed to the same expression in most other macaques, inwhich it signifies submission. How do we know this? Quantitative analysis of natural social interaction sequences among Sulawesi macaque demonstrates that the bared-teeth display
predicts the onset of affinitive contact between sender andaddressee, hence that it likely is associated with a positivesocial attitude. In these macaques, teeth-baring often occurs
mutually between individuals. In the better known rhesus
macaque, in contrast, teeth-baring is given exclusively by subordinate to dominant individuals – hence never mutually –and is a common response to threats and intimidations . . .
The colloquial term “fear grimace” for all teeth-baring
expressions derives from the familiarity of researchers withthe rhesus monkey – the most common laboratory primate
in the West – rather than from a comprehensive look at the
primate order, in which this expression has a variety of meanings. (de Waal 2003, p. 10)

One of the important points and reasons for quoting thispassage at length is to note that the use of teeth in a
gesture such as the smile, and therefore the meaning(s)of the smile, varies even within closely related primatesand depends upon the type of social structure in which
they live. We can be motivated by this comparison,noting that the showing of teeth in the gesture called thesmile is very complex. This variation exists even acrosshuman cultures. For instance, some Asian cultures teachthe covering of the teeth during smiling and laughing,whereas other cultures do not.
The possibility that the same expression, such as thesmile or the laugh, can have different meanings acrossand within cultures is further expressed by this quote:
The laughing expression of apes is clearly homologous withthat of our own species: the laugh derives from a widespreadmammalian play expression. As we have seen, however, homology does not necessarily imply that the expression functionsin the same way in all hominoids (i.e., humans and apes). In
bonobos and chimpanzees laughing is closely tied to play,whereas in our own species it occurs under a much wider
range of circumstances. Playful interaction is obviously
included and can be considered the original laughing
context, but we use the same expression also in bonding (i.e.,“laughing with”) and, sometimes, as a hostile signal (i.e.,“laughing at”)” (de Waal 2003, p. 17).

These three categories of laughter conform to our own
typology for the smile. Furthermore, we suggest that therelationship between laughing and smiling in humans may
be a closer one than it is for other primates. Indeed, there
may be a relationship between the human smile and theshow of teeth as in threat, as suggested by Ohala;
however, it seems likely, as we think may be implied by
the previous quote, that the different human smiles havetheir basis in laughter (which also shows teeth). This does
not deny the possibility, discussed by Liu, Ge, Luo, &Luo (Liu et al.), that some cultures hide the teeth possibly
to avoid miscommunication of threat or else a violation of anundesired association to animals.
Among the many ways of validating this smile typology,
we have recently begun a reanalysis of data collected forother theoretical purposes (Deborah Prentice, personalcommunication). The experiments in which the data
were collected involved the presentation of norm-conforming and norm non-conforming oral statements byindividuals who were ostensibly peers (students at the466

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

same university) of the participants. As the participants listened to the norm conforming and non-conforming statements, their faces were recorded by video, and four facialmuscles were recorded with electromyography techniques. In addition to measures of the face, measures ofthe participants’ moods and their reactions to the peers
whose comments they were listening to were also taken.
One of the most striking things is that when listening toboth conforming and non-conforming statements, participants smiled. The difference was that their perceptions of
the non-conforming peers were negative. Their smiles
were smiles of derision, not affiliation. We can test thisimpression by relating the use of specific muscles to the
feelings and the judgments of the participants to ask
whether individuals smile both out of affiliation and superiority and whether those smiles have different properties.
Maybe different muscles are employed, and maybe thebody is used differently (which we can code from the videos).
Evaluating the characteristics of smiles associated with
judgments of enjoyment, affiliation, and dominance, will
force us to push the SIMS to be more specific in its statements of the brain systems that ground smile meaning, ofcourse. In their commentary, Mann & Choe argue that,
although SIMS holds that the meaning of smiles can begrounded in embodied simulation, it is not specificenough about which sensory-motor systems ground themeanings of smiles (a related argument is presented by
Chang & Vermeulen). Mann & Choe propose that,
although some meaning can be captured by the facial
sensory-motor system, important aspects of smile
meaning cannot be captured without grounding smiles
with respect to high-level actions making up the behavioral
context.
Mann & Choe indeed suggest an interesting way toground the meaning of smiles in higher-level actions of
the perceiver of the smile. They propose that when a perceiver sees an enjoyment smile, the objective is to maintainthe enjoyment smile (maintain sensory invariance), forinstance, by telling a joke. The meaning of a smile canthen be grounded in the (higher-level) actions of the perceiver, which maintain the smile in the sender. In our target
article, we focused mainly on the affective feedback of the
facial sensory-motor system as a consequence of facial simulation. In that case, the meaning of the smile is grounded in
the affective output of the simulation. In addition, we argue
that the meaning of a smile can be grounded in the social
and behavioral context in which a smile occurs. For
example, a smile shown by a salesclerk who tries to sell apair of shoes is seen as less genuine compared to a smilethat is shown by a person who just sold a pair of shoes (Maringer et al., in press). Hence, similar to what Mann & Choe
propose, we do argue that the meaning of smiles can begrounded in high-level actions of the sender (rather thanthe perceiver). Sensory invariance in this case might relate
to the different contexts and behaviors that are likely to beaccompanied by similar facial expressions. In this way, thecommentary by Mann & Choe has extended our link
between meaning of smiles and behavioral context of thesender to the link between meaning of smiles and the behavioral context of the perceiver. We believe that both behaviors (perceiver’s and sender’s behaviors) can be used to
ground meaning of facial expressions.
Along with refining and validating the smile typology,
it will be pertinent to specify the nature and role ofResponse/Niedenthal et al.: The Simulation of Smiles (SIMS) model
facial mimicry in classification, recognition, and interpretation of these facial expressions. Although we do believethat the existing data suggest a role for mimicry in theseprocesses, along with Hamon-Hill & Barresi we feelthat the role is far from clear. In addition to the problems
raised by their commentary, even the question of thepotential roles of motor efference versus reafference isstill an open one.
R4. Top-down processes in smile interpretation
Commentators Evers, Noens, Steyaert, & Wagemans(Evers et al.); Lakens & Ruys; Lobmaier & Fischer;and Morsella, Montemayor, Hubbard, & Zarolia(Morsella et al.) worry in different ways about integrating
top-down processes and individual differences in SIMS.
Fortunately, the former is an important element of themodel, and we have already advanced some ideas about
the latter, which we elaborate upon in the section 4.
Top-down processes include the application of beliefs,
stereotypes, expectations, and motivated biases. We havenoted that such processes can provide important input
into the interpretation of smile meaning. For example,
Maringer et al. (in press) have shown that when people
believe smiles are not genuine read-outs of positive
feeling, these beliefs influence judgments of how
genuine a given smile is (in a particular culture) whenmimicry is inhibited. In fact, Hess et al. (1998) foundthat just evoking the notion that expressions may not begenuine, by asking participants to rate their genuineness,eliminated mimicry to the same expressions that were
mimicked when participants rated the emotion expressed.
Further, Halberstadt et al. (2009) have shown that when
individuals believe that an ambiguous facial expression is
actually communicative of a specific emotion, they later
mimic the ambiguous expression in terms of that specificemotion.
Granted that top-down processes affect the lower-level
perceptual and motor processes involved in encoding
facial expression and deducing meaning, in our view the
problem is how top-down and bottom-up processes communicate. Our solution was to suggest that both concepts
and emotions can be grounded in states of sensory-motor
systems and that interactions between these proceduresarise naturally from this account. As we noted in
section 5, paragrah 4,
the embodied account provides a natural way to link conceptual knowledge about smiles and the related social situation
to the actual perception of smiles. Rather than assuming thata smile activates an amodal knowledge structure to represent
its meaning, the embodied simulation account proposes
instead that a smile triggers a simulation of a smile experiencethat includes emotion, motor activity, and somatosensory
experience [. . .]. Once this simulation becomes active, it provides a conceptual interpretation of the perceived smile,
going beyond the information given to place it in a context.

Of course, when an individual is not behaviorally interacting with a smile (eye contact and mimicry are inhibited), it is certainly possible that conceptual knowledge
represented in language does all of the work (Boroditsky& Prinz 2008). For example, if one believes that Tony
Blair’s smile is a smile of dominance, then without
paying any attention to the smile whatsoever, one isquite capable of applying that concept as expressed inlanguage.
In fact, as we note in discussing the SIMS model, there
are a number of different ways that can be used to decode
the meaning of expressions, and situational knowledge, or
the knowledge about the emotionality of members of agroup (see Hess & Kirouac 2000; Hess et al. 2009b), canreplace simulation in situations where gaze contact isinhibited. Yet, even in such cases, simulation may stillplay a role. For example, Houde et al. (2009) have demonstrated that providing observers with verbal labels describing the emotional state of a person who shows a neutral
expression, elicits emotion congruent facial expressions –in essence, mimicry – in the observer.
This issue is also addressed in research on disorders
such as autism and of culture, where it is pertinent toestablish which of the critical behaviors – eye contact
and facial mimicry – is present or absent for those populations, and what beliefs are held, ready to be appliedinstead of or together with an embodied simulation.
Note also that eye contact was proposed as a sufficientand developmentally primary trigger to embodied simulation of the smile. Research on individual differences
will also need to determine other sufficient triggers of
embodiment (such as motivation; see Mondillon et al.
2007). This is particularly true if it turns out to be thecase that perceivers of dominance smiles tend not tomake eye contact (even if ever so fleetingly) with those
who display such expressions, as is asserted by Huang &Galinsky.
Winkielman, who focuses specifically on research on
autism, rightfully points out that we are a long way fromunderstanding the conditions that tend to favor an embodied simulation strategy for processing facial expression ofemotion. We agree that the importance of successful communication, as sometimes determined by the significanceof the relationship between the expresser and the perceiver, is a factor that determines the type of processing thatgrounds judgments of smile meaning. Individuals may
devote more resources to understanding those withwhom they have an intimate or interdependent relationship (e.g., Zajonc et al. 1988). But a rigorous and theoretically concise account of why that is so will be required.
One avenue is to integrate some of the insights from thecommentary by Vigil & Coulombe, particularly as
regards the role of attachment in these processes. We donote that according to Vigil & Coulombe our central
claim is that “simulative sensitivities modulate the autonoetic representation or epistemic ‘meaning’ of otherpeople’s facial expressions”; however, we would not articulate the central claim of our target article in that way as thestatement only addresses a small part of the model. Furthermore, like Winkielman, we do believe that the roleof mimicry in interpreting the meaning of facial expression
has received some impressive supportive evidence (e.g.,
from Maringer et al., in press; discussed earlier and in
the target article).
R5. What is it about eye contact?
Aside from our proposed functional categorization ofsmiles and its links to social-attributional judgments, theidea that eye contact has developed as a developmentally
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

467

Response/Niedenthal et al.: The Simulation of Smiles (SIMS) model
basic and subsequently sufficient trigger of embodied
simulation gave rise to the most energetic discussion.
There is much to work on here.
Senju & Johnson, for instance, have suggested that the
SIMS model be integrated with their fast-track modulator(FTM) model. The FTM model holds that eye contact ismediated by a subcortical face detection pathway hypothesized to involve the superior colliculus, pulvinar, andamygdala (Senju & Johnson 2009). The root of this subcortical pathway is assumed to be constituted of alphaganglion cells, at the origin of magnocellular layers andLSF subsequent cortical pathways, which are very fast
but provide only low spatial frequency (LSF) visual information. This subcortical pathway has been confirmed forauditory information (Campeau & Davis 1995; Doron &Ledoux 1999) and visual information (Doron & Ledoux
1999; Linke et al. 1999; Shi & Davis 2001) in rats.
Concerning humans, different studies suggest the existence of a preferential link between LSF information andthe emotional system, particularly threat detection. Thisplausible preferential link was obtained on the basis ofneuroimaging (Morris et al. 1999; Pourtois et al. 2005;
Vuilleumier et al. 2003), neural-network modeling (Mermillod, in press a; Mermillod et al. 2009), and behavioral
experiments (Bocanegra & Zeelenberg, 2009; Holmes
et al. 2005; Mermillod et al., in press b). However,whereas these studies hint at a preferential link between
LSF visual information and emotional processes, possibly
occurring at the level of the amygdala, they do not constitute formal evidence for the subcortical pathway assumed
in Ledoux’s (1996) model. Therefore, it is speculative toincorporate this pathway in the SIMS model at this time.
However, this is obviously a very interesting way todevelop the SIMS model and, more specifically, the complementarities between fast and automatic processes of
relevant emotional stimuli in the environment, potentiallyoperating at the level of subcortical structures and more
complex associative processes occurring at the level of cortical areas. The interesting contribution of the FTM modelis that it specifies the relationship between early visual
processes and subsequent emotional processes related toeye contact. This link between perception and emotions
should therefore be investigated.
Similarly, the question of automatic and unconscious
response to eye gaze was raised by Chatelle, Laureys,
Majerus, & Schnakers (Chantelle et al.) with regard
to severely brain-injured patients. These commentators
propose that minimally conscious state (MCS) patients
could be differentiated from vegetative state (VS) patients
based on eye gaze. They propose that intentionality anddifferent level of consciousness could be assessed though
eye gaze. For our model, this raises the question of the
consciousness of different level of processing. In relationship with the previous comment proposed by Senju &Johnson, we suggest the possibility of a direct, automatic,
but also unconscious subcortical pathway for eye gazeorientation. This hypothesis can be supported by researchon emotional blindsight (de Gelder et al. 1999; Pegna et al.
2004), showing that hemianopsic patients might be able todetect emotions presented in their blind visual field above
chance (de Gelder et al. 1999). An important point is thatthese patients are not conscious of this blind perception
and feel that they respond at chance level. In otherwords, this subcortical pathway (if it exists) seems to
468

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

operate beyond the scope of consciousness. Combining
this finding with the FTM model (Senju & Johnson2009), we can assume that automatic but uncontrolled
eye gaze direction could be directed by subcortical structures like the superior colliculus, the pulvinar, and possibly
the amygdala for automatic responses to environmental
stimuli in VS patients, whereas MCS patients might beable to use more neural resources involving cortical andother subcortical structures. As suggested by Chatelle
et al., we believe that investigation of the neural underpinnings of directed versus automatic eye gaze in VS versus
MCS patients could be a very interesting way to investigate the neural basis of consciousness.
Finally, as highlighted by Chakrabarti’s comments, amodel of smile perception involving the question of eye
contact necessarily raises the question of parsimony.
More than 30 emotional states can be associated withsmiles (Golan & Baron-Cohen 2006), and it would berather implausible to associate one neural module pertype of emotional feeling. The goal of the SIMS model,
however, was to propose general principles, associatedwith different neural pathways, to account for theemotional processing of smiles. We hope that this parsimonious approach, based on scientific evidence in support ofour new theoretical framework, will be able to account for
the larger set of empirical data relating to smiles. Amongthem, and as proposed by Chakrabarti, we will have todetermine whether not only does eye contact act as a
trigger for subsequent amygdala activity, but, conversely,
amygdala activity can boost the search for relevant social
cues and, therefore, eye contact. Chakrabarti also raises
the question of determining whether embodied simulation
processes are emotion specific. The SIMS model focuseson smiles, but as noted in the target article, we assume
that similar processes are likely to occur in the recognitionof other emotional expressions.
R6. Moderators and individual differences
The SIMS model was developed with the goal of defining
general processes by which individuals attribute meaning
to a smile. In particular, the model highlights the role ofeye contact, mimicry, and the induction of specific brain
states. However, along with our commentators, we encourage discussion of the potential impact of individual
differences. Foremost amongst these are the differencesof gender and culture, as emphasized by several commentators (Bouissac; Caldara; Conty, Grézes, &Sander (Conty et al.); Liu et al.; and Simpson &Fragaszy).
The current development of the SIMS model does notdiscuss those influences in detail, even though we specifically note that “it is essential to note that cultural differences may modulate our account.” Clearly, we agreewith the notion that individual differences have a role toplay and can be expected to moderate some of the processes we discuss. As Covas-Smith, Fine, Glenberg,
Keylor, Li, Marsh, Osborne, Soliman, & Yee (CovasSmith et al.) note, cultural differences in eye contact preferences should entrain differential predictions based on
the SIMS model. The question then arises – do weknow enough about these differences to accurately
predict their effect? And here the issue becomes rather

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
more complex. Caldara provides evidence that membersof collectivist cultures fixate faces around the nose area –that is, do not actually seek full eye contact. By contrast,
Yuki et al. (2007) note that Japanese participants weigheye information more heavily than mouth information,and similar data are reported by Liu et al. as being
under review. Just this short summary of data presented
in the commentaries shows that at this point we may not
be able to predict the impact of culture on eye gaze and,
consequently, on its role for smile interpretation. Whatis needed are intercultural studies focusing not only on
emotion decoding accuracy but also more precisely on
the process of interpreting facial expressions. The SIMS
model provides a framework for such research.
For example, Hess and Kirouac (2000) have proposedthat when individuals do not know each other, they tend
to resort to stereotype knowledge about social groupmembers when decoding facial expressions. We havealready considered whether this is a general rule orwhether this fact indeed varies across culture; and how
this fits with the SIMS. One of our major cultural hypotheses relates to the processing of in-group versus out-group
facial expressions (e.g., Niedenthal & Maringer 2009). The
foundational history of some societies – sometimes called“settler societies” where the focus is on the fact that a land
occupied by an indigenous peoples was taken over andsettled by people from other cultures and nationalities –may strongly influence the key behaviors of SIMS.
The other individual difference highlighted by commentators is that of gender differences (Simpson & Fragaszy). Should we expect embodiment to play a differentrole for men and women? Simpson & Fragaszy point out
that there are differences between men and women inemotion recognition accuracy in some studies and theseare positively correlated with looks to the eye (Hall et al.
2010). They also note that women who have been given
testosterone show reduced mimicry (Hermans et al.
2006). In fact, a number of top-down processes, such as
motivation and the observers’ own beliefs and values, arelikely to modulate behaviors linked to mimicry and contagion (Hess & Fischer, under review). In this context it isalso interesting to note that men and women differ intheir capacity for interoception (e.g., Harver et al. 1993);
and it has been suggested that women are more likely tobase at least the perception of their own emotions onsocial context cues than men are (Pennebaker & Roberts
1992). This opens the door to the possibility that women
may also use social context cues and emotion knowledge
to a larger extent when considering the meaning ofsmiles. These issues are certainly important and should
be investigated in the framework of the SIMS model.
Yet, the factors discussed above only affect quantitative
aspects of the process and do not require the addition of
new processes into the SIMS model.

R7. Extensions
Some of our commentators have suggested ways to push
SIMS as we engage in the process of testing and refining
the model (Alibali & Hostetter; Bartlett; Basso &Oullier; Briñol, DeMarree, & Smith [Briñol et al.];Kiverstein & Zamuner).

For instance, Bartlett argues for the need to consider
time. By “time,” she means that part of the developmentof a complex model that integrates behaviors and multiple
brain circuits involves a specification of the timing of theseinteractions and the neural activations. Silvan S. Tomkins
was an early advocate of the notion that emotions aredefined in part in terms of the timing of their onset andtheir duration (Tompkins 1962; 1963). These temporalcomponents constitute the very form of the emotion. We
agree that timing principles will be an important part ofthe model and that timing gone awry will be at the basisof dysfunction in emotional information processing.
Another extension is proposed by Alibali & Hostetter,and also by Basso & Oullier, who point to the fact thatemotions are also expressed in gesture and in the voice.
Embodied simulation of these parts of emotional experience will also play a role in emotional information processing per se and will interact with the simulation, andtherefore, the understanding of facial expression. Different weights and roles of the many cues to emotion canin part be determined by examining the cases of mismatch
between one (facial expression) and the other (emotional
gesture). Another important question will be why
emotion is communicated often in more than one expressive channel. Why do we have emotional gesture andprosody if we already have the exquisitely nuanced face?
We feel that the SIMS model presents a valuable framework for the study of these and other pertinent processes.
NOTE
1. An intended negative outcome for the smiler would be
unusual and probably related to a non-typical state, as one
smiles before committing an act that one knows will be punishedor will lead to self-harm.

References[The letters “a” and “r” before author’s initials stand for target article andresponse references, respectively.]
Abe, J. A., Beetham, M. & Izard, C. (2002) What do smiles mean? An analysis interms of differential emotions theory. In: An empirical reflection on the smile,ed. M. H. Abel, pp. 83 – 110. Edwin Mellen Press. [aPMN]
Abel, M. H. (2002) The elusive nature of smiling. In: An empirical reflection on thesmile, ed. M. H. Abel, pp. 1 – 13. Edwin Mellen Press. [aPMN]
Achaibou, A., Pourtois, G., Schwartz, S. & Vuilleumier, P. (2008) Simultaneousrecording of EEG and facial muscle reactions during spontaneous emotional
mimicry. Neuropsychologia 46:1104 – 13. [aPMN]
Adams, R. B., Jr., Gordon, H. L., Baird, A. A., Ambady, N. & Kleck, R. E. (2003)
Effects of gaze on amygdala sensitivity to anger and fear faces. Science
300(5625):1536. [JSL, aPMN]
Adams, R. B., Jr. & Kleck, R. E. (2003) Perceived gaze direction and the processingof facial displays of emotion. Psychological Science 14(6):644– 47. [BChan, LC]
Adams, R. B., Jr. & Kleck, R. E. (2005) Effects of direct and averted gaze on theperception of facially communicated emotion. Emotion 5(1):3 – 11. [BChan,LC]
Adams, R. B., Pauker, K. & Weisbuch, M. (2010) Looking the other way: The role ofgaze direction in the cross-race memory effect. Journal of Experimental Social
Psychology 46(2):478– 81. [LC]Adolphs, R. (2002) Recognizing emotion from facial expressions: Psychological andneurological mechanisms. Behavioral and Cognitive Neuroscience Reviews
1:21 – 62. [aPMN]
Adolphs, R. (2003) Cognitive neuroscience of human behavior. National Review of
Neuroscience 4:165– 78. [aPMN]
Adolphs, R. (2006) Perception and emotion. Current Directions in Cognitive
Science 15:222 – 26. [aPMN]
Adolphs, R. (2008) Fear, faces, and the human amygdala. Current Opinion in
Neurobiololgy 18(2):166– 72. [aPMN]
BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

469

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Adolphs, R., Damasio, H., Tranel, D., Cooper, G. & Damasio, A. R. (2000) A rolefor somatosensory cortices in the visual recognition of emotion as revealed by
3-D lesion mapping. Journal of Neuroscience 20:2683 – 90. [aPMN]
Adolphs, R., Damasio, H., Tranel, D. & Damasio, A. R. (1996) Cortical systems forthe recognition of emotion in facial expressions. Journal of Neuroscience
16:7678 – 87. [aPMN]
Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P. & Damasio, A. R.
(2005) A mechanism for impaired fear recognition after amygdala damage.
Nature 433(7021):68– 72. [CL, aPMN]
Adolphs, R., Gosselin, F., Buchanan, T., Tranel, D., Schyns, P. & Damasio, A.
(2005) A mechanism for impaired fear recognition after amygdala damage.
Nature 433:68 –72. [aPMN]
Adolphs, R. & Tranel, D., eds. (2000) The amygdala. A functional analysis. Oxford
University Press. [CL]
Adolphs, R., Tranel, D., Damasio, H. & Damasio, A. R. (1994) Impaired recognitionof emotion in facial expressions following bilateral damage to the human
amygdala. Nature 372(6507):669– 72. [CL, aPMN]
Allen, C. & Bekoff, M. (1995) Function, natural design, and animal behavior: Philosophical and ethological considerations. Perspectives on Ethology 11:1 –46.
[rPMN]
Amaral, D. G., Schumann, C. M. & Nordahl, C. W. (2008) Neuroanatomy ofautism. Trends in Neurosciences 31:137– 45. [KE]Ambadar, Z., Cohn, J. F. & Reed, L. I. (2009) All Smiles are not createdequal: Morphology and timing of smiles perceived as amused, polite,
and embarrassed/nervous. Journal of Nonverbal Behavior 33(1):17 – 34.
[DAS]
Ambadar, Z., Schooler, J. W. & Cohn, J. F. (2005) Deciphering the enigmatic face:
The importance of facial dynamics in interpreting subtle facial expressions.
Psychological Science 16:403 – 10. [CH-H]
Anderson, J. R., Qin, Y., Sohn, M.-H., Stenger, V. A. & Carter, C. S. (2003) An
information-processing model of the BOLD response in symbol manipulation
tasks. Psychonomic Bulletin and Review 10:241 –61. [aPMN]
Andrew, R. J. (1963) The origin and evolution of the calls and facial expressions of
the primates. Behaviour 20:1 – 109. [JJO]
Argyle, M. (1972) The psychology of interpersonal behavior. Penguin Books.
[aPMN]
Argyle, M. & Cook, M. (1976) Gaze and mutual gaze. Cambridge University Press.
[BChan, CMC-S]
Ashforth, B. E. & Humphrey, R. H. (1993) Emotional labor in service roles: The
influence of identity. The Academy of Management Review 18(1):88– 115. [FB]Atkinson, A. (2007) Face processing and empathy. Empathy in mental illness, pp.
360 – 85. Cambridge University Press. [aPMN]
Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A., Moscovitch, M. & Bentin, S. (2008) Angry, disgusted, or afraid? Psychological
Science 19(7):724. [BChak]
Bach, P., Peatfield, N. A. & Tipper, S. P. (2007) Focusing on body sites: The role ofspatial attention in action perception. Experimental Brain Research
178(4):509– 17. [BChak]
Bailey, A. J., Braeutigam, S., Jousmaki, V. & Swithenby, S. J. (2005) Abnormal
activation of face processing systems at early and intermediate latency inindividuals with autism spectrum disorder: A magnetoencephalographic study.
European Journal of Neuroscience 21:2575 – 85. [AS]Barresi, J. & Moore, C. (1996) Intentional relations and social understanding. Behavioral and Brain Sciences 19(1):107– 22. [CH-H]
Barresi, J. & Moore, C. (2008) The neuroscience of social understanding. In: The
shared mind: Perspectives on intersubjectivity, ed. J. Zlatev, T. Racine, C.
Sinha & E. Itkonen, pp. 39– 66. John Benjamins. [CH-H]
Barrett, K. C. (2002) Smiling in children: Displays and their meanings. In: An
empirical reflection on the smile, ed. M. H. Abel, pp. 137 –54. Edwin Mellen
Press. [aPMN]
Barrett, L. F. (2006) Are emotions natural kinds? Perspectives on Psychological
Science 1:28– 58. [J-MF-D]
Barsalou, L. W. (1999) Perceptual symbol systems. Behavioral and Brain Sciences22(4):577 – 660. [MWA, FB, EM, aPMN]
Barsalou, L. W. (2005) Situated conceptualization. In: Handbook of categorizationin cognitive science, ed. H. Cohen & C. Lefebvre, pp. 619 – 50. Elsevier.
[aPMN]
Barsalou, L. W. (2008) Grounded cognition. Annual Review of Psychology 59:617 –45. [JSL, aPMN]
Barsalou, L. W., Niedenthal, P. M., Barbey, A. & Ruppert, J. (2003) Social embodiment. In: The psychology of learning and motivation, vol. 43, ed. B. Ross, pp.
43 –92. Academic Press. [aPMN]
Barsalou, L. W., Santos, A., Simmons, W. K. & Wilson, C. D. (2008) Language andsimulation in conceptual processing. In: Symbols, embodiment, and meaning,ed. M. De Vega, A. M. Glenberg & A. C. Graesser, pp. 245 – 83. Oxford University Press. [aPMN]
Bartlett, M. & Whitehill, J. (in press) Automated facial expression measurement:
Recent applications to basic research in human behavior, learning, and470

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

education. In: Handbook of face perception, ed. A. Calder, G. Rhodes, J. V.
Haxby & M. H. Johnson. Oxford University Press. [MSB]
Bastiaansen, J. A. C. J., Thiox, M. & Keyers, C. (2009) Evidence for mirror systems
in emotions. Philosophical Transactions of the Royal Society 364:2391 – 404.
[aPMN]
Bavelas, J., Black, A., Lemery, C. R. & Mullett, J. (1986) “I show how you feel”:
Motor mimicry as a communicative act. Journal of Personality and Social
Psychology 50:322 – 29. [aPMN]
Becker, D. V., Kenrick, D. T., Neuberg, S. L., Blackwell, K. C. & Smith, D. M.
(2007) The confounded nature of angry men and happy women. Journal of
Personality and Social Psychology 92:179 – 90. [ES]Belt, V., Richardson, R. & Webster, J. (2002) Women, social skills, and interactive
service work in telephone call centres. New Technology, Work and Employment 17(1):20 – 34. [FB]
Berger, C. R. (1994) Power, dominance, and social interaction. In: Handbook ofinterpersonal communication, 2nd edition, ed. M. L. Knapp & G. R. Miller, pp.
450 – 507. Sage. [LH]
Bernstein, M. J., Young, S. G., Brown, C. M., Sacco, D. F. & Claypool, H. M. (2008)
Adaptive responses to social exclusion: Social rejection improves detection ofreal and fake smiles. Psychological Science 19(10):981– 83. [CL, aPMN]
Bertrand, M. (1969) The behavioural repertoire of the stumptail macaque: A
descriptive and comparative study. Bibliotheca Primatalogica 11:1 –273. [LH]
Biele, C. & Grabowska, A. (2006) Sex differences in perception of emotion intensity
in dynamic and static facial expressions. Experimental Brain Research 171:1–
6. [ES]
Bindemann, M., Burton, A. M. & Langton, S. R. H. (2008) How do eye gaze andfacial expression interact? Visual Cognition 16(6):708. [MM]Blairy, S., Herrera, P. & Hess, U. (1999) Mimicry and the judgment of emotional
facial expressions. Journal of Nonverbal Behavior 23(1):5 – 41. [CL, CH-H,
aPMN]
Blais, C., Jack, R. E., Scheepers, C., Fiset, D. & Caldara, R. (2008) Culture shapes
how we look at faces. PLoS ONE 3(8):e3022. [doi:10.1371/journal.pone.0003022] [RC, CMC-S]
Blass, E. M. & Camp, C. A. (2001) The ontogeny of face recognition: Eye contact
and sweet taste induce face preference in 9- and 12-week-old human infants.
Developmental Psychology 37:762 – 74. [aPMN]
Bocanegra, B. R. & Zeelenberg, R. (2009) Emotion improves and impairs earlyvision. Psychological Science 20:707 – 13. [rPMN]Bogart, K. R. & Matsumoto, D. (2010) Facial mimicry is not necessary to recognize
emotion: Facial expression recognition by people with Moebius syndrome.
Social Neuroscience 5:241– 51. [EM]Bohrn, I., Carbon, C. C. & Hutzler, F. (2010) Mona Lisa’s smile – Perception or
deception? Psychological Science 21:378 – 80. [J-MF-D]
Boksem, M. A. S., Smolders, R. & de Cremer, D. (2009) Social power andapproach-related neural activation. Social Cognitive and Affective Neuroscience Advance Access. Published online on March 20, 2009. [aPMN]
Bolwig, N. (1964) Facial expressions in primates with remarks on a parallel development in certain carnivores (A preliminary report on work in progress). Behaviour 22:167 – 92. [JJO]
Boly, M., Faymonville, M. E., Schnakers, C., Peigneux, P., Lambermont, B., Phillips, C., Lancellotti, P., Luxen, A., Lamy, M., Moonen, G., Maquet, P. &Laureys, S. (2008) Perception of pain in the minimally conscious state withPET activation: An observational study. The Lancet Neurology 7:1013 – 20.
[CC]
Boraston, Z., Corden, B., Miles, L., Skuse, D. & Blakemore, S. (2008) Brief report:
Perception of genuine and posed smiles by individuals with autism. Journal ofAutism and Developmental Disorders 38:574 – 80. [aPMN]
Boroditsky, L. & Prinz, J. (2008) What thoughts are made of. In: Embodied
grounding: Social, cognitive, affective, and neuroscientific approaches, ed. G.
Semin & E. R. Smith, pp. 98 – 117. Cambridge University Press. [rPMN]
Boroditsky, L. & Ramscar, M. (2002) The roles of body and mind in abstract
thought. Psychological Science 13:185 – 89. [EM]Bouissac, P. (2001) The visual role of the sclera and the teeth in facial interaction.
In: Oralite et gestualite, ed. C. Cave, I. Guaitella & S. Santi, pp. 161 – 66.
L’Harmattan. [PBou]
Bouissac, P. (2005) What is a trustworthy face? Available at: http://www.semioticon.com/virtuals/risk/Trustworthyface.pdf. [PBou]
Bourgeois, P. & Hess, U. (2008) The impact of social context on mimicry. BiologicalPsychology 77:343 – 52. [aPMN]
Bourgeron, T. (2009) A synaptic trek to autism. Current Opinion in Neurobiology
19:231 – 34. [KE]
Bower, G. (1981) Mood and memory. American Psychologist 36:129 – 48. [aPMN]
Bowers, D., Bauer, R. M., Coslett, H. B. & Heilman, K. M. (1985) Processing offaces by patients with unilateral hemisphere lesions: Dissociation between
judgments of facial affect and facial identity. Brain and Cognition 4:258 – 72.
[aPMN]
Breiter, H. C., Etcoff, N. L., Whalen, P. J., Kennedy, W. A., Rauch, S. L., Buckner,
R. L., Strauss, M. M., Hyman, S. E. & Rosen, B. R. (1996) Response andReferences/Niedenthal et al.: The Simulation of Smiles (SIMS) model
habituation of the human amygdala during visual processing of facial
expression. Neuron 17:875 – 87. [aPMN]
Briñol, P. & DeMarree, K. G., eds. (in press) Social metacognition. Psychology
Press. [PBri]
Briñol, P. & Petty, R. E (2003) Overt head movements and persuasion: A selfvalidation analysis. Journal of Personality and Social Psychology 84:1123 – 39.
[PBri]
Briñol, P. & Petty, R. E. (2008) Embodied persuasion: Fundamental processes bywhich bodily responses can impact attitudes. In: Embodiment grounding:Social, cognitive, affective, and neuroscientific approaches, ed. G. R. Semin &E. R. Smith, pp. 184 – 207. Cambridge University Press. [PBri]
Briñol, P., Petty, R. E. & Wagner, B. C. (2009) Body postures effects on selfevaluation: A self-validation approach. European Journal of Social Psychology
39:1053 – 64. [PBri]
Brown, W. M., Palameta, B. & Moore, C. (2003) Are there nonverbal cues tocommitment? An exploratory study using the zero-acquaintance video presentation paradigm. Evolutionary Psychology 1:42 – 69. [MM]Bruce, V. & Young, A. (1986) Understanding face-recognition. British Journal of
Psychology 77:305 – 27. [BChak]
Bruno, M., Vanhaudenhuyse, A., Schnakers, C., Boly, M., Gosseries, O., Demertzi,A., Majerus, S., Moonen, G., Hustinx, R. & Laureys, S. (2010) Visual fixation in
the vegetative state: An observational case series PET study. BMC Neurology
10(1):35. [CC]Buchanan, T. W., Tranel, D. & Adolphs, R. (2009) The human amygdala in social
function. In: The human amygdala, ed. P. J. Whalen & E. A. Phelps, pp. 289 –
318. Guilford Press. [PBou]
Buck, R. (1984) The communication of emotion. Guilford Press. [aPMN]
Buck, R. (1991) Social factors in facial display and communication: A reply to Chovil
and others. Journal of Nonverbal Behavior 15:155 – 62. [aPMN]
Burgoon, J. K., Buller, D. B. & Woodall, W. G. (1996) Nonverbal communication:
The unspoken dialog, vol. 2, pp. 259 – 92. McGraw-Hill. [aPMN]
Cacioppo, J., Petty, R., Losch, M. & Kim, H. (1986) Electromyographic activity overfacial muscle regions can differentiate the valence and intensity of affective
reactions. Journal of Personality and Social Psychology 50:260 – 68. [BChan,aPMN]
Cahill, L. (2006) Why sex matters for neuroscience. Nature Reviews Neuroscience
7:477– 84. [ES]Caldara, R. & Abdi, H. (2006) Simulating the “other-race” effect with autoassociative neural networks: Further evidence in favor of the face-space model.
Perception 35(5):659– 70. [RC]Caldara, R., Rossion, B., Bovet, P. & Hauert, C. A. (2004) Event-related potentials
and time course of the “other-race” face classification advantage. NeuroReport15(5):905 – 10. [RC]Caldara, R., Thut, G., Servoir, P., Michel, C. M., Bovet, P. & Renault, B. (2003)
Face versus non-face object perception and the “other-race” effect: A spatiotemporal event-related potential study. Clinical Neurophysiology 114(3):515 –28. [RC]
Caldara, R., Zhou, X. & Miellet, S. (2010) Putting culture under the “spotlight” reveals
universal information use for face recognition. PLoS ONE 5(3):e9708. [RC]Calder, A. J., Keane, J., Cole, J., Campbell, R. & Young, A. W. (2000a) Facial
expression recognition by people with Mobius syndrome. Cognitive Neuropsychology 17:73– 87. [aPMN]
Calder, A. J., Keane, J., Lawrence, A. D. & Manes, F. (2004) Impaired recogniton ofanger following damage to the ventral striatum. Brain 127:1958 – 69. [arPMN]
Calder, A. J., Keane, J., Manes, F., Antoun, N. & Young, A. W. (2000b) Impaired
recogniton and experience of disgust following brain injury. Nature Neuroscience 3:1077 – 78. [aPMN]
Calder, A. J., Lawrence, A. D. & Young, A. W. (2001) Neuropsychology of fear andloathing. Nature Reviews Neuroscience 2(5):352 – 63. [aPMN]
Calder, A .J. & Young, A. W. (2005) Understanding the recognition of facial identityand facial expression. Nature Review Neuroscience 6:641 – 51. [aPMN]
Calder, A. J., Young, A. W., Rowland, D., Perrett, D. I., Hodges, J. R. & Etcoff, N.
L. (1996) Face perception after bilateral amygdala damage: Differentiallysevere impairment of fear. Cognitive Neuropsychology 13:699 – 745. [aPMN]
Campeau, S. & Davis, M. (1995) Involvement of subcortical and cortical afferents tothe lateral nucleus of the amygdala in fear conditioning measured with fearpotentiated startle in rats trained concurrently with auditory and visual conditioned stimuli. Journal of Neuroscience 15:2312– 27. [rPMN]
Campos, J. J., Mumme, D., Kermoian, R. & Campos, R. G. (1994) A functionalist
perspective on the nature of emotion. The Japanese Journal of Research on
Emotions 2(1):1 – 20. [JJO]
Cannon, P. R., Hayes, A. E. & Tipper, S. P. (2009) An electromyographic investigation of the impact of task relevance on facial mimicry. Cognition andEmotion 23:918 – 29. [BChan, ES]
Carr, L., Iacoboni, M., Dubeau, M. C., Mazziotta, J. C. & Lenzi, G. L. (2003)
Neural mechanisms of empathy in humans: A relay from neural systems forimitation to limbic areas. Proceedings of the National Academy of Sciences USA
100:5497 – 502. [CH-H, aPMN]

Cary, M. S. (1978) The role of gaze in the initiation of conversation. Social Psychology Quarterly 41:269 – 71. [BChan]
Cashdan, E. (2004) Smiles, speech, and body posture: How women and men display
sociometric status and power. Journal of Nonverbal Behavior 22:209 – 28.
[aPMN]
Centorrino, S., Djemai, E., Hopfensitz, A., Milinski, M. & Seabright, P. (2010)
Smiling is a costly signal of cooperation opportunities: Experimental evidence
from a trust game. Unpublished manuscript. [SC]
Chakrabarti, B., Bullmore, E. & Baron-Cohen, S. (2006) Empathising with basic
emotions: Common and discrete neural substrates. Social Neuroscience 1(3 –4):364– 84. [BChak]
Chakrabarti, B., Kent, L., Suckling, J., Bullmore, E. T. & Baron-Cohen, S. (2006)
Variations in human cannabinoid receptor (CNR1) gene modulate striatal
response to happy faces. European Journal of Neuroscience 23:1944– 48.
[aPMN]
Chance, M. R. A. (1962) The interpretation of some agonistic postures: The role of“cut-off” acts and postures. Symposium of the Zoological Society 8:71– 89.
[LH]
Chartrand, T. L. & Bargh, J. A. (1999) The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology
76:893 – 910. [PBri]
Chawarska, K. & Shic, F. (2009) Looking but not seeing: Atypical visualscanning and recognition of faces in 2 and 4-year old children with Autism
Spectrum Disorder. Journal of Autism and Developmental Disorders
39:1663– 72. [AS]Chen, Y. H., Dammers, J., Boers, F., Leiberg, S., Edgar, J. C., Roberts, T. P. L. &Mathiak, K. (2009) The temporal dynamics of insula activity to disgust andhappy facial expressions: A magnetoencephalography study. NeuroImage
47:1921– 28. [J-MF-D]
Cheng, Y., Chou, K.-H., Decety, J., Chen, I.-Y., Hung, D., Tzeng, O. J.-L. & Lin,
C.-P. (2009) Sex differences in the neuroanatomy of human mirror-neuron
system: A voxel-based morphometric investigation. Neuroscience 158:713– 20.
[ES]
Choe, Y. & Smith, N. H. (2006) Motion-based autonomous grounding: Inferring
external world properties from internal sensory states alone. In: Proceedings ofthe 21st National Conference on Artificial Intelligence (AAAI 2006), ed. Y. Gil& R. Mooney, pp. 936 – 41. AAAI Press. [TAM]
Choe, Y., Yang, H.-F. & Chern-Yeow Eng, D. (2007) Autonomous learning of thesemantics of internal sensory states based on motor exploration. International
Journal of Humanoid Robotics 4:211– 43. [TAM]
Chong, S. E., Werker, J. F., Russell, J. A. & Carroll, J. M. (2003) Three facial
expressions mothers direct to their infants. Infant and Child Development
12(3):211 – 32. [J-MF-D]
Chong, T. T. J., Williams, M. A., Cunnington, R. & Mattingley, J. B. (2008) Selective
attention modulates inferior frontal gyrus activity during action observation.
NeuroImage 40(1):298 – 307. [BChak]
Clore, G. L. & Storbeck, J. (2006) Affect as information about liking, efficacy, andimportance. In: Hearts and minds: Affective influences on social cognition andbehaviour, ed. J. Forgas, pp. 123 – 42. Psychology Press. [aPMN]
Cockburn, J., Bartlett, M., Tanaka, J., Movellan, J., Pierce, M. & Schultz, R. (2008)
SmileMaze: A tutoring system in real-time facial expression perception andproduction for children with autism spectrum disorder. Paper presented at theInternational Conference on Automatic Face and Gesture Recognition,Workshop on Facial and Bodily Expressions for Control and Adaptation ofGames, Amsterdam, The Netherlands, September 17 –19, 2008. [MSB]
Conty, L., Dezecache, G. & Grèzes, J. (2010) The spatio-temporal integration ofsocial visual cues: A study coupling EEG and fMRI. Poster presented at the
16th International Conference on Functional Mapping of the Human Brain,
Barcelona, Spain, June 6 –10, 2010. Abstract forthcoming in a supplement of
NeuroImage. [LC]Coussi-Korbel, S. (1994) Learning to outwit a competitor in mangabeys (Cercocebus torquatus torquatus). Journal of Comparative Psychology 108:164– 71.
[aPMN]
Cristinzio, C., N’Diaye, K., Seeck, M., Vuilleumier, P. & Sander, D. (2010) Integration of gaze direction and facial expression in patients with unilateral
amygdala damage. Brain 133(Part 1):248 – 61. [LC]Csibra, G. (2007) Action mirroring and action interpretation: An alternative
account. In: The sensorimotor foundations of higher cognition. Attention andperformance XXII, ed. P. Haggard, Y. Rossetti & M. Kawato, pp. 435 – 59.
Oxford University Press. [JK]Csibra, G. & Gergely, G. (2009) Natural pedagogy. Trends in Cognitive Sciences
13(4):148 – 53. [LC]D’Entremont, B., Hains, S. & Muir, D. (1997) A demonstration of gaze following in
3- to 6-month olds. Infant Behavior and Development 20:569 – 72. [LC]
Dalton, K. M., Nacewicz, B. M., Johnstone, T., Schaefer, H. S., Gernsbacher, M. A.,
Goldsmith, H. H., Alexander, A. & Davidson, R. J. (2005) Gaze fixation and theneural circuitry of face processing in autism. Nature Neuroscience 8:519 – 26.
[BChak, aPMN]

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

471

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Damasio, A. R., Grabowski, T. J., Bechara, A., Damasio, H., Ponto, L. L. B., Parvizi,
J. & Hichwa, R. D. (2000) Subcortical and cortical brain activity during thefeeling of self-generated emotions. Nature Neuroscience 3:1049 – 56. [aPMN]
Dapretto, M., Davies, M. S., Pfeifer, J. H., Scott, A. A., Sigman, M., Bookheimer, S.
Y. & Iacoboni, M. (2005) Understanding emotions in others: Mirror neuron
dysfunction in children with autism spectrum disorders. Nature Neuroscience
9:28 – 30. [PW]
Darwin, C. (1872) The expression of the emotions in man and animals. John Murray.
[JJO]
Darwin, C. (1872/1965) The expression of emotions in man and animals. John Murray/
University of Chicago Press. (Original work published in 1872). [J-MF-D]
Darwin, C. (1872/1998) The expression of the emotions in man and animals. Oxford
University Press. [aPMN]Davidson, R. J. (1993) The neuropsychology of emotion and affective style. In:
Handbook of emotion, ed. M. Lewis & J. M. Haviland, pp. 143 – 54. Guilford
Press. [aPMN]Davidson, R. J., Ekman, P., Saron, C., Senulis, J. & Friesen, W. (1990) Approachwithdrawal and cerebral asymmetry: Emotional expression and brain physiology I. Journal of Personality and Social Psychology 58:330 – 41. [aPMN]Davidson, R. J. & Hugdahl, K., eds. (1995) Brain asymmetry. MIT Press. [BChan]Davidson, R. J. & Irwin, W. (1999) The functional neuroanatomy of emotion andaffective style. Trends in Cognitive Science 3:11 – 21. [aPMN]de Gelder, B., Vroomen, J., Pourtois, G. & Weiskrantz, L. (1999) Non-conscious
recognition of affect in the absence of striate cortex. NeuroReport 10:3759 – 63.
[rPMN]de Waal, F. B. M. (1989) Peacemaking among primates. Harvard University Press.
[LH]
de Waal, F. B. M. (2003) Darwin’s legacy and the study of primate visual communication. Annals of the New York Academy of Sciences 1000:7 –31. [rPMN]
Decety, J. & Chaminade, T. (2003) Neural correlates of feeling sympathy.
Neuropsychologia 41:127 – 38. [aPMN]
Decety, J. & Chaminade, T. (2004) The neurophysiology of imitation and intersubjectivity. In: ed. S. Hurley & N. Chater. Perspectives on imitation: Fromneuroscience to social science, pp. 119 – 40. MIT Press. [aPMN]
Decety, J. & Grèzes, J. (1999) Neural mechanisms subserving the perception ofhuman actions. Trends in Cognitive Science 3:172 – 78. [aPMN]
Decety J. & Grèzes, J. (2006) The power of simulation: Imagining one’s own andother’s behavior. [Special Issue]. Social Cognitive Neuroscience of Cognitive
Brain Research 1079:4 – 14. [aPMN]
Deery S. & Kinnie N. (2002) Call centres and beyond: Athematic evaluation.
Human Resource Management Journal 12:3 – 13. [FB]
DeMarree, K. G., Briñol, P., Petty, R. E. & Smith, K. R. (2010) Facial validation
through metacognitive processes. Unpublished manuscript, Texas Tech University. [PBri]
Dick, A. S., Goldin-Meadow, S., Hasson, U., Skipper, J. I. & Small, S. L. (2009) Cospeech gestures influence neural activity in brain regions associated withprocessing semantic information. Human Brain Mapping 30(11):3509 – 26.
[MWA]
Dimberg, U. & Petterson, M. (2000) Facial reactions to happy and angry facial
expressions: Evidence for right hemisphere dominance. Psychophysiology
37:693 – 96. [DL]Dimberg, U. & Thunberg, M. (1998) Rapid facial reactions to emotional facial
expressions. Scandinavian Journal of Psychology 39(1):39 – 45. [aPMN]
Dimberg, U., Thunberg, M. & Elmehed, K. (2000) Unconscious facial reactions toemotional facial expressions. Psychological Science 11:86– 89. [BChan, DL]
Dinstein, I., Gardner, J. L., Jazayeri, M. & Heeger, D. J. (2008) Executed andobserved movements have different distributed representations in humanaIPS. The Journal of Neuroscience 28:11231 –39. [aPMN]
Donovan, W. & Leavitt, L. (1980) Physiologic correlates of direct and averted gaze.
Biological Psychology 10:189 – 99. [aPMN]Doron, N. N. & Ledoux, J. E. (1999) Organization of projections to the lateral
amygdala from auditory and visual areas of the thalamus in the rat. Journal ofComparative Neurology 412:383– 409. [rPMN]
Duchenne de Boulogne, C.-B. (1862) The mechanism of human facial expression.
Jules Renard. [aPMN]
École Nationale Supériure des Beaux-Arts (1999) Duchenne de Boulogne 1806 –1875. Beaux-Arts de Paris. [aPMN]
Edinger, J. A. & Patterson, M. L. (1983) Nonverbal involvement and social control.
Psychological Bulletin 93(1):30 – 56. [LH]
Effron, D. A., Niedenthal, P. M., Gil, S. & Droit-Volet, S. (2006) Embodied temporal perception of emotion. Emotion 6:1 – 9. [ES]Eibl-Eibesfeldt, I. (1971) Love and hate: The natural history of behavior patterns.
Holt, Rinehard & Winston. [JJO]
Eibl-Eibesfeldt, I. (1972) Similarities and differences between cultures in
expressive movements. Non-verbal communication. Cambridge University
Press. [aPMN]
Ekman, P. (1989) The argument and evidence about universals in facial expressionsof emotion. In: Handbook of psychophysiology: The biological psychology of472

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

the emotions and social processes, ed. H. Wagner & A. Manstead, pp. 143 – 64.
John Wiley. [aPMN]
Ekman, P. (1994) Strong evidence for universals in facial expressions: A reply to
Russell’s mistaken critique. Psychological Bulletin 115:268– 87. [J-MF-D,aPMN]
Ekman, P. (1997) Should we call it expression or communication? Innovations in
Social Science Research 10(4):333 –44. [MM]Ekman, P. (2001) Telling lies: Clues to deceit in the marketplace, politics, andmarriage. W.W. Norton. [PBou, aPMN]
Ekman, P., Davidson, R. & Friesen, W. V. (1990) The Duchenne smile: Emotional
expression and brain physiology, II. Journal of Personality and Social Psychology 58:342 – 53. [J-MF-D, arPMN]
Ekman, P. & Friesen, W. V. (1978) Facial action coding system: A technique for themeasurement of facial movement. Consulting Psychologists Press. [BChak,aPMN]
Ekman, P. & Friesen, W. V. (1982) Felt, false and miserable smiles. Journal of
Nonverbal Behavior 6:238– 52. [arPMN]
Ekman, P., Friesen, W. V. & Ancoli, S. (1980) Facial signs of emotional experience.
Journal of Personality and Social Psychology 39:1125 – 34. [BChan, aPMN]
Ekman, P. & Heider, K. G. (1988) The universality of a contempt expression: A
replication. Motivation and Emotion 12:303 – 308. [aPMN]
Elfenbein, H. A. & Ambady, N. (2002) On the universality and cultural specificity ofemotion recognition: A meta-analysis. Psychological Bulletin 128:203– 35.
[J-MF-D]
Elliott, R., Friston, K. J. & Dolan, R. J. (2000) Dissociable neural responses inhuman reward systems. Journal of Neuroscience 20:6159 –65. [aPMN]
Ellsworth, P. & Carlsmith, J. M. (1973) Eye contact and gaze aversion in an
aggressive encounter. Journal of Personality and Social Psychology 28(2):280–
92. [LH]
Elsabbagh, M., Volein, A., Csibra, G., Holmboe, K., Garwood, H., Tucker, L.,
Krljes, S., Baron-Cohen, S., Bolton, P., Charman, T., Baird, G. & Johnson, M.
H. (2009) Neural correlates of eye gaze processing in the infant broader autism
phenotype. Biological Psychiatry 65:31 – 38. [AS]
Enticott, P. G., Johnston, P. J., Herring, S. E., Hoy, K. E. & Fitzgerald, P. B. (2008)
Mirror neuron activation is associated with facial emotion processing. Neuropsychologia 46:2851– 54. [CH-H]
Erdelyi, M. H. (1974) A new look at the new look: Perceptual defense and vigilance.
Psychological Review 81(1):1 – 25. [JSL]
Fadiga, L., Fogassi, L., Pavesi, G. & Rizzolatti, G. (1995) Motor facilitation during
action observation: A magnetic stimulation study. Journal of Neurophysiology
73:2608 –11. [aPMN]
Farroni, T., Csibra, G., Simion, F. & Johnson, M. H. (2002) Eye contact detection inhumans from birth. Proceedings of the National Academy of Sciences USA
99(14):9602 – 605. [LC, aPMN]
Farroni, T., Mansfield, E. M., Lai, C. & Johnson, M. H. (2003) Infants perceiving
and acting on the eyes: Tests of an evolutionary hypothesis. Journal of Experimental Child Psychology 85(3):199 – 212. [LC]Farroni, T., Massaccesi, S., Pividori, D. & Johnson, M. (2004) Gaze following in
newborns. Infancy 5(1):39– 60. [aPMN]
Fehr, E. & Gächter, S. (2000) Cooperation and punishment in public goods
experiments. The American Economic Review 90(4):980– 94. [SC]Feldman, R. & Eidelman, A. I. (2009) Biological and environmental initial conditions shape the trajectories of cognitive and social-emotional development
across the first years of life. Developmental Science 12(1):194– 200. [JES]
Feldman, R., Granat, A., Pariente, C., Kanety, H., Kuint, J. & Gilboa-Schechtman,
E. (2009) Maternal depression and anxiety across the postpartum year andinfant social engagement, fear regulation, and stress reactivity. Journal of theAmerican Academy of Child and Adolescent Psychiatry 48(9):919 –27. [JES]
Feldman, R., Greenbaum, C. W. & Yirmiya, N. (1999) Mother-infant affect synchrony as an antecedent of the emergence of self-control. Developmental
Psychology 35(1):223 –31. [JES]
Feldman-Barrett, L. & Niedenthal, P. M. (2004) Valence focus and perceptions offacial affect. Emotion 4:266– 74. [aPMN]
Fernández-Dols, J. M. & Carroll, J. M. (1997) Is the meaning perceived in facial
expression independent of its context? In: The psychology of facial expression,
ed. J. A. Russell & J. M. Fernández-Dols, pp. 275 – 94. Cambridge University
Press. [J-MF-D]
Fernández-Dols, J. M. & Russell, J. A. (2003) Emotion, affect, and mood in social
judgments. In: Handbook of psychology, vol. 5: Personality and social psychology, ed. T. Millon & M. J. Lerner, pp. 283 – 98. Wiley. [J-MF-D]
Fernández-Dols, J. M., Carrera, P., Barchard, K. & Gacitua, M. (2008) False recognition of facial expressions of emotion: Causes and consequences. Emotion
8:530– 39. [J-MF-D]
Feygin, D. L., Swain, J. E. & Leckman, J. F. (2006) The normalcy of neurosis:
Evolutionary origins of obsessive-compulsive disorder and related behaviors.
Progress in Neuro-Psychopharmacology and Biological Psychiatry 30(5):854–
64. [JES]
Fineman, S., ed. (2000) Emotion in organizations. Sage. [FB]

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Fischer, A. & Roseman, I. J. (2007) Beat them or ban them: Characteristics andsocial functions of anger and contempt. Journal of Personality and Social
Psychology 93:103 – 15. [aPMN]
Fischer, M. H. & Zwaan, R. A. (2008) Embodied language – A review of the role ofthe motor system in language comprehension. Quarterly Journal of Experimental Psychology 61(6):825– 50. [JSL]
Fiske, A. P. (1992) The four elementary forms of sociality: Framework for a unified
theory of social relations. Psychological Review 99(4):689– 723. [LH]
Fiske, S. T., Cuddy, A. J. C. & Glick, P. (2006) Universal dimensions of social cognition: warmth and competence. Trends in Cognitive Sciences 11:77–83. [JMV]
Fitzgerald, D. A., Angstadt, M., Jelsone, L. M., Nathan, P. J. & Phan, K. L. (2006)
Beyond threat: Amygdala reactivity across multiple expressions of facial affect.
Neuroimage 30:1441– 48. [aPMN]
Flack, J. C. & de Waal, F. B. M. (2007) Context modulates signal meaning inprimate communication. Proceedings of the National Academy of Science USA
104(5):1581 – 86. [MM]Fodor, J. A. (1975) The language of thought. Crowel. [aPMN]
Fodor, J. A. (1985) Précis of the modularity of mind. Behavioral and Brain Sciences
8:1– 42. [DL]Fogel, A., Nelson-Goens, G., Hsu, H. & Shapiro, A. (2000) Do different infant
smiles reflect different positive emotions? Social Development 9:497– 520.
[aPMN]
Fraley, R. C., Niedenthal, P. M., Marks, M. J., Brumbaugh, C. C. & Vicary, A.
(2006) Adult attachment and the perception of facial expressions of emotion:
Probing the hyperactivating strategies underlying anxious attachment. Journalof Personality 74:1163 – 90. [aPMN]Frank, M. G. (2002) Smiles, lies, and emotion. In: An empirical reflection on thesmile, ed. M. H. Abel, pp. 15– 44. Edwin Mellen Press. [BChan, aPMN]Frank, M. G., Ekman, P. & Friesen, W. V. (1993) Behavioral markers and recognizability of the smile of enjoyment. Journal of Personality and Social Psychology 64:83– 93. [aPMN]Frank, M., Ekman, P. & Friesen, W. (1997) Behavioral markers and recognizabilityof the smile of enjoyment. What the face reveals: Basic and applied studies of
spontaneous expression using the Facial Action Coding System (FACS), pp.
217 – 42. Oxford University Press. [aPMN]Frank, M. & Stennett, J. (2001) The forced-choice paradigm and the perception of
facial expression of emotion. Journal of Personality and Social Psychology
80:75– 85. [aPMN]Frank, T. (1988) Passions within reason: The strategic role of the emotions. W.
W. Norton. [SC]Fridlund, A. (1991) The sociality of solitary smiling: Potentiation by an implicit
audience. Journal of Personality and Social Psychology 60:229 – 40. [aPMN]
Fridlund, A. J. (2002) The behavioral ecology view of smiling and other facial
expressions. In: An empirical reflection on the smile, ed. M. H. Abel. Edwin
Mellen Press. [aPMN]
Froggatt, K. (1998) The place of metaphor and language in exploring nurses’
emotional work. Journal of Advanced Nursing 28:332 –38. [FB]
Fusar-Poli, P., Placentin, A., Carletti, F., Landi, P., Alle, P., Surguladze, S., Benedetti, F., Abbamont, M., Gasparott, R., Baral, F., Perez, J., McGuire, P. &Politi, P. (2009) Functional atlas of emotional faces processing: A voxel-based
meta-analysis of 105 functional magnetic resonance imaging studies. Journal of
Psychiatry and Neuroscience 34(6):418– 32. [aPMN]
Gallese, V. (2003) The roots of empathy: The shared manifold hypothesis and theneural basis of intersubjectivity. Psychopathology 36:171 – 80. [aPMN]
Gallese, V. (2005) ’Being like me’: Self-other identity, mirror neurons, and empathy.
Perspectives on imitation: From neuroscience to social science, vol. 1. Mechanisms of imitation and imitation in animals, pp. 101 – 118. MIT Press. [aPMN]
Gallese, V. (2007) Embodied simulation: From mirror neuron systems to interpersonal relations. Novartis Foundation Symposium 2278:3 – 12. [aPMN]
Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. (1996) Action recognition in the
premotor cortex. Brain 119:593 – 609. [aPMN]
Gallese V. & Goldman, A. (1999) Mirror neurons and the simulation theory ofmind-reading. Trends in Cognitive Science 2:493– 500. [aPMN]
Gallese, V. & Lakoff, G. (2005) The brain’s concepts: The role of the sensory-motor
system in conceptual knowledge. Cognitive Neuropsychology 22:455– 79.
[aPMN]
Gans, N., Koole, G. & Mandelbaum, A. (2003) Telephone call centers: Tutorial,
review, and research Prospects. Manufacturing and Service Operations Management 5(2):79– 141. [FB]
Geary, D. C. (2005) The origin of the mind: Evolution of brain, cognition, andgeneral intelligence. American Psychological Association. [JMV]George, N. & Conty, L. (2008) Facing the gaze of others. Neurophysiologie Clinic/
Clinical Neurophysiology 38(3):197– 207. [LC]George, N., Driver, J. & Dolan, R. J. (2001) Seen gaze-direction modulates fusiform
activity and its coupling with other brain areas during face processing. NeuroImage 13:1102– 12. [AS]
Gepner, B. & Feron, F. (2009) Autism: A world changing too fast for a mis-wired
brain? Neuroscience and Biobehavioral Reviews 33:1227 – 42. [MSB]

Giacino, J., Ashwal, S., Childs, N., Cranford, R., Jennett, B., Katz, D., Kelly, J.,
Rosenberg, J., Whyte, J. & Zafonte, R. (2002) The minimally conscious state:
Definition and diagnostic criteria. Neurology 58:349 – 53. [CC]Giacino, J., Kalmar, K. & Whyte, J. (2004) The JFK coma recovery scale-revised:
Measurement characteristics and diagnostic utility. Archives of Physical
Medicine and Rehabilitation 85(12):2020– 29. [CC]Glenberg, A. M. (1997) What memory is for. Behavioral and Brain Sciences 20:1–
55. [MWA, EM]
Golan, O. & Baron-Cohen, S. (2006) Systemizing empathy: Teaching adults withAsperger’s Syndrome and High Functioning Autism to recognize emotions
using interactive multimedia. Development and Psychopathology 18(2):589–
617. [BChak, rPMN]
Goldman, A. I. (2006) Simulating minds: The philosophy, psychology, and neuroscience of mindreading. Oxford University Press. [J-MF-D]
Goldman, A. I. & Sripada, C. S. (2005) Simulationist models of face-based emotion
recognition. Cognition 94(3):193 – 213. [BChak, CH-H, aPMN, ES]
Graham, J. A. & Heywood, S. (1976) The effects of elimination of hand gesture andof verbal codability on speech performance. European Journal of Social Psychology 5:189– 95. [MWA]
Grandey, A., Fisk, G., Mattila, A., Jansen, K. & Sideman, L. (2005) Is “service with asmile” enough? Authenticity of positive displays during service encounters.
Organizational Behavior and Human Decision Processes 96(1):38 – 55. [FB]
Gullberg, M. & Holmqvist, K. (1999) Keeping an eye on gestures: Visual perceptionof gestures in face-to-face communication. Pragmatics & Cognition 7:35 – 63.
[MWA]
Gullberg, M. & Kita, S. (2009) Attention to speech-accompanying gestures: Eyemovements and information uptake. Journal of Nonverbal Behavior 33(4):251 – 77. [MWA]
Hadjikhani, N., Hoge, R., Snyder, J. & de Gelder, B. (2008) Pointing with the eyes:
The role of gaze in communicating danger. Brain and Cognition 68(1):1 – 8.
[LC]
Haidt, J. & Keltner, D. (1999) Culture and facial expression: Open-ended methods
find more expressions and a gradient of recognition. Cognition and Emotion
13:225 – 66. [aPMN]
Hains, S. M. J. & Muir, D. W. (1996) Infant sensitivity to adult eye direction. Child
Development 67:1940– 51. [aPMN]
Haith, M. M. (1972) The forgotten message of the infant smile. Merrill-Palmer
Quarterly 18:321. [aPMN]
Halberstadt, J. & Niedenthal, P. M. (2001) Effects of emotion concepts on perceptual memory for emotional expressions. Journal of Personality and Social
Psychology 81:587 – 98. [aPMN]
Halberstadt, J., Winkielman, P., Niedenthal, P. & Dalle, N. (2009) Emotional
conception: How embodied emotion concepts guide perception and facial
action. Psychological Science 20:1254 – 61. [J-MF-D, DL, arPMN]Hall, J. A., Coats, E. J. & LeBeau, L. S. (2005) Nonverbal behavior and the verticaldimension of social relations: A meta-analysis. Psychological Bulletin131(6):898 – 924. [LH]
Hall, J. A., Hogan, T. G. & Carter, J. D. (2002) Assigned and felt status in relation toobserver-coded and participant-reported smiling. Journal of Nonverbal Behavior 26(2):63– 81. [MM]Hall, J. K., Hutton, S. B. & Morgan, M. J. (2010) Sex differences in scanning faces:
Does attention to the eyes explain female superiority in facial expression recognition? Cognition and Emotion 24:629 – 37. [doi:10.1080/
02699930902906882]. [rPMN, ES]
Hamann, S. B., Stefanacci, L., Squire, L. R., Adolphs, R., Tranel, D., Damasio, H. &Damasio, A. (1996) Recognizing facial emotion. Nature 379:497. [aPMN]
Hamon-Hill, C. & Barresi, J. (2008a) I feel the same way: Recognizing facial
emotion as appropriate to context. [Abstract] Canadian Psychology Abstracts
49:2a, 11. [CH-H]
Hamon-Hill, C. & Barresi, J. (2008b) I remember your smile: Recognizing facial
emotions from a brief acquaintance. [Abstract]. Abstracts of the 2008 CSBBCS(Canadian Society for Brain, Behaviour, and Cognitive Science) Annual
Meeting. Canadian Journal of Experimental Psychology 62(4):261–321. [CH-H]
Hamon-Hill, C. & Barresi, J. (2009) Disrupted embodied simulation while judging
appropriateness of facial emotion to context. Paper presented at the Joint
International Conference with the Canadian Society for Brain, Behavior, andCognitive Sciences and the Experimental Psychology Society, York, United
Kingdom, July 8-10, 2009. [CH-H]
Harmon-Jones, E. & Segilman, J. (2001) State anger and prefrontal brain activity:
Evidence that insult-related relative left-prefrontal activation is associated withexperienced anger and aggression. Journal of Personality and Social Psychology 80:797– 803. [BChan]
Harnad, S. (1990) The symbol grounding problem. Physica D 42:335 – 46. [EM]Harver, A., Katkin, E. S. & Bloch, E. (1993) Signal-detection outcomes on heartbeat
and respiratory resistance detection tasks in male and female subjects. Psychophysiology 30:223 – 30. [rPMN]
Hasson, O. (1994) Cheating signals. Journal of Theoretical Biology 167:223– 38.
[MM]

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

473

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Hatfield, E., Cacioppo, J. T. & Rapson, R. L. (1992) Primitive emotional contagion.
In: Review of personality and social psychology: Emotion and social behavior,
vol. 14, ed. M. S. Clark, pp. 151 – 77. Sage. [aPMN]
Hatfield, E., Cacioppo, J. T. & Rapson R. L. (1993) Emotional contagion. Current
Directions in Psychological Science 2:96 – 99. [aPMN]
Haxby, J., Hoffman, E. & Ida Gobbini, M. (2000) The distributed human neuralsystem for face perception. Trends in Cognitive Sciences 4:223 – 33. [BChak]
Heberlein, A. S. & Atkinson, A. (2009) Neuroscientific evidence for simulation andshared substrates in emotion recognition: Beyond faces. Emotion Review
1:162 –77. [J-MF-D, aPMN]
Heberlein, A. S., Padon, A. A., Gillihan, S. J., Farah, M. J. & Fellows, L. K. (2008)
Ventromedial frontal lobe plays a critical role in facial emotion recognition.
Journal of Cognitive Neuroscience 20:721 – 33. [J-MF-D]
Hecht, M. A. & LaFrance, M. (1998) License or obligation to smile: The effect ofpower and sex on amount and type of smiling. Personality and Social Psychology Bulletin 24(12):1332 – 42. [aPMN]
Hennenlotter, A., Dresel, C., Castrop, F., Ceballos-Baumann, A. O., Wohlschläger,
A. M. & Haslinger, B. (2009) The link between facial feedback and neuralactivity within central circuitries of emotion: New insights from Botulinum
Toxin-induced denervation of frown muscles. Cerebral Cortex 19:537–42.
[aPMN]
Hennenlotter, A., Schroeder, U., Erhard, P., Catrop, F., Haslinger, B., Stoecker,
D., Lange, K. W. & Ceballos-Baumann, A. O. (2005) A common neural basis
for receptive and expressive communication of pleasant facial affect. NeuroImage 26:581– 91. [JK, aPMN]
Hermans, E. J., Putman, P. & van Honk, J. (2006) Testosterone administration
reduces empathetic behavior: A facial mimicry study. Psychoneuroendocrinology 31:859 –66. [rPMN, ES]
Hess, U., Adams, R. B., Jr. & Kleck, R. E. (2005) Who may frown and who should
smile? Dominance, affiliation, and the display of happiness and anger. Cognition and Emotion 19:515– 36. [aPMN]
Hess, U., Adams, R. B., Jr. & Kleck, R. E. (2007) When two do the same, it might
not mean the same: The perception of emotional expressions shown by men
and women. In: Group dynamics and emotional expression, pp. 33– 50.
Cambridge University Press. [aPMN]
Hess, U. Adams, R. B., Jr. & Kleck, R. E. (2009a) The categorical perception ofemotions and traits. Social Cognition 27:319 – 25. [aPMN]
Hess, U., Adams, R. B., Jr. & Kleck, R. E. (2009b) The face is not an empty
canvas: How facial expressions interact with facial appearance. PhilosophicalTransactions of the Royal Society B: Biological Sciences 364:3497 – 504.
[rPMN]
Hess, U., Banse, R. & Kappas, A. (1995) The intensity of facial expression isdetermined by underlying affective state and social situation. Journal of Personality and Social Psychology 69:280 – 88. [aPMN]
Hess, U., Beaupre, M. & Cheung, N. (2002) Who to whom and why – Culturaldifferences and similarities in the function of smiles. In: An empiricalreflection on the smile, ed. M. Abel, pp. 187 – 216. Edwin Mellen Press.
[aPMN, DAS]
Hess, U. & Blairy, S. (2001) Facial mimicry and emotional contagion to dynamic
emotional facial expressions and their influence on decoding accuracy. International Journal of Psychophysiology 40:129 – 41. [CH-H, aPMN]
Hess, U., Blairy, S. & Kleck, R. E. (2000) The influence of facial emotion displays,
gender, and ethnicity on judgments of dominance and affiliation. Journal of
Nonverbal Behavior 24(4):265 – 83. [BChak]
Hess, U., Blairy, S. & Philippot, P. (1999) Facial mimicry. In: The social context of
nonverbal behavior, ed. P. Philippot, R. Feldman & E. Coats, pp. 213 – 41.
Cambridge University Press. [aPMN]
Hess, U. & Bourgeois, P. (in press) You smile – I smile: Emotion expression insocial interaction. Biological Psychology. [aPMN]
Hess, U. & Fischer, A. (under review) Emotional contagion: What you see is notalways what you feel. [rPMN]
Hess, U. & Kirouac, G. (2000) Emotion expression in groups. In: Handbook of
emotion, 2nd edition, ed. M. Lewis & J. Haviland-Jones, pp. 368 – 81. Guilford
Press. [rPMN]
Hess, U. & Kleck, R. E. (1990) Differentiating emotion elicited and deliberateemotional facial expressions. European Journal of Social Psychology 20:369 –85. [BChan, aPMN]
Hess, U., Philippot, P. & Blairy, S. (1998) Facial reactions to emotional facial
expressions: Affect or cognition? Cognition and Emotion 12:509 – 32. [rPMN]
Hilgard, E. R. (1987) Psychology in America: A historical survey. Harcourt Brace,
Jovanovich. [EM]Hirshleifer, J. (1987) On the emotions as guarantors of threats and promises. In: The
latest on the best: Essays on evolution and optimality, ed. John Dupré. MIT
Press. [SC]
Hochschild, A. R. (1979) Emotion work, feeling rules, and social structure. American Journal of Sociology 85(3):551. [FB]
Hochschild, A. R. (1983) The managed heart : Commercialization of human feeling.
University of California Press. [FB]474

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Hochschild, A. R. (2005) On the edge of the time bind: Time and market culture.
Social Research 72(2):339– 54. [FB]
Hoehl, S. & Striano, T. (2008) Neural processing of eye gaze and threat-related
emotional facial expressions in infancy. Child Development 79(6):1752– 60.
[LC]
Holle, H., Gunter, T. C., Rüschemeyer, S.-A., Hennenlotter, A. & Iacoboni, M.
(2008) Neural correlates of the processing of co-speech gestures. NeuroImage
39:2010 –24. [MWA]
Holmes, A., Green, S. & Vuilleumier, P. (2005) The involvement of distinct visualchannels in rapid attention towards fearful facial expressions. Cognition andEmotion 19:899 – 922. [rPMN]
Hooker, C. I., Paller, K. A., Gitelman, D. R., Parrish, T. B., Mesulam, M. & Reber,
P. J. (2003) Brain networks for analyzing eye gaze. Cognitive Brain Research
17(2):406 – 18. [BChak, aPMN]
Hostetter, A. B. & Alibali, M. W. (2008) Visible embodiment: Gestures as simulated
action. Psychonomic Bulletin and Review 15:495 –514. [MWA]
Hostetter, A. B. & Alibali, M. W. (2010) Language, gesture, action! A test of thegesture as Simulated Action framework. Journal of Memory and Language
63:245 – 57. [MWA]
Houde, S., Simard, A. & Hess, U. (2009) I show you how you feel: Mimicry as afunction of assigned emotional state. Paper presented at the 49th Annual
Meeting of the Society for Psychophysiological Research, Berlin, Germany,
October 21-24, 2009. [rPMN]
Iacoboni, M., Molnar-Szakacs, I., Gallese, V., Buccino, G., Mazziotta, J. C. & Rizzolatti, G. (2005) Grasping the intentions of others with one’s own mirror
neuron system. PLoS Biology 3(3):529 –35. [CL]Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, J. C. & Rizzolatti,
G. (1999) Cortical mechanisms of human imitation. Science 286(5449):2526–
28. [CL, aPMN]
Iizuka, Y. (1992) Extraversion, introversion, and visual interaction. Perceptual andMotor Skills 74:43– 50. [aPMN]
Izard, C. (1971) The face of emotion. Appleton Century Crofts. [aPMN, JJO]
Jack, R. E., Blais, C., Scheepers, C., Schyns, P. G. & Caldara, R. (2009) Cultural
confusions show that facial expressions are not universal. Current Biology
19(18):1543 – 48. [RC, CL]
Jacob, P. (2008) What do mirror neurons contribute to human social cognition?
Mind and Language 23:190 – 223. [aPMN]
Jacob, P. (2009) The tuning-fork model of human social cognition: A critique.
Consciousness and Cognition 18:229 – 43. [aPMN]
Jakobs, E., Manstead, A. S. R. & Fischer, A. H. (1999) Social motives and emotional
feelings as determinants of facial displays: The case of smiling. Personality andSocial Psychology Bulletin 25:424 – 35. [aPMN]
Jeannerod, M. (2001) Neural simulation of action: A unifying mechanism for motor
cognition. NeuroImage 14:S103– S109. [MWA]Johnson, M. H. (2005) Subcortical face processing. Nature Reviews Neuroscience
6:766– 74. [AS]Johnson-Laird, P. & Oatley, K. (1989) The language of emotions: An analysis of asemantic field. Cognition and Emotion 3:81 –123. [aPMN]
Jörn, P. W., Scharlemann, C. C., Eckel, A. K. & Wilson, R K. (2001) The value of asmile: Game theory with a human face. Journal of Economic Psychology
22(5):617 – 40. [TAM]
Kaffman, A. & Meaney, M. J. (2007) Neurodevelopmental sequelae of postnatal
maternal care in rodents: Clinical and research implications of molecularinsights. Journal of Child Psychology and Psychiatry 48(3 – 4):224 – 44. [JES]
Kagan, J. (2007) What is emotion? History, measures, and meanings. Yale University Press. [J-MF-D]
Kalat, J. W. & Shirota, N. M. (2006) Emotion, Thomson Wadsworth. [CMC-S]
Kawagoe, R., Takikawa, Y. & Hikosada, O. (1998) Expectation of reward
modulates cognitive signals in the basal ganglia. Nature Neuroscience 1:411 –16. [aPMN]
Keillor, J. M., Barrett, A. M., Crucian, G. P., Kortenkamp, S. & Heilman, K. M.
(2002) Emotional experience and perception in the absence of facial feedback.
Journal of the International Neuropsychological Society 8(1):130 – 35. [aPMN]
Kelly, D. J., Miellet, S. & Caldara, R. (2010) Culture shapes eye movements forvisually homogeneous objects. [Original Research Article]. Frontiers in Perception Science 1:6. [RC]Keltner, D. (1995) Signs of appeasement: Evidence for the distinct displays ofembarrassment, amusement, and shame. Journal of Personality and Social
Psychology 68(3):441 –54. [aPMN]
Keltner, D. & Gross, J. J. (1999) Functional accounts of emotion. Cognition andEmotion 13:467 – 80. [rPMN]
Keltner, D. & Haidt, J. (1999) Social function of emotions at four levels of analysis.
Cognition and Emotion 13:505 –21. [arPMN]
Keysers, C. & Gazzola, V. (2006) Towards a unifying neural theory of social cognition. Progress in Brain Research 156:379 – 401. [JSL]
Keysers, C. & Gazzola, V. (2007) Integrating simulation and theory of mind: Fromself to social cognition. Trends in Cognitive Sciences 11(5):194 – 96. [CL,aPMN]

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Keysers, C. & Perrett, D. (2004) Demystifying social cognition: A Hebbian perspective. Trends in Cognitive Science 8(11):501– 507. [BChak]
Keysers, C., Wicker, B., Gazzola, V., Anton, J. L., Fogassi, L. & Gallese, V. (2004) A
touching sight: SII/PV activation during the observation and experience of
touch. Neuron 42(2):335 – 46. [BChak, aPMN]
Kilts, C. D., Egan, G., Gideon, D. A., Ely, T. D. & Hoffman, J. M. (2003) Dissociable neural pathways are involved in the recognition of emotion in static anddynamic facial expressions. NeuroImage 18:156 – 68. [CH-H]
Kim, P., Feldman, R., Leckman, J. F., Mayes, L. C. & Swain, J. E. (under review)
Breastfeeding, brain activation to own infant cry, and maternal sensitivity.
Journal of Child Psychology and Psychiatry. [JES]
Kim, P., Leckman, J. F., Mayes, L. C., Newman, M. A., Feldman, R. & Swain, J. E.
(2010) Perceived quality of maternal care in childhood and structure andfunction of mothers’ brain. Developmental Science 13(4):662– 73. [JES]
Kimbara, I. (2006) On gestural mimicry. Gesture 6(1): 39 – 61. [MWA]
Kimbara, I. (2008) Gesture form convergence in joint description. Journal of
Nonverbal Behavior 32(2): 123 –31. [MWA]
Kirouac, G. & Hess, U. (1999) Group membership and the decoding of nonverbal
behavior. In: The social context of nonverbal behavior, pp. 182 – 210. Cambridge University Press. [aPMN]
Kitayama, S., Duffy, S., Kawamura, T. & Larsen, J. (2003) Perceiving an object andits context in different cultures: A cultural look at new look. Psychological
Science 14:201 –208. [CMC-S]
Kleinhans, N. M., Richards, T., Sterling, L., Stegbauer, K. C., Mahurin, R.,
Johnson, L. C., Greenson, J., Dawson, G. & Aylward, E. (2008) Abnormal
functional connectivity in autism spectrum disorders during face processing.
Brain 131:1000 – 12. [AS]Klineberg, O. (1940) Social psychology. Holt. [aPMN]
Klinnert, M., Campos, J., Sorce, J., Emde, R. & Svejda, M. (1983) Emotions as
behavior regulators in infancy: Social referencing in infancy. In: Emotion:
Theory, research and experience, ed. R. Plutchik & H. Kellerman, pp. 57 –86.
Academic Press. [aPMN]
Klucharev, V. & Sams, M. (2004) Interaction of gaze direction and facial expressions
processing: ERP study. NeuroReport 15:621 – 25. [aPMN]
Knapp, M. L. & Hall, J. A. (2005) Nonverbal communication in human interaction.
Wadsworth. [RC]Kosslyn, S. M. (1976) Can imagery be distinguished from other forms of internal
representation? Evidence from studies of information retrieval time. Memory
and Cognition 4:291– 97. [aPMN]
Kövecses, Z. (1990) Emotion concepts. Springer. [FB]Kövecses, Z. (2008) Metaphor and emotion. In: The Cambridge handbook of
metaphor and thought. ed. J. R. W. Gibbs, pp. 380 – 96. Cambridge University
Press. [FB]
Kraut, R. E. & Johnston, R. E. (1979) Social and emotional messages of smiling: An
ethological approach. Journal of Personality and Social Psychology 37:1539–
53. [aPMN]
Kringelbach, M. L. (2005) The human orbitofrontal cortex: Linking reward tohedonic experience. Nature Reviews Neuroscience 6(9):691– 702. [JES]
Krumhuber, E. & Kappas, A. (2005) Moving smiles: The role of dynamic components for the perception of the genuineness of smiles. Journal of Nonverbal
Behavior 29:3– 24. [J-MF-D, aPMN]
Krumhuber, E. G. & Manstead, A. S. R (2009) Can Duchenne smiles be feigned?
New evidence on felt and false smiles. Emotion 9:807– 20. [J-MF-D]
Krumhuber, E., Manstead, A. S. R., Cosker, D., Marshall, D., Rosin, P. L. &Kappas, A. (2007) Facial dynamics as indicators of trustworthiness and cooperative behavior. Emotion 7(4):730– 35. [SC, J-MF-D]
Krumhuber, E., Manstead, A. S. R. & Kappas, A. (2007) Temporal aspects of facial
displays in person and expression perception: The effects of smile dynamics,
head-tilt, and gender. Journal of Nonverbal Behavior 31:39 –56. [aPMN, ES]
Kunz, M., Prkachin, K. & Lautenbacher, S. (2009) The smile of pain. Pain 145:273 –75. [J-MF-D]
LaBarre, W. (1947) The culture basis of emotions and gestures. Journal of Personality 16:49– 68. [aPMN]
LaFrance, M. & Hecht, M. A. (1999) Option or obligation to smile: The effect ofpower and gender on facial expression. In: The social context of non-verbal
behaviour. Studies in emotion and social interactions, ed. P. Philippot, R. S.
Feldman & E. J. Coats, pp. 45 – 70. Cambridge University Press. [MM]Laird, J. D., Alibozak, T., Davainis, D., Deignan, K., Fontanella, K., Hong, J.,
Levy, B. & Pacheco, C. (1994) Individual differences in the effects of spontaneous mimicry on emotional contagion. Motivation and Emotion 18:231 –46.
[aPMN]
Lakin, J. L., Chartrand, T. L. & Arkin, R. M. (2008) I am too just like you: Nonconscious mimicry as an automatic behavioral response to social exclusion.
Psychological Science 19:816 – 22. [ES]Lakoff, G. (1987) Women, fire, and dangerous things: What categories reveal about
the mind. University of Chicago Press. [FB]
Lakoff, G. (1993) The synthax of metaphorical semantic role. In: Semantics and thelexicon, ed. J. Pustejovsky, pp. 27 – 36. Kluwer Academic. [FB]

Lakoff, G. & Johnson, M. (1980) Metaphors we live by. University of Chicago Press.
[FB]
Lakoff, G. & Johnson, M. (1999) Philosophy in the flesh: The embodied mind and its
challenge to western thought. Basic Books. [FB]
Landauer, T. K. & Dumais, S. T. (1997) A solution to Plato’s problem: The latentsemantic analysis theory of acquisition, induction, and representation ofknowledge. Psychological Review 104:211 – 40. [EM]
Lane, R., Chua, P. & Dolan, R. (1999) Common effects of emotional valence,arousal and attention on neural activation during visual processing of pictures.
Neuropsychologia 37:989 –97. [aPMN]
Lane, R. D., Reiman, E. M., Ahern, G. L., Schwartz, G. E. & Davidson, R. J. (1997)
Neuroanatomical correlates of happiness, sadness, and disgust. American
Journal of Psychiatry 154:926– 33. [aPMN]
Larson, E. B. & Yao, X. (2005) Clinical empathy as emotional labor in the patient–physician relationship. Journal of the American Medical Association
293(9):1100 – 106. [FB]
Laureys, S., Faymonville, M. E., Peigneux, P., Damas, P., Lambermont, B., Del
Fiore, G., Degueldre, C., Aerts, J., Luxen, A. & Franck, G. (2002) Corticalprocessing of noxious somatosensory stimuli in the persistent vegetative state.
NeuroImage 17:732 – 41. [CC]Laureys, S., Faymonville, M., Degueldre, C., Fiore, G., Damas, P., Lambermont,B., Janssens, N., Aerts, J., Franck, G., Luxen, A., Moonen, G., Lamy, M. &Maquet, P. (2000) Auditory processing in the vegetative state. Brain
123(8):1589 – 601. [CC]
Lawrence, A. D., Calder, A. J., McGowan, S. W. & Grasby, P. M. (2002) Selective
disruption of the recognition of facial expressions of anger. NeuroReport
13:881 – 84. [aPMN]
Lawrence, A. D., Chakrabarti, B. & Calder, A. J. (2004) Looking at happy andsad faces: An fMRI study. Paper presented at the Annual Meeting of
the Cognitive Neuroscience Society, San Diego, CA, April 18– 22, 2004.
[aPMN]
Lawrence, E., Shaw, P., Baker, D., Baron-Cohen, S. & David, A. (2004) Measuring
empathy – Reliability and validity of the empathy quotient. Psychological
Medicine 34:911 – 19. [BChak]
Leary, T. (1957) Interpersonal diagnosis of personality. Ronald. [JMV]
Leavitt, H. J. (2005) Top down: Why hierarchies are here to stay and how to manage
them more effectively. Harvard Business School Press. [LH]
Leckman, J. F. & Mayes, L. C. (1999) Preoccupations and behaviors associated withromantic and parental love. Perspectives on the origin of obsessive-compulsive
disorder. Child and Adolescent Psychiatry Clinics of North America 8(3):635–
65. [JES]
Ledoux, J. (1996) The emotional brain: The mysterious underpinnings of emotional
life. Simon & Shuster. [rPMN]
LeDoux, J. E. (1996) The emotional brain: The mysterious underpinnings ofemotional life. Simon and Schuster. [EM]LeDoux, J. E. (2007) Emotional memory. Scholarpedia 2(7):180. [aPMN]
Lee, T.-W., Dolan, R. J. & Critchley, H. D. (2007) Controlling emotional
expression: Behavioral and neural correlates of nonimitative emotional
responses. Cerebral Cortex 18(1):104– 13. [aPMN]
Lee, T.-W., Josephs, O., Dolan, R. J. & Critchley, H. D. (2006) Imitating
expressions: Emotion-specific neural substrates in facial mimicry. Social Cognitive and Affective Neuroscience 1(2):122 – 35. [aPMN]
Lenzi, D., Trentini, C., Pantano, P., Macaluso, E., Iacoboni, M., Lenzi, G. L. &Ammaniti, M. (2009) Neural basis of maternal communication and emotional
expression processing during infant preverbal stage. Cerebral Cortex 19(5):1124– 33. [rPMN, JES]
Leonard, C. M., Rolls, E. T., Wilson, F. A. & Baylis, G. C. (1985) Neurons in the
amygdala of the monkey with responses selective for faces. Behavioural Brain
Research 15(2):159 – 76. [PBou]
Leslie, K. R., Johnson-Frey, S. H. & Grafton, S. T. (2004) Functional imaging offace and hand imitation: Towards a motor theory of empathy. NeuroImage
21:601 – 607. [aPMN]
Likowski, K. U., Mühlberger, A., Seibt, B., Pauli, P. & Weyers, P. (2008) Modulation of facial mimicry by attitudes. Journal of Experimental Social Psychology
44:1065– 72. [aPMN]
Linke, R., De Lima, A. D., Schwegler, H. & Pape, H. C. (1999) Direct synaptic
connections of axons from superior colliculus with identified thalamo-amygdaloid projection neurons in the rat: Possible substrates of a subcortical visual
pathway to the amygdala. Journal of Comparative Neurology 403:158 –70.
[rPMN]
Lipp, O. V., Price, S. M. & Tellegen, C. L. (2009) No effect of inversion on attentional and affective processing of facial expressions. Emotion 9(2):248 –59.
[LH]
Lishner, D. A., Cooter, A. B. & Zald, D. H. (2008) Rapid emotional contagion andexpressive congruence under strong test conditions. Journal of Nonverbal
Behavior 32:225 –39. [J-MF-D]
Liu, C., Ge, Y., Mai, X. & Luo, Y. (under review) Eyes are windows to the Chinese
soul: Evidence from the detection of real and fake smiles. [CL]

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

475

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Lobmaier, J. S. & Perrett, D. I. (in press) The world smiles at me: Self-referential
positivity bias when interpreting direction of attention. Cognition andEmotion. [JSL]
Lobmaier, J. S., Tiddeman, B. & Perrett, D. I. (2008) Emotional expression
modulates perceived gaze direction. Emotion 8(4):573 – 77. [JSL]
Lohaus, A., Keller, H. & Voelker, S. (2001) Relationships between eye contact,maternal sensitivity, and infant crying. International Journal of Behavioral
Development 25:542 – 48. [aPMN]Longo, M. R. & Bertenthal, B. I. (2009) Attention modulates the specificity of
automatic imitation to human actors. Experimental Brain Research 192:739–44.
[DL]
Magee, J. C. & Galinsky, A. D. (2008) Social hierarchy: The self-reinforcing natureof power and status. Academy of Management Annals 2:351 – 98. [LH]
Magnee, M. J. C. M., de Gelder, B., van Engeland, H. & Kemner, C. (2007) Facial
electromyographic responses to emotional information from faces and voices
in individuals with pervasive developmental disorder. Journal of Child Psychology and Psychiatry 48:1122– 30. [PW]
Magnée, M. J., Stekelenburg, J. J., Kemner, C. & de Gelder, B. (2007) Similar facial
electromyographic responses to faces, voices, and body expressions. NeuroReport 18:369 – 72. [AS]Mahon, B. Z. & Caramazza, A. (2008) A critical look at the embodied cognition
hypothesis and a new proposal for grounding conceptual content. Journal of
Physiology – Paris 102:59 – 70. [EM]Maringer, M., Krumhuber, E., Fischer, A. H. & Niedenthal P. M. (in press) Beyond
smile dynamics: Mimicry and beliefs in judgments of smiles. Emotion. [BChan,
CL, arPMN]
Markman, A. B. & Dietrich, E. (2000) Extending the classical view of representation. Trends in Cognitive Sciences 4:470 – 75. [EM]Markus, H. R. & Kitayama, S. (1991) Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review 98:224– 53. [RC]Marsh, A. A., Adams, R. B. & Kleck, R. E. (2005) Why do fear and anger look theway they do? Form and social function in facial expressions. Personality andSocial Psychology Bulletin 31:73– 86. [JMV]Marshall, J. (2003) The canonical Smiley (and 1-line symbol) list. Available at: http://
www.astro.umd.edu/~marshall/smileys.html. [CL]Massimelli, M. (2007) The anencephalic newborn: Medical/legal and bioethicalissues. Panminerva Medica 49(2):83 – 96. [CC]Mathews, A., Fox, E., Yiend, J. & Calder, A. (2003) The face of fear: Effects ofeye gaze and emotion on visual attention. Visual Cognition 10(7):823– 35.
[JSL]
Matsumoto, D., Yoo, S. H. & Fontaine, J. (2008) Mapping expressive differences
around the world: The relationship between emotional display rules andindividualism versus collectivism. Journal of Cross-Cultural Psychology
39(1):55 – 74. [CMC-S]
McArthur, L. Z. & Baron, R. M. (1983) Toward an ecological theory of social
perception. Psychological Review 90:215 – 38. [aPMN]
McIntosh, D. (1996) Facial feedback hypotheses: Evidence, implications, anddirections. Motivation and Emotion 20:121 – 47. [aPMN]
McIntosh, D. N. (2006) Spontaneous facial mimicry, liking and emotional contagion. Polish Psychological Bulletin 37:31 – 42. [aPMN]
McIntosh, D. N., Reichmann-Decker, A., Winkielman, P. & Wilbarger, J. L. (2006)
When the social mirror breaks: Deficits in automatic, but not voluntary,
mimicry of emotional facial expressions in autism. Developmental Science
9:295 –302. [aPMN, AS, PW]
McNeil, N. M., Alibali, M. W. & Evans, J. L. (2000) The role of gesture in children’scomprehension of spoken language: Now they need it, now they don’t. Journalof Nonverbal Behavior 24:131 – 50. [MWA]
McNeill, D. (in press) Gesten der Macht und die Macht der Gesten (Gestures ofpower and the power of gestures) In: Gesten, Inszenierung, Aufführung undPraxis (Gesture, Staging, Performance, and Practice), ed. E. Fischer-Lichte &C. Wulf.. Wilhelm Fink. [MWA]
Mehrabian, A. (1969) Significance of posture and position in the communicationof attitude and status relationships. Psychological Bulletin 71(5):359– 72.
[LH]
Mehu, M. & Dunbar, R. I. M. (2008) Naturalistic observations of smiling andlaughter in human group interactions. Behaviour 145:1747 – 80. [MM]Mehu, M., Grammer, K. & Dunbar, R. I. M. (2007) Smiles when sharing. Evolution
and Human Behavior 28(6):415– 22. [SC, MM]
Melville, H. (1852/1996) Pierre, or, the ambiguities. Penguin. [aPMN]
Menzel, E. W., Jr. (1973) Leadership and communication in young chimpanzees.
In: Precultural primate behavior, ed. J. E. W. Menzel, pp. 192 – 225. Karger.
[aPMN]
Menzel, E. W., Jr. (1974) A group of young chimpanzees in a one-acre field. In:
Behavior of nonhuman primates, ed. A. M. Schrier & F. Stollnitz, pp. 83– 153.
Academic Press. [aPMN]
Mermillod, M., Bonin, P., Mondillon, L., Alleysson, D. & Vermeulen, N. (in press a)
Coarse scales are sufficient for efficient categorization of emotional facial
expressions: Evidence from neural computation. Neurocomputing. [rPMN]

476

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Mermillod, M., Droit-Volet, S., Devaux, D., Schaefer, A. & Vermeulen, N. (in pressb) Are coarse scales sufficient for fast detection of visual threat? Psychological
Science. [rPMN]
Mermillod, M., Vuilleumier, P., Peyrin, C., Alleysson, D. & Marendaz, C. (2009)
The importance of low spatial frequency information for recognizing fearful
facial expressions. Connection Science 21(1):75 – 83. [rPMN]
Mesquita, B., Barrett, L. F. & Smith, E. R., eds. (2010) The mind in context.
Guilford Press. [DL]Messinger, D. S., Fogel, A. & Dickson, K. L. (2001) All smiles are positive, but somesmiles are more positive than others. Developmental Psychology 375:642– 53.
[aPMN]
Michel, C., Caldara, R. & Rossion, B. (2006a) Same-race faces are perceived more
holistically than other-race faces. Visual Cognition 14(1):55 – 73. [RC]Michel, C., Rossion, B., Han, J., Chung, C. S. & Caldara, R. (2006b) Holistic processing is finely tuned for faces of one’s own race. Psychological Science
17(7):608 – 15. [RC]Milders, M., Crawford, J. R., Lamb, A. & Simpson, S. A. (2003) Differential deficits
in expression recognition in gene-carriers and patients with Huntington’sdisease. Neuropsychologia 41:1484– 92. [J-MF-D]
Miles, L. & Johnston, L. (2007) Detecting happiness: Perceiver sensitivity toenjoyment and non-enjoyment smiles. Journal of Nonverbal Behavior 31:259–
75. [aPMN]
Minagawa-Kawai, Y., Matsuoka, S., Dan, I., Naoi, N., Nakamura, K. & Kojima, S.
(2009) Prefrontal activation associated with social attachment: Facial-emotion
recognition in mothers and infants. Cerebral Cortex 19:284 –92. [aPMN]
Minagawa-Kawai, Y., Matsuoka, S., Dan, I., Naoi, N., Nakamura, K. & Kojima, S.
(2009) Prefrontal activation associated with social attachment: Facial-emotion
recognition in mothers and infants. Cerebral Cortex 19(2):284– 92. [JES]
Mobbs, D., Greicius, M. D., Abdel-Azim, E., Menon, V. & Reiss, A. L. (2003)
Humor modulates the mesolimbic reward centers. Neuron 40:1041 – 48.
[aPMN]
Moffatt, C. A. & Nelson, R. J. (1992) May/December romance: Adaptive significance non probabilis est. Behavioral and Brain Sciences 15:106 – 7. [rPMN]
Mojzisch, A., Schilbach, L., Helmert, J., Pannasch, S., Velichkovsky, B. & Vogeley,
K. (2006) The effects of self-involvement on attention, arousal, and facial
expression during social interaction with virtual others: A psychophysiologicalstudy. Social Neuroscience 1:184– 95. [aPMN]
Mondillon, L., Niedenthal, P. M., Gil, S. & Droit-Volet, S. (2007) Imitation and ingroup versus outgroup members’ facial expressions of anger: A test with a timeperception task. Social Neuroscience 2:223 – 37. [arPMN]Montague, D. P. & Walker-Andrews, A. S. (2001) Peekaboo: A new look at infants’
perception of emotion expressions. Developmental Psychology 37(6):826 –38.
[LC]
Montepare, J. M. & Dobish, H. (2003) The contribution of emotion perceptions andtheir overgeneralizations to trait impressions. Journal of Nonverbal Behavior
27:237 – 54. [JMV]Montgomery, K. J., Isenberg, N. & Haxby, J. V. (2007) Communicative hand gestures and object-directed hand movements activated the mirror neuron
system. Social Cognitive and Affective Neuroscience 2:114 – 22. [MWA]
Morris, J. S., Friston, K. J., Buchel, C., Frith, C. D., Young, A. W., Calder, A. J. &Dolan, R. J. (1998) A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain 121:47 – 57. [aPMN]
Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W., Calder, A. J. &Dolan, R. J. (1996) A differential neural response in the human amygdala tofearful and happy facial expressions. Nature 383:812– 15. [aPMN]
Morris, J. S., Öhman, A. & Dolan, R. J. (1999) A subcortical pathway to the right
amygdala mediating “unseen” fear. Proceedings of the National Academy of
Sciences USA 96:1680 – 85. [rPMN]
Morris, M. W. & Keltner, D. (2000) How emotions work: The social functions ofemotional expression in negotiations. Research in Organizational Behavior
22:1 – 50. [FB]
Morris, M. W. & Peng, K. (1994) Culture and cause: American and Chinese attributions for social and physical events. Journal of Personality and Social Psychology 64(6):949 – 71. [CMC-S]
Morton, E. W. (1977) On the occurrence and significance of motivation-structural
rules in some bird and mammal sounds. American Naturalist 111:855–69. [JJO]
Multi-Society Task Force on PVS (1994) Medical aspects of the persistent vegetative state, I. New England Journal of Medicine 330(21):1499 – 508. [CC]
Murphy, F. C., Michael, A., Robbins, T. W. & Sahakian, B. J. (2003) Neuropsychological impairment in patients with major depressive disorder: The effects
of feedback on task performance. Psychological Medicine 33:455 –67. [aPMN]
N’Diaye, K., Sander, D. & Vuilleumier, P. (2009) Self-relevance processing in thehuman amygdala: Gaze direction, facial expression, and emotion intensity.
Emotion 9(6):798 – 806. [LC, MM]
Niedenthal, P. M. (2007) Embodying emotion. Science 316:1002 – 1005. [PBou, JMF-D, aPMN]
Niedenthal, P. M. (2008) Emotion concepts. In: Handbook of emotion, 3rd edition,
ed. M. Lewis, J. M. Haviland-Jones & L. F. Barrett. Guilford Press. [aPMN]

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Niedenthal, P. M., Barsalou, L. W., Ric, F. & Krauth-Gruber, S. (2005a) Embodiment in the acquisition and use of emotion knowledge. In: Emotion: Conscious and unconscious, ed. L. Feldman-Barrett, P. M. Niedenthal & P.
Winkielman, pp. 21– 50. Guilford Press. [arPMN]
Niedenthal, P. M., Barsalou, L. W., Winkielman, P., Krauth-Gruber, S. & Ric, F.
(2005b) Embodiment in attitudes, social perception, and emotion. Personality
and Social Psychology Review 9:184 – 211. [aPMN]
Niedenthal, P. M., Brauer, M. Halberstadt, J. & Innes-Ker, A. (2001) When did hersmile drop? Facial mimicry and the influence of emotional state on thedetection of change in emotional expression. Cognition and Emotion 15:853 –64. [CH-H, aPMN, ES]
Niedenthal, P. M., Halberstadt, J. B., Margolin, J. & Innes-Ker, A. H. (2000)
Emotional state and the detection of change in facial expression of emotion.
European Journal of Social Psychology 30:211 – 22. [JSL, aPMN]
Niedenthal, P. M., Hess, U. & Miyamoto, Y. (in preparation) Interpreting the smile
across cultures. [RC, aPMN]
Niedenthal, P. M., Kruth-Gruber, S. & Ric, F. (2006) The psychology of emotion:
Interpersonal, experiential, and cognitive approaches. Principles of Social
Psychology Series. Psychology Press. [rPMN]
Niedenthal, P. M. & Maringer, M. (2009) Embodied emotion considered. Emotion
Review 1:122– 28. [rPMN]
Niedenthal, P. M., Winkielman, P., Mondillon, L. & Vermeulen, N. (2009)
Embodiment of emotion concepts. Journal of Personality and Social Psychology 96(6):1120 – 36. [J-MF-D, aPMN]
Nitschke, J. B., Nelson, E. E., Rusch, B. D., Fox, A .S., Oakes, T. R. & Davidson,
R. J. (2004) Orbitofrontal cortex tracks positive mood in mothers viewing
pictures of their newborn infants. NeuroImage 21(2):583– 92. [aPMN, JES]
Noriuchi, M., Kikuchi, Y. & Senoo, A. (2008) The functional neuroanatomy ofmaternal love: Mother’s response to infant’s attachment behaviors. BiologicalPsychiatry 63(4):415 – 23. [DL, JES]
Oberman, L. M., Hubbard, E. M., McCleery, J. P., Ramachandran, V. S. & Pineda,
J. A. (2005) EEG evidence for mirror neuron dysfunction in autism. Cognitive
Brain Research 24:190 – 98. [PW]
Oberman, L. M., Ramachandran, V. S. & Pineda, J. A. (2008) Modulation of mu
suppression in children with autism spectrum disorders in response to familiar
or unfamiliar stimuli: The mirror neuron hypothesis. Neuropsychologia
46:1558 – 65. [PW]
Oberman, L. M., Winkielman, P. & Ramachandran, V. S. (2007) Face to face:
Blocking facial mimicry can selectively impair recognition of emotional
expressions. Social Neuroscience 2:167– 78. [CH-H, aPMN, ES, PW]
Oberman, L. M., Winkielman, P. & Ramachandran, V. S. (2009) Slow echo: Facial
EMG evidence for the delay of spontaneous, but not voluntary, emotional
mimicry in children with autism spectrum disorders. Developmental Science
12:510 – 20. [MSB, AS, PW]
O’Doherty, J., Critchley, H., Deichmann, R. & Dolan, R. J. (2003) Dissociating
valence of outcome from behavioral control in human orbital and ventral
prefrontal cortices. Journal of Neuroscience 23:7931 –39. [aPMN]O’Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J. & Andrews C. (2001)
Abstract reward and punishment representations in the human orbitofrontal
cortex. Nature Neuroscience 4(1):95– 102. [aPMN]
Ohala, J. J. (1984) An ethological perspective on common cross-language utilizationof F0 of voice. Phonetica 41:1 –16. [JJO]
Öhman, A., Lundqvist, D. & Esteves, F. (2001) The face in the crowd revisited: A
threat advantage with schematic stimuli. Journal of Personality and Social
Psychology 80(3):381 – 96. [LH, aPMN]
Okun, M., Bowers, D., Springer, U., Shapira, N., Malone, D., Rezai, A., Nuttin, B.,
Heilman, K. M., Morecraft, R. J., Rasmussen, S. A., Greenberg, B. D., Foote,
K D. Goodman, W. K. (2004) What’s in a “Smile”?: Intra-operative observations of contralateral smiles induced by deep brain stimulation. Neurocase
10(4):271 – 79. [aPMN]
Olsson, A. & Phelps, E. A. (2004) Learned fear of “unseen” faces after
Pavlovian, observational, and instructed fear. Psychological Science 15:822 –28.
[EM]
Olsson, A. & Phelps, E. A. (2007) Social learning of fear. Nature Neuroscience
10:1095 – 102. [EM]Ortony, A., Clore, G. & Foss, M. (1987) The referential structure of the affective
lexicon. Cognitive Science 11:341– 64. [aPMN]
Osgood, C. E., Suci, G. J. & Tannenbaum, P. H. (1957) The measurement of
meaning. University of Illinois Press. [LH]
Ouellette, N., Hamon-Hill, C. & Barresi, J. (2010) Differential effect of disrupted
simulation during emotion recognition in dynamic and static facial expressions.
Poster presented at 22nd Annual Convention of the Association for Psychological Science, Boston, MA, May 27 – 30, 2010. [CH-H]
Owen, A., Coleman, M., Boly, M., Davis, M., Laureys, S. & Pickard, J. (2006)
Detecting awareness in the vegetative state. Science 313:1402. [CC]Owren, M. J. & Rendall, D. (2001) Sound on the rebound: Bringing form andfunction back to the forefront in understanding nonhuman primate vocal signaling. Evolutionary Anthropology 10:58– 71. [rPMN]

Palermo, R. & Rhodes, G. (2007) Are you always on my mind? A review
of how face perception and attention interact. Neuropsychologia 45:75 – 92.
[DL]
Panksepp, J., Nelson, E. & Siviy, S. (1994) Brain opioids and mother-infant social
motivation. Acta Paediatrica Supplement 397:40 – 46. [JES]
Parkinson, B. (1996) Emotions are social. British Journal of Psychology 87:663– 83.
[DL]
Parkinson, J. A., Cardinal, R. N. & Everitt, B. J. (2000) Limbic cortical – ventral
striatal systems underlying appetitive conditioning. Progress in Brain Research
126:263– 85. [aPMN]
Parr, L. A. & Waller, B. (2006) Understanding chimpanzee facial expression:
Insights into the evolution of communication. Social Cognitive and Affective
Neuroscience 1:221– 28. [aPMN]
Patterson, M. L. (1982) A sequential functional model of nonverbal exchange.
Psychological Review 89:231– 49. [aPMN]
Patterson, M. L. (1983) Nonverbal behavior: A functional perspective. Springer.
[aPMN]
Pegna, A. J., Khateb, A., Lazeyras, F. & Seghier, M. L. (2004) Discriminating
emotional faces without primary visual cortices involves the right amygdala.
Nature Neuroscience 8:24 – 25. [rPMN]
Peña Cervel, S. (2001) A cognitive approach to the role of body parts in the conceptualization of emotion metaphors. Epos 17:245 – 60. [FB]
Pennebaker, J. W. & Roberts, T. A. (1992) Toward a his and hers theory of emotion:
Gender differences in visceral perception. Journal of Social and Clinical Psychology 11:199 – 212. [rPMN]
Petty, R. E., Briñol, P. & Tormala, Z. L. (2002) Thought confidence as a determinant of persuasion: The self-validation hypothesis. Journal of Personality andSocial Psychology 82:722 – 41. [PBri]
Petty, R. E. & Cacioppo, J. T. (1986) The elaboration likelihood
model of persuasion. Advances in Experimental Social Psychology
19:123 – 205. [PBri]
Phillips, M. L., Bullmore, E. T., Howard, R., Woodruff, P. W. R., Wright, I. C.,
Williams, S. C. R., Simmons, A., Andrew, C., Brammer, M. J. & David, A. S.
(1998) Investigation of facial recognition memory and happy and sad facial
expression perception: An fMRI study. Psychiatry Research: Neuroimaging
83:127 – 38. [aPMN]
Pitcher, D., Garrido, L., Walsh, V. & Duchaine, B. (2008) TMS disrupts the perception and embodiment of facial expressions. Journal of Neuroscience
28(36):8929 – 33. [aPMN]
Pollack, A. (1996) Happy in the East (^_^) or smiling in the West:-). The New YorkTimes, August 12, 1996, p. D5. [CL]Pourtois, G., Dan, E. S., Grandjean, D., Sander, D. & Vuilleumier, P. (2005)
Enhanced extrastriate visual response to bandpass spatial frequency filteredfearful faces: Time course and topographic evoked-potentials mapping.
Human Brain Mapping 26:65– 79. [rPMN]
Pourtois, G., Sander, D., Andres, M., Grandjean, D., Reveret, L., Olivier, E. &Vuilleumier, P. (2004) Dissociable roles of the human somatosensory andsuperior temporal cortices for processing social face signals. European Journalof Neuroscience 20(12):3507 – 15. [LC, aPMN, DAS]
Pourtois, G. & Vuilleumier, P. (2006) The perception of fear in faces: Involuntary
and unconscious responses in the human brain as revealed by functional
imaging. In: Fear in cognitive neurosciences, pp. 63– 95. Nova Science Publishers. [aPMN]
Preuschoft, S. & van Hooff, J. A. R. A. M. (1997) The social function of “smile” and“laughter”: Variations across primate species and societies. In: Non-verbal
communication: Where nature meets culture, ed. U. Segerstrale & P. Molnàr,
pp. 171 – 89. Erlbaum. [MM, aPMN]
Pugh, S. D. (2001) Service with a smile: Emotional contagion in the service
encounter. Academy of Management Journal 44(5):1018 – 27. [FB]
Putman, P., Hermans, E. & van Honk, J. (2006) Anxiety meets fear in perception of
dynamic expressive gaze. Emotion 6(1):94– 102. [JSL]
Rafaeli, A. (1989) When clerks meet customers: A test of variables related to emotional
expressions on the job. Journal of Applied Psychology 74(3):385–93. [FB]
Rafaeli, A. & Sutton, R. I. (1990) Busy stores and demanding customers: How dothey affect the display of positive emotion? Academy of Management Journal
33(3):623 – 37. [FB]
Rapcsak, S. Z., Galper, S. R., Comer, J. F., Reminger, S. L., Nielsen, L., Kaszniak,
A. W., Verfaellie, M., Laguna, J. F., Labiner, D. M. & Cohen, R. A. (2000) Fearrecognition deficits after focal brain damage: A cautionary note. Neurology
54:575 – 81. [J-MF-D]
Rauch, S. L., Shin, L. M., Dougherty, D. D., Alpert, N. M., Orr, S. P., Lasko, M.,
Macklin, M. L., Fischman, A. J. & Pitman, R. K. (1999) Neural activation
during sexual and competitive arousal in healthy men. Psychiatry Research,
Neuroimaging Section 911 – 10. [aPMN]
Reddy, V. (2003) On being the object of attention: Implications for self-otherconsciousness. Trends in Cognitive Science 7(9):397 – 402. [LC]
Redican, W. K. (1975) Facial expression in non human primates. In: Primate behavior, vol. 4, ed. L. A. Rosenblum, pp. 103 – 94. Academic Press. [LH]

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

477

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Redoute, J., Stoleru, S., Gregoire, M., Costes, N., Cinotti, L., Lavenne, F., Le Bars,
D., Forest, M. & Pujol, J. (2000) Brain processing of visual sexual stimuli inhuman males. Human Brain Mapping 11:162 – 77. [aPMN]
Rees, C. E., Knight, L. V. & Wilkinson, C. E. (2007) Doctors being up there and webeing down here: A metaphorical analysis of talk about student/doctor-patient
relationships. Social Science & Medicine 65(4):725 – 37. [FB]Richardson, H. J. & Howcroft, D. (2006) The contradictions of CRM - A criticallens on call centres. Information and Organization 16(1):56 – 81. [FB]Richeson, J., Todd, A., Trawalter, S. & Baird, A. (2008) Eye-gaze direction modulates race-related amygdala activity. Group Processes and Intergroup Relations
11:233 – 46. [aPMN]
Rinn, W. E. (1991) Neuropsychology of facial expression. In: Fundamentals of
nonverbal behavior, ed. R. S. Feldman & B. Rimé, pp. 3 – 30. Press Syndicateof the University of Cambridge. [aPMN]
Rizzolatti, G. & Craighero, L. (2004) The mirror-neuron system. Annual Review of
Neuroscience 27:169 – 92. [aPMN]
Rizzolatti, G. & Fabbri-Destro, M. (2008) The mirror neuron system and its role insocial cognition. Current Opinion in Neurobiology 18:179 – 84. [KE]Rizzolatti, G., Fadiga L., Gallese, V. & Fogassi, L. (1996) Premotor cortex and therecognition of motor actions. Cognitive Brain Research 3:131 – 41. [aPMN]
Rizzolatti, G., Fogassi, L. & Gallese, V. (2001) Neurophysiological mechanisms
underlying the understanding and imitation of action. Nature Reviews
Neuroscience 2:661– 70. [MWA]
Rolls, E. (2000) The orbitofrontal cortex and reward. Cerebral Cortex 10(3):284–
94. [aPMN]
Rolls, E. T. (2004) The functions of the orbitofrontal cortex. Brain and Cognition
55:11 –29. [aPMN]
Rosenberg, S., Nelson, C. & Vivekananthan, P. (1968) A multidimensional approachto the structure of personality impressions. Journal of Personality and Social
Psychology 9:283– 94. [JMV]
Russell, J. A. (1994) Is there universal recognition of emotion from facial
expression? A review of the cross-cultural studies. Psychological Bulletin
115:102 – 41. [J-MF-D]
Russo, N. (1975) Eye contact, interpersonal distance, and the equilibrium theory.
Journal of Personality and Social Psychology 31:497 –502. [aPMN]
Rutherford, M. D. & McIntosh, D. N. (2007) Rules versus prototype matching:
Strategies of perception of emotional facial expressions in the autism spectrum.
Journal of Autism and Developmental Disorders 37:187 – 96. [PW]
Ruys, K. I. & Aarts, H. (in press) When competition merges people’s behavior:Interdependency activates shared action representations, Journal of Experimental Social Psychology. DOI: 10.1016/j.jesp.2010.05.016. [DL]Ruys, K. I. & Stapel, D. A. (2008a) Emotion elicitor or emotion-messenger? Subliminal exposure to two faces of facial expressions. Psychological Science
19:593 – 600. [DL]Ruys, K. I. & Stapel, D. A. (2008b) The secret life of emotions. Psychological
Science 19:385 – 91. [DL]Saether, L. Van Belle, W., Laeng, B., Brennen, T. & Øvervoll, M. (2009) Anchoringgaze when categorizing faces’ sex: Evidence from eye-tracking data. Vision
Research 49:2870– 80. [doi:10.1016/j.visres.2009.09.001]. [ES]Sander, D., Grafman, J. & Zalla, T. (2003) The human amygdala: An evolved system
for relevance detection. Reviews in the Neurosciences 14(4):303– 16. [LC,aPMN]Sander, D., Grandjean, D., Kaiser, S., Wehrle, T. & Scherer, K. R. (2007) Interaction effects of perceived gaze direction and dynamic facial expression: Evidence for appraisal theories of emotion. European Journal of Cognitive
Psychology 19(3):470 – 80. [LC, MM]
Sato, W., Kochiyama, T., Uono, S. & Yoshikawa, S. (2010) Amygdala integrates
emotional expression and gaze direction in response to dynamic facial
expressions. NeuroImage 50(4):1658 –65. [LC]
Sauter, D. A. (2010) More than happy: The need for disentangling positive
emotions. Current Directions in Psychological Science 19(1):36 – 40. [DAS]
Sauter, D. A. (in press) Are positive vocalizations perceived as communicating happiness across cultural boundaries? Communicative & Integrative Biology. [DAS]
Sauter, D. A., Eisner, F., Ekman, P. & Scott, S. K. (2010) Cross-cultural recognitionof basic emotions through nonverbal emotional vocalizations. Proceedings of
the National Academy of Sciences USA 107(6):2408 – 12. [DAS]
Scharlemann, J. P. W., Eckel, C. C., Kacelnik, A. & Wilson, R. K. (2001) The valueof a smile: Game theory with a human face. Journal of Economic Psychology
22(5):617 – 40. [TAM]
Scherer, K. R. (1987) Towards a dynamic theory of emotion: The component
process model of affective states. Geneva Studies in Emotion and Communication 1:1 – 98. Retrieved from http://www.unige.ch/fapse/emotion/publications/pdf/tdte_1987.pdf. [rPMN]
Scherer, K. R. (2001) Appraisal processes in emotion: Theory, methods, research.
Oxford University Press. [JSL]
Schilbach, L., Eickhoff, S. B., Mojzisch, A. & Vogeley, K. (2008) What’s in a smile?
Neural correlates of facial embodiment during social interaction. Social
Neuroscience 3:37 – 50. [DL, aPMN]

478

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Schilbach, L., Wohlschlaeger, A., Kraemer, N., Newen, A., Shah, N. & Fink, G.,(2006) Being with virtual others: Neural correlates of social interaction. Neuropsychologia 44:718 – 30. [aPMN]
Schmid-Mast, M. & Hall, J. A. (2004) When is dominance related to smiling?
Assigned dominance, dominance preference, trait dominance, and gendermoderators. Sex Roles 50(5 – 6):387 – 99. [MM]Schnakers, C., Perrin, F., Schabus, M., Majerus, S., Ledoux, D., Damas, P., Boly,
M., Vanhaudenhuyse, A., Bruno, M. A, Moonen, G. & Laureys, S. (2008)
Voluntary brain processing in disorders of consciousness. Neurology 71:1614–
20. [CC]
Schneider, K. & Josephs, I. (1991) The expressive and communicative functions ofpreschool children’s smiles in an achievement situation. Journal of Nonverbal
Behavior 15:185 –98. [J-MF-D]
Schore, A. (2001) Effects of a secure attachment relationship on right brain
development, affect regulation, and infant mental health. Infant Mental Health
Journal 22:7 –66. [aPMN]
Schrammel, F., Pannasch, S., Graupner, S.-T., Mojzisch, A. & Velichkovsky, B. M.
(2009) Virtual friend or threat? The effects of facial expressions and gazeinteraction on psychophysiological responses and emotional experience. Psychophysiology 46:922 – 31. [BChan, aPMN]
Schulte-Rüther, M., Markowtsch, H. J., Shah, N. J., Fink, G. R. & Piefke, M. (2008)
Gender differences in brain networks supporting empathy. NeuroImage
42:393 – 403. [ES]Schultz, R., Gauthier, I., Klin, A., Fulbright, R., Anderson, A. & Volkmar, F. (2000)
Abnormal ventral temporal cortical activity during face discrimination amongindividuals with autism and Asperger syndrome. Archives of General Psychiatry 57:331 – 40. [aPMN]
Schultz, W. (2007) Multiple dopamine functions at different time courses. Annual
Review of Neuroscience 30:259 – 88. [PBou]
Schwarz, N. (1990) Feelings as information: Informational and motivational functions of affective states. In: Foundations of social behavior, vol. 2E, ed. T.
Higgins & R. Sorrentino, pp. 527 – 61. Guilford Press. [JSL]
Schyns, P. G., Bonnar, L. & Gosselin, F. (2002) Show me the features! Understanding recognition from the use of visual information. Psychological Science
13:402 – 409. [ES]Semin, G. R. & Cacioppo, J. T. (2008) Grounding social cognition: Synchronization,
entrainment, and coordination. In: Embodied grounding: Social, cognitive,
affective, and neuroscientific approaches, ed. G. R. Semin & E. R. Smith, pp.
119 – 47. Cambridge University Press. [aPMN]
Senior, C., Phillips, M. L., Barnes, J. & David, A. S. (1999) An investigation into theperception of dominance from schematic faces: A study using the World-WideWeb. Behavior Research Methods, Instruments and Computers 31:341 – 46.
[aPMN]
Senju, A. & Johnson, M. A. (2009a) The eye contact effect: Mechanisms anddevelopment. Trends in Cognitive Sciences 13:127 – 34. [rPMN, AS]
Senju, A. & Johnson, M. H. (2009b) Atypical eye contact in autism: Models,mechanisms and development. Neuroscience and Biobehavioral Reviews
33:1204 –14. [AS]Senju, A., Yaguchi, K., Tojo, Y. & Hasegawa, T. (2003) Eye contact does notfacilitate detection in children with autism. Cognition 89:B43– 51. [AS]Shi, C. & Davis, M. (2001) Visual pathways involved in fear conditioning measured
with fear potentiated startle: Behavioral and anatomic studies. Journal of
Neuroscience 21:9844– 55. [rPMN]
Shiota, M. N., Campos, B. & Keltner, D. (2003) The faces of positive emotion:
Prototype displays of awe, amusement, and pride. Annals of the New York
Academy of Sciences 1000(1):296 – 99. [DAS]Singer, T. (2006) The neuronal basis and ontogeny of empathy and mind reading:
Review of the literature and implications for future research. Neuroscience andBiobehavioral Reviews 30:855 – 63. [ES]Singer, T., Seymour, B., O’Doherty, J., Kaube, H., Dolan, R. & Frith, C. (2004)
Empathy for pain involves the affective but not sensory components of pain.
Science 303(5661):1157– 67. [BChak]
Smith, E. & Semin, G. (2007) Situated social cognition. Current Directions in
Psychological Science 16:132 – 35. [aPMN]
Smith, K. R., DeMarree, K. G., Briñol, P. & Petty, R. E. (2010) Mimicry: A selfvalidation analysis. Unpublished manuscript, Texas Tech University. [PBri]
Smith, M. L., Cottrell, G. W., Gosselin, F. & Schyns, P. G. (2005) Transmitting anddecoding facial expressions. Psychological Science 16(3):184 – 89. [CL]Smith, M. L., Cottrell, G., Gosselin, F. & Schyns, P. G. (2005) Transmitting anddecoding facial expressions of emotions. Psychological Science 16:184 – 89.
[aPMN]
Solomon, K. O. & Barsalou, L. W. (2004) Perceptual simulation in property verification. Memory and Cognition 32:244 – 59. [aPMN]
Soltis, J. (2004) The signal functions of early infant crying. Behavioral and Brain
Sciences 27(4):443– 58; discussion 459 – 90. [JES]
Sonnby-Borgström, M. (2002) Automatic mimicry reactions as related to differences in emotional empathy. Scandinavian Journal of Psychology 43:433 – 43.
[aPMN]

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
Soussignan, R. (2002) Duchenne smile, emotional experience, andautonomic reactivity: A test of the facial feedback hypothesis. Emotion
2:52 – 74. [aPMN]
Spence, M. (1973) Job market signaling. The Quarterly Journal of Economics
87(3):355 – 74. [SC]Spezio, M., Adolphs, R., Hurley, R. & Piven, J. (2007a) Abnormal use of facial
information in high-functioning autism. Journal of Autism and Developmental
Disorders 37:929 – 39. [aPMN]
Spezio, M. L., Huang, P.-Y. S., Castelli, F. & Adolphs, R. (2007b) Amygdala damage
impairs eye contact during conversations with real people. Journal of Neuroscience 27:3994 – 97. [aPMN]
Stel, M. & van Knippenberg, A. (2008) The role of facial mimicry in the recognitionof affect. Psychological Science 19:984 – 85. [aPMN, ES]
Stel, M., van den Heuvel, C. & Smeets, R. (2008) Facial feedback mechanisms in
autistic spectrum disorders. Journal of Autism and Developmental Disorders
38:1250 – 58. [PW]
Strack, F. & Deutsch, R. (2004) Reflective and impulsive determinants of social
behavior. Personality and Social Psychology Review 8:220 –47. [aPMN]
Strathearn, L., Fonagy, P., Amico, J. & Montague, P. R. (2009) Adult attachment
predicts maternal brain and oxytocin response to infant cues. Neuropsychopharacology 34(12):2655 – 66. [rPMN]Straube, T., Weisbrod, A., Schmidt, S., Raschdorf, C., Preul, C., Mentzel, H. J. &Miltner, W. H. R. (2010) No impairment of recognition and experience ofdisgust in a patient with a right-hemispheric lesion of the insula and basal
ganglia. Neuropsychologia 48:1735– 41. [J-MF-D]
Strayer, J. (1993) Children’s concordant emotions and cognitions in response toobserved emotions. Child Development 64(1):188– 201. [aPMN]
Striano, T., Kopp, F., Grossmann, T. & Reid, V. (2006) Eye contact influences
neural processing of emotional expressions in 4-month-old infants. Social
Cognitive and Affective Neuroscience 1:87 – 94. [aPMN]
Surakka, V. & Hietanen, J. (1998) Facial and emotional reactions to Duchenne andnon-Duchenne smiles. International Journal of Psychophysiology 29(1):23–
33. [aPMN]
Surguladze, S. A., Brammer, M. J., Young, A. W., Andrew, C., Travis, M. J., Williams, S. C. & Phillips, M. L. (2003) A preferential increase in the extrastriate
response to signals of danger. NeuroImage 19:1317 – 28. [aPMN]
Sutton, R. I. (1991) Maintaining norms about expressed emotions: The case of billcollectors. Administrative Science Quarterly 36(2):245. [FB]
Sutton, R. I. & Rafaeli, A. (1988) Untangling the relationship between displayedemotions and organizational sales: The case of convenience stores. Academy ofManagement Journal 31(3):461 – 87. [FB]
Swain, J. E. (2008) Baby stimuli and the parent brain: Functional neuroimaging of
the neural substrates of parent – infant attachment, Psychiatry (Edgmont) 5(8):28– 36. [rPMN, JES]
Swain, J. E. (2010) The human parental brain: In vivo neuroimaging. Progress in
Neuro-Psychopharmacology & Biological Psychiatry. 2010 Oct 29. [Epubahead of print]
Swain, J. E. & Lorberbaum, J. P. (2008) Imaging the human parental brain. In:
Neurobiology of the parental brain, ed. R. S. Bridges, pp. 83– 100. Elsevier.
[JES]
Swain, J. E., Lorberbaum, J. P., Kose, S. & Strathearn, L. (2007) Brain basis of earlyparent – infant interactions: Psychology, physiology, and in vivo functional
neuroimaging studies. Journal of Child Psychology and Psychiatry 48(3 –4):262 –87. [JES]
Swain, J. E., Mayes, L. C. & Leckman, J. F. (2004) The development of parentinfant attachment through dynamic and interactive signaling loops of care andcry. Behavioral and Brain Sciences 27(4):472 – 73. [JES]
Symons, L. A., Hains, S. M. J. & Muir, D. W. (1998) Look at me: 5-month-old
infants’s sensitivity to very small deviations in eye-gaze during social interactions. Infant Behavioral Development 21:531 – 36. [aPMN]
Tager-Flusberg, H. (2007) Evaluating the theory-of-mind hypothesis of autism.
Current Directions in Psychological Science 16:311 – 15. [KE]Taylor, P. & Bain, P. (1999) “An assembly line in the head”: Work andemployee relations in the call centre. Industrial Relations Journal30(2):101 – 17. [FB]
Theoret, H., Halligan, E., Kobayashi, M., Fregni, F., Tager-Flusberg, H. & PascualLeone, A. (2005) Impaired motor facilitation during action observation in
individuals with autism spectrum disorder. Current Biology 15:R84 – R85.
[PW]
Thibault, P., Levesque, M., Gosselin, P. & Hess, U. (2008) Cultural aspects of smile
authenticity. Paper presented at the 29th International Congress of Psychology, Berlin, Germany. July 20 – 25, 2008. [aPMN]
Thibault, P., Levesque, M., Gosselin, P. & Hess, U. (submitted) The Duchenne
marker is NOT a universal signal of smile authenticity: But it can be learned!
[aPMN]
Tiedens, L. Z. & Fragale, A. R. (2003) Power moves: Complementarity in dominantand submissive nonverbal behavior. Journal of Personality and Social Psychology 84(3):558 – 68. [LH]

Tinbergen, N. (1963) On aims and methods of ethology. Zeitschrift für Tierpsychologie 20:410 –33. [MM]Tipples, J., Atkinson, A. P. & Young, A. W. (2002) The eyebrow frown: A salientsocial signal. Emotion 2:288– 96. [aPMN]
Todorov, A. (2008) Evaluating faces on trustworthiness: An extension of systems forrecognition of emotions signaling approach/avoidance behaviors. Annuals of
the New York Academy of Sciences 1124:208 –24. [JMV]Tomalski, P., Johnson, M. H. & Csibra, G. (2009) Temporal-nasal asymmetry of
rapid orienting to face-like stimuli. NeuroReport 20:1309– 12. [AS]Tomkins, S. S. (1962) Affect imagery consciousness: Vol. 1. The positive affects.
Springer. [rPMN]Tomkins, S. S. (1963) Affect imagery consciousness: Vol. 2. The negative affects.
Springer. [rPMN]
Tracy, J. L. & Robins, R. W. (2004) Show your pride: Evidence for a discreteemotion expression. Psychological Science 15:194 – 97. [aPMN]
Tracy, J. L. & Robins, R. W. (2008) The nonverbal expression of pride: Evidence forcross-cultural recognition. Journal of Personality and Social Psychology
94:516 – 30. [aPMN]
Trevarthen, C. (1974) Conversations with a two-month-old. New Scientist 2:230–
35. [aPMN]
Tronick, E. Z. (1989) Emotions and emotional communication in infants. American
Psychologist 44(2):112– 19. [JES]
Tronick, E., Als, H., Adamson, L., Wise, S. & Brazelton, T. (1978) The infant’s
response to entrapment between contradictory messages in face to faceinteraction. Journal of the American Academy of Child Psychiatry 17:1 –13.
[aPMN]
Tsao, D. Y. & Livingstone, M. S. (2008) Mechanisms of face perception. Annual
Review of Neuroscience 31:411 – 37. [PBou]
Turella, L., Pierno, A., Tubaldi, F. & Castiello, U. (2009) Mirror neurons inhumans: Consisting or confounding evidence? Brain and Language 108:10–
21. [aPMN]
van Baaren, R. B., Maddux, W. W., Chartrand, T. L., de Bouter, C. & van Knippenberg, A. (2003) It takes two to mimic: Behavioral consequences of selfconstruals. Journal of Personality and Social Psychology 84:1093– 102. [DL]van der Gaag, C., Minderaa, R. & Keysers, C. (2007) Facial expressions: What themirror neuron system can and cannot tell us. Social Neuroscience 2:179 – 222.
[aPMN]
Van Hooff, J. A. R. A. M. (1967) The facial displays of the catarrhine monkeys andapes. In: Primate ethology, ed. D. Morris, pp. 7 – 68. Weidenfield & Nicolson.
[LH]
van Hooff, J. A. R. A. M. (1976) The comparison of the facial expressions in man andhigher primates. In: Methods of inference from animal to human behavior, ed.
M. von Cranach, pp. 165 –96. Aldine. [aPMN]
Vanhaudenhuyse, A., Schnakers, C., Bredart, S. & Laureys, S. (2008) Assessment ofvisual pursuit in post-comatose states: Use a mirror. Journal of Neurology
Neurosurgery and Psychiatry 79(2):223. [CC]Vatsyayan, K. (1996) Bharata, the Natyasastra. Sahitya Akademi. [BChak]
Vazire, S., Naumann, L. P., Rentfrow, P. J. & Gosling, S. D. (2009) Smiling reflects
different emotions in men and women. Behavioral and Brain Sciences 32:403–
405. [J-MF-D]
Vigil, J. M. (2009) A socio-relational framework of sex differences in the expressionof emotion. Behavioral and Brain Sciences 32:375 – 428. [J-MF-D, ES, JMV]
Vizioli, L., Foreman, K., Rousselet, G. A. & Caldara, R. (2010) Inverting faces elicits
sensitivity to race on the N170 component: A cross-cultural study. Journal of
Vision 10(1):1 – 23. [RC]von Cranach, M. (1971) The role of orienting behavior in human interaction. In:
Behavior and environment: The use of space by animal and man, ed. A. H.
Esser, pp. 217 – 37. Plenum Press. [aPMN]
Vrij, A., Semin, G. R. & Bull, R. (2006) Insight into behavior displayed during
deception. Human Communication Research 22(4):544– 62. [SC]
Vuilleumier, P., Armony, J. L., Driver, J. & Dolan, R. J. (2003) Distinct spatial
frequency sensitivities for processing faces and emotional expressions. Nature
Neuroscience 6:624– 31. [rPMN]
Wacker, J., Chavanon, M. L., Leue, A. & Stemmler, G. (2010) Trait BIS predicts
alpha asymmetry and P300 in a go/no-go task. European Journal of Personality
24:85– 105. [rPMN]
Waller, B. & Dunbar, R. I. M. (2005) Differential behavioural effects of silent bared
teeth display and relaxed open mouth display in chimpanzees (Pan troglodytes). Ethology 111:129– 42. [aPMN]
Walsh, D. & Hewitt, J. (1985) Giving men the come-on: Effect of eye contact andsmiling in a bar environment. Perceptual and Motor Skills 61:873 – 74. [aPMN]
Whalen, P. J., Davis, F. C., Oler, J. A., Kim, H., Kim, M. J. & Neta, M. (2009)
Human amygdala response to facial expressions of emotion. In: The human
amygdala, ed. P. Whalen & E. A. Phelps, pp. 265 –88. Guilford Press. [PBou]
Whalen, P. J. & Kleck, R. E. (2008) The shape of faces (to come). Nature Neuroscience 11(7):739 –40. [LH]
Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. & Jenike, M. A.
(1998) Masked presentations of emotional facial expressions modulate

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

479

References/Niedenthal et al.: The Simulation of Smiles (SIMS) model
amygdala activity without explicit knowledge. Journal of Neuroscience 18:411 –18. [aPMN]
Whalen, P. J., Shin, L. M., McInerney, S. C., Fischer, H., Wright, C. I. & Rauch, S.
L. (2001) A functional MRI study of human amygdala responses to facial
expressions of fear vs. anger. Emotion 1:70 – 83. [aPMN]
Wicker, B., Keysers, C., Plailly, J., Royet, J. P., Gallese, V. & Rizzolatti, G. (2003)
Both of us disgusted in my insula: The common neural basis of seeing andfeeling disgust. Neuron 40:655 – 64. [BChak, aPMN]
Wild, B., Erb, M. & Bartels, M. (2001) Are emotions contagious? Evoked emotions
while viewing emotionally expressive faces: Quality, quantity, time course, andgender differences. Psychiatry Research 102:109 – 24. [aPMN]
Willems, R. M., Özyürek, A. & Hagoort, P. (2007) When language meets action:
The neural integration of gesture and speech. Cerebral Cortex 17:2322– 33.
[MWA]
Williams, J., Whitenb, A., Suddendorfc, T. & Perrett, D. I. (2001) Imitation, mirror
neurons and autism. Neuroscience and Biobehavioral Reviews 25(4): 287 – 95.
[MSB]
Winkielman, P., McIntosh, D. N. & Oberman, L. (2009) Embodied and disembodied emotion processing: Learning from and about typical and autistic individuals. Emotion Review 2:178– 90. [aPMN, PW]
Winston, J. S., O’Doherty, J. & Dolan, R. J. (2003) Common and distinct neural
responses during direct and incidental processing of multiple facial emotions.
NeuroImage 2084 – 97. [aPMN]
Wiseman, R. L. & Pan, X. (2004) Smiling in the People’s Republic of China and theUnited States: Status and situational influences on the social appropriateness of
smiling. Intercultural Communication Studies 13(1):1– 18. [CMC-S]
Wojciszke, B. (2005) Morality and competence in person and self perception.
European Review of Social Psychology 16:155 – 88. [JMV]
Woodworth, R. S. (1915) A revision of imageless thought. Psychological Review
22:1 – 27. [EM]Working Party of the Royal College of Physicians (2003) The vegetative state:
Guidance on diagnosis and management. Clinical Medicine 3:249– 54. [CC]xinhuanet. (2008) The mystery of the small points in those ancient ladies’ face at the
Beijing Olympics opening ceremony. Available at: http://cul.shangdu.com/
recommend/20080811-10249/index.shtml. [CL]Xu, X., Zuo, X., Wang, X. & Han, S. (2009) Do you feel my pain? Racial groupmembership modulates empathic neural responses. Journal of Neuroscience29(26):8525– 29. [RC]480

BEHAVIORAL AND BRAIN SCIENCES (2010) 33:6

Yabar, Y., Johnston, L., Miles, L. & Peace, V. (2006) Implicit behavioral mimicry:
Investigating the impact of group membership. Journal of Nonverbal Behavior
30:97– 113. [J-MF-D]
Yang, T. T., Menon, V., Eliez, S., Blasey, C., White, C. D., Reid, A. J., Gotlib, I. H. &Reiss, A. L. (2002) Amygdalar activation associated with positive and negative
facial expressions. NeuroReport 13:1737 – 41. [aPMN]
Yu, N. (2008) Metaphor form body and culture. In: The Cambridge handbook of
metaphor and thought, ed. J. R. W. Gibbs, pp. 247 –61. Cambridge University
Press. [FB]
Yuki, M., Maddux, W. W. & Masuda, T. (2007) Are the windows to the soul thesame in the East and West? Cultural differences in using the eyes and mouth ascues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology 43(2):303– 11. [CL]Yuki, M., Maddux, W. W. & Masuda, T. (2007) Are the windows to the soul thesame in the East and West? Cultural differences in using the eyes and mouth torecognize emotions in Japan and the United States. Journal of Experimental
Social Psychology 43:202 – 11. [rPMN]
Zahavi, A. (1975) Mate selection: A selection for a handicap. Journal of Theoretical
Biology 53(1): 205 – 14. [SC]Zajonc, R. B., Adelmann, P. K., Murphy, S. T. & Niedenthal, P. M. (1987) Convergence in the physical appearance of spouses: An implication of thevascular theory of emotional efference. Motivation and Emotion 11:335 – 46.
[arPMN]
Zajonc, R., Murphy, S. & Inglehart, M. (1989) Feeling and facial efference:
Implications of the vascular theory of emotion. Psychological Review 96:395–
416. [aPMN]
Zapf, D. (2002) Emotion work and psychological well-being: A review of the literature and some conceptual considerations. Human Resource Management
Review 12(2): 237 –68. [FB]
Zebrowitz, L. A., Kikuchi, M. & Fellous, J. M. (2010) Facial resemblance toemotions: Group differences, impression effects, and race stereotypes. Journalof Personality and Social Psychology 98:175 –89. [J-MF-D]
Zink, C. F., Tong, Y., Chen, Q., Bassett, D., Stein, J. L. & Meyer-Lindenberg, A.
(2008) Know your place: Neural processing of social hierarchy in humans.
Neuron 58: 273 – 83. [aPMN]
Zwaan, R. A. (2008) Experiential traces and mental simulations in language comprehension. In: Symbols, embodiment, and meaning, ed. M. DeVega, A. M.
Glenberg & A. C. Graesser, pp. 165 – 80. Oxford University Press. [EM]