Journal of Vision (2018) 18(9):5, 1–27

1

Tracking the recognition of static and dynamic facial
expressions of emotion across the life span
Anne-Raphaëlle Richoz
Junpeng Lao
Olivier Pascalis
Roberto Caldara

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland
LPNC, University of Grenoble Alpes, Grenoble, France$

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland
LPNC, University of Grenoble Alpes, Grenoble, France
Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

The effective transmission and decoding of dynamic
facial expressions of emotion is omnipresent andcritical for adapted social interactions in everyday life.
Thus, common intuition would suggest an advantage
for dynamic facial expression recognition (FER) overthe static snapshots routinely used in mostexperiments. However, although many studies
reported an advantage in the recognition of dynamic
over static expressions in clinical populations, results
obtained from healthy participants are contrasted. To
clarify this issue, we conducted a large cross-sectional
study to investigate FER across the life span in orderto determine if age is a critical factor to account forsuch discrepancies. More than 400 observers (agerange 5–96) performed recognition tasks of the six
basic expressions in static, dynamic, and shuffled(temporally randomized frames) conditions,normalized for the amount of energy sampled overtime. We applied a Bayesian hierarchical step-linear
model to capture the nonlinear relationship between
age and FER for the different viewing conditions.
Although replicating the typical accuracy profiles of
FER, we determined the age at which peak efficiency
was reached for each expression and found greater
accuracy for most dynamic expressions across the life
span. This advantage in the elderly population was
driven by a significant decrease in performance forstatic images, which was twice as large as for the
young adults. Our data posit the use of dynamic
stimuli as being critical in the assessment of FER in
the elderly population, inviting caution when drawing$

conclusions from the sole use of static face images tothis aim.

Introduction
Human faces convey a wealth of dynamic signalsthat are critical for an adequate and rapid categorization of the emotional states of others. Yet the vast
majority of studies investigating expression recognition
have relied on static images that commonly display the
apex or the highest state of a given expression. In
everyday life, however, facial expressions are rarelytransmitted and decoded through static snapshots ofinternal states. Natural human interactions are a highly
dynamic (and multimodal) phenomenon with faces
evolving over time while transmitting distinct signals toconvey diverse emotional states. Dynamic expressions
provide observers with additional cues related to theirinherent temporal properties, such as their unfolding
speed (slow vs. fast; Bould & Morris, 2008; Bould,
Morris, & Wink, 2008; Kamachi et al., 2001), rise time(from the neutral to the highest state; R. E. Jack,
Garrod, & Schyns, 2014; Recio, Schacht, & Sommer,2013) or intensity (Bould et al., 2008), critical for anadequate categorization. Therefore, dynamic faces arericher and ecologically more valid depictions of the way
expressions are encountered in everyday life compared
to static images (e.g., Johnston, Mayes, Hughes, &Citation: Richoz, A.-R., Lao, J., Pascalis, O., & Caldara, R. (2018). Tracking the recognition of static and dynamic facial expressionsof emotion across the life span. Journal of Vision, 18(9):5, 1–27, https://doi.org/10.1167/18.9.5.
https://doi.org/1 0. 11 67 /1 8 .9 .5

Received February 27, 2018; published September 7, 2018ISSN 1534-7362 Copyright 2018 The Authors

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

Young, 2013; Paulmann, Jessen, & Kotz, 2009;
Trautmann, Fehr, & Herrmann, 2009). Interestingly,
from an evolutionary perspective, humans have had
more experience with dynamic faces as static pictures
only appeared during the last century with the adventof photography and the rapid expansion of digital tools
and social networks. The decoding of static faces is alsoa learnt behavior that develops throughout life. As thehuman visual system is, from birth on, steadily
stimulated by dynamic signals from faces with minimal
exposure to static faces, common intuition would
suggest the existence of a particular expertise to decodesuch events with the presence of an advantage for therecognition of dynamic over static expressions.
Previous studies that have attempted to investigate
this question have yielded inconsistent ﬁndings (for areview, see Alves, 2013; Fiorentini & Viviani, 2011;
Kätsyri, 2006; Krumhuber, Kappas, & Manstead,2013). Some behavioral studies have revealed anadvantage (e.g., Ambadar, Schooler, & Cohn, 2005;
Cunningham & Wallraven, 2009; Giard & Peronnet,
1999; Knappmeyer, Thornton, & Bülthoff, 2003;
Paulmann et al., 2009; Wehrle, Kaiser, Schmidt, &Scherer, 2000), whereas others have revealed that the
beneﬁts of dynamic cues in facial expression recognition may be minimal (e.g., Gold et al., 2013) orinexistent (e.g., Fiorentini & Viviani, 2011). These
contrasting ﬁndings suggest that the dynamic advantage for facial expression recognition is not as
straightforward as it may appear. Rather, it seems thatthe physical properties of the stimuli presented as well
as clinical or neuropsychological conditions inﬂuence
the extent to which dynamic displays lead to processing
beneﬁts (Ambadar et al., 2005; Bould et al., 2008;
Wallraven, Breidt, Cunningham, & Bülthoff, 2008).
Several studies have shown that the beneﬁcial effects
of dynamic events are particularly relevant in suboptimal situations in which the physical information
available is limited (Ambadar et al., 2005; Bould et al.,2008), deteriorated, or blurred (Ehrlich, Schiano, &Sheridan, 2000; Kätsyri & Sams, 2008; Wallraven et al.,2008). For example, Wallraven et al. (2008) found thatdynamic events increased recognition accuracy ofcomputer-animated facial expressions whose texture or
shape were systematically degraded. Similarly, bycomparing the ability of observers to recognize
expressions from schematic and natural faces, Kätsyri
and Sams (2008) and Ehrlich et al. (2000) discovered arecognition advantage for dynamic expressions with
schematic but not natural faces. Along the same lines,
other studies have revealed that dynamic events provide
compensatory cues when subtle facial expressions arepresented (Ambadar et al., 2005; Bould et al., 2008).
With subtle expressions, additional temporal information may be essential to disambiguate the uncertainty
introduced by the lack of intensity.

2

Similarly, an advantage is noticeable when it comesto clinical conditions as dynamic information provides
compensatory cues in suboptimal situations. Dynamic
presentations facilitate the recognition of facial expressions in adults and children with intellectual
disability (Harwood, Hall, & Shinkﬁeld, 1999), pervasive developmental disorder (Uono, Sato, & Toichi,
2010), and autism (Back, Ropar, & Mitchell, 2007;
Gepner, Deruelle, & Grynfeltt, 2001; Tardif, Lainé,
Rodriguez, & Gepner, 2007; but see Kätsyri, Saalasti,
Tiippana, von Wendt, & Sams, 2008, for Asperger
syndrome). In neuropsychology, several brain-injury
studies have shown increased recognition performance
when dynamic expressions were used (Adolphs, Tranel,& Damasio, 2003; Humphreys, Donnelly, & Riddoch,
1993; Richoz, Jack, Garrod, Schyns, & Caldara, 2015).
For example, Humphreys et al. (1993) reported the caseof an agnosic patient who was signiﬁcantly impaired at
identifying facial identity and facial expressions when
exposed to static images. In contrast, his performance
was proﬁcient when asked to judge a subset of facial
expressions (i.e., smiling, frowning, or surprise) fromdynamic faces animated by light dots. On the same line,
we recently investigated the ability of a prosopagnosic
patient—the well-studied case of PS—with multiple
and extensive brain lesions in the occipitotemporal
cortex to recognize facial expressions from static anddynamic faces. Our ﬁndings revealed that the patient
PS was selectively impaired in decoding static expressions while showing normal performance for the
decoding of dynamic emotional expressions. Thisobservation favors the existence of distinct representational systems for static and dynamic expressions or
dissociable cortical pathways to access them (Richoz etal., 2015). Noteworthy, the advantage for processing
dynamic faces in PS is related to a suboptimal
information use for static (i.e., bias toward the mouth)
compared to dynamic faces (i.e., all face features; Fiset
et al., 2017).
Although several neuropsychological studies haveshown that the dynamic properties of human facial
expressions provide signiﬁcant processing advantages,
other behavioral studies involving healthy observers
suggest that this might not be the case (Bould &Morris, 2008, for expressions of high intensity; Christie& Bruce, 1998; Fiorentini & Viviani, 2011; Gold et al.,2013; Jiang et al., 2014; Kamachi et al., 2001,
experiment 2). By using a threshold model, Fiorentini
and Viviani (2011), for example, reported that neither
reaction times nor identiﬁcation accuracy were moreaccurate for the dynamic as compared to the static
expressions. Similar ﬁndings were reported in a later
study by Gold et al. (2013). Their results revealed thatrecognition rates were nearly identical when participants were exposed to static, dynamic, shufﬂed(temporally randomized expressions), or reversed

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

expressions. This suggests that the temporal properties
provided by moving faces are not necessary forobservers to reliably categorize emotional expressions.
Altogether, these studies suggest that a healthy visualsystem seems to be powerful enough to efﬁciently
recognize intense expressions from static faces, leaving
only a nonsigniﬁcant beneﬁt to the processing ofdynamic facial expressions. By contrast, in clinicalconditions, the muscular movements associated withthe temporal unfolding of an expression may force theobservers to shift their attention to different facial
features. This may enhance attention and motor
simulations (A. Wood, Lupyan, Sherrin, & Niedenthal,
2016; A. Wood, Rychlowska, Korb, & Niedenthal,2016) in fragile or neurologically impaired face systems,
which may explain the increased performance withdynamic signals in these populations.
Interestingly, there are stages in healthy observers
during which the perceptual system is also particularly
fragile or immature. For example, from early infancy tolate adolescence, the brain undergoes a wide array of
anatomical and functional changes as it develops (e.g.,
Blakemore, 2012; Blakemore & Choudhury, 2006;
Casey, Tottenham, Liston, & Durston, 2005; Durston
et al., 2001). Similarly, during normal aging, thecognitive functions decline, which is induced by agerelated loss of synaptic contacts, neural apoptosis (e.g.,Raz, 2000; Rossini, Rossi, Babiloni, & Polich, 2007),
reduction in cerebral blood ﬂow (e.g., Chen, Rosas, &Salat, 2011), or volume reduction in different brain
regions (e.g., amygdala, hippocampus, frontal cortex;
Calder et al., 2003; C. R. Jack et al., 1997; Navarro &Gonzalo, 1991; Ruffman, Henry, Livingstone, &Phillips, 2008). Considering the increased vulnerabilityof the brain under neural architectural changes(Andersen, 2003; Hof & Morrison, 2004), it is possiblethat healthy young children and normal aging adults
also beneﬁt from the presentation of dynamic faces.
However, only a few developmental studies havecompared facial expression recognition in children
using both static and dynamic stimuli (Nelson,
Hudspeth, & Russell, 2013; Nelson & Russell, 2011).
These studies yielded equivocal results, none of them
revealing a signiﬁcant advantage for dynamic overstatic stimuli; two studies even pointed to differences
favoring static stimuli (Nelson & Russell, 2011; Widen& Russell, 2015). Nevertheless, most of these studies
tested facial expression recognition with the use of asingle actor and provided additional information about
face, body movements, and vocal intonations, which
may have facilitated expression recognition. In the
aging literature, a small number of studies examinedfacial expression recognition with static and dynamic
faces (Grainger, Henry, Phillips, Vanman, & Allen,
2015; Krendl & Ambady, 2010; Sze, Goodkind,
Gyurak, & Levenson, 2012). Although most of these3

studies pointed to a dynamic advantage for therecognition of facial expressions, they (a) did not use a
database of static and dynamic stimuli controlled forthe amount of low–level visual information carried overtime (Grainger et al., 2015; Sze et al., 2012), (b) werelimited to a subset of emotional expressions (Krendl &Ambady, 2010), (c) included participants in only one
condition (Krendl & Ambady, 2010), or (d) relied on
dynamic movies that were not displaying naturalexpressions (Grainger et al., 2015). These methodological issues considerably limit ﬁrm conclusions on thepotential beneﬁts of dynamic cues for the recognitionof facial expressions in elderly people.
Developmental studies have reported an early tuning
to culturally speciﬁc expressions (Geangu et al., 2016;
for a review, see Caldara, 2017) and emotion-dependent differences in the development of facial expression
recognition abilities with some expressions being
recognized earlier (e.g., happiness) than others (e.g.,fear) (Durand, Gallay, Seigneuric, Robichon, & Baudouin, 2007; Gao & Maurer, 2010; Gross & Ballif,
1991; Herba & Phillips, 2004; Rodger, Vizioli, Ouyang,& Caldara, 2015). Similarly, studies with elderly people
have shown that the recognition of some expressions
decreases with increasing age, and the recognition ofothers remains stable or even improves (Calder et al.,
2003; MacPherson, Phillips, & Della Sala, 2002;
Sullivan & Ruffman, 2004b; Zhao, Zimmer, Shen,
Chen, & Fu, 2016). Most of these studies were,
however, conducted with static posed images, and only
little is known about the effects of aging on therecognition of genuine dynamic emotional expressions.
To ﬁll this gap in the developmental literature, weinvestigated whether the advantage for dynamic stimuli
extends to other populations with immature (i.e., young
children) or fragile (i.e., elderly adults) face-processing
systems. We conducted a large cross-sectional study
involving more than 400 observers (age range 5–96) inorder to investigate facial expression recognition fromearly to elderly age. Observers performed categorization tasks of the six basic expressions (anger, disgust,fear, happiness, sadness, and surprise) in three conditions: static, dynamic, and shufﬂed (temporally randomized frames; Gold et al., 2013). Importantly, we
relied on a speciﬁc database of static, dynamic, andshufﬂed stimuli created by Gold et al. (2013). Our
experimental choice was driven by the fact that theseauthors also used an ideal observer model to objectively
measure the amount of low-level physical information
carried by the stimuli. It is worth noting that moststudies investigating the presence of a dynamic
advantage (e.g., Ambadar et al., 2005; Bould & Morris,
2008; Bould et al., 2008; Cunningham & Wallraven,2009; Fiorentini & Viviani, 2011; Kätsyri & Sams,
2008) directly compared participants’ recognition rates
in the static and dynamic conditions without control-

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

4

The experiment script, raw data, and analysis codes
are open to access on Github (https://github.com/
iBMLab/Static_dynamic).

olds (N ¼ 31, 27 females). From the age of 21 to the ageof 96, six different groups were created: 21- to 30-yearolds (N ¼ 31, 23 females), 31- to 40-year-olds (N ¼ 23,
13 females), 41- to 50-year-olds (N ¼ 33, 22 females),
51- to 60-year-olds (N ¼ 30, 18 females), 61- to 80-yearolds (N ¼ 31, 25 females), and 81- to 96-year-olds (N ¼40, 30 females).
All participants had normal or corrected-to-normal
vision with no neurological or psychiatric history.
Children were recruited from primary and high schools
in the area of Fribourg, Switzerland. Parental consent
was required for all children under the age of 16.
Participants older than 16 were recruited at the
University of Fribourg through social networks oradvertisements. Observers from the university obtained
course credits for their participation. All participants
signed a consent form that described the main goals of
our experiment.
Elderly people were recruited and tested in senior
housing in the Fribourg region. We used the MiniMental State Examination (Folstein, 1975) in order todetermine the eligibility of the elderly people aged 60
and over. This brief cognitive screening test, which has
been extensively used and validated since its creation in
1975, allows the assessment of different cognitive
functions, such as memory, orientation, attention,
language, and recall, through 11 questions with amaximum score of 30. Elderly people with a scorebelow 24 were excluded from our study (N ¼ 3) as thisscore has been set as the most commonly used cutoff
score for cognitive impairment (Mitchell, 2009). The
ethical committee of the department of psychology of
the University of Fribourg approved the study reported
here.

Participants

Stimuli

A total of 444 healthy observers participated in thecurrent study. Subjects who did not respond at least
once to all expressions on the ﬁrst condition/block were
excluded from the analyses (N ¼ 32), leaving a total
number of 412 participants. Their exclusion is based on
the difﬁculty to determine whether they actually did notrecognize the expression presented or did not correctly
understand the task. A future research paper will
investigate the systematic errors of the participants thatwere excluded.
We intended to collect data from 20 participants ineach age group ranging from 5 to 96 years of age. The
groups were comprised as follows: 5- to 6-year-olds (N¼ 27, 17 females), 7- to 8-year-olds (N ¼ 24, 17 females),9- to 10-year-olds (N ¼ 22, 11 females), 11- to 12-yearolds (N ¼ 22, 14 females), 13- to 14-year-olds (N ¼ 24,
10 females), 15- to 16-year-olds (N ¼ 21, eight females),17- to 18-year-olds (N ¼ 21, 16 females), 19- to 20-year-

We used the same stimuli as those used by Gold et al.
(2013). In order to create their database, Gold et al.
asked eight actors (four females) to reproduce the six
basic facial expressions of emotion as naturally as
possible (i.e., anger, disgust, fear, happiness, sadness,and surprise; Ekman & Friesen, 1976). The facial
expressions of emotion started from a neutral state andnaturally evolved into a full expression. Their apex
state (i.e., the point at which an expression reached its
fully articulated state) was determined by two raters.
The dynamic faces evolved from a neutral state to afull-blown expression at a frame rate of 30 frames/s. All
expressions reached their apex within 30 frames. If the
fully articulated expression was reached before 30
frames, one to four supplementary apex frames wereappended, but as the actors were asked to maintain theapex for several seconds, this happened for only sevenout of 48 movies (for more details, see Gold et al.,ling the amount of low-level information physicallyavailable to the observers. As mentioned by Gold et al.,the absence of an objective measure of stimulus
information makes it difﬁcult, in most cases, todetermine whether increased recognition rates are due
to adequate categorization skills, to the amount ofphysical information available, or to a combination ofboth factors. By comparing human expression recognition scores with the performance of a statisticallyideal observer, Gold et al. reported that their dynamic
stimuli did not provide additional low-level information than what was already offered by their static
snapshots (for additional information, see Gold et al.,2013). In addition to this approach, we modeled therelationship between age and facial-expression recognition by using a hierarchical Bayesian approach with a
step-linear model. Our results revealed emotion-speciﬁc
advantages for dynamic stimuli. More speciﬁcally,
although participants displayed nearly identical categorization performance for the static and dynamic
expressions of fear and sadness, all the other expressions were more readily labeled as correct whenfeaturing dynamic displays. Overall, the results of thisstudy provide a comprehensive and detailed view of theway in which static and dynamic expressions arerecognized across the human life span.

Material and methods

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara2013). Faces were presented in black and white andcropped at the hairline to present only the internal
facial features. Previous experiments have shown thatexternal features attract children’s attention (Leitzke &Pollak, 2016). Moreover, the faces were centered andseen through an oval aperture, which was placed in the
middle of a gray-colored background. The borders of
the oval aperture were slightly blurred in order toproduce a progressive transition between the background and the faces (Gold et al., 2013). The faces were
resized from the original experiment and each measured 768 pixels in height and 768 pixels in width. They
subtended a visual angle of 128 on the screen at aviewing distance of 65 cm. All faces were equated forluminance and contrast.
Based on these dynamic sequences, Gold et al. (2013)
generated two other sets of stimuli: a set of frozen
images (static condition) and a set of temporally
randomized dynamic frames (shufﬂed condition) (seeFigure 1; supplementary videos related to this article
can be found under the speciﬁc links). In the static
condition, movies were created by taking the apex
frame of each dynamic sequence and replicating it 30
times in a row. In the shufﬂed condition, movies weregenerated by randomly selecting the individual framesof the dynamic sequences. This condition was originallydesigned to assess whether human observers weresensitive to the temporal development of an expression
over time (i.e., order of frames). The results reported byGold et al. revealed that recognition efﬁciency did not
signiﬁcantly differ between the dynamic and shufﬂed
expressions in young adults, suggesting that young
adults are insensitive to the temporal properties
associated with the unfolding of an expression.
We normalized all stimuli for their low-levelproperties and the amount of energy sampled over timeeven for the static condition for every frame. Moreconcretely, the video stimuli were normalized across all
frames and all expressions using the SHINE toolbox
with the default option (Willenbockel et al., 2010). In
order to partly account for the differences in visual
input between static and dynamic stimuli, we computed
the raw pixel intensity differences between each frameof the dynamic movies. We, thus, added these intensity
differences to each frame at random permuted locations in the static images. It is worth noting that
normalization after adding the noise would defeat thispurpose as the difference between each frame will not
be straightforwardly comparable with the natural lowlevel differences in the dynamic stimuli. However, with
our approach, we could ensure that all the frames for
all the faces in all conditions have equal low-levelproperties (luminance and contrast). The stimuli weredisplayed on a color liquid-crystal display with aresolution of 1,440 3 900 pixels and a refresh rate of 60
Hz. The whole experiment was programmed in

5

Figure 1. Examples of the stimuli used in our study. Face
identities and facial expressions used for the study with eachactor (column) and the six expressions (row: anger, disgust, fear,
happiness, sadness, surprise). Please note that we inserted
noise in the static condition in order to normalize the amountof energy sampled over time across conditions. For an
illustrative purpose, see an example for anger for the static(http://perso.unifr.ch/roberto.caldara/JoV/Anger-static.mov),
dynamic (http://perso.unifr.ch/roberto.caldara/JoV/Angerdynamic.mov), and shuffled conditions (http://perso.unifr.ch/roberto.caldara/JoV/Anger-shuffled.mov). The stimuli wereadapted with permission from Gold et al. (2013).

MATLAB (MATLAB 2014B; MathWorks, Natick,MA) using the Psychophysics Toolbox (PTB-3; Brainard, 1997; Kleiner et al., 2007).

Procedure
Participants were told that they would see faces
expressing different kinds of emotions on a computer
screen and their task would be to categorize them as
accurately as possible, according to the six following
possibilities: anger, disgust, fear, happiness, sadness,and surprise.
In order to familiarize children with the faces andensure that they understood the conceptual meaning of
all expressions, we presented them with printed sheetsof the different expressions and asked them to tell us
how the person presented on the image was feeling.
All participants sat 65 cm away from a computer
screen in a quiet room. Each trial started with a whiteﬁxation cross presented at the center of the screen for
500 ms. The stimuli were then presented in a random
order, one at a time, in the center of the computer
screen for a duration of 1 s each (for a schematic
representation of the procedure, see Figure 2). We used
the same stimuli presentation time in all three

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

6

Figure 2. Schematic representation of the procedure. Each trial began with a white fixation cross that was presented for 500 ms,
followed by a face presented for 1 s, which expressed one of the six basic facial expressions of emotion: anger, disgust, fear,
happiness, sadness, and surprise. After each trial, participants were asked to categorize the previously seen expression.

conditions in order to fully replicate the study by Goldet al. (2013). Note that a presentation time of 1 s was
also previously used in other studies with dynamic faces(Adolphs et al., 2003; Recio et al., 2013; Richoz et al.,2015). After each presentation, a response window was
displayed on the screen and remained there until theparticipant answered. Observers categorized eachstimulus by using a computer keyboard on which welabeled the keys accordingly. They could press a keylabeled ‘‘I don’t know’’ if they were unsure, had not had
enough time to see the expression, or did not know theanswer. We decided to introduce an ‘‘I don’t know’’
option in order to reduce the noise and response bias
produced by the lack of such a key. We gave ourparticipants as much time as required to categorize theexpressions and told them that judgment accuracy was
important, not the response time. Children under theage of 10, participants who were not familiar withcomputers, and elderly people over 65 gave theiranswers verbally to the experimenter who keyed themin. No feedback was provided. The stimuli were
blocked by condition. Each condition consisted of two
blocks of 48 trials (eight actors, six expressions)presented twice (96 expressions for each condition) fora total of 288 trials. Participants took part in all three
conditions in a counterbalanced random order. The
testing was done in one session for adolescents andadults, two or three sessions for participants under 10
or over 65. Before starting the testing phase, participants completed 12 practice trials for each condition.

Data analysis
Data analysis was performed in Python using
Jupyter Notebook. Summary statistics by groups are
displayed as confusion matrices (Supplementary Figure
S1A through E) and line plots (Figure 4) for eachcondition.
Bayesian modeling was performed using PyMC3
version 3.2, and the results were displayed using Seabornand Matplotlib. The main aim of the current study was
to determine the underlying function between expression-recognition ability and age, conditioned on thebasis of different types of visual stimuli. More speciﬁcally, we were interested in modeling expression-

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

7

Figure 3. A conceptual representation of the step-linear model. We are interested in the posterior distribution of the peak efficiency
and the contrasts between the posterior distribution of different slopes and the different intercepts.

recognition ability as a function of age, expression, andstimuli type (static, dynamic, or shufﬂed):
Recognition Ability ¼ fðage; expression; stimuli typeÞ:Here, recognition abilities were measured using thecorrect identiﬁcation (i.e., hit). Importantly, as thetarget function f is a nonlinear function, in order tocapture the increase and then the decrease in therecognition abilities displayed in the data, we constructed a simple step-linear function with two linear
equations. The ﬁrst equation captures the increase inrecognition abilities before a break point, deﬁned as the

Figure 4. Accuracy across age groups for each expression in the three different conditions. Error bars show 95% bootstrap confidence
interval for the mean. Age groups were created as follows: 5–6, 7–8, 9–10, 11–12, 13–14, 15–16, 17–18, 19–20, 21–30, 31–40, 41–50,
51–60, 61–70, 71–80, 81–96.

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

momentum in age capturing peak efﬁciency, whereas
the second equation captures the decrease in recognition abilities.
f1 ðage; expression; stimuli typeÞ; age , s;f2 ðage; expression; stimuli typeÞ; age  s:
Here, the break point s is expressed as a latentvariable that is estimated from the model. Both f1 andf2 are linear functions of age during the recognition of
speciﬁc expressions and stimuli type. Thus, the slope ofthe function f1 and f2 (coefﬁcient for age) captures thechange in recognition abilities, whereas the intercept of
the function captures the general recognition abilities
before and after the age s. We estimated the general
dynamic advantage by computing contrasts of theintercepts between the different stimuli type (i.e., static,
dynamic, shufﬂed) and quantiﬁed the interaction
between stimuli type and age (i.e., whether there is astronger dynamic advantage in young/old age) bycomputing contrasts of the slopes. Importantly, considering that the break point s could occur at different
age stages among the expressions, we modeled each
expression independently, thus turning the targetfunction into
f1 ðage; stimuli typejexpressionÞ; age , s;f2 ðage; stimuli typejexpressionÞ; age  s;where recognition ability of different types of stimuli isa step-linear function of age conditioned on a speciﬁc
expression.
In practice, we formulated functions f1 and f2 aslogistic regressions with the function output being the
success probability p in each trial in the binomial
distribution. The total number of correct responses for
one participant during the presentation of one expression and one stimuli type follows a binomial distribution:k ; Binomialðp; nÞ:Thus, this is an extended beta-binomial model withlatent variables. The full model is formulated as below:
i for each task (dynamic, static, shufﬂe), j for eachparticipant.
Hyper-priors of the slope
lb ; Student t(3, 0, 10)
rb ; Half Normal (10)Hyper-prior of the breakpoints ; Uniform (0, 100)Hyper-prior of the recognition ability at s
h ; Uniform (0, 1)
j ; Uniform (0, Nt)(reparameterized as the mode of beta distribution)8

a ¼ h 3 (j – 2) þ 1
b ¼ (1  h) 3 (j – 2) þ 1
For each stimuli type i  static, dynamic, shufﬂed:
Prior of the recognition ability at s
hi ; Beta (a, b)
Priors of the break point
si ; Normal (s, 10)Priors of the slopes (a indicates before age s)ba;bi ; Normal (lb, rb)
The intercepts before and after age sa;bba;bi ¼ logit(hi)  si 3 bi
Linear function andtransform invlogitbai ; bai if age , sibi, Intercepti ¼
bbi ; bbi if age  si
hi,j ¼ bi  agej þ Interceptiŷi,j ¼ invlogit(hi,j)Observed accurate categorizations
ki,j ; Binomial (ŷi,j, ni,j)As shown above, the slope of each condition isregularized using a weakly informative hyper-prior. The
prior of each slope is a normal distribution with themean distributed as a zero mean Student t distribution
with three degrees of freedom and 10 standard
deviations and the standard deviations distributed as ahalf-normal distribution. The hyper-prior of the breakpoint s is a uniform distribution from 0 to 100, which is
the overall mean of the condition-speciﬁc break pointthat follows a normal distribution with 10 standard
deviations as prior. Importantly, the intercept of the twolinear functions f1 and f2 is determined by therecognition ability h at the break point s. The conditionspeciﬁc recognition ability hi follows a Beta distribution
as prior. Moreover, we reparameterized the beta
distribution by the mode h and the concentration j(Kruschke, 2014, cf. equation 9.4, p. 223). Here, themode h follows a uniform prior between zero and one,and j follows a uniform prior with two as minimum andthe number of trials as maximum.
The probabilistic model was built using PyMC3, andwe sampled from the posterior distribution using NUTS
with automatic differentiation variational inference
initialization. We ran four Markov chain Monte Carlochains with 3,000 samples each; the ﬁrst 1,000 samples
were used for tuning the mass matrix and step size for
NUTS and were discarded following this. Model
convergence was diagnosed by computing Gelman and
Rubin’s (1992) convergence diagnostic (R-hat), examining the effective sample size, inspecting the mixing of
the traces, and checking whether there is any divergent
sample that has been returned from the sampler.
From the posterior distribution, we estimated (a) the
peak efﬁciency, namely the point at which observers’
recognition performance reaches its maximum before

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

9

Figure 5. Anger. The posterior model fit (solid line) for the expression of anger with the individual performance (scatter plot) and thegroup average performance (dots with error bars) is given here. The overall peak efficiency is shown as the red vertical dashed line,
and the condition-specific peak efficiencies are represented by the black dashed lines.

declining; (b) the steepness of increase and decrease inrecognition abilities; (c) differences in the steepness ofincrease and decrease between different conditions (e.g.,
dynamic vs. static); and (d) the overall processing
advantage of the dynamic over the static and the
shufﬂed stimuli. By performing statistical inference
directly on the full posterior distribution, we were able toproperly quantify the dynamic stimuli effects and theirassociated uncertainty. In fact, the slopes of the linear
relationship before and after the peak efﬁciency could
provide estimations of the developmental trajectory andcan be useful to make predictions on the performance ofany new participant from any age. Here, however, wewere only interested in comparing the slope difference
between dynamic and static faces. A conceptual
representation of the model is provided in Figure 3.

Results
The group average categorization performance foreach condition is presented in Figure 4. The nonlinear
relationship between age and recognition ability isclearly demonstrated with differences among conditions clearly visible for some expressions. When themodel returns a concave pattern, we refer to the breakpoint as a peak efﬁciency. This value relates to thepoint at which recognition performance reaches its
apex, also relating to the age at which observers are themost efﬁcient.
For the Bayesian modeling, trace plots, posterior
distributions for the key parameters in the model,
contrasts of interest, and full numerical reports of theparameter estimations are available in the supplementary results. Below, we report the key ﬁndings of thestep-linear model.

Anger
The posterior model ﬁt for the raw data is shown in
Figure 5. By sampling the full posterior distribution, we
estimated that the overall recognition ability for the
expression of anger peaks at age 36.13 [22.23, 51.21],(bracket shows 95% highest posterior density interval).
The posterior expectation of the age at which observers
are the most efﬁcient is given as follows: dynamic 39.17[31.03, 46.84], static 35.70 [23.59, 46.41], and shufﬂed
33.22 [18.13, 49.62]. The overall recognition ability ofanger at peak efﬁciency is 0.605 [0.350, 0.872], and theaverage peak accuracy for each condition is given as
follows: dynamic 0.660 [0.627, 0.696], static 0.592[0.554, 0.628], and shufﬂed 0.539 [0.489, 0.587]. On
average, participants showed better performance in the
dynamic condition as compared to the static and the
shufﬂed conditions, both before (dynamic–static: 0.075[0.042, 0.106], dynamic–shufﬂed: 0.096 [0.060, 0.132])
and after (dynamic–static: 0.043 [0.012, 0.072], dynamic–shufﬂed: 0.085 [0.051, 0.118]) peak efﬁciency. In
contrast, the difference between the static and shufﬂed
conditions is quite small (shufﬂed–static before peak
efﬁciency: 0.021 [0.061, 0.019], after peak efﬁciency:
0.042 [0.070, 0.013]). The slopes of the step-linear
functions are the following: dynamic 0.0069 [0.0011,
0.0156], static 0.0107 [0.0003, 0.0239], shufﬂed 0.0040[0.0072, 0.0180] before peak efﬁciency and dynamic
0.0251 [0.0300, 0.0201], static 0.0183 [0.0236,
0.0123], shufﬂed 0.0159 [0.0211, 0.0108] after
peak efﬁciency. Moreover, the differences of the slope
across different conditions are mostly negligible; mostof the posterior contrasts are distributed around zerowith the exception of the contrast: dynamic–shufﬂed:
0.0092 [0.0162, 0.0020] after peak efﬁciency (Figure 5).

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

10

Figure 6. Disgust. The posterior model fit (solid line) of the expression of disgust with the individual performance (scatter plot) andthe group average performance (dots with error bars) is given here. The overall peak efficiency is shown as the red vertical dashedline, and the condition-specific peak efficiencies are represented by the black dashed lines.

DisgustFear

The overall recognition ability for the expression of
disgust peaks at age 18.87. The posterior expectation of
the age at which observers are the most efﬁcient is given
as follows: dynamic 18.02 [15.38, 20.75], static 19.71[18.19, 21.38], and shufﬂed 18.14 [16.55, 19.67]. The
overall recognition ability of the expression of disgust
at peak efﬁciency is 0.644 [0.380, 0.920], and theaverage peak accuracy for each condition is thefollowing: dynamic 0.665 [0.644, 0.687], static 0.724[0.702, 0.744], and shufﬂed 0.500 [0.475, 0.522]. On
average, participants showed better performance in the
dynamic and static conditions as compared to the
shufﬂed condition, both before (dynamic–shufﬂed:
0.199 [0.160, 0.240], static–shufﬂed: 0.183 [0.146,
0.217]) and after (dynamic–shufﬂed: 0.191 [0.170,
0.214], static–shufﬂed: 0.152 [0.125, 0.177]) peak
efﬁciency. The difference between the dynamic and thestatic conditions is quite small before peak efﬁciency(dynamic–static: 0.016 [0.037, 0.072]); it is, however,substantial after peak efﬁciency (0.040 [0.013, 0.069]).
The slopes of the step-linear functions are thefollowing: dynamic 0.0494 [0.0318, 0.0680], static
0.0824 [0.0671, 0.0993], shufﬂed 0.0686 [0.0511, 0.0860]
before peak efﬁciency and dynamic 0.0104 [0.0130,
0.0078], static 0.0225 [0.0254, 0.0196], shufﬂed
0.0128 [0.0152, 0.0102] after peak efﬁciency.
Moreover, the slopes of the static condition are steeper
than the ones in the dynamic and shufﬂed conditions.
The contrasts of the slopes before peak efﬁciency are
given as follows: dynamic–static: 0.0330 [0.0577,
0.0096], shufﬂed–static: 0.0138 [0.0360, 0.0123];and the contrasts of the slopes after peak efﬁciency are
the following: dynamic–static: 0.0121 [0.0084, 0.0161],
shufﬂed–static: 0.0097 [0.0059, 0.0135] (Figure 6).

The overall recognition ability of the expression offear peaks at around age 20.83. The posterior
expectation of the age at which observers are the most
efﬁcient is given as follows: dynamic 20.87 [18.71,
23.18], static 19.72 [17.63, 21.43], and shufﬂed 21.79[20.17, 23.51]. The overall recognition ability of fear atpeak efﬁciency is 0.446 [0.168, 0.697]; the average peakaccuracy for each condition is the following: dynamic
0.399 [0.372, 0.430], static 0.416 [0.390, 0.443], andshufﬂed 0.526 [0.494, 0.557]. On average, participants
showed better performance in the shufﬂed condition
compared to the other two conditions, both before(shufﬂed–dynamic: 0.056 [0.013, 0.099], shufﬂed–static:
0.055 [0.024, 0.083]) and after (shufﬂed–dynamic: 0.094[0.071, 0.118], shufﬂed–static: 0.113 [0.089, 0.135]) peak
efﬁciency; however, the difference between the dynamic
and static conditions is quite small (dynamic–static
before peak efﬁciency: 0.002 [0.043, 0.033], after
peak efﬁciency: 0.018 [0.003, 0.040]). The slopes of allconditions are comparable: dynamic 0.0706 [0.0546,
0.0867], static 0.0927 [0.0747, 0.1139], shufﬂed 0.0913[0.0764, 0.1056] before peak efﬁciency and dynamic
0.0151 [0.0186, 0.0115], static 0.0189 [0.0224,
0.0157], shufﬂed 0.0176 [0.0213, 0.0141] after
peak efﬁciency. The maximum contrasts of the slopes
before peak efﬁciency is given as follows: dynamic–
static: 0.022 [0.0473, 0.0022], and the maximum
contrasts of the slopes after peak efﬁciency is thefollowing: dynamic–static: 0.0038 [0.0010, 0.0086](Figure 7).

Happiness
Unlike for the other facial expressions, the overall
recognition ability for the expression of happiness is

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

11

Figure 7. Fear. The posterior model fit (solid line) of the expression of fear with the individual performance (scatter plot) and thegroup average performance (dots with error bars) is presented here. The overall peak efficiency is shown as the red vertical dashedline, and the condition-specific peak efficiencies are represented by the black dashed lines.

near ceiling at a very young age, declining slowly
throughout the life span. Therefore, the age of the peak
efﬁciency could not be identiﬁed for this facial
expression. Nonetheless, our model identiﬁes a breakpoint at around age 57.98 with a large uncertainty.
Importantly, the accuracy rate estimated at this breakpoint is not the apex in recognition performance, butrather the start of the decline (i.e., the model did not
return a concave pattern). The posterior expectation of
the age at this break point is given as follows: dynamic
61.27 [24.18, 93.42], static 50.25 [35.90, 62.01], andshufﬂed 62.22 [21.95, 81.00]. The overall recognition
ability of this expression at the break point is 0.855[0.664, 0.999] with dynamic 0.895 [0.819, 0.972], static
0.898 [0.868, 0.931], and shufﬂed 0.660 [0.541, 0.896].
Overall, participants performed better in the dynamic
condition as compared to the static and shufﬂed
conditions, both before (dynamic–static: 0.040 [0.027,
0.054], dynamic–shufﬂed: 0.106 [0.063, 0.143]) and after(dynamic–static: 0.060 [0.029, 0.095], dynamic–shufﬂed: 0.269 [0.203, 0.319]) the break point. Participantsalso performed better in the static than in the shufﬂed
condition (static–shufﬂed before the break point: 0.066[0.018, 0.105] and after the break point: 0.209 [0.156,
0.255]). The slopes of the step-linear functions are given
as follows: dynamic 0.0248 [0.0335, 0.0121], static
0.0078 [0.0172, 0.0007], shufﬂed 0.0306 [0.0375,
0.0163] before the break point and dynamic 0.0239[0.0387, 0.0023], static 0.0298 [0.0381, 0.0217],
shufﬂed 0.0175 [0.0355, 0.0026] after the breakpoint. The differences of the slopes across the different
conditions are mostly negligible. Most of the posterior
contrasts are distributed around zero with the largest
contrasts being the following before the break point:
dynamic–static: 0.0170 [0.0297, 0.0036] and shufﬂed–static: 0.0228 [0.0353, 0.0087] (Figure 8).

Figure 8. Happiness. The posterior model fit (solid line) of the expression of happiness with the individual performance (scatter plot)
and the group average performance (dots with error bars) is presented here. The overall break point is shown as the red verticaldashed line, and the condition-specific break points are represented by the black dashed lines.

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

12

Figure 9. Sadness. The posterior model fit (solid line) of the expression of sadness with the individual performance (scatter plot) andthe group average performance (dots with error bars). The overall peak efficiency is shown as the red vertical dashed line, and thecondition-specific peak efficiencies are represented by the black dashed lines.

Sadness
The overall recognition ability for the expression ofsadness peaks at the age of 28.96. The posterior
expectation of the age at which observers are the most
efﬁcient is given as follows: dynamic 27.87 [21.44,
33.90], static 28.12 [23.50, 32.62], and shufﬂed 30.52[26.05, 34.74]. The overall recognition ability of sadness
at peak efﬁciency is 0.638 [0.408, 0.888]; the averagepeak accuracy for each condition is the following:
dynamic 0.605 [0.572, 0.636], static 0.631 [0.602, 0.660],
and shufﬂed 0.653 [0.622, 0.681]. The categorization
accuracy rates of all conditions are comparable both
before and after peak efﬁciency. The maximum
contrasts of the average performance before peak
efﬁciency is given as follows: shufﬂed–static 0.0277[0.0011, 0.0561]; the maximum contrasts of theaverage performance after peak efﬁciency is thefollowing: shufﬂed–static 0.0097 [0.0156, 0.0362].
Similarly, all conditions show comparable slopes:
dynamic 0.0075 [0.0031, 0.0181], static 0.0235 [0.0117,
0.0347], shufﬂed 0.0179 [0.0079, 0.0285] before peak
efﬁciency and dynamic 0.0223 [0.0262, 0.0186],
static 0.0265 [0.0306, 0.0223], shufﬂed 0.0302[0.0346, 0.0261] after peak efﬁciency. The maximum
contrast between slopes before peak efﬁciency is thefollowing: dynamic–static: 0.0160 [0.0320, 0.0010];
the maximum contrast after peak efﬁciency is given as
follows: dynamic–shufﬂed: 0.0079 [0.0022, 0.0137](Figure 9).

Surprise
The overall recognition ability of surprise peaks atage 22.47. The posterior expectation of the age at whichobservers are the most efﬁcient is given as follows:
dynamic 23.55 [20.52, 26.91], static 24.30 [20.36, 28.26],

and shufﬂed 19.34 [17.34, 21.76]. The overall recognition ability of surprise at peak efﬁciency is 0.692 [0.466,
0.953]; the average peak accuracy for each condition is
dynamic 0.758 [0.735, 0.783], static 0.700 [0.673, 0.725],
and shufﬂed 0.575 [0.552, 0.599]. On average, participants showed the best performance in the dynamic
condition, and the worst in the shufﬂed condition. The
results were the following: dynamic–static: 0.075 [0.048,
0.101], static–shufﬂed: 0.093 [0.058, 0.123] before peak
efﬁciency and dynamic–static: 0.107 [0.082, 0.133],
static–shufﬂed: 0.172 [0.146, 0.195] after peak efﬁciency. The slopes of the step-linear functions are dynamic
0.0442 [0.0315, 0.0565], static 0.0442 [0.0311, 0.0577],
shufﬂed 0.0530 [0.0381, 0.0692] before peak efﬁciency
and dynamic 0.0126 [0.0164, 0.0092], static
0.0175 [0.0213, 0.0133], shufﬂed 0.0190 [0.0220,
0.0163] after peak efﬁciency. The slope between age andaccuracy is similar across all conditions before peak
efﬁciency, whereas after peak efﬁciency, the dynamic
condition shows the most gradual slope: static–
dynamic: 0.0048 [0.0104, 0.0004], shufﬂed–dynamic:
0.0064 [0.0109, 0.0018] (Figure 10).
To highlight the key results of our study, we
illustrated the results of the previous statistical analyses
comparing the dynamic and static tasks in Figure 11.
The posterior estimation of peak efﬁciency ages in therecognition of all the expressions except happiness are
summarized in Figure 11A. The estimated age is the
youngest for disgust and fear, whereas the oldest foranger (which is also the expression driving the largest
uncertainty in the estimation). Moreover, the dynamic
and static conditions do not signiﬁcantly modulate thiseffect.
The differences in recognition between the dynamic
and static conditions are summarized in Figure 11B,
which shows performance both before and after theestimated peak efﬁciency. As reported below, therecognition performance of the facial expression of

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

13

Figure 10. Surprise. The posterior model fit (solid line) for the expression of surprise with the individual performance (scatter plot)
and the group average performance (dots with error bars) are given here. The overall peak efficiency is shown as the red verticaldashed line, and the condition-specific peak efficiencies are represented by the black dashed lines.

Figure 11. Summary of the key findings. (A) The posterior estimation of peak efficiency ages in the recognition of different facial
expressions of emotion. (B) The posterior estimation of the dynamic advantage before and after the peak efficiency age (represented
by the average difference between the correct categorization of dynamic and static facial expressions). The recognition of the happy
expression could not be identified as the performance for this facial expression was already at ceiling in the early age we tested. The
dots show the posterior expectation, the bold horizontal line shows the 50% highest posterior density, and the thin horizontal line
shows the 95% highest posterior density. Nonoverlapping lines indicate a significant difference between two conditions.

surprise shows the largest dynamic advantage, whereas
fear is, on average, similar in both conditions.

Discussion
Our results present a ﬁne-grained developmental
tracking of human observers’ ability to recognize the six
basic emotions when presented with varying temporal

properties: dynamic, static, and shufﬂed. Previous
studies in the literature examined expression recognition
by using arbitrary age groups: 10-year bins (Williams etal., 2009), stages of life (Horning, Cornwell, & Davis,2012), or largely different age groups (e.g., 18–30, 58–70,
Calder et al., 2003) while revealing either expressionrecognition improvement (Rodger et al., 2015) or decline(Calder et al., 2003; MacPherson et al., 2002; Malatesta,
Izard, Culver, & Nicolich, 1987; Moreno, Borod,
Welkowitz, & Alpert, 1993; Ruffman et al., 2008;

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

Sullivan & Ruffman, 2004a). In contrast, our approachinnovates by estimating the continuous developmental
trajectory of facial expression recognition (from increase
to decline) by considering age as a continuum, ranging
from 5 to 96 years.
Using a Bayesian approach, we estimated for eachcondition and each expression individually the associated uncertainty and (a) the peak efﬁciency, namely the
point at which observers’ recognition performancereaches its maximum before declining; (b) the steepnessof increase and decrease in recognition abilities; (c)differences in the steepness of increase and decrease
between different conditions (e.g., dynamic vs. static);and (d) the overall processing advantage of the
dynamic over the static and the shufﬂed stimuli. We
now discuss, in turn, each of these ﬁndings and theirimplications.

Recognition trajectory across development:
Increase, peak efficiency, and decrease
Our ﬁndings reveal unique developmental proﬁlesand peak efﬁciency for the static, dynamic, and shufﬂed
versions of each individual expression. Herein, we focus
on the dynamic and static trajectories and thedifferences between both conditions in a more detailed
manner (i.e., static and dynamic). The results of the
shufﬂed condition are brieﬂy considered at the end of
the discussion.

Efficiency: Increase
In both static and dynamic conditions, the sharpest
rises in accuracy were observed for fear followed by
disgust and, to a lesser extent, surprise. These ﬁndings
mirror the results of a previous developmental study
that investigated the effects of age on the developmentof emotion processing in children, revealing that
increasing age produced signiﬁcant improvements in
the recognition of fear and disgust (Herba & Phillips,
2004). We observed a more gradual increase for sadness
and anger but only in the static condition. Finally, we
did not observe any increase for the expression ofhappiness regardless of the experimental condition.
The steepest increase observed for fear might beaccounted for by the very low recognition rates
observed for this expression in young children, reaching
only 13% in the 5–6 age group in the static condition(15% in the dynamic condition; Supplementary Figure
S1A). The expression of fear has been regularly
reported in developmental (Herba & Phillips, 2004;
Rodger et al., 2015; Widen, 2013), neuropsychological(Adolphs et al., 2003; Richoz et al., 2015), andbehavioral studies (Calder et al., 2003) as being the14

most difﬁcult expression to effectively recognize among
all the expressions—a difﬁculty that is puzzling
considering the evolutionary importance of an adequate and rapid categorization of this expression forsurvival. Importantly, however, a poor performance forthe recognition of some expressions does not necessarily mean that these expressions are not detected asrecent studies have shown a dissociation between thesetwo processes (Smith & Rossit, 2018; Sweeny, Suzuki,
Grabowecky, & Paller, 2013). For example, Smith andRossit (2018) have shown that the emotional expressionof fear is better detected than recognized. Also, among
the basic expressions, fear is probably the one thattransmits the strongest multisensory perceptual cues.
Multisensory and contextual information, such asenvironmental threats, may, therefore, play a crucialrole in the decoding of this expression and be essential
for an adequate categorization.
Consistent with our ﬁndings, fear has been observed
to display a sharp increase in some prior developmental
studies (Herba, Landau, Russell, Ecker, & Phillips,
2006; Vicari, Reilly, Pasqualetti, Vizzotto, & Caltagirone, 2000), and other studies have revealed more
gradual improvements (Gao & Maurer, 2009; Thomas,
De Bellis, Graham, & LaBar, 2007) or stable, albeit low
task performance from early childhood to adulthood(Rodger et al., 2015). Differences across studies may be
attributed to methodological considerations and taskdifferences as recognition rates have been proven to betask dependent (e.g., Montirosso, Peverelli, Frigerio,Crespi, & Borgatti, 2010; Vicari et al., 2000) withperformance variations occurring even within the same
study when the task is changed (Vicari et al., 2000).
Importantly, the ﬁndings reported here provide further
evidence that the recognition of fear has a special status
within the framework of facial-expression recognition(Richoz et al., 2015; Rodger et al., 2015).
Disgust also showed a steep increase in recognition
accuracy, following a similar trajectory as fear. In linewith our ﬁndings, steep improvements from childhood
to adulthood were previously observed for disgust in astudy by Rodger et al. (2015), which measured thequantity of information necessary for an observer toaccurately recognize facial expressions, as well as inearlier studies that investigated expression recognition
with matching (Herba et al., 2006) or labeling tasks(Vicari et al., 2000). As mentioned by Vicari et al.
(2000), the steep improvement observed for disgust inchildren aged 5 to 10 may occur owing to the greater
lexico-semantic abilities in older children. It might also
be plausible that the very distinctive facial conﬁgurations of disgust convey signals about potentiallycontaminated food. These signals are crucial from an
evolutionary perspective and, hence, the need to rapidly
improve in the detection of this expression in order tostay away from harmful substances.

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

Finally, our ﬁndings also revealed a sharp increase
for surprise in both the static and dynamic conditions.
Interestingly, the expression of surprise was alreadywell recognized in very young children aged 5 to 6 withrecognition rates of 60% for the dynamic stimuli (55%for the static images; Supplementary Figure S1A). Highrecognition rates in young children would rather accord
with a more gradual developmental trajectory as
suggested by prior research that investigated therecognition of surprise from 5 up to 18 years of age(Rodger et al., 2015). Interestingly, however, the sharp
increase observed for surprise in the current study may
be accounted for by the very high recognition rates
observed for this expression in participants above theage of 18, reaching up to 79% in the 21–30 age groupfor the dynamic stimuli (67% for the static images;
Supplementary Figure S1C).
We also observed a gradual increase for anger andsadness although only in the static condition. Theseﬁndings are generally consistent with previous reports(Herba et al., 2006; Rodger et al., 2015, for sad; Vicari
et al., 2000). In the dynamic condition, children aged 5
to 6 were nearly as effective in recognizing anger (62%;Supplementary Figure S1A) as young adults aged 17–
18 (64%; Supplementary Figure S1B), 21–30 (66%;
Supplementary Figure S1C), or 31–40 (66%; Supplementary Figure S1D). The same pattern was observed
for sadness with identical recognition rates for young
children aged 5–6 (56%) and young adults in the 17–18
age group.
Finally, our results did not reveal an increase forhappiness in either condition with task performanceremaining stable over ages and with our peak efﬁciency
revealing the peak of the decline. The absence ofimprovement observed for happiness may be explainedby the very high recognition accuracy already found inyoung children for this expression, which leaves little
scope for improvement. Our ﬁndings for happiness are
consistent with previous studies that revealed thatchildren as young as 5 years of age recognize theexpression of happiness just as effectively as adults(Gao & Maurer, 2009; Gross & Ballif, 1991; Herba &Phillips, 2004) even when the presentation time was as
fast as 500 ms (Rodger et al., 2015). In order to capture
the increase in recognition performance for happiness,
we should have started earlier, under 5 years of age, or
adopt alternative approaches. Limiting the available
information by degrading the signal (Rodger et al.,2015), controlling for spatial frequency use (Gao &Maurer, 2011), or modifying the intensity of facial
expressions (Gao & Maurer, 2009, 2010; Rodger, Lao,
& Caldara, 2018) all represent alternative techniques topotentially reveal the improvement in the recognitionof the emotional expression of happiness. Therefore,
future studies using those approaches represent a
promising route to assess and eventually reveal

15

developmental differences in the trajectory of thedecoding of happiness.
Additionally, it is worth noting that our ﬁndings
revealed differences in the steepness of increase between
the static and dynamic conditions for the expressions of
disgust and sadness, the increase being steeper with thestatic stimuli. These ﬁndings might be accounted for by
the low recognition rates found for the static images of
disgust and sadness in very young children. An
exposure to static images of disgust and sadness israther uncommon in everyday life, particularly foryoung children, whereas an exposure to the dynamic
versions of these expressions might be more frequent
for children when their schoolfellows or siblings dislike
some particular food they have to eat (disgust) or when
they cry or express their sorrow (sadness).

Peak efficiency
The data-driven identiﬁcation of the peak efﬁciency,
the point at which observers’ recognition performancereaches its optimum before declining, revealed a seriesof novel interesting ﬁndings. To the best of ourknowledge, this is the ﬁrst study that has effectively
isolated the age at which observers are the most
efﬁcient for the recognition of the basic facial
expressions of emotion across the life span. We
observed the earliest peak efﬁciencies for both the staticand dynamic expressions of disgust (18–20 years) andfear (19–21 years) in young adults, followed by surprise(23–25 years) and sadness (27–28 years). Peak efﬁciency
for the static expression of anger was found at 35 yearsof age, whereas the recognition of its dynamic version
was reached at 39 years. The latest break point thatemerged from our data was observed for the dynamic
expression of happiness at around 61 years of age (50years for the static version). It is worth noting that the
break points found for each expression were nearly thesame in both conditions with the exception of anger
and happiness, which reached their break points laterwith dynamic expressions.
There are two explanations for the very early peak
efﬁciencies found for fear and disgust. First, as
mentioned above, from an evolutionary perspective,these two expressions convey important signals about
potential dangers or harmful substances, both important for survival. Disgust and fear can, therefore, be
expected to reach their peak rapidly in order to ensure
survival. Second, for fear and disgust, the point in timeat which the peak efﬁciency emerges may be driven by
the inherent properties of those expressions. Stimuli
that are difﬁcult to recognize for young people might beeven more difﬁcult for elderly people as difﬁcult tasks
are likely to be more sensitive to cognitive decline(Calder et al., 2003; Ruffman et al., 2008). Changes in

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

the slope of the lines may, therefore, be expected tooccur earlier with difﬁcult tasks. We examined response
biases for each expression, computing confusion
matrices across different age groups (see Supplementary Figure S1A through E). The confusion matrices
found for fear and disgust indeed revealed that these
two expressions were particularly difﬁcult for ourobservers to identify. Disgust was commonly confusedwith anger with confusion rates ranging up to 28% in
the 5–6 age group for the dynamic stimuli (Supplementary Figure S1A) and 20% for the 71–80 age group(Supplementary Figure S1E). Previous studies also
reported marked confusion between disgusted andangry faces, which were interpreted as a general bias
toward angry faces (Recio et al., 2013). Such a bias
could explain the stable and high recognition rates
found in the current research for angry faces fromchildhood onward. Other confusion was observed
between fear and surprise. In line with previous studies(Rodger et al., 2015), fear was found to be the mostfrequently confounded expression among all age
groups with confusion rates reaching up to 53% for the
dynamic expression of surprise in the 5–6 age group(Supplementary Figure S1A) or even 63% in the 71–80
age group (Supplementary Figure S1E). As mentioned
by Calder et al. (2003), age-related cognitive decline
may reinforce these confusions due to perceptual orconceptual difﬁculties (i.e., fear and surprise areconceptually very close and share facial signals that aremorphologically similar; Delis et al., 2016). Note alsothat the reverse confusion was much less common.
When presented with surprise, the confusion rates
observed for fear reached only 3% for the dynamic
stimuli (4.6% for the static expressions) in the 5–6 age
group and 5% (3.3% for the static expressions) inelderly people aged 71–80.
Interestingly, our ﬁndings also revealed a later
emergence of the peak efﬁciency for anger compared tothe other expressions. As mentioned before, recognition abilities for anger showed no increase in the
dynamic condition, task performance being alreadyhigh in young children, and displayed only a slightincrease for the static condition, recognition rates being
also high in young children. A potential, although
speculative, explanation for this observation may lie in
the fact that we are daily exposed to the expression ofanger, arguing with our partners, children, colleagues—
an exposure that might postpone the recognition
decrease of this expression and, therefore, the changes
in the slope of the line.
Finally, the latest break point found for happinessmay be accounted for by the ceiling effect found for thisexpression from childhood onward.
Altogether, this second set of ﬁndings offers novel
insights into the development of human facial-expression recognition. As observed, facial-expression recog-

16

nition develops following emotion-dependent
trajectories that do not necessarily all reach their peak
efﬁciency in early adulthood as predicted by previous
studies (Calder et al., 2003; De Sonneville et al., 2002;
Horning et al., 2012; Williams et al., 2009). The optimallevel of task performance can indeed be reached at avery late point in development, depending also on thevery nature of the diagnostic information of the facial
expression and its temporal properties and evolutionary value.

Efficiency: Decrease
Finally, we observed differences in the steepness ofdecrease in recognition performance across emotions
and conditions. In the dynamic condition, the steepest
decreases were observed for anger, happiness, andsadness and less severe decreases for fear and surprise.
Disgust showed the least severe decrease in thiscondition. Different patterns were observed in thestatic condition, the steepest decline being for happiness followed by sadness and disgust. Less severe
decreases were found for fear and anger, whereas theleast severe decrease was observed for the expressionof surprise. Similarly to the differences in the steepnessof increase observed between static and dynamic
conditions, differences in the steepness of decrease
were observed between both conditions for the
expression of disgust. The recognition of the static
expression of disgust decreased from 51% to 34%
between the ages of 61 and 70 and 81 and 90, whereas
recognition accuracy of its dynamic version remained
relatively stable (decrease from 55 % to 52%;
Supplementary Figure S1E).
This pattern of results posits that the recognition of
facial expression declines over time, which is consistent with previous models of aging. These models
suggest that age-related structural changes in different
brain regions, particularly in frontal and temporal
volumes as well as changes in neurotransmitters(Calder et al., 2003; Ruffman et al., 2008) might beresponsible for older adults’ impairment in therecognition of facial expression. For example, theamygdala, which plays a crucial role in the processingof fear and sadness (e.g., Adolphs et al., 2005; Yang etal., 2002), undergoes severe atrophy with age andbecomes progressively less responsive to negative
stimuli (De Winter et al., 2016; Mather et al., 2004;
Ruffman et al., 2008). In contrast, the insula and basal
ganglia, which underlie the processing of disgust, seemto be less vulnerable to aging as evidenced by thepreserved ability to recognize this expression in older
adults (Calder et al., 2003; Horning et al., 2012;
Ruffman et al., 2008). Interestingly, our ﬁndings
revealed the least severe decrease in recognition

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

performance for the dynamic expression of disgust.
However, in contrast to previous studies that showedno reduction in the recognition of some expressions oreven some improvements with increasing age (e.g.,
Calder et al., 2003), our ﬁndings revealed steep-tomoderate decreases for all the expressions even for
disgust being usually preserved in elderly people(Calder et al., 2003; Horning et al., 2012; Ruffman etal., 2008).
Methodological considerations may be responsible
for the differences observed between the current study
and previous ones. Indeed, previous studies investigated facial expression recognition across groups of ages
(Calder et al., 2003), stages of life (Horning et al.,2012), or decades (Williams et al., 2009), whereas ourstudy investigated elderly people’s ability to categorize
emotions by considering age as a continuum. Thismethodological approach overcomes the problem of
deﬁning arbitrary age boundaries, which are routinely
used in the literature to relate to critical developmental
ages.
Furthermore, the variability in ﬁndings between thecurrent research and previous neuropsychological andbehavioral studies can be accounted for by the age
ranges tested across the studies. For example, Calder etal. (2003) observed improved recognition abilities for
disgust in their older adult age group, spanning fromage 58 to 70 (mean age 65). In contrast, in our study, wetested participants up to the age of 96, giving rise to thepossibility that the decline for disgust appears at a later
point in development. This assumption is in line with aprevious study that showed a decrease in the recognition of disgust in elderly people aged 80 to 91 (Williams
et al., 2009).
Additionally, the stimuli used across the different
studies might also have impacted expression-recognition performance. In the current study, we used a
speciﬁc database of emotional expressions that are lessprototypical than the Ekman and Friesen (1976)
standard set of facial photographs used in previous
research (Calder et al., 2003; McDowell, Harrison, &Demaree, 1994; Sullivan & Ruffman, 2004b). Moreover, in contrast to previous reports that used only
static images displaying the apex or the highest stateof an emotional expression (Calder et al., 2003;
Ruffman et al., 2008), we tested facial expression
recognition with static, dynamic, and shufﬂed stimuli.
Importantly, our stimuli were controlled for theamount of low-level discriminative information carried over time. In other words, the quantity of lowlevel information carried by our static, dynamic, andshufﬂed stimuli was identical across conditions andtasks (Gold et al., 2013). In line with previous studies(Krendl & Ambady, 2010; Sze et al., 2012), we foundthat elderly people were impaired in recognizing static
but not dynamic expressions. However, in contrast to17

the ﬁndings reported by Krendl and Ambady (2010),
we also observed steep-to-moderate declines for all theexpressions even in the dynamic condition. However,in their study, participants were provided withadditional aiding cues, such as body-related information or contextual cues, which might have facilitatedexpression recognition given that the perception of aparticular expression is strongly inﬂuenced by thecontext in which it occurs (Barrett & Kensinger, 2010;
Horning et al., 2012). For example, Aviezer et al.
(2008) found more consistent recognition performancefor fear when person-related or contextual information was provided to the participants.
Finally, the divergence between our ﬁndings andthose of previous research may also be due to the small
number of trials presented (Horning et al., 2012;
Moreno et al., 1993) as well as the differences in thesettings used with some studies relying on laboratory
settings (Calder et al., 2003; Horning et al., 2012) andothers on online tasks (Williams et al., 2009).

Static versus dynamic expressions
A dynamic advantage before peak efficiency
Our ﬁndings revealed a dynamic face advantage forthe recognition of anger, surprise, and happinessbefore peak efﬁciency. These results are inconsistent
with previous developmental studies, which revealed
that dynamic presentations did not increase children’s
recognition performance (Nelson et al., 2013; Nelson& Russell, 2011; Widen & Russell, 2015), and withsome experiments showing even an overall advantage
for static expressions (Nelson & Russell, 2011, study
1; Widen & Russell, 2015). Such advantage also
differs from the results reported by previous studies inyoung and healthy adults (e.g., Christie & Bruce,
1998; Jiang et al., 2014; Kätsyri & Sams, 2008),
showing that the recognition of facial expressions is
not facilitated by the dynamic information provided
by moving faces.
The lack of consistency between these studies andthe present work may be accounted for by methodological factors. For instance, in some of the aforementioned developmental studies, only a single actor
was selected to record the facial expressions (Nelson etal., 2013; Nelson & Russell, 2011), raising thepossibility that the results found could be biased by
the acting performance. Compared to the Ekman andFriesen (1976) standard set of facial expressions, theexpressions of the single actor used in the study by
Nelson and Russell (2011) were indeed more readilylabeled as correct by adults as they were perceived asclearer and more intense. Asking children to catego-

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

rize facial expressions of a single actor in theirdynamic and static forms might also have impacted
their recognition performance because they might
have been more likely to choose the same label in boththe conditions by using a picture-matching strategy. In
addition, compared to the current research in whichchildren were asked to choose the correct answer
among six possibilities, previous developmental studies used free labeling as a measure of recognition(Nelson & Russell, 2011; Widen & Russell, 2015),
which raises the possibility that vocabulary performance rather than children’s true ability to understand the emotions of others were tested.
Importantly, most developmental studies that revealed an overall static advantage for facial expression
recognition (Nelson & Russell, 2011; Widen & Russell,2015) directly compared children’s performance forstatic expressions to their scores with dynamic expressions. In most cases, these direct comparisons can beproblematic because they make it difﬁcult to determine
whether increased recognition rates are caused bypsychological or physical factors (Gold et al., 2013).
For instance, Nelson and Russell (2011) and Widen
and Russell (2015) created their static images bypresenting a single frame of the highest amplitude ofthe dynamic sequences, a procedure that might havecreated ‘‘optimal’’ static images. The overall staticadvantage found in their research may be due to anincreased quantity of discriminative information provided by the stimuli rather than an enhanced psychological ability to perceive the static expressions. In
order to control for this general confounding ofphysical and psychological factors, we decided to rely
on a database of stimuli created by Gold et al. (2013),
who controlled for the amount of low-level information
carried by their stimuli over time by carefully dissociating these two factors with the use of a psychophysicalapproach. Compared to previous studies (Nelson et al.,2013; Nelson & Russell, 2011; Widen & Russell, 2015),
our results offer a more reliable view and a better
understanding of the way in which temporal properties
inﬂuence facial expression recognition from childhood
onward.

A dynamic advantage after peak efficiency
Our results revealed processing beneﬁts of dynamic
stimuli after peak efﬁciency for all the expressions withthe exception of sadness and fear. Interestingly, ourdata evidenced that these results were driven by asuboptimal performance for the recognition of static
expressions in elderly people rather than increased
abilities to recognize dynamic expressions (see Supplementary Table S1 for the example of surprise).

18

In everyday life, facial expressions are dynamic
events that unfold over time in some particular ways,
representing a richer and more valid approach to study
facial expression recognition. Previous fMRI studies
have also suggested that different neural substrates
underlie the processing of dynamic and static expressions (e.g., Johnston et al., 2013; Kessler et al., 2011;
LaBar, Crupain, Voyvodic, & McCarthy, 2003; Paulmann et al., 2009; Sato, Kochiyama, Yoshikawa,
Naito, & Matsumura, 2004; Schultz & Pilz, 2009;
Trautmann et al., 2009). Dynamic faces have beenfound to selectively elicit higher neural responses in theposterior superior temporal sulcus, in the anterior
superior temporal sulcus, and in the inferior frontal
gyrus (Bernstein, Erez, Blank, & Yovel, 2017; Fox,
Iaria, & Barton, 2009; Pitcher, Dilks, Saxe, Triantafyllou, & Kanwisher, 2011). A very recent fMRI study
that used multivoxel pattern analysis revealed thatdynamic expressions were associated with increased
activation in both face-selective and motion-selective
areas as well as higher categorization accuracies
compared to static expressions (Liang et al., 2017).
Given that dynamic faces elicit higher neural responses(Bernstein et al., 2017; Fox et al., 2009; Pitcher et al.,2011) and cause the activation of a wider network ofregions in the brain (Arsalidou, Morris, & Taylor,2011; Liang et al., 2017), their decoding may be lessvulnerable to age-related degeneration compared to thedecoding of static images.
In contrast, suboptimal performance for static
stimuli could be explained by age-related structuralchanges in brain regions responsible for the processingof static emotional expressions. For example, De
Winter et al. (2016) recently evidenced that age-induced
atrophy to the amygdala of patients with frontotemporal dementia affected emotion processing in distantface-selective areas. More speciﬁcally, their ﬁndings
evidenced a positive correlation between gray mattervolume in the left amygdala and emotion-related brain
activity in the fusiform face area, a core region in theface-processing network involved in the decoding ofstatic stimuli (Pitcher et al., 2011) and emotional
expressions (Ganel, Valyear, Goshen-Gottstein, &Goodale, 2005; Xu & Biederman, 2010).
Dynamic faces also include information that cannot
be completely rendered by static images, forcing theobservers to shift their attention to different facial
features. Multiple shifts on different facial areas arelikely to beneﬁt expression recognition given that the
facial signals critical for the recognition of emotional
expressions can be found throughout the face. In
clinical conditions or normal aging populations, whenslower or suboptimal processing takes place, dynamic
stimuli may provide additional cues, attracting andholding attention as well as enhancing motor simulations (Rymarczyk, Biele, Grabowska, & Majczynski,

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

2011; Sato, Fujimura, & Suzuki, 2008; Sato &Yoshikawa, 2007). The increased attention inherently
elicited by moving faces may compensate for theapparent age-related deﬁcits found in elderly populations on expression-recognition tasks with static
images. Dynamic face stimuli may naturally drive thefocus of attention toward the diagnostic information ina bottom-up fashion (i.e., the mouth for surprise),
whereas static face images require the observers tomove toward those features based on top-downinternal representations.
It is also important to note that the advantage weobserved for dynamic expressions cannot simply be
attributed to an overall larger amount of discriminative
information carried by the dynamic stimuli. As
reported above, the stimuli used in the current
experiment were created by Gold et al. (2013), whoused an ideal observer approach to effectively measure
the amount of information provided by the stimuli.
Gold et al.’s ﬁndings reveal that their dynamic stimuli
did not offer more discriminative information to theobservers as compared to their static images. Thus, thedynamic advantage for anger, disgust, happiness, andsurprise observed in our participants is unlikely to be
the result of physical factors. Rather, this dynamic
advantage most probably comes from an adequate
ability to use the available perceptual and diagnostic
information.
We did not ﬁnd a dynamic advantage for fear andsadness before or after peak efﬁciency. From asociobiological perspective, the expression of fear is
critical for human survival (LoBue, 2010), and it has
been shown to potentiate early visual processing ofperceptual events (e.g., Phelps, Ling, & Carrasco, 2006)
and enhance attention (e.g., Carlson & Mujica-Parodi,
2015; Pourtois, de Gelder, Bol, & Crommelinck, 2005).
Interestingly, using a single-trial repetition suppression
approach, we very recently revealed that this expression
boosts the early coding of individual faces regardless ofattentional constraints (Turano et al., 2017). Similarﬁndings were reported in a very recent developmental
study that examined detection thresholds for happy
and fearful faces presented with noise. The superior
ability for detecting fearful faces was observed alreadyin infants aged 3.5 months (Bayet et al., 2017). Our
brain may be particularly tuned to recognize thisexpression regardless of its temporal properties. This
assumption may explain the absence of a dynamic
advantage for the decoding of this expression. However, enhanced processing of fear would also predict
increased categorization performance, a prediction thatis inconsistent with our ﬁndings. Our results indeed
revealed very low recognition rates throughout the life
span. Interestingly, in a recent study, Sweeny et al.
(2013) evidenced that the detection of a fearful facedissociates from its categorization. Fearful faces were19very rapidly and accurately detected even whenpresented for only 10 ms, whereas their categorization
rates were near chance level. Our brain might,
therefore, be specially tuned to detect this expression
independently from its categorization (see also Smith &Rossit, 2018). Also, as mentioned above, among all
expressions, fear is probably the most powerful fortransmitting multisensory information. Broader contextual information may, therefore, be necessary toreliably categorize it. This assumption is in line with an
emerging literature that suggests that isolated facialsignals may not be sufﬁcient for observers to adequately perceive the emotions of fear and disgust andthat additional information regarding the context inwhich the expression occurs is critical (e.g., Barrett &Kensinger, 2010).
The absence of a dynamic advantage for theprocessing of sadness is consistent with previous
ﬁndings, which revealed that the expression of sadness
is better recognized through static pictures (Bould etal., 2008; Recio et al., 2013; Widen & Russell, 2015) or
when evolving slowly (Kamachi et al., 2001; Recio etal., 2013). Ekman (2003) suggested that, among all theexpressions, sadness is the one lasting the longest overtime, a property that may explain why slowness or
stillness may increase recognition performance. Our
results further conﬁrm that the idiosyncratic propertiesof this expression are inherently slow.
We should acknowledge that we did not assess
whether elderly people’s cognitive abilities inﬂuenced
recognition performance. Fluid intelligence (e.g.,
Horning et al., 2012; Sullivan & Ruffman, 2004a),
processing speed (e.g., Orgeta & Phillips, 2007), verbal
memory (e.g., MacPherson et al., 2002), or discrimination of visual information (Mill, Allik, Realo, &Valk, 2009) are all cognitive faculties that are critical
for the recognition of human facial expressions andhave been found to decrease with increasing age (e.g.,
Mill et al., 2009; Salthouse, 2004). Because our stimuli
were only presented for 1 s, reduced processing speed inelderly people may have inﬂuenced their recognition
performance for static face images as the number offacial features extracted from the faces in this limited
presentation time might have been lower than that inyounger adults. Interestingly, a recent cross-sectional
study that examined the inﬂuence of different cognitive
abilities on facial-expression recognition observed thatthese faculties contributed to the performance but did
not fully account for the impairments observed in older
adults (Horning et al., 2012). Additionally, in a later
study, Zhao et al. (2016) observed that the slower
processing speed in elderly people was not responsible
for facial expression–recognition deﬁcits (but see
Suzuki & Akiyama, 2013; West et al., 2012).
It is also worth noting that differences in recognition
abilities could stem from differences in the cohorts that

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

were tested, such as educational shifts, cultural norms,
or social differences. To the best of our knowledge, noprior research has ever examined the extent to which
the recognition of facial expression is inﬂuenced bythese cohort effects (Ruffman et al., 2008).

The shuffled condition
We introduced this condition to fully replicate thestudy conducted by Gold et al. (2013) without having
clear predictions for this experimental condition. Thiscondition was originally designed to assess whether
human observers were sensitive to the temporaldevelopment of an expression over time (i.e., order offrames). Our results revealed similar developmental
trajectories for the six basic expressions in the shufﬂed
condition (i.e., increase, peak efﬁciency, and decrease)although recognition rates were generally lower thanthose observed in the other conditions. More specifically, our ﬁndings revealed a recognition advantage forthe dynamic expressions of anger, disgust, happiness,and surprise over the shufﬂed ones and a recognition
advantage for the static expressions of happiness,
disgust, and surprise over the shufﬂed ones. In contrast,
we observed better recognition performance for the
expression of fear in the shufﬂed compared to the staticor dynamic conditions. As reported by our participants, this advantage for fear could be accounted for by
the properties of the stimuli themselves. The shufﬂed
expressions were generated by temporally randomizing
the frames of the dynamic movies. This procedure leads
to the impression that the actors performing theemotional expressions are shaking giving the feelingthat they are afraid.
Interestingly, differences in recognition performanceacross conditions are inconsistent with the results
reported by Gold et al. (2013), who observed similar
performance in all three conditions. In that prior study,
however, the authors did not consider the recognition
rates of the individual expressions effectively, collapsingthem across the six expressions in each condition. Ourﬁndings offer, therefore, new evidence that the temporalprogression of information (i.e., the order of the frames)
provided by genuine natural expressions is more
important for the recognition of some expressions (e.g.,anger, disgust, happiness, surprise) than others (e.g., fear,sadness). Given the very nonecological nature of thestimuli, we do not further discuss these results as theircontribution is limited from a theoretical point of view.

Methodological considerations
In the current study, we used a hierarchical Bayesianmodel with weakly informative priors. The ﬂexibility

20

and power of the Bayesian approach in dealing with
time-series data and building nonlinear models was also
recently demonstrated in emotion research (e.g., see
Krone, Albers, Kuppens, & Timmerman, 2017, for an
application in personal emotion dynamics) thanks to
the rapid development in probabilistic language programing. It is worth noting that there are alternative
candidate models that could capture the nonlinear
relationship between age and some psychological or
behavioral measurements, including latent growth
curve models (for an introductory text, see Duncan,
Duncan, & Strycker, 2013); generalized additive mixed
models, including spline regression; and quadratic
linear mixed-effect models (S. N. Wood, 2006). Some ofthese models have been previously applied to investigate similar questions, such as the estimation of thepeak efﬁciency of diverse recognition abilities (i.e.,change-point estimation, e.g., Cohen, 2012; Cudeck &Klebe, 2002). The Bayesian modeling framework weused here provides a coherent mathematical language
to describe our model and assumption while giving the
ﬂexibility to potentially extend part of the component
to build more complex models. Moreover, it allowed us
to properly quantify the uncertainty and regulate theestimation across different conditions using hyperpriors.
To identify inverse U-shape patterns, such as thoseobserved in our study, previous modeling methods
occasionally involved the testing of a quadratic
relationship (e.g., a signiﬁcant regression coefﬁcient ofage2) even if such practice is not always valid(Simonsohn, 2017). Instead, Simonsohn (2017) suggested ﬁtting two separate linear models and comparing the coefﬁcients of the two slopes as a more
valid alternative. Although our model is conceptuallysimilar to Simonsohn’s model, there are two majordifferences. First, the inference proposed by Simonsohn involved multiple model-ﬁtting steps by initially
identifying the break point (i.e., peak efﬁciency in ourcase) and then estimating the coefﬁcients of the twolinear functions. In contrast, with a full model thatjointly estimates the break point and the linear
functions, we could better estimate the parameters andquantify the associated uncertainty. Second, theintercepts of our step-linear function are linked andrepresented as one value (i.e., the recognition ability at
peak efﬁciency), whereas in Simonsohn’s model thetwo linear functions are not linked. The linked linear
function is more appropriate in our case as it isunlikely to have a sudden increase or decrease inrecognition ability in a short span during naturaldevelopment. Nonetheless, an implicit yet important
assumption present in both models is that the peak
efﬁciency is found somewhere in the middle of the life
span (or, more precisely, not at either of the two
extrema). Indeed, if the peak efﬁciency is at the lower

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

or upper limit (e.g., too young or too old), theparameter estimation may not be accurate.
Our model estimation performed well except for the
expression of happiness because of the ceiling effect weobserved for this expression. The divergence in the
trace and the multimodal in the posterior distributionof the peak efﬁciency both indicate that the current
step-linear model is not the best suited to representchanges in recognition abilities across the life span forthis expression. Currently, all the expressions are
estimated independently. Although modeling this way
is easier to interpret, we ignored the random effect inthe subjects across the expressions. Future studies arenecessary to take into account the random effect fromeach subject (intercept and slope). This could be doneby directly modeling the full confusion matrix fromeach subject (instead of only looking at the diagonal in
the current study), presumably with some matrix
decomposition trick or a dirichlet-categorical model.
Finally, our model allowed us to estimate the overall
advantage of one condition over another before andafter the peak efﬁciency. However, because we decided
to consider the age as a continuum and not rely on
speciﬁc age groups on the basis of arbitrary boundaries,
our model did not allow us to ﬁnely estimate at which
precise age the dynamic advantage emerges or disappears.

Conclusions
Current knowledge about facial expression recognition primarily arises from studies that use static
images. In our daily life, however, natural faces are
dynamic; they evolve over time in some particular
ways to convey crucial information for adapted social
behaviors. Prior studies investigating the importanceof dynamic cues for the processing of facial expressions have yielded equivocal results with some studies
suggesting that dynamic expressions are more readily
recognizable than static images and others suggestingthat they are not. In order to clarify these results andto determine if age is a critical factor to account forsuch discrepancies, we conducted a large crosssectional study to investigate the recognition of facial
expressions by participants aged 5 to 96. More than
400 observers were instructed to categorize static,
dynamic, and shufﬂed expressions according to the six
basic expressions. Our ﬁndings revealed that, regardless of the age of the observer or temporal condition,
happiness was the best recognized facial expression,
whereas fear was the most difﬁcult to effectively
categorize as this expression was commonly confusedwith surprise. Bayesian modeling allowed us toquantify the steepness of increase and decrease in21

performance for each individual expression in eachcondition. Our results also reveal a data-driven
estimation of the peak efﬁciency for every expression
and, ﬁnally, provide new evidence for a dynamic
advantage for facial expression recognition, stronger
for some expressions than others and more importantaround speciﬁc points in the life course. Notably,
performance for static images was less effective in theelderly population. Altogether, our ﬁndings highlightthe importance of using ecologically valid faces in
exploring the recognition of facial expressions andinvite caution while drawing conclusions from studies
that use only static images to this aim.
Keywords: facial expressions of emotion, life span,static, dynamic, aging

Acknowledgments
We would like to thank Prof. Jason Gold and hiscolleagues from Indiana University, Bloomington, forproviding us with the stimuli used in this study. Veryspecial thanks go to all our participants, especially thechildren and adolescents who participated in our study
from the following schools: Ecole Enfantine et Primairede Marly Grand Pré and Cycle d’Orientation de
Domdidier in the area of Fribourg, Switzerland. We
would also like to thank all the teachers for theirpatience and help, especially the head teachers, Claude
Meuwly from Marly and Chantal Vienny Guerry from
Domdidier, as well as technician Zvonko Traykoski for
meticulously organizing our visits and ensuring thateverything worked well. We would also like to express
our gratitude to the retirement homes Foyer deBouleyres and Maison Bourgeoisiale in Bulle andFoyer St-Martin in Cottens, especially to ChristianRime, Véronique Castella, Isabelle Montagnon, andPhilippe Bourquin for letting us test their residents. We
would also like to thank Maria Teresa Turano andProf. Maria Pia Viggiano from Florence, Léa Poitrine,
Claudia Wyler, Qendresa Shkodra, Vanessa Ferrari,Pauline Rotztetter, Pauline Schaller, Linda Pigozzi,
Lauriane Beffa, Hugo Najberg, Christel Aichele, andMartina Studer for their precious help with testing.
This study was supported by grant F14/06 from theRectors’ Conference of Swiss Universities (CRUS)
awarded to ARR and by a grant from the Swiss
National Science Foundation (n8100014_156490/1)
awarded to RC. The publication fees were covered by
the Comité des Alumni et Amis de l’Université deFribourg.
Commercial relationships: none.
Corresponding authors: Anne-Raphaëlle Richoz;
Roberto Caldara.

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

Email: anne-raphaelle.richoz@unifr.ch; roberto.
caldara@unifr.ch.
Address: Department of Psychology, University of
Fribourg, Fribourg, Switzerland.

References
Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel,D., Schyns, P., & Damasio, A. R. (2005, January
6). A mechanism for impaired fear recognition after
amygdala damage. Nature, 433(7021), 68–72.
Adolphs, R., Tranel, D., & Damasio, A. R. (2003).
Dissociable neural systems for recognizing emotions. Brain and Cognition, 52(1), 61–69.
Alves, N. T. (2013). Recognition of static and dynamic
facial expressions: A study review. Estudos dePsicologia (Natal), 18(1), 125–130.
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005).
Deciphering the enigmatic face: The importance offacial dynamics in interpreting subtle facial expressions. Psychological Science, 16(5), 403–410.
Andersen, S. L. (2003). Trajectories of brain development: Point of vulnerability or window of opportunity? Neuroscience & Biobehavioral Reviews,
27(1), 3–18.
Arsalidou, M., Morris, D., & Taylor, M. J. (2011).
Converging evidence for the advantage of dynamic
facial expressions. Brain Topography, 24(2), 149–
163.
Aviezer, H., Hassin, R. R., Ryan, J., Grady, C.,
Susskind, J., Anderson, A., . . . Bentin, S. (2008).
Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological
Science, 19(7), 724–732.
Back, E., Ropar, D., & Mitchell, P. (2007). Do the eyes
have it? Inferring mental states from animated faces
in autism. Child Development, 78(2), 397–411.
Barrett, L. F., & Kensinger, E. A. (2010). Context isroutinely encoded during emotion perception.
Psychological Science, 21(4), 595–599.
Bayet, L., Quinn, P. C., Laboissière, R., Caldara, R.,
Lee, K., & Pascalis, O. (2017). Fearful but not
happy expressions boost face detection in humaninfants. Proceedings of the Royal Society of London
B, 284(1862), 1–9.
Bernstein, M., Erez, Y., Blank, I., & Yovel, G. (2017).
The processing of dynamic faces in the humanbrain: Support for an integrated neural frameworkof face processing. BioRxiv :140939.
Blakemore, S.-J. (2012). Imaging brain development:
The adolescent brain. NeuroImage, 61(2), 397–406.

22

Blakemore, S.-J., & Choudhury, S. (2006). Development of the adolescent brain: Implications for
executive function and social cognition. Journal ofChild Psychology and Psychiatry, 47(3–4), 296–312.
Bould, E., & Morris, N. (2008). Role of motion signals
in recognizing subtle facial expressions of emotion.
British Journal of Psychology, 99(2), 167–189.
Bould, E., Morris, N., & Wink, B. (2008). Recognising
subtle emotional expressions: The role of facial
movements. Cognition and Emotion, 22(8), 1569–
1587.
Brainard, D. H., (1997). The psychophysics toolbox.
Spatial Vision, 10, 433–436.
Caldara, R. (2017). Culture reveals a flexible system forface processing. Current Directions in Psychological
Science, 26(3), 249–255.
Calder, A. J., Keane, J., Manly, T., Sprengelmeyer, R.,
Scott, S., Nimmo-Smith, I., & Young, A. W.
(2003). Facial expression recognition across theadult life span. Neuropsychologia, 41(2), 195–202.
Carlson, J. M., & Mujica-Parodi, L. R. (2015).
Facilitated attentional orienting and delayed disengagement to conscious and nonconscious fearfulfaces. Journal of Nonverbal Behavior, 39(1), 69–77.
Casey, B., Tottenham, N., Liston, C., & Durston, S.
(2005). Imaging the developing brain: What have
we learned about cognitive development? Trends in
Cognitive Sciences, 9(3), 104–110.
Chen, J. J., Rosas, H. D., & Salat, D. H. (2011). Ageassociated reductions in cerebral blood flow are
independent from regional atrophy. NeuroImage,55(2), 468–478.
Christie, F., & Bruce, V. (1998). The role of dynamic
information in the recognition of unfamiliar faces.
Memory & Cognition, 26(4), 780–790.
Cohen, P. (2012). Applied data analytic techniques forturning points research. New York: Routledge.
Cudeck, R., & Klebe, K. J. (2002). Multiphase mixedeffects models for repeated measures data. Psychological Methods, 7(1), 41–63.
Cunningham, D. W., & Wallraven, C. (2009). Dynamic
information for the recognition of conversational
expressions. Journal of Vision, 9(13):7, 1–17,
https://doi.org/10.1167/9.13.7. [PubMed] [Article]
Delis, I., Chen, C., Jack, R. E., Garrod, O. G., Panzeri,
S., & Schyns, P. G. (2016). Space-by-time manifold
representation of dynamic facial expressions foremotion categorization. Journal of Vision, 16(8):14,
1–20, https://doi.org/10.1167/16.8.14. [PubMed][Article]
De Sonneville, L., Verschoor, C., Njiokiktjien, C., Op
het Veld, V., Toorenaar, N., & Vranken, M. (2002).

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

Facial identity and facial emotions: Speed, accuracy, and processing strategies in children and adults.
Journal of Clinical and Experimental Neuropsychology, 24(2), 200–213.
De Winter, F.-L., Van den Stock, J., de Gelder, B.,
Peeters, R., Jastorff, J., Sunaert, S., & hellip;
Vandenbulcke, M. (2016). Amygdala atrophy
affects emotion-related activity in face-responsive
regions in frontotemporal degeneration. Cortex, 82,
179–191.
Duncan, T. E., Duncan, S., & Strycker, L. A. (2013).
An introduction to latent variable growth curvemodeling: Concepts, issues, and application. New
York: Routledge Academic.
Durand, K., Gallay, M., Seigneuric, A., Robichon, F.,& Baudouin, J.-Y. (2007). The development offacial emotion recognition: The role of configuralinformation. Journal of Experimental Child Psychology, 97(1), 14–27.
Durston, S., Pol, H. E. H., Casey, B., Giedd, J. N.,
Buitelaar, J. K., & Van Engeland, H. (2001).
Anatomical MRI of the developing human brain:
What have we learned? Journal of the American
Academy of Child & Adolescent Psychiatry, 40(9),1012–1020.
Ehrlich, S. M., Schiano, D. J., & Sheridan, K. (2000).
Communicating facial affect: It’s not the realism,
it’s the motion. In Proceedings of ACM CHI 2000
Conference on Human Factors in Computing Systems (pp. 252–253), New York, NY: ACM.
Ekman, P. (2003). Emotions revealed. Understanding
faces and feelings. London: The Orion PublishingGroup Ltd.
Ekman, P., & Friesen, W. V. (1976). Measuring facial
movement. Environmental Psychology and Nonverbal Behavior, 1(1), 56–75.
Fiorentini, C., & Viviani, P. (2011). Is there a dynamic
advantage for facial expressions? Journal of Vision,
11(3):17, 1–15, https://doi.org/10.1167/11.3.17.
[PubMed] [Article]
Fiset, D., Blais, C., Royer, J., Richoz, A.-R., Dugas,G., & Caldara, R. (2017). Mapping the impairment
in decoding static facial expressions of emotion inprosopagnosia. Social Cognitive and Affective
Neuroscience, 12(8), 1334–1341.
Folstein, M. F. (1975). A practical method for grading
the cognitive state of patients for the clinician.
Journal of Psychiatric Research, 12, 189–198.
Fox, C. J., Iaria, G., & Barton, J. J. (2009). Defining
the face processing network: Optimization of thefunctional localizer in fMRI. Human Brain Mapping, 30(5), 1637–1651.

23

Ganel, T., Valyear, K. F., Goshen-Gottstein, Y., &Goodale, M. A. (2005). The involvement of the‘‘fusiform face area’’ in processing facial expression.
Neuropsychologia, 43(11), 1645–1654.
Gao, X., & Maurer, D. (2009). Influence of intensity on
children’s sensitivity to happy, sad, and fearful
facial expressions. Journal of Experimental Child
Psychology, 102(4), 503–521.
Gao, X., & Maurer, D. (2010). A happy story:
Developmental changes in children’s sensitivity tofacial expressions of varying intensities. Journal of
Experimental Child Psychology, 107(2), 67–86.
Gao, X., & Maurer, D. (2011). A comparison of spatial
frequency tuning for the recognition of facial
identity and facial expressions in adults andchildren. Vision Research, 51(5), 508–519.
Geangu, E., Ichikawa, H., Lao, J., Kanazawa, S.,
Yamaguchi, M. K., Caldara, R., & Turati, C.
(2016). Culture shapes 7-month-olds perceptual
strategies in discriminating facial expressions ofemotion. Current Biology, 26(14), R663–R664.
Gelman, A., & Rubin, D. B. (1992). Inference fromiterative simulation using multiple sequences. Statistical Science, 7(4), 457–472.
Gepner, B., Deruelle, C., & Grynfeltt, S. (2001).
Motion and emotion: A novel approach to thestudy of face processing by young autistic children.
Journal of Autism and Developmental Disorders,
31(1), 37–45.
Giard, M. H., & Peronnet, F. (1999). Auditory-visual
integration during multimodal object recognition inhumans: A behavioral and electrophysiologicalstudy. Journal of Cognitive Neuroscience, 11(5),473–490.
Gold, J. M., Barker, J. D., Barr, S., Bittner, J. L.,
Bromfield, W. D., Chu, N., . . . Srinath, A. (2013).
The efficiency of dynamic and static facial expression recognition. Journal of Vision, 13(5):23, 1–12,
https://doi.org/10.1167/13.5.23. [PubMed] [Article]
Grainger, S. A., Henry, J. D., Phillips, L. H., Vanman,
E. J., & Allen, R. (2015). Age deficits in facial affectrecognition: The influence of dynamic cues. Journals of Gerontology Series B: Psychological Sciences
and Social Sciences, 72(4), 622–632.
Gross, A. L., & Ballif, B. (1991). Children’s understanding of emotion from facial expressions andsituations: A review. Developmental Review, 11(4),368–398.
Harwood, N. K., Hall, L. J., & Shinkfield, A. J. (1999).
Recognition of facial emotional expressions frommoving and static displays by individuals withmental retardation. American Journal on Mental
Retardation, 104(3), 270–278.

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

Herba, C., Landau, S., Russell, T., Ecker, C., &Phillips, M. L. (2006). The development of emotion–processing in children: Effects of age, emotion,and intensity. Journal of Child Psychology andPsychiatry, 47(11), 1098–1106.
Herba, C., & Phillips, M. (2004). Annotation: Development of facial expression recognition fromchildhood to adolescence: Behavioural and neurological perspectives. Journal of Child Psychology
and Psychiatry, 45(7), 1185–1198.
Hof, P. R., & Morrison, J. H. (2004). The aging brain:
Morphomolecular senescence of cortical circuits.
Trends in Neurosciences, 27(10), 607–613.
Horning, S. M., Cornwell, R. E., & Davis, H. P. (2012).
The recognition of facial expressions: An investigation of the influence of age and cognition. Aging,
Neuropsychology, and Cognition, 19(6), 657–676.
Humphreys, G. W., Donnelly, N., & Riddoch, M. J.
(1993). Expression is computed separately fromfacial identity, and it is computed separately formoving and static faces: Neuropsychological evidence. Neuropsychologia, 31(2), 173–181.
Jack, C. R., Petersen, R. C., Xu, Y. C., Waring, S. C.,
O’Brien, P. C., Tangalos, E. G., . . . Kokmen, E.
(1997). Medial temporal atrophy on MRI in
normal aging and very mild Alzheimer’s disease.
Neurology, 49(3), 786–794.
Jack, R. E., Garrod, O. G., & Schyns, P. G. (2014).
Dynamic facial expressions of emotion transmit anevolving hierarchy of signals over time. Current
Biology, 24(2), 187–192.
Jiang, Z., Li, W., Recio, G., Liu, Y., Luo, W., Zhang,D., & Sun, D. (2014). Time pressure inhibits
dynamic advantage in the classification of facial
expressions of emotion. PLoS One, 9(6), 1–7.
Johnston, P., Mayes, A., Hughes, M., & Young, A. W.
(2013). Brain networks subserving the evaluation of
static and dynamic facial expressions. Cortex,
49(9), 2462–2472.
Kamachi, M., Bruce, V., Mukaida, S., Gyoba, J.,
Yoshikawa, S., & Akamatsu, S. (2001). Dynamic
properties influence the perception of facial expressions. Perception, 30(7), 875–887.
Kätsyri, J. (2006). Human recognition of basic emotions
from posed and animated dynamic facial expressions.
Unpublished doctoral dissertation, Helsinki University of Technology. Retrieved from https://
aaltodoc.aalto.fi/bitstream/handle/123456789/
2807/isbn951228538X.
pdf?sequence¼1&isAllowed¼yKätsyri, J., Saalasti, S., Tiippana, K., von Wendt, L., &Sams, M. (2008). Impaired recognition of facial

24

emotions from low-spatial frequencies in Asperger
syndrome. Neuropsychologia, 46(7), 1888–1897.
Kätsyri, J., & Sams, M. (2008). The effect of dynamics
on identifying basic emotions from synthetic andnatural faces. International Journal of HumanComputer Studies, 66(4), 233–242.
Kessler, H., Doyen-Waldecker, C., Hofer, C., Hoffmann, H., Traue, H. C., & Abler, B. (2011). Neuralcorrelates of the perception of dynamic versus static
facial expressions of emotion. Psychosocial Medicine, 8, 1–8.
Kleiner, M., Brainard, D., Pelli, D., Ingling, A.,
Murray, R., & Broussard, C. (2007). What’s new in
Psychtoolbox-3. Perception, 36(14), 1–16.
Knappmeyer, B., Thornton, I. M., & Bülthoff, H. H.
(2003). The use of facial motion and facial form
during the processing of identity. Vision Research,
43(18), 1921–1936.
Krendl, A. C., & Ambady, N. (2010). Older adults’
decoding of emotions: Role of dynamic versus
static cues and age-related cognitive decline.
Psychology and Aging, 25(4), 788–793.
Krone, T., Albers, C. J., Kuppens, P., & Timmerman,
M. E. (2018). A multivariate statistical model for
emotion dynamics. Emotion, 18(5), 739–754.
Krumhuber, E. G., Kappas, A., & Manstead, A. S.
(2013). Effects of dynamic aspects of facial
expressions: A review. Emotion Review, 5(1), 41–46.
Kruschke, J. (2014). Doing Bayesian data analysis: A
tutorial with R, JAGS, and Stan (2nd ed.). New
York: Academic Press.
LaBar, K. S., Crupain, M. J., Voyvodic, J. T., &McCarthy, G. (2003). Dynamic perception of facial
affect and identity in the human brain. Cerebral
Cortex, 13(10), 1023–1033.
Leitzke, B. T., & Pollak, S. D. (2016). Developmental
changes in the primacy of facial cues for emotion
recognition. Developmental Psychology, 52(4), 572–
581.
Liang, Y., Liu, B., Xu, J., Zhang, G., Li, X., Wang, P.,& Wang, B. (2017). Decoding facial expressions
based on face–selective and motion–sensitive areas.
Human Brain Mapping, 38(6), 3113–3125.
LoBue, V. (2010). What’s so scary about needles andknives? Examining the role of experience in threat
detection. Cognition and Emotion, 24(1), 180–187.
MacPherson, S. E., Phillips, L. H., & Della Sala, S.
(2002). Age, executive function and social decision
making: A dorsolateral prefrontal theory of cognitive aging. Psychology and Aging, 17(4), 598–609.
Malatesta, C. Z., Izard, C. E., Culver, C., & Nicolich,M. (1987). Emotion communication skills in young,

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

middle-aged, and older women. Psychology andAging, 2(2), 193–203.
Mather, M., Canli, T., English, T., Whitfield, S., Wais,
P., Ochsner, K., . . . Carstensen, L. L. (2004).
Amygdala responses to emotionally valenced stimuli in older and younger adults. Psychological
Science, 15(4), 259–263.
McDowell, C. L., Harrison, D. W., & Demaree, H. A.
(1994). Is right hemisphere decline in the perceptionof emotion a function of aging? International
Journal of Neuroscience, 79(1–2), 1–11.
Mill, A., Allik, J., Realo, A., & Valk, R. (2009). Agerelated differences in emotion recognition ability: A
cross-sectional study. Emotion, 9(5), 619–630.
Mitchell, A. J. (2009). A meta-analysis of the accuracyof the mini-mental state examination in thedetection of dementia and mild cognitive impairment. Journal of Psychiatric Research, 43(4), 411–
431.
Montirosso, R., Peverelli, M., Frigerio, E., Crespi, M.,& Borgatti, R. (2010). The development of dynamic
facial expression recognition at different intensitiesin 4- to 18-year-olds. Social Development, 19(1), 71–
92.
Moreno, C., Borod, J. C., Welkowitz, J., & Alpert, M.
(1993). The perception of facial emotion across theadult life span. Developmental Neuropsychology,
9(3–4), 305–314.
Navarro, C., & Gonzalo, L. M. (1991). Cambios en elcomplejo amigdalino humano debidos a la edad
[Changes in the human amygdaloid complex due toage]. Revista de Medicina de la Universidad deNavarra, 35, 7–12.
Nelson, N. L., Hudspeth, K., & Russell, J. A. (2013). A
story superiority effect for disgust, fear, embarrassment, and pride. British Journal of Developmental Psychology, 31(3), 334–348.
Nelson, N. L., & Russell, J. A. (2011). Putting motion
in emotion: Do dynamic presentations increasepreschooler’s recognition of emotion? Cognitive
Development, 26(3), 248–259.
Orgeta, V., & Phillips, L. H. (2007). Effects of age andemotional intensity on the recognition of facial
emotion. Experimental Aging Research, 34(1), 63–79.
Paulmann, S., Jessen, S., & Kotz, S. A. (2009).
Investigating the multimodal nature of humancommunication: An ERP study. Journal of Psychophysiology, 23(2), 63–76.
Phelps, E. A., Ling, S., & Carrasco, M. (2006).
Emotion facilitates perception and potentiates the25

perceptual benefits of attention. Psychological
Science, 17(4), 292–299.
Pitcher, D., Dilks, D. D., Saxe, R. R., Triantafyllou,
C., & Kanwisher, N. (2011). Differential selectivity
for dynamic versus static information in faceselective cortical regions. NeuroImage, 56(4), 2356–
2363.
Pourtois, G., de Gelder, B., Bol, A., & Crommelinck,
M. (2005). Perception of facial expressions andvoices and of their combination in the humanbrain. Cortex, 41(1), 49–59.
Raz, N. (2000). Aging of the brain and its impact on
cognitive performance: Integration of structuraland functional findings. In F. I. M. Craik & T. A.
Salthouse (Eds.), The handbook of aging andcognition (pp. 1–90). Mahwah, NJ: Lawrence
Erlbaum Associates.
Recio, G., Schacht, A., & Sommer, W. (2013).
Classification of dynamic facial expressions ofemotion presented briefly. Cognition & Emotion,
27(8), 1486–1494.
Richoz, A.-R., Jack, R. E., Garrod, O. G., Schyns, P.
G., & Caldara, R. (2015). Reconstructing dynamic
mental models of facial expressions in prosopagnosia reveals distinct representations for identityand expression. Cortex, 65, 50–64.
Rodger, H., Lao, J., & Caldara, R. (2018). Quantifying
facial expression signal and intensity use during
development. Journal of Experimental Child Psychology, 174, 41–59.
Rodger, H., Vizioli, L., Ouyang, X., & Caldara, R.
(2015). Mapping the development of facial expression recognition. Developmental Science, 18(6),926–939.
Rossini, P. M., Rossi, S., Babiloni, C., & Polich, J.
(2007). Clinical neurophysiology of aging brain:
From normal aging to neurodegeneration. Progress
in Neurobiology, 83(6), 375–400.
Ruffman, T., Henry, J. D., Livingstone, V., & Phillips,
L. H. (2008). A meta-analytic review of emotion
recognition and aging: Implications for neuropsychological models of aging. Neuroscience & Biobehavioral Reviews, 32(4), 863–881.
Rymarczyk, K., Biele, C., Grabowska, A., & Majczynski, H. (2011). EMG activity in response tostatic and dynamic facial expressions. International
Journal of Psychophysiology, 79(2), 330–333.
Salthouse, T. A. (2004). What and when of cognitiveaging. Current Directions in Psychological Science,13(4), 140–144.
Sato, W., Fujimura, T., & Suzuki, N. (2008). Enhanced
facial EMG activity in response to dynamic facial

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

expressions. International Journal of Psychophysiology, 70(1), 70–74.
Sato, W., Kochiyama, T., Yoshikawa, S., Naito, E., &Matsumura, M. (2004). Enhanced neural activity inresponse to dynamic facial expressions of emotion:
An fMRI study. Brain Research Cognitive Brain
Research, 20(1), 81–91.
Sato, W., & Yoshikawa, S. (2007). Spontaneous facial
mimicry in response to dynamic facial expressions.
Cognition, 104(1), 1–18.
Schultz, J., & Pilz, K. S. (2009). Natural facial motion
enhances cortical responses to faces. Experimental
Brain Research, 194(3), 465–475.
Simonsohn, U. (2017). Two-Lines: A valid Alternative
to the Invalid Testing of U-Shaped Relationships
with Quadratic Regressions. SSRN Electronic
Journal, 1–29. Retrieved from: https://ssrn.com/
abstract¼3021690.
Smith, F. W., & Rossit, S. (2018). Identifying anddetecting facial expressions of emotion in peripheral vision. PLoS One, 13(5), e0197160.
Sullivan, S., & Ruffman, T. (2004a). Emotion recognition deficits in the elderly. International Journal of
Neuroscience, 114(3), 403–432.
Sullivan, S., & Ruffman, T. (2004b). Social understanding: How does it fare with advancing years?
British Journal of Psychology, 95(1), 1–18.
Suzuki, A., & Akiyama, H. (2013). Cognitive aging
explains age-related differences in face-based recognition of basic emotions except for anger anddisgust. Aging, Neuropsychology, and Cognition,20(3), 253–270.
Sweeny, T. D., Suzuki, S., Grabowecky, M., & Paller,
K. A. (2013). Detecting and categorizing fleeting
emotions in faces. Emotion, 13(1), 76–91.
Sze, J. A., Goodkind, M. S., Gyurak, A., & Levenson,
R. W. (2012). Aging and emotion recognition: Not
just a losing matter. Psychology and Aging, 27(4),940–950.
Tardif, C., Lainé, F., Rodriguez, M., & Gepner, B.
(2007). Slowing down presentation of facial movements and vocal sounds enhances facial expression
recognition and induces facial–vocal imitation inchildren with autism. Journal of Autism andDevelopmental Disorders, 37(8), 1469–1484.
Thomas, L. A., De Bellis, M. D., Graham, R., &LaBar, K. S. (2007). Development of emotional
facial recognition in late childhood and adolescence. Developmental Science, 10(5), 547–558.
Trautmann, S. A., Fehr, T., & Herrmann, M. (2009).
Emotions in motion: Dynamic compared to static
facial expressions of disgust and happiness reveal

26

more widespread emotion-specific activations.
Brain Research, 1284, 100–115.
Turano, M. T., Lao, J., Richoz, A. R., De Lissa, P.,
Desgosciu, S. B. A., Viggiano, M. P., & Caldara, R.
(2017). Fear boosts the early neural coding of faces.
Social, Cognitive, and Affective Neuroscience,
12(12), 1959–1971.
Uono, S., Sato, W., & Toichi, M. (2010). Brief report:
Representational momentum for dynamic facial
expressions in pervasive developmental disorder.
Journal of Autism and Developmental Disorders,
40(3), 371–377.
Vicari, S., Reilly, J. S., Pasqualetti, P., Vizzotto, A., &Caltagirone, C. (2000). Recognition of facial
expressions of emotions in school–age children: Theintersection of perceptual and semantic categories.
Acta Paediatrica, 89(7), 836–845.
Wallraven, C., Breidt, M., Cunningham, D. W., &Bülthoff, H. H. (2008). Evaluating the perceptual
realism of animated facial expressions. ACM
Transactions on Applied Perception, 4(4), 1–20.
Wehrle, T., Kaiser, S., Schmidt, S., & Scherer, K. R.
(2000). Studying the dynamics of emotional expression using synthesized facial muscle movements. Journal of Personality and Social
Psychology, 78(1), 105–119.
West, J. T., Horning, S. M., Klebe, K. J., Foster, S. M.,
Cornwell, R. E., Perrett, D., Burt, D. M., & Davis,
H. P. (2012). Age effects on emotion recognition in
facial displays: From 20 to 89 years of age.
Experimental Aging Research, 38(2), 146–168.
Widen, S. C. (2013). Children’s interpretation of facial
expressions: The long path from valence-based to
specific discrete categories. Emotion Review, 5(1),72–77.
Widen, S. C., & Russell, J. A. (2015). Do dynamic
facial expressions convey emotions to children
better than do static ones? Journal of Cognition andDevelopment, 16(5), 802–811.
Willenbockel, V., Sadr, J., Fiset, D., Horne, G. O.,
Gosselin, F., & Tanaka, J. W. (2010). Controlling
low-level image properties: The SHINE toolbox.
Behavior Research Methods, 42(3), 671–684.
Williams, L. M., Mathersul, D., Palmer, D. M., Gur,
R. C., Gur, R. E., & Gordon, E. (2009). Explicit
identification and implicit recognition of facial
emotions: I. Age effects in males and females across
10 decades. Journal of Clinical and Experimental
Neuropsychology, 31(3), 257–277.
Wood, A., Lupyan, G., Sherrin, S., & Niedenthal, P.
(2016). Altering sensorimotor feedback disrupts
visual discrimination of facial expressions. Psychonomic Bulletin & Review, 23(4), 1150–1156.

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018

Journal of Vision (2018) 18(9):5, 1–27

Richoz, Lao, Pascalis, & Caldara

Wood, A., Rychlowska, M., Korb, S., & Niedenthal,P. (2016). Fashioning the face: Sensorimotor
simulation contributes to facial expression recognition. Trends in Cognitive Sciences, 20(3), 227–
240.
Wood, S. N. (2006). Generalized additive models: An
introduction with R. Boca Raton, FL: CRC
Press.
Xu, X., & Biederman, I. (2010). Loci of the releasefrom fMRI adaptation for changes in facial
expression, identity, and viewpoint. Journal of27

Vision, 10(14):36, 1–13, https://doi.org/10.1167/10.
14.36. [PubMed] [Article]
Yang, T. T., Menon, V., Eliez, S., Blasey, C., White,
C. D., Reid, A. J., . . . Reiss, A. L. (2002).
Amygdalar activation associated with positive andnegative facial expressions. Neuroreport, 13(14),1737–1741.
Zhao, M.-F., Zimmer, H. D., Shen, X., Chen, W., &Fu, X. (2016). Exploring the cognitive processes
causing the age-related categorization deficit in therecognition of facial expressions. Experimental
Aging Research, 42(4), 348–364.

Downloaded From: https://jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/937491/ on 09/08/2018