Journal of Vision (2024) 24(1):7, 1–22

1

Recognizing facial expressions of emotion amid noise: A
dynamic advantage
Anne-Raphaëlle Richoz

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

Lisa Stacchi

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

Pauline Schaller

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

Junpeng Lao

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

Michael Papinutto

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

Valentina Ticcinelli

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

Roberto Caldara

Eye and Brain Mapping Laboratory (iBMLab),
Department of Psychology, University of Fribourg,
Fribourg, Switzerland

Humans communicate internal states through complex
facial movements shaped by biological and evolutionary
constraints. Although real-life social interactions are
ﬂooded with dynamic signals, current knowledge on
facial expression recognition mainly arises from studies
using static face images. This experimental bias mightstem from previous studies consistently reporting thatyoung adults minimally beneﬁt from the richer dynamic
over static information, whereas children, the elderly,
and clinical populations very strongly do (Richoz, Jack,
Garrod, Schyns, & Caldara, 2015, Richoz, Jack, Garrod,
Schyns, & Caldara, 2018b). These observations point to anear-optimal facial expression decoding system in young
adults, almost insensitive to the advantage of dynamic
over static cues. Surprisingly, no study has yet tested theidea that such evidence might be rooted in a ceilingeﬀect. To this aim, we asked 70 healthy young adults toperform static and dynamic facial expression recognitionof the six basic expressions while parametrically andrandomly varying the low-level normalized phase andcontrast signal (0%–100%) of the faces. As predicted,
when 100% face signals were presented, static anddynamic expressions were recognized with equal
eﬃciency with the exception of those with the mostinformative dynamics (i.e., happiness and surprise).
However, when less signal was available, dynamic
expressions were all better recognized than their static
counterpart (peaking at ∼20%). Our data show that
facial movements increase our ability to eﬃciently
identify emotional states of others under the
suboptimal visual conditions that can occur in everyday
life. Dynamic signals are more eﬀective and sensitive
than static ones for decoding all facial expressions ofemotion for all human observers.

Citation: Richoz, A.-R., Stacchi, L., Schaller, P., Lao, J., Papinutto, M., Ticcinelli, V., & Caldara, R. (2024). Recognizing facial expressions of emotion amid noise: A dynamic advantage. Journal of Vision, 24(1):7, 1–22, https://doi.org/10.1167/jov.24.1.7.
https://doi.org/10.1167/jov.24.1.7

Received June 19, 2023; published January 10, 2024

This work is licensed under a Creative Commons Attribution 4.0 International License.
Downloaded from jov.arvojournals.org on 01/11/2024

ISSN 1534-7362 Copyright 2024 The Authors

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

Introduction
Human facial movements transmit a wealth ofdynamic signals that readily provide crucial information
about other people’s emotional states. The temporal
dynamics of facial expressions of emotion are finely
optimized to hierarchically transmit biologically rooted
and socially adaptive signals over time (Jack, Garrod,& Schyns, 2014). In everyday life, we are flowed bydynamic emotional signals steadily stimulating thevisual system over the course of social interactions.
As a consequence, the frequency of exposure to
dynamic faces is markedly higher compared with static
ones. Yet, most studies investigating facial expression
recognition (FER) have relied on the sole use of static
face images; thus, very little is known about how thehuman brain decodes and processes dynamic facial
expressions of emotion. This experimental bias mightstem from previous studies consistently reportingthat young adults minimally benefit from the richer
dynamic over static information, whereas children,
the elderly and, clinical populations very strongly
do (Richoz, Jack, Garrod, Schyns, & Caldara, 2015;
Richoz, Jack, Garrod, Schyns, & Caldara, 2018b).
Dynamic face stimuli are also more difficult to control;
experiments using dynamic faces take longer, and there
are fewer databases available for dynamic comparedwith static faces. In fact, as previously pointed out
by Dobs, Bulthoff, & Schultz, (2018), controlling therole of motion and form information in dynamic
ecological natural stimuli is challenging. Point lights
can provide and test the importance of facial motion
information, but they are highly unnatural. Morphing
techniques might help to control motion while
approximately preserving form. However, as noted by
Dobs et al. (2018), they only represent a coarse linear
approximation of natural face motion, which might lead
to less accurate emotion recognition than their naturalcounterparts (Cosker, Krumhuber, & Hilton, 2010;
Korolkova, 2018; Wallraven, Breidt, Cunningham, &Bülthoff, 2018). Synthetic faces might help to solve thischallenge, but they suffer in terms of ecological validity.
Despite these methodological challenges, the lack ofstudies using dynamic natural faces is still surprisingbecause evolutionary and ontogenetic perspectives
would posit an advantage in processing dynamic faces
for all human observers.
From an evolutionary perspective, humans have
had more extensive experience with dynamic faces as
the perception of static faces emerged only recently
in human history, with the first existing portraits
originating in ancient Egypt approximately 5,000 years
ago. For hundreds of centuries, this form of art wasa privilege of the nobles and powerful. Although
exposure to paintings and statues increased throughout
history, it is only during the last century that humans

Downloaded from jov.arvojournals.org on 01/11/20242

have become more and more confronted with static
face images with the advent of photography and therapid expansion of digital technologies. Critically,
daily exposures to static faces occurred only during
the last 20 years with the use of smartphones andthe diffusion of pictures and selfies throughout social
networks and media content. However, during thevery last few years, dynamic social media messages
are massively overtaking static ones over the internet(i.e., Instagram, TikTok). The exposure to static face
images in social media remains limited compared withdynamic face signals. Moreover, from an ontogenetic
perspective, infants are mostly deprived of any
exposure to static faces during their first year of life.
Infants rapidly tune to cultural differences in facial
expressions of emotion based on learning cues indynamic signals (Geangu et al., 2016). Although daily
exposure and evolutionary and ontogenetic perspectives
would predict an advantage in processing dynamic
over static facial expressions, this question remains
unclear.
To date, a relatively modest number of studies
have investigated this question, yielding contradictory
findings (for a review, see Alves, 2013; Fiorentini &Viviani, 2011; Kätsyri, 2006; Krumhuber, Kappas,& Manstead, 2013). Although some studies did not
find a dynamic advantage in healthy young adults(e.g., Fiorentini & Viviani, 2011; Gold et al., 2013;
Kätsyri & Sams, 2008), an advantage was observed inclinical (e.g., Atkinson, Dittrich, Gemmell, & Young,
2014; Kan, Mimura, Kamijima, & Kawamura, 2004;
Schaefer, Baumann, Rich, Luckenbaugh, & Zarate,2010) and neuropsychological populations (Adolphs,
Tranel, & Damasio, 2003; Humphreys, Donnelly, &Riddoch, 1993; Richoz et al., 2015). For example,
Yitzhak, Gilaie-Dotan, & Aviezer, (2018) investigatedemotion recognition with patient L.G., a single caseof developmental visual agnosia and prosopagnosia.
Their results revealed improved recognition scores whensubtle nonstereotypical expressions were presented
with dynamic compared with static faces. We observed
similar findings in another neuropsychological study
examining FER in patient P.S., a pure case of acquired
prosopagnosia with bilateral occipitotemporal lesions
anatomically sparing the regions critical for decoding
facial expressions. Although patient P.S. was selectively
impaired in categorizing several static expressions, her
performance was within the average range with dynamic
faces (Richoz et al., 2015). In contrast with L.G.’s
relatively poor dynamic advantage, the gain for patient
P.S. was prominent; she reached maximum accuracy for
all dynamic expressions except for fear. Her impaired
performance for static expressions can be accounted
for by a suboptimal use of facial information withstatic faces, for which she only relies on the lower partof the face (Fiset et al., 2017). More important, P.S.’
peculiar brain lesions point to the existence of distinct

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

cortical pathways for processing static and dynamic face
information (Bernstein, Erez, Blank, & Yovel, 2018;
Duchaine & Yovel, 2015; Fox, Iaria, & Barton, 2009;
Pitcher, Dilks, Saxe, Triantafyllou, & Kanwisher, 2011).
P.S.’s advantage for the recognition of dynamic facial
expressions might rely on an intact functional corticalpathway directly connecting the early visual cortex tothe posterior superior temporal sulcus, and subsequent
processing in the anterior superior temporal sulcus(Richoz et al., 2015).
Pursuing these observations, we recently investigated
FER across the lifespan from 5 to 96 years of age.
We aimed to empirically probe the hypothesis thatthe dynamic advantage occurs only in populations
with fragile face processing systems—with difficulties
extracting facial information—such as very young
children, whose system is yet to fully mature, andelderly populations whose system is declining (Richoz
et al., 2018b). Our findings supported our hypothesis;
we observed a clear dynamic advantage in both young
children and the elderlies, whereas young adults
exhibited only a minimal benefit for dynamic stimuli
and only for a very limited number of expressions.
These findings are consistent with previous behavioral
studies revealing that the latter population does notbenefit from the presentation of dynamic signals(Fiorentini & Viviani, 2011; Gold et al., 2013; Kätsyri& Sams, 2008). For example, Fiorentini and Viviani(2011) reported similar identification accuracies andreaction times for both static and dynamic expressions
in a study using a threshold model with morphed
expressions. Using an ideal observer approach toobjectively assess how much information is carried bytheir stimuli across varying tasks and conditions, Goldet al. (2013) further evidenced identical recognition
scores for static, dynamic, and even shuffled andreversed expressions in which movie frames wererandomly presented or temporally reversed. Altogether,
these findings suggest a near-optimal FER system inhealthy young adults—as also argued by fast periodic
visual stimulation studies that revealed specific brain
responses to basic emotional facial expressions at asingle glance (e.g., Dzhelyova, Jacques, & Rossion,
2017; Poncet, Baudouin, Dzhelyova, Rossion, & Leleu,2019).
In optimal situations and with expressions ofhigh intensity, the visual system of healthy young
adults seems thus to be powerful enough to efficiently
categorize static emotional expressions, leaving only
little scope for improvement when dynamic faces areshown. As suggested in previous reviews (Dobs et al.,2018; O’Toole, Roark, & Abdi, 2002), dynamic cues are
beneficial only in suboptimal visual conditions in which
the facial information is limited. This has been shownin several studies in healthy young adults with degraded(Ambadar, Schooler, & Cohn, 2005; Bould & Morris,
2008; Bould, Morris, & Wink, 2008) or blurred stimuli(Ehrlich, Schiano, & Sheridan 2000; Kätsyri & Sams,
Downloaded from jov.arvojournals.org on 01/11/20243

2008; Wallraven, Breidt, Cunningham, & Bülthoff,2008).
For example, Wallraven et al. (2008) revealed
that dynamic faces led to more accurate recognition
scores when facial features, such as texture, shape, or
motion information, were gradually degraded over four
different blur levels. In a further study, Cunningham andWallraven (2009) revealed an overall better recognition
performance for dynamic over static expressions
when the faces were presented with varying spatial
information (point-light, wireframe, and full-surface
faces). Although performance was near chancelevel for static faces in the point-light condition, therecognition of dynamic expressions was overall higher
and remarkably similar across all three experimental
manipulations. This report suggests that motion
information can mitigate the negative consequences ofdegrading, blurring, or changing the texture of faceand shape information. The facilitative effects driven by
dynamic stimuli were further observed in studies using
complex stimuli such as nonstereotypical (Yitzhak et al.,2018), conversational (Cunningham &Wallraven, 2009),
subtle (Ambadar et al., 2005; Bould et al., 2008), or
genuine versus deliberate expressions (Namba, Kabir,
Miyatani, & Nakao, 2018; Zloteanu, Krumhuber, &Richardson, 2018). With these expressions, additional
information provided by moving faces might benecessary to compensate for the ambiguity elicited bycomplexity or the lack of intensity (Krumhuber et al.,2013).
To sum up, previous studies either failed to reporta dynamic advantage in healthy young adults, whichis possibly reflecting a ceiling effect owing to anear-optimal facial expression decoding system in thispopulation or reported the existence of very limited
dynamic advantage for FER in healthy young adults,
restricted to situations in which FER is disturbed
through stimulus manipulation. This observation
raises the question of how much signal is needed toelicit a dynamic advantage. To clarify this question,
we parametrically and randomly manipulated thesignal (i.e., phase coherence) of the facial expressions
presented to the observers to determine how muchsignal they need to recognize the emotional expressions
accurately (from 0% to 100%). We implemented asimilar approach to the one we used in several previous
studies (Bayet et al., 2017; Rodger, Vizioli, Ouyang,& Caldara, 2015; Rodger, Lao, & Caldara, 2018;
Rodger et al., 2021; Wyssen et al., 2019) yet adapted
it to make it more suitable for dynamic faces. Morespecifically, we parametrically manipulated the phasecoherence (i.e., the quantity of signal) by using an
adaptation of the weighted mean phase technique by
Dakin, Hess, Ledgeway, & Achtman, (2002) (see also
Rainer, Augath, Trinath, & Logothetis, 2001; Rousselet,
Pernet, Bennett, & Sekuler, 2008). Importantly, we
implemented an approach equating all the dynamic
and static stimuli for their low-level image properties

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

(i.e., spatial frequency spectrum and contrast). Eachstimulus was then characterized by its level of signal
with 0% corresponding to completely de-phased images
and 100% to natural images (100% of phase coherence).
On each trial, the level of signal was determinedby a uniform or adaptive sampling, with the latterbeing based on participants’ previous performance.
Seventy-one healthy young adults performed FER
tasks of the six basic expressions (i.e., anger, disgust,fear, happiness, sadness, surprise) in static and dynamic
conditions. We relied on a database of the six basic
static and dynamic facial expressions of emotion)
created by (Gold et al., 2013). Our methodologicalchoice was driven by the fact that these authors used
an ideal observer model to objectively measure theamount of low-level physical information in the stimuli,
as previous studies did not control for this factor (e.g.,
Ambadar et al., 2005; Bould et al., 2008; Cunningham& Wallraven, 2009; Fiorentini & Viviani, 2011).
Importantly, this database has been successfully usedin several studies (Richoz et al., 2018a; Richoz et al.,2018b) and led to similar FER performance profiles
as those observed with the seminal Pictures of Facial
Affect database developed by Ekman and Friesen, 1976;
Ekman and Friesen, 1978, corroborating its validity.

Methods
Participants
Seventy-one healthy young adults participatedin the study (M = 20.9 years; SD = 1.7, 16 males).
All participants had normal or corrected-to-normal
visual acuity and did not suffer from neurological,
developmental, or psychiatric disorders. They were
all Caucasian and have always lived in a Caucasianculture. We decided to control for the ethnicity ofour participants, given that cultural factors have an
influence on FER (Jack, Caldara, & Schyns, 2012;
Jack, Garrod, Yu, Caldara, & Schyns, 2012), andthis can be observed already very early in infancy(Geangu et al., 2016). Most of our participants were
university students (67) and received course credits
for their participation. Before starting the experiment,
all participants signed a consent form explaining themain goals of the study. The Ethical Committee of
the Department of Psychology of the University of
Fribourg in Switzerland approved the current study.
All participants provided written informed consent inaccordance with the Declaration of Helsinki.

Stimuli
We used a set of static and dynamic stimuli createdby Gold et al. (2013). They generated their databaseby asking eight actors (four females) to perform the
Downloaded from jov.arvojournals.org on 01/11/2024

4

six basic facial expressions of emotion (i.e., anger,
disgust, fear, happiness, sadness, and surprise) (Ekman& Friesen, 1976). All dynamic expressions started
with a neutral face and naturally evolved into a fully
articulated expression over the course of 30 frames.
The apex of each expression (i.e., the point at which an
expression reached its highest state) was determined by
two raters. Each expression was presented for 1000ms
at a frame rate of 30 frames/second, and all expressions
reached their apex within 30 frames. If an expression
was fully articulated before reaching 30 frames or ifhead movements or other artifacts occurred, one to four
supplementary apex frames were added to the dynamic
movie to reach the duration of 1s (this happened only
for seven out of 48 movies; for more details, see Goldet al., 2013). All dynamic expressions were presented
as front-view grayscale faces cropped at the hairline toshow only the internal facial features to the observers.
In addition, the faces were displayed within an oval
frame, located in the center of a dark grey uniformbackground. The edges of the oval frame were slightly
blurred to create a gradual transition between the facial
expressions and the grey-colored background (Goldet al., 2013). The faces subtended a visual angle of 12°
at a viewing distance of 65 cm from the screen and theimages were 768 pixels in height and 768 pixels in width.
The static facial expression movies were created byreplicating the final frame of each dynamic movie 30
times in a row (see Figure 1; video examples of thestimuli can be found under the links).
We used the SHINE toolbox (Willenbockel et al.,2010) to normalize all stimuli for their low-levelproperties (spatial frequency and contrast) and theamount of energy sampled over time, even for thestatic facial expressions (see Richoz et al., 2018b). This
procedure allowed us to ensure that all the frames for all
the faces in all conditions had equal low-level properties
over time.
The stimuli were presented to the observers withvarying levels of phase coherence (i.e., signal). The
percentage of phase coherence varied between 100% fornatural images to 0% for completely de-phased images.
To parametrically manipulate the phase coherence, we
adapted the weighted mean phase technique by Dakinet al., 2002; see also (Rainer et al., 2001; Rousselet et al.,2008) so that our faces were dephased while contrast
and luminance were maintained constant across alllevels of signal. More specifically, the final image (Ifinal )shown to observers was obtained by combing an
original image with 100% phase coherence (Iinitial ) witha noise image (Inoise ) as it follows:
I f inal = w Iinitial + (1 − w) Inoisew [0, 1]

where w defines the degree of phase coherence of the
final image.
Inoise was built through the inverse fast Fouriertransform using the same amplitude as the original
image (ainitial ), but a different phase (ϕ noise ). Specifically,

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

5

Figure 1. (A) Static examples of one identity expressing the six basic emotions at diﬀerent levels of phase signal. The rows represent
the six basic expressions (anger, disgust, fear, happiness, sadness, and surprise), and the columns the diﬀerent levels of signal (0%,20%, 40%, 50%, 60%, 80%, and 100%). We adapted the stimuli with permission from Gold et al. (2013). Further illustrative video
examples can be seen with the following links for the static condition—Movie 1 and the dynamic condition Movie 3. (B) A schematic
illustration of the adaptive sampling approach.

these two components of the noise image were obtainedusing the following equations:ainitial = abs [F F T (Iinitial )]⎧−1 Sϕrandom⎪+ π if Cϕrandom < 0
tan
⎪
⎪
Cϕ
⎪
⎨
S random ϕϕnoise = tan−1 Cϕrandom + 2π if Sϕrandom < 0 and Cϕrandom > 0
random⎪⎪⎪ −1 Sϕrandom ⎪⎩tanotherwiseCϕrandomThis procedure was applied to each frame of the
dynamic movies while ϕ random was kept constant withinbut varied between trials.
In the static condition, each frame was combinedwith the same Inoise as its corresponding frame in the
dynamic condition. This result was possible because
the generation of ϕ random was seed controlled and wastherefore the same across conditions for identical levelsof phase coherence. This procedure ensured that the
low-level properties of each frame of the movies andthe information revealed by the noise were identical inboth conditions.

where:

Methods
Sϕrandom = sin (ϕrandom )Cϕrandom = cos (ϕrandom ) .

Downloaded from jov.arvojournals.org on 01/11/2024

As shown in Figure 1, the movies presented to theobservers were characterized by their level of signal,

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

with 0% signal corresponding with a completely noisy
pattern and 100% signal with natural images. We
determined the amount of signal presented using two
different procedures across two sessions: a uniform
and an adaptive sampling. Under uniform sampling
conditions, participants were shown images masked
by a random amount of noise that was sampledindependently for each trial, participant, expression,and condition from a uniform distribution ranging
between 0% and 100%. Although all participants
were not necessarily presented with precisely thesame percentage of noise, this approach allowed toevenly sample the whole space when considering all
observers. Our hypothesis for the impact of the signal
level on response accuracy was an S-shaped response,
where low and high percentages of signal would leadto a predominance of wrong and correct answers,
respectively. However, the level of signal at which thetransition between these two extremes would occur
was unknown. Therefore, to capture this moment more
precisely, we adopted an adaptive sampling approachthat modelled the amount of signal to be shown basedon observers’ previous responses. To this aim, on a
separate testing session, we first presented 20 trials witha signal level drawn from a uniform distribution. Then
we used curve fitting and inverse transform sampling toincrease the likelihood of sampling a signal percentage
around the point of the curve’s maximum slope. This
process was repeated after each of the remaining
trials. Importantly, this adaptive sampling procedure
was implemented for each participant and expression
separately.
The stimuli were shown on a color liquid-crystal
display with a resolution of 1,440 × 900 pixels and arefresh rate of 60 Hz. The experiment was programmed
in MATLAB (2014) using the Psychophysics Toolbox(Brainard & Vision, 1997; Kleiner et al., 2007).

Procedure
We informed our participants that they would
be exposed to faces expressing different emotional
expressions on a computer screen and that their task
would be to categorize them as accurately as possible,
according to the six basic facial expressions: anger,
disgust, fear, happiness, sadness, and surprise. All
participants sat 65 cm away from a computer screen ina quiet room at the University of Fribourg. Each trial
started with a white fixation cross presented for 500 ms
on a grey background at the centre of the screen. Facial
expressions were then presented in random order at thecentre of the screen, one at a time, for 1 second eachand at a signal strength estimated by the uniform or
the adaptive sampling (for a schematic representationof the procedure, see Figure 1). The same presentation
time was used in both conditions. We decided to use a

Downloaded from jov.arvojournals.org on 01/11/2024

6

1-second stimulus presentation time as it was previously
used in several different studies with dynamic faces(Adolphs et al., 2003; Recio, Schacht, & Sommer, 2013;
Richoz et al., 2015). After each stimulus, a response
window appeared at the centre of the screen andremained there until the participant gave their answer
by pressing the correct key on a labelled keyboard. We
gave the observers as much time as needed to familiarize
themselves with the different possible responses andtheir corresponding keys and told them that reaction
time was not important for the current experiment. No
feedback was given to the observers for their answers.
If participants did not know the answer or did not haveenough time to judge an expression, they could press
an, “I don’t know”–labelled key on the keyboard. Sucha key was proposed to the participants to reduce thenoise and response bias produced by the absence of this
response possibility. Participants performed 768 trials
presented with the uniform sampling of the noise and768 trials with the adaptive sampling of the noise, fora total of 1,536 trials divided into 2 different sessions
that took place on different days. Each session lasted
for approximately 50 minutes. The 768 trials included 8
identities expressing 6 expressions 8 times in static anddynamic conditions (8 × 6 × 8 × 2). The 768 trials weredivided into 10 blocks of 77 stimuli (75 for the last one).
The stimuli were not blocked by condition, and eachblock presented a random ratio of dynamic and static
faces. Before starting the testing phase, participants
completed 12 practice trials in each condition.

Data analysis
Data analysis was performed in R (The R Foundationfor Statistical Computing, Vienna, Austria). First, we
computed the average accuracy percentage by using
a 2% signal window, from 0% to 2% of signal to
98% to 100% of the signal, which resulted in 50 bins.
This procedure was performed twice: once for the
dynamic and the static condition independently of theexpressions and once for each condition and expression
separately.
The binned data were then fitted using a threeparameter Weibull type II curve. Using each model,
we estimated the psychometric curve as a function of
signal at a resolution of 0.01% (i.e., resulting in 10,001
points between 0% and 100% of signal). These values
were used to compare dynamic and static conditions(i.e., dynamic–static) in terms of accuracy, as well as todetermine the amount of signal at which the greatest
difference between both conditions occurred. This
dynamic–static difference was subsequently comparedacross expressions.
Additionally, we estimated and compared between
conditions the amount of signal required to reach
99% of the upper asymptote (i.e., ceiling point) andJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

to surpass chance level (i.e., 1/6). Finally, using theestimated models’ parameters, we differentiated eachcurve and extracted the slope as a function of signal ata resolution of 0.01. Using these values, we determined
the maximum slope, which we then compared between
conditions for each expression separately.
To determine the statistical significance of thecomparisons, we used percentile bootstrap. First,
we sampled subjects with replacement and binned
the data. We then fitted the Weibull type II curve,
estimated the same measures as for the original
data, and computed the differences of interest (i.e.,
dynamic–static or between-expression differences).
This process was repeated 9,999 times allowing us tobuild a 95% confidence interval (CI) for each difference,
which was considered as statistically significant if its
CI did not encompass 0. Importantly, the CI was
adjusted using Bonferroni correction to accountfor multiple comparisons when testing each of thesix basic expressions and the overall condition (i.e.,
7 contrasts) or cross-comparing facial expressions(i.e., 15 contrasts). This correction was implemented
by dividing the alpha level of 5% in the CI by the
number of contrasts performed. In this case, the CI willcorrespond to 100, the corrected alpha level. To simplify
the reading, we refer to this statistical adjustment for
the number of inferences made as the corrected CI.
Finally, to better understand the data, we used
observers’ binned data to determine, in percentages,
how often each expression was recognized as the correct
one or as another one. These values were used to build
confusion plots (see Supplementary Figures S1 andS2) and matrices to display participants’ choices as afunction of the condition, expression, and amount ofsignal presented. All data used in statistical analyses andrepresented in figures are available online on the OpenScience Framework repository at: https://osf.io/3269r/?view_only=23d291246d3644f898a21c08729d6dae.

Results
Figure 2 shows curve fitting across expressions andconditions. Data follow the expected S-shaped patterns,
as accuracy grows from 0% when 0% of signal isshown, to a ceiling performance when 100% of signal
is presented. Visual exploration of the raw data andcurve fitting shows that the ceiling points as well as
the rate of growth and quantity of signal required toaccomplish this transition vary across both expressions
and conditions (Figure 2).
As mentioned elsewhere in this article, to evaluate
differences between conditions and expressions
statistically, we used percentile bootstrap to build acorrected CI. We considered a difference significant if
its corrected CI did not encompass zero. This statistical

Downloaded from jov.arvojournals.org on 01/11/2024

7

processing was applied on the data predicted by the
curve that was fitted on the raw data.

Dynamic vs. static FER performance acrosssignal percentage: The dynamic advantage
The difference in FER performance between dynamic
and static conditions was computed by subtracting, foreach binned signal percentage, accuracy in the static
condition from modelled accuracy in the dynamic
condition. Consequently, any positive difference
reflects a recognition advantage of dynamic over static
expressions, and any negative difference an advantage
of static over dynamic expressions.
Overall, comparisons between fitted curves show thata significant dynamic advantage emerges as early as
0.01% of signal. The advantage then grows and peaks
at 18.6% of signal before decreasing and disappearing
from 37.36% of signal onward (Figure 3). Although asimilar pattern in the dynamic advantage is found acrossall expressions, some differences can be observed. The
advantage onset occurs later in terms of signal for anger(9.65%), disgust (2.65%), sadness (7.64%), and surprise(3.16%) (Figure 3). As reported in Table 1, the amountof signal at which the greatest dynamic advantage
occurs also varies across expressions. Additionally,
for two expressions out of six (i.e., happiness andsurprise) the dynamic advantage persists until 100% of
signal, and for one expression (i.e., fear), it converts
into a static advantage from 22.95% of signal onward(Figure 3, Figure 4, and Table 2).
Across all expressions, the largest dynamic advantage
was observed for happiness (mean = 58.02%; 95%
CI [51.79, 64.88]) and the smallest for fear (mean =12.24%; 95% CI [7.01, 17.02]) (Figure 3, Figure 5A, andTable 1). Comparisons between expressions revealed
that the magnitude of the dynamic advantage wassignificantly different across all of them, except for the
disgust–anger, disgust–sadness, and happiness–surprisecontrasts (Figure 5B). Assessing the percentage ofsignal at which the maximum dynamic advantage
occurs shows that it first emerges for fear, followed byhappiness, and surprise and it appears last for sadness,anger, and disgust (Table 1).

Signal needed to reach the ceiling point (i.e.,
99% of the maximum recognition performance)
As illustrated in Figure 2, participants’ accuracyshows an initial increase as a function of signal
percentage, followed later by a plateau. To assess theminimum amount of signal required by participants toreach their maximum recognition performance (i.e., theceiling point), we determined, for each expression andJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

8

Figure 2. Raw data and curve ﬁtting for dynamic and static conditions overall and for each expression separately. Dots represent raw
data, while lines represent ﬁtted curves. The vertical dotted lines mark the amount of signal needed to reach the ceiling point (i.e.,
99% of the maximum performance). The horizontal dotted line indicates chance level. Triangles below the x axis mark the presence ofa signiﬁcant diﬀerence in terms of accuracy between dynamic and static conditions.

condition, the signal percentage at which performancereached 99% of the curve’s upper asymptote.
Results show that both overall and for each
expression independently, except for fear and disgust,participants needed significantly less signal to reach
99% of their maximum recognition performance in the
dynamic condition (Figures 6, 7). More specifically,

Downloaded from jov.arvojournals.org on 01/11/2024

this difference was significant overall (M = −14.66,
corrected CI [−18.37, –4.00]) and for anger (M =–22.05, corrected CI [−34.61, –9.55]), happiness (M= –8.29, corrected CI [−13.54, –1.92]), sadness (M =–6.62, corrected CI [−14.7, –0.45]), and surprise (M =–16.06, corrected CI [−33.02, –11.31]) (Figure 6 andTable 2).

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

9

Figure 3. Dynamic–static curve diﬀerence overall and across each expression separately as a function of signal percentage. Lines show
the dynamic-static diﬀerence between ﬁtted curves as a function of signal percentage. Shaded areas around the curve indicate thecorrected CI. Red shading indicates that the CI does not include 0 (i.e., statistically signiﬁcant), which is illustrated here as thehorizontal dashed line. Gray shading indicates that the CI includes 0, and the diﬀerence is therefore nonsigniﬁcant. CI = conﬁdence
interval.

Signal needed to surpass chance level
Finally, we examined the amount of signal required
to surpass chance level in the dynamic and static
conditions. We defined chance level performance as
the scores at 16.66%, which corresponds with one overDownloaded from jov.arvojournals.org on 01/11/2024

six possible answers. The first score above 16.66%
was considered as the value surpassing chance level.
Results show that both overall and for each expression
individually, observers needed significantly less signal
to reach chance level in the dynamic compared with thestatic condition (p < 0.001) (Figure 8 and Table 2).

Journal of Vision (2024) 24(1):7, 1–22

Overall
Anger
DisgustFear
Happiness
Sadness
Surprise

Richoz et al.

Magnitude of the
maximum dynamic
advantage

Signal at maximum
dynamic advantage

33.83 [30.91–37.34]
36.66 [30.68–42.91]
27.85 [21.23–35.13]
12.24 [7.01–17.02]
58.02 [51.79–64.88]
23.05 [17.58–29.82]
50.10 [43.43–56.16]

18.60 [18.03–19.26]
22.76 [21.50–23.79]
22.73 [21.62–23.66]
14.23 [13.18–15.91]
15.98 [15.44–16.59]
21.13 [20.03–22.17]
18.50 [17.60–19.46]

Table 1. Magnitude of the dynamic advantage and level ofsignal at which it occurs. Note. For the amplitude of the
maximum dynamic advantage, we report the corrected CI. For
the signal at which such advantage occurs, we report the 95%
CI. CI = conﬁdence interval.

Growth rate across dynamic and static
conditions
To assess the rate of maximum accuracy increase
as a function of the signal presented, we extracted theslope of the curve for each expression in each condition
separately. Results show that overall and for each
expression separately, except for fear, signal increase led
to a steeper accuracy increase in the dynamic comparedwith the static condition (Figure 9). However, thestatistical comparison of the maximum slope between
conditions was significant only when all expressions
were pooled together (M = 1.44, corrected CI [0.83,
2.29]), and only for anger (M = 2.65, corrected CI[1.32, 5.09]) and surprise (M = 3.62, corrected CI [1.79,
6.01]) when the expressions were considered separately(Table 2).

10Discussion
This study provides a novel fine-grained parametricalmapping of young adults’ ability to categorize static anddynamic expressions from low to full signal strength.
By using an innovative psychophysical approach,we parametrically and randomly manipulated thequantity of signal available to the observers. We relied
on a database of stimuli that was created by Goldet al. (2013), who ensured that the low-level physicalinformation carried by static and dynamic faces wasequal in both conditions. Moreover, because all stimuli
were equated for their low-level properties, we could
identify the genuine quantity of signal required forour observers to effectively categorize the six basic
expressions in static and dynamic conditions. This
precise and novel approach allowed us to clarify
whether the very limited to noninexistent advantage
for the recognition of dynamic expressions previously
reported in young adults (e.g., Gold et al., 2013;
Fiorentini & Viviani, 2011; Jiang et al., 2014; Richoz
et al., 2018b) is rooted in a ceiling effect owing to
the experimental conditions typically found in FER
tasks.

A dynamic advantage for all expressions withlow signal
First and foremost, our findings revealed no
overall beneficial effect of motion in healthy young
adults when the stimuli were presented with 100%of phase signal, as recognition scores were very
similar in both conditions. With these results, weFigure 4. Accuracy level at 100% of signal. Accuracy levels in the dynamic and static conditions are reported overall and for each
expression independently. Error bars represent the 95% CI. The * indicates a signiﬁcant diﬀerence between conditions based on thecorrected CI. CI = conﬁdence interval.

Downloaded from jov.arvojournals.org on 01/11/2024

Journal of Vision (2024) 24(1):7, 1–22

Accuracy at 100% signal
StaticOverall
69.57 [68.17, 70.82]
Anger
73.79 [70.38, 77.04]
Disgust72.95 [70.10, 75.51]
Fear
53.13 [49.65, 55.98]
Happiness
94.84 [92.91, 96.41]
Sadness
60.76 [58.01, 63.52]
Surprise69.49 [66.69, 72.75]
Dynamic
Overall
70.26 [68.82, 71.53]
Anger
75.04 [71.52, 78.21]
Disgust72.11 [69.33, 74.67]
Fear
46.38 [43.02, 49.2]
Happiness
97.07 [96.23, 97.81]
Sadness
62.70 [59.93, 65.43]
Surprise73.59 [70.62, 76.79]
Dynamic–static diﬀerences
Overall
0.69 [−0.30, 1.62]
Anger
1.24 [−1.70, 3.90]
Disgust−0.84 [−2.82, 1.2]
Fear
−6.75 [−9.59, −3.81]
Happiness
2.23 [0.6, 4.39]
Sadness
1.94 [−0.09, 3.87]
Surprise4.09 [1.48, 6.72]

Richoz et al.

11

Maximum slope

Signal at ceiling point(99% of maximum
performance)

Signal at chance level4.17 [3.86, 4.50]
3.02 [2.70, 3.51]
4.62 [4.16, 5.19]
5.42 [4.33, 7.20]
8.29 [7.15, 9.71]
4.20 [3.57, 4.94]
3.47 [2.95, 4.15]

39.77 [34.13, 41.71]
50.63 [43.27, 58.35]
38.80 [36.17, 43.46]
25.09 [22.79, 28.99]
29.64 [27.22, 32.84]
35.27 [31.54, 29.74]
40.38 [37.48, 50.8]

16.60 [16.04, 17.26]
17.96 [17.10, 19.04]
20.21 [19.18, 21.28]
15.45 [14.36, 16.63]
14.69 [13.96, 15.48]
18.45 [17.45, 19.62]
18.11 [17.18, 19.11]

5.61 [5.15, 6.27]
5.67 [4.72, 7.27]
5 [4.15, 6.15]
3.67 [3.04, 4.96]
9.81 [8.63, 11.30]
5.31 [4.51, 6.51]
7.09 [5.88, 8.75]

25.11 [23.65, 30.16]
28.58 [25.02, 34.32]
31.54 [28.53, 37.6]
24.17 [21.14, 30.83]
21.35 [19.63, 24.93]
28.65 [25.05, 31.67]
24.32 [19.85, 26.79]

10.55 [10.02, 11.2]
12.56 [11.65, 13.57]
14.66 [13.8, 15.66]
12.23 [11.34, 13.65]
8.35 [7.69, 9.10]
14.38 [13.38, 15.45]
9.72 [8.98, 10.49]

1.44 [0.83, 2.29]
2.65 [1.32, 5.09]
0.38 [−1.07, 2.21]−1.75 [−4.09, 0.15]
1.52 [−0.94, 4.18]
1.11 [−0.16, 2.80]
3.62 [1.79, 6.01]−14.66 [−18.37, −4]−22.05 [−34.61, −9.55]−7.26 [−15.17, 1.98]−0.92 [−7.05, 8.03]−8.29 [−13.54, −1.92]−6.62 [−14.7, −0.45]−16.06 [−33.02, −11.31]−6.05 [−6.70, −5.43]−5.40 [−6.91, −3.99]−5.55 [−7.07, −4.02]−3.22 [−4.19, −1.89]−6.34 [−7.49, −5.18]−4.07 [−5.56, −2.89]−8.39 [−9.96, −6.91]

Table 2. Measures [95% CI] estimated from the ﬁtted curves. Note. For static and dynamic FER performance, we report the 95% CI.
For dynamic–static diﬀerences, we report CI after alpha Bonferroni correction for multiple comparisons (i.e., the corrected CI). Bold
indicates signiﬁcant dynamic–static diﬀerences. A positive diﬀerence in accuracy and slope and a negative diﬀerence in signal bothindicate an advantage for the dynamic condition. CI = conﬁdence interval.

replicate previous findings using identical (Goldet al., 2013) or similar stimuli (intense expressions)and methodological paradigms (optimal viewing
conditions) (e.g., Bould & Morris, 2008; Jiang
et al., 2014; Yitzhak et al., 2018). However, when
examining each expression independently, our findings
revealed a dynamic advantage with full signal strengthfor happiness and surprise. Although some priorstudies (e.g., Gold et al., 2013) only examined theoverall dynamic advantage without considering
each expression individually, the current results are
consistent with the ones we reported in a previous
study revealing a dynamic over static advantage
for happiness and surprise in healthy young adults(Richoz et al., 2018b). Thus, for young adults, optimal
dynamic signals offer processing benefits only for a
few expressions. For most emotions, the additional
temporal information provided by dynamic faces suchas muscular changes, temporal evolution, and velocity
are not necessary for young adults to recognize them

Downloaded from jov.arvojournals.org on 01/11/2024

effectively when they are presented in optimal viewing
conditions.
Second, we examined whether the near-optimal facial
expression decoding system of healthy young adults
is genuinely insensitive to the richness of dynamic
signals or whether the very limited dynamic advantage
observed in previous studies is rooted in a ceiling effectowing to the experimental settings typically found in
FER tasks. In other words, we tested the idea that a
dynamic advantage is present for all expressions but
can only be revealed with the use of more sensitive
suboptimal visual signals that can occur in everyday life
(distance, occlusion, etc.). To do so, we parametricallyand randomly manipulated the signal (0%–100%) of
the facial expressions presented to the observers byusing a computerized psychophysical technique togenerate unique noise patterns for each frame of the
dynamic movies. We then applied the identical noise
patterns to the corresponding static frame to ensurethat the information available at each level of signal wasJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

12

Figure 5. Dynamic advantage across expressions and cross-expression comparisons. (A) Bar plots represent the magnitude of the
maximum dynamic advantage for each expression. Error bars represent the corrected CI. (B) The matrix represents the
cross-expression comparisons of the magnitude of the maximum dynamic advantage. The signiﬁcance of the contrasts wasdetermined using the corrected CI and are color coded according to diﬀerent alpha thresholds. CI = conﬁdence interval.

Figure 6. Minimum amount of signal needed by observers to reach ceiling points. The minimum amount of signal needed to reach theceiling points (i.e., 99% of the maximum recognition performance) in both dynamic and static conditions are reported overall and foreach expression independently. Error bars represent the 95% CI. The start indicates signiﬁcant diﬀerences between conditions basedon the corrected CI. CI = conﬁdence interval.

identical in both conditions. We found that all dynamic
facial expressions were better decoded than their static
counterparts when presented with low signal. This
dynamic advantage appeared with as little as 1% of
signal for some expressions, gradually increased, peaked,
and decreased with increasing signals, disappearing
entirely at 38% of signal. Only happiness and surprise
showed a different pattern, with a dynamic advantage

Downloaded from jov.arvojournals.org on 01/11/2024

persisting until 100% of signal. Note also that theinitial dynamic advantage observed for fear changed
into a static advantage at approximately 22% of signal.
These findings suggest that dynamic cues provide
additional emotion-related information that facilitate
the recognition of all six basic emotional expressions in
suboptimal visual conditions. Importantly, in healthy
young adults, motion-related cues are beneficial for theJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

13

Figure 7. Minimum amount of signal needed by observers to reach the ceiling point (i.e., 99% of their maximum performance). Each
expression is displayed with the level of signal at its ceiling point (i.e., 99% of maximum performance). Top and bottom rows illustrate
the static and dynamic conditions, respectively.

recognition of anger, disgust, fear, and sadness only
when static information is insufficient, compensating
for the deleterious consequences of degraded or missing
visual information.

Two perceptual processes might explain how facial
dynamics improve emotion recognition in suboptimalsituations. First, the saliency of change may naturally
drive the attention of observers toward the diagnostic

Figure 8. Minimum amount of signal needed by observers to surpass chance level. The minimum amount of signal needed, surpass
chance level in both dynamic and static conditions is reported overall and for each expression independently. Error bars represent the
95% CI. The * indicates signiﬁcant diﬀerences between conditions based on the corrected CI. CI = conﬁdence interval.

Downloaded from jov.arvojournals.org on 01/11/2024

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

14

Figure 9. Maximum slope across conditions for each expression and overall. The maximum slopes of the ﬁtted curves in the dynamic
and static conditions are reported overall and for each expression independently. Error bars represent the 95% CI. The * indicatessigniﬁcant diﬀerences between conditions based on the corrected CI. CI = conﬁdence interval.

information in a bottom-up fashion (i.e., the mouth forhappiness), while with static expressions, participants
are required to direct their attention toward those facial
features based on top-down internal representations.
Secondly, motion signals, such as the direction ofchange, the temporal evolution of an emotional
expression, the velocity of muscular changes andcontractions, provide additional diagnostic informationthat might be critical to support adequate expression
categorization in nonoptimal visual conditions(Kamachi et al., 2001; Kamachi et al., 2013; Yitzhak
et al., 2018).
In addition, several neuroimaging studies that haveexamined the neural underpinnings of the dynamic
advantage have shown that dynamic expressions
involve dissociable neural pathways and elicit broader
activations compare to static expressions (e.g.,
Johnston, Mayes, Hughes, & Young, 2013; Kessler
et al., 2011; LaBar, Crupain, Voyvodic, & McCarthy,
2003; Paulmann, Jessen, & Kotz, 2009; Sato et al.,
2004; Schultz & Pilz, 2009; Trautmann, Fehr, &Herrmann, 2009). For example, Sato et al. (2004) havereported enhanced activations to dynamic comparedwith static displays in right-lateralized occipital andtemporal cortices comprising the inferior occipitalgyri, middle temporal gyri, and fusiform gyri. In
contrast, the perception of static displays has beenshown to activate a network of motor, prefrontal,
and parietal regions, typically involved in motor
imagery (Kilts et al., 2003). More recently, Liang et al.
(2017) have shown that dynamic compared with static
expressions were associated with higher recognition
accuracies and more robust neural responses inface-selective areas (occipital face area, fusiform face

Downloaded from jov.arvojournals.org on 01/11/2024

area, posterior superior temporal sulcus), as well as inmotion-sensitive regions. These findings suggest thatdomain–general motion-sensitive areas that are notface specific are also strongly involved in decoding
dynamic facial expressions. Observers’ enhanced
ability to accurately decode dynamic expressions
presented with a very low signal could be explainedby the larger and more sensitive cortical network
dedicated to their processing as compared with static
expressions.
Note that our behavioral findings are in line with asmall number of studies that have investigated staticand dynamic FER in suboptimal visual situations
with degraded or blurred faces (Ehrlich et al., 2000;
Kätsyri & Sams, 2008; Wallraven et al., 2008). For
instance, using computer-animated faces, Wallraven
et al. (2008) revealed that motion cues enhanced
the recognition of facial affects when texture or
shape information was systematically degraded or
blurred. If dynamic cues were not provided, degrading
face information significantly affected expression
recognition. However, one major issue of altering
spatial frequency information (i.e., blurring faces) isthat the recognition of some facial expressions canbe more affected than others, because the diagnostic
spatial frequencies are different across emotions(Plouffe-Demers et al., 2019; Schyns, Petro, & Smith,
2009; Tian et al., 2018). To overcome this limitation, inthe current study, we used a psychophysical approachthat normalized spatial frequency information for
all expressions and experimental conditions. Thismethodological choice provides a more reliable viewof how dynamic cues offer processing benefits for therecognition of all six basic facial expressions on theJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

continuum from low to full-strength signal and may
help to clarify past inconsistencies observed acrossstudies.

Maximum sensitivity to dynamic signals
Having established a dynamic advantage for therecognition of all facial expressions with low signal,
we then estimated for each expression the maximum
sensitivity to dynamic signals by determining thepoint at which the dynamic gain reaches its maximum
before declining. We also estimated the quantity ofsignal necessary to reach this peak. To the best of ourknowledge, this study is the first that has effectively
quantified the strength of the dynamic gain for each
expression and the availability of visual information
at this point. We observed the strongest dynamic gain
for happiness followed by surprise with differences inaccuracy of 58% and 50% between static and dynamic
conditions at the maximum dynamic gain, respectively.
The quantity of signal necessary to reach the maximum
dynamic gain was around 16% for happiness andaround 19% for surprise. We observed the maximum
dynamic gain for anger, disgust, and sadness between
21% and 23% of signal. Although significantly weaker
than the dynamic gain for happiness and surprise, thedifferences in recognition performance between staticand dynamic conditions at the maximum dynamic
gain were 23% for sadness, 28% for disgust, and 37%
for anger. Finally, we observed a very distinctive
trajectory for fear with an early significant dynamic
gain (12% differences in recognition accuracy) peaking
at 14% of signal before shifting toward a significant
static advantage between 23% and 100% of signal.
Notably, the maximum dynamic gain observed forfear was significantly weaker than for all the otherexpressions.
The stronger dynamic advantage observed for some
expressions over others might be explained by the
diagnostic information embedded in the temporal
evolution of these expressions. Our results here suggest
that, among all expressions, the dynamics of happinessand surprise signals are the most informative. Thesesignals might be particularly salient and act as attention
grabbers when very low information is available to theobservers, making them more detectable than static
peak frames in suboptimal conditions. Kamachi et al.
(2013) have also shown that quick dynamic events tend
to be categorized as surprising or happy events, whereas
slow or static events are more likely to be categorized assad. The very strong dynamic advantage observed forhappiness and surprise could thus also be due to the
distinct temporal properties of these expressions being
inherently dynamic and rapid (see also Bould et al.,2008; Jack et al., 2014; Yitzhak et al., 2018).

Downloaded from jov.arvojournals.org on 01/11/2024

15

The maximum dynamic gain we found for anger
and disgust was also strong—albeit significantly lower
than for happiness and surprise. This finding waspaired with a shift in the quantity of signal necessary
to reach the maximum dynamic gain for these two
expressions (i.e., more signal was needed). Previous
studies have shown that the emotional expression of
disgust is frequently confounded with anger (e.g.,
Recio et al., 2013; Richoz et al., 2018b; Rodger et al.,2015) that could be explained partly by the sharedmuscular action units between these two expressions(e.g., Poncet et al., 2021). More specifically, Jack et al.
(2014) have shown that the confusion between anger
and disgust occurs because both expressions sharesimilar signals in early dynamics (nose wrinkler andlip funneler), suggesting that late dynamic signals arenecessary to disambiguate those expressions. The signal
shift we observed here could be accounted for by thenecessity of revealing additional diagnostic information
critical to disentangle the ambiguity triggered by thoseexpressions. Additionally, we could also speculate thatthe early onset in the maximum dynamic gain observedfor happiness–surprise over anger–disgust could relyon their frequency of exposure. During everyday lifesocial interactions, we routinely smile to our friends,
feeling joy or often expressing wonderment, a positive
sentiment of surprise (Vrticka, Lordier, Bediou, &Sander, 2014). In contrast, we rarely face anger anddisgust.
Interestingly, we observed a dynamic gain forsadness, an expression that has been previously shown
to be better recognized through static face images(Bould et al., 2008; Recio et al., 2013; Richoz et al.,2018a; Widen & Russell, 2015) or when evolving very
slowly (Kamachi et al., 2001; Recio et al., 2013; Richoz
et al., 2018b; Widen & Russell, 2015). For instance, ina previous cross-sectional study, we failed to report a
dynamic advantage for the categorization of sadness at
any age (Richoz et al., 2018b). Ekman (2003) suggested
that, among all expressions, sadness is the one lasting
the longest over time, a property that may explain
why slowness or stillness may increase recognition
performance. Although this explanation might clarify
the absence of a dynamic gain in optimal visualconditions and with intense expressions, the current
results provide new evidence that the recognition ofsadness benefits from dynamic cues when the diagnostic
information is not fully available.
Finally, our findings revealed a very distinct
trajectory for fear, supporting previous evidence that
this expression has a special status within the frameworkof FER (Richoz et al., 2015; Rodger et al., 2015). We
only observed a dynamic advantage for fear with a very
low signal, peaking rapidly before converting into astatic advantage. This initial dynamic advantage could
be due to an increased saliency elicited by the wide
and rapid opening of the eyes when a very low signal

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

is available to the observers (see Liu et al., 2022). Ourdata also revealed a static advantage for recognizing
fear between 23% and 100% of signal. Althoughcounterintuitive at first sight, this static advantage
could be explained by the diagnostic information
conveyed by the emotional expression of fear overtime. Using Bayesian classifiers, Jack et al. (2014)
revealed that fear and surprise share similar muscular
activations (upper lid raise, jaw drop) in early signaling
dynamics, leading to systematic confusion between
those two emotion categories. The critical diagnostic
information (eyebrow raiser; Jack et al., 2014) thatallows to accurately distinguish both expressions
becomes fully available only in later signaling dynamics.
Static expressions of fear, displaying the fully evolved
late signaling dynamics for 1 second, are maximally
informative and could thus be advantageous for the
categorization of this expression (see also, Richoz et al.,2018b). Furthermore, given its unique evolutionary
significance (i.e., indication of danger), the decodingof fear might recruit additional brain regions or
faster neural pathways (e.g., the amygdala) that mightshortcut the presumably longer processing trajectoryof dynamic faces (e.g., Adolphs, 2008; Furl, Henson,
Friston, & Calder, 2013). For instance, Furl et al. (2013)
have shown that the amygdala plays a critical role inthe decoding of static and dynamic fearful expressions
by recruiting distinct brain areas in a context-sensitive
fashion (form, or motion) to enhance and optimize
their processing. With dynamic faces the amygdalatargets the superior temporal sulcus and V5, both
involved in the encoding of motion information (e.g.,
Pitcher et al., 2011; Schultz & Pilz, 2009), whereas withstatic expressions the amygdala selectively targets thefusiform face area, an area dedicated to the processingof facial identity (e.g., Haxby, Hoffman, & Gobbini,2000) and static facial expressions (e.g., Ganel, Valyear,
Goshen-Gottstein, & Goodale, 2005). These findings
suggest that the amygdala guides and controls how
socially salient information is visually encoded bymodulating its connections to dorsal and ventral brain
regions.

Static and dynamic emotion recognition
trajectories from low to full signal
To further examine the specific static and dynamic
emotion recognition trajectories from low to full signal,
we estimated for each expression the quantity of signal
required to surpass chance level, defined as the firstscore above 16.66% (1/6 possible answers). Strikingly,
we observed that the amount of signal necessary tosurpass chance level was significantly higher for all
expressions in the static compared with the dynamic
condition. These findings further confirm the existenceof a dynamic advantage for recognizing all facial

Downloaded from jov.arvojournals.org on 01/11/2024

16

expressions and reinforce the notion that dynamic
faces are richer, ecologically more valid depictions ofreal-life face representations, enhancing recognition
performance in suboptimal situations.
We also estimated a ceiling point that we definedas the value at which participants reached 99%of their maximum recognition performance andagain determined the quantity of signal necessary
to reach this point. Our findings revealed that moresignal was required to reach the ceiling point in thestatic compared with the dynamic condition. Whenconsidering each expression individually, we could
evidence that this was true for anger, happiness, andsurprise. In addition, our data also evidenced that fear
and happiness were the first two expressions to reach
their ceiling points, regardless of the condition. In otherwords, less signal was required by the participants toreach their maximum recognition performance for these
two expressions. From an evolutionary perspective, theemotional expression of fear transmits critical signals to
detect dangers and avoid harmful situations. Therefore,
the biological relevance and importance for humansurvival of this expression could explain why observers
reached their maximum recognition performance very
rapidly and needed less signal compared with theother expressions. As for happiness, the very earlyceiling points observed in both conditions might beaccounted for by the high frequency of exposure to thisemotion during everyday life social interactions (Calvo,
Avero, Fernández-Martín, & Recio, 2016), as well as
to the orthogonal muscular activations elicited by thisexpression. This physiological pattern results in a very
effective transmission of diagnostic facial information,
hence leading to an early peak in maximum recognition
accuracy. The dynamic advantage we observed forthe decoding of facial expressions of emotion mightarise from the integration of form and movement, aspreviously shown for face identification (Dobs, Bulthoff,& Schultz, 2016; O’Toole, Roark, & Abdi, 2002). Futurestudies manipulating these facial information properties
are necessary to clarify this question.
Finally, to provide an estimate of the nature ofemotion recognition process (i.e., categorical vs.
continuous), we examined the steepness of increase
in recognition performance across emotions andconditions as a function of signal. To do so, we
extracted the slope of the curve for each expression ineach condition. This procedure allowed us to examine
how quickly the transition happens from very low
to very high recognition accuracy in each condition.
Overall, our data revealed a steeper increase in the
dynamic compared with the static condition, suggesting
that, with an increase of signal, participants quickly
transition to much higher recognition performance withmoving compared with still faces. When considering
each expression separately, we observed a significantly
steeper increase in the dynamic compared with theJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

static condition for anger and surprise. Similar results
were observed for all the other expressions, except for
fear, yet the differences between conditions were notsignificant. This pattern of results posits that humanobservers are more sensitive to signal changes withdynamic compared with static expressions. For thelatter, the increase of visual information more slowly
benefits the participants.
Altogether, this last set of findings provides somenovel insights into the unfolding of static and dynamic
FER as a function of the amount of visual signal
available, thereby clarifying some of the contrasted anddiscrepant results previously reported in the literature.
As observed, the dynamic advantage depends on thevery nature of the diagnostic information available.
Driven by experience and shaped by evolutionary andontogenetic dynamics, the human visual system is
optimally tuned to successfully categorize all dynamic
expressions in suboptimal situations. Our findings
also support previous neurofunctional explanations
suggesting the existence of distinct cortical pathways
for processing static and dynamic face information(Duchaine & Yovel, 2015).

17

Note also that we cannot rule out that the noise
patterns affected the perceptual strategies used byour participants to recognize the facial expressions.
However, as mentioned elsewhere in this article, we
generated the noise using a seed-controlled procedure,so that corresponding static and dynamic stimuli wereassigned with the same noise patterns for identicalsignal levels. As such, if holistic strategies were affectedby the noise patterns added to the stimuli, then theexpressions would have been affected identically instatic and dynamic conditions, given that the same noise
patterns were used in both conditions. Future work will
examine the fixation distribution of our participants toshed further light on facial information use during thistask.
Finally, it would be interesting to examine whether
the emotion-specific dynamic vs. static advantages
observed in the current study generalize to otherdatabases of facial expressions of emotion, populations(Quesque et al., 2022), and cultures (e.g., Caldara, 2017;
Jack, Blais, Scheepers, Schyns, & Caldara, 2009).

Conclusions
Methodological considerations and future
directions
In the current study, we used a computerized
psychophysical paradigm to parametrically andrandomly manipulate the signal of the static anddynamic expressions presented to the observers. We
used uniform and adaptive sampling to determine
the level of signal presented in each trial. Under
uniform sampling conditions, the amount of noise was
randomly sampled for each participant from a uniform
distribution ranging between 0% and 100%. Although
this methodological choice allowed us to examine
the dynamic over static advantage across signal, it
did not allow us to precisely determine participants
perceptual thresholds for an effective recognition of thesix basic expressions in static and dynamic conditions,especially not at the single subject level. Further
studies implementing threshold-seeking algorithms
are necessary to address this question and examinequantitative individual differences in static and dynamic
facial signal use.
In addition, uniform and adaptive sampling
procedures allowed us to limit the number of trials while
evenly sampling the whole space when all observers
were considered together. Yet, with such procedures,
we did not get the same amount of data points acrossall the levels of signal. Therefore, in our future study,
we will address this limitation by defining precisely
specific levels of signal that will be presented to all ourparticipants.

Downloaded from jov.arvojournals.org on 01/11/2024

Prior studies investigating the dynamic advantage forthe recognition of facial expressions of emotion haveyielded contrasted findings, with some suggesting thatdynamic cues provide processing benefits, while otherssuggesting they do not (Fiorentini & Viviani, 2011;
Gold et al., 2013) or only in specific populations (Alves,2013). To further clarify this question, we examinedstatic and dynamic FER across signal (0%–100%)by parametrically and randomly manipulating thequantity of visual information available to theobservers. Our findings revealed that facial movements
provide additional cues for the recognition of all
facial expressions in suboptimal situations (i.e., withlow signal), allowing observers to disentangle theambiguity triggered by insufficient, lacking, or degraded
information. In contrast, in optimal situations, thevisual system of healthy young observers is powerfulenough to efficiently categorize static emotional
expressions, with dynamic faces enhancing recognition
performance only for happiness and surprise with 100%of signal. By extracting the slope of the curves, ouranalyses allowed us to further estimate the steepnessof increase in recognition performance as a functionof signal for each expression in each condition. We
also examined the quantity of signal necessary tosurpass chance level and to reach a ceiling point inrecognition performance. Overall, we observed thesteepest accuracy increment in the dynamic condition;less signal was also required to surpass chance level andto reach a ceiling point in recognition performance inthis condition. Altogether, our findings confirm theJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

existence of a dynamic advantage for the recognitionof facial expressions of emotion, but also evidence
that this advantage depends on the very nature of thevisual information available. In line with evolutionary
and ontogenetic perspectives as well as neurofunctional
explanations, dynamic signals are more effective andsensitive than static inputs to reliably categorize facial
expressions of emotion for all human observers. Our
study increases our understanding of the recognitionof static and dynamic expressions and offers a new
approach to precisely map FER deficits in specificpopulations.
Keywords: facial expressions of emotion, static,
dynamic, parametric noise, psychophysics

Acknowledgments
The authors express their deepest gratitude to Jason
Gold and his colleagues from Indiana University,
Bloomington, for sharing with us the stimuli used inthis study. We also thank all our participants, as wellas Marie Chardonnens, for her previous help withtesting.
Supported by the grant no 10001C_201145 from the
Swiss National Science Foundation awarded to RC andARR as project partners.
Commercial relationships: none.
Corresponding author: Anne-Raphaëlle Richoz.
Email: roberto.caldara@unifr.ch.
Address: Eye and Brain Mapping Laboratory(iBMLab), Department of Psychology, University of
Fribourg, Rue P.-A.-de-Faucigny 2, Fribourg 1700,
Switzerland.

References
Adolphs, R., Tranel, D., & Damasio, A. R. (2003).
Dissociable neural systems for recognizing
emotions. Brain and Cognition, 52(1), 61–69,
https://doi.org/10.1016/S0278-2626(03)00009-5.
Adolphs, R. (2008). Fear, faces, and the human
amygdala. Current Opinion in Neurobiology, 18(2),
166–172, https://doi.org/10.1016/j.conb.2008.06.
006.
Alves, N. T. (2013). Recognition of static and dynamic
facial expressions: A study review. Estudos dePsicologia, 18(1), 125–130.
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005).
Deciphering the enigmatic face: The importanceof facial dynamics in interpreting subtle facial

Downloaded from jov.arvojournals.org on 01/11/202418

expressions. Psychological Science, 16(5), 403–410,
https://doi.org/10.1111/j.0956-7976.2005.01548.x.
Atkinson, A. P., Dittrich, W. H., Gemmell, A. J., &Young, A. W. (2004). Emotion perception fromdynamic and static body expressions in point-lightand full-light displays. Perception, 33(6), 717–746,
http://www.ncbi.nlm.nih.gov/pubmed/15330366.
Bayet, L., Quinn, P. C., Laboissière, R., Caldara, R.,
Lee, K., & Pascalis, O. (2017). Fearful but not happy
expressions boost face detection in human infants.
Proceedings of the Royal Society B, 284(1862), 1–9,
https://doi.org/10.1098/rspb.2017.1054.
Bernstein, M., Erez, Y., Blank, I., & Yovel, G. (2018).
An integrated neural framework for dynamic andstatic face processing. Scientific Reports, 8(7036),
1–10, https://doi.org/:10.1038/s41598-018-25405-9.
Bould, E., & Morris, N. (2008). Role of motion signals
in recognizing subtle facial expressions of emotion.
British Journal of Psychology, 99(2), 167–189,
https://doi.org/10.1348/000712607X206702.
Bould, E., Morris, N., & Wink, B. (2008). Recognising
subtle emotional expressions: The role of facial
movements. Cognition and Emotion, 22(8), 1569–
1587, https://doi.org/10.1080/02699930801921156.
Brainard, D. H., & Vision, S. (1997). The psychophysics
toolbox. Spatial Vision, 10, 433–436.
Caldara, R. (2017). Culture reveals a flexible
system for face processing. Current Directionsin Psychological Science, 26(3), 249–255,
https://doi.org/10.1177/0963721417710036.
Calvo, M. G., Avero, P., Fernández-Martín, A., &Recio, G. (2016). Recognition thresholds for staticand dynamic emotional faces. Emotion, 16(8),1186–1200, https://psycnet.apa.org/doi/10.1037/
emo0000192.
Cosker, D., Krumhuber, E., & Hilton, A. (2010).
Perception of linear and nonlinear motion
properties using a FACS validated 3D facialmodel. In Proceedings of the 7th Symposium on
Applied Perception in Graphics and Visualization(pp. 101–108). Los Angeles, California,
http://dx.doi.org/10.1145/1836248.1836268.
Cunningham, D. W., & Wallraven, C. (2009). Dynamic
information for the recognition of conversational
expressions. Journal of Vision, 9(13), 1–17,
https://doi.org/10.1167/9.13.7.
Cunningham, D. W., & Wallraven, C. (2009).
The interaction between motion and formin expression recognition. Proceedings of
the 6th Symposium on Applied Perception inGraphics and Visualization, 41–44. Crete, Greece,
https://doi.org/10.1145/1620993.1621002.
Dakin, S., Hess, R., Ledgeway, T., & Achtman,
R. (2002). What causes non-monotonic

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

tuning of fMRI response to noisy images?
Current Biology, 12(14), R476–R477, https://doi.org/10.1016/s0960-9822(02)00960-0.
Dobs, K., Bulthoff, I., & Schultz, J. (2016). Identity
information content depends on the type of
facial movement. Scientific Reports, 6, 34301,
https://doi.org/10.1038/srep34301.
Dobs, K., Bulthoff, I., & Schultz, J. (2018). Useand usefulness of dynamic face stimuli for faceperception studies-a review of behavioral findings
and methodology. Frontiers in Psychology, 9, 1355,
https://doi.org/10.3389/fpsyg.2018.01355.
Duchaine, B., & Yovel, G. (2015). A revisedneural framework for face processing. Annual
Review of Vision Science, 1, 393–416, https://doi.org/10.1146/annurev-vision-082114-035518.
Dzhelyova, M., Jacques, C., & Rossion, B. (2017). Ata single glance: Fast periodic visual stimulation
uncovers the spatio-temporal dynamics ofbrief facial expression changes in the humanbrain. Cerebral Cortex, 27(8), 4106–4123,
https://doi.org/10.1093/cercor/bhw223.
Ehrlich, S. M., Schiano, D. J., & Sheridan, K. (2000,
April 1–6). Communicating facial affect: It’s not
the realism, it’s the motion. Proceedings of ACM
CHI 2000 Extended Abstracts on Human Factors
in Computing Systems, 252–253. New York, New
York, https://doi.org/10.1145/633292.633439.
Ekman, P. (2003). Emotions revealed: Understanding
faces and feelings to improve communication andemotional life. New York: Times Books/Henry Holtand Co.
Ekman, P., & Friesen, W. V. (1976). Measuring
facial movement. Environmental Psychology
and Nonverbal Behavior, 1(1), 56–75, https://doi.org/10.1007/BF01115465.
Ekman, P., & Friesen, W. V. (1976). Pictures of facial
affect. Palo Alto: Consulting Psychologists Press.
Ekman, P., & Friesen, W. V. (1978). Facial action coding
system. Palo Alto: Consulting Psychologists Press,
https://psycnet.apa.org/doi/10.1037/t27734-000.
Fiorentini, C., & Viviani, P. (2011). Is there a dynamic
advantage for facial expressions? Journal of Vision,
11(3), 1–15, https://doi.org/10.1167/11.3.17.
Fiset, D., Blais, C., Royer, J., Richoz, A.-R.,
Dugas, G., & Caldara, R. (2017). Mapping theimpairment in decoding static facial expressionsof emotion in prosopagnosia. Social Cognitive
and Affective Neuroscience, 12(8), 1334–1341,
https://doi.org/10.1093/scan/nsx068.
Fox, C. J., Iaria, G., & Barton, J. J. (2009). Defining
the face processing network: Optimizationof the functional localizer in fMRI. Human

Downloaded from jov.arvojournals.org on 01/11/2024

19

Brain Mapping, 30(5), 1637–1651, https://doi.org/10.1002/hbm.20630.
Furl, N., Henson, R. N., Friston, K. J., & Calder,
A. J. (2013). Top-down control of visual
responses to fear by the amygdala. Journalof Neuroscience, 33(44), 17435–17443, https://doi.org/10.1523/JNEUROSCI.2992-13.2013.
Ganel, T., Valyear, K. F., Goshen-Gottstein, Y., &Goodale, M. A. (2005). The involvement of the“fusiform face area” in processing facial expression.
Neuropsychologia, 43(11), 1645–1654, https://doi.org/10.1016/j.neuropsychologia.2005.01.012.
Geangu, E., Ichikawa, H., Lao, J., Kanazawa, S.,
Yamaguchi, M. K., Caldara, R., . . . Turati, C.
(2016). Culture shapes 7-month-olds’ perceptual
strategies in discriminating facial expressions ofemotion. Current Biology, 26(14), R663–R664,
https://doi.org/10.1016/j.cub.2016.05.072.
Gold, J. M., Barker, J. D., Barr, S., Bittner, J. L.,
Bromfield, W. D., Chu, N., . . . Srinath, A. (2013).
The efficiency of dynamic and static facial
expression recognition. Journal of Vision, 13(5),1–12, https://doi.org/10.1167%2F13.5.23.
Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000).
The distributed human neural system for faceperception. Trends in Cognitive Science, 4(6),223–233, https://doi.org/10.1016/s1364-6613(00)01482-0.
Humphreys, G. W., Donnelly, N., & Riddoch, M. J.
(1993). Expression is computed separately fromfacial identity, and it is computed separately formoving and static faces: Neuropsychologicalevidence. Neuropsychologia, 31(2), 173–181,
http://www.ncbi.nlm.nih.gov/pubmed/8455786.
Jack, R. E., Blais, C., Scheepers, C., Schyns, P.
G., & Caldara, R. (2009). Cultural confusions
show that facial expressions are not universal.
Current Biology, 19(18), 1543–1548, https://doi.org/10.1016/j.cub.2009.07.051.
Jack, R. E., Caldara, R., & Schyns, P. G. (2012).
Internal representations reveal cultural diversity
in expectations of facial expressions of emotion.
Journal of Experimental Psychology: General,141(1), 19–25, http://dx.doi.org/10.1037/a0023463.
supp.
Jack, R. E., Garrod, O. G., & Schyns, P. G.
(2014). Dynamic facial expressions of emotion
transmit an evolving hierarchy of signals
over time. Current Biology, 24(2), 187–192,
https://doi.org/10.1016/j.cub.2013.11.064.
Jack, R. E., Garrod, O. G., Yu, H., Caldara, R.,& Schyns, P. G. (2012). Facial expressions ofemotion are not culturally universal. Proceedingsof the National Academy of Sciences of theJournal of Vision (2024) 24(1):7, 1–22

Richoz et al.

United States of America, 109(19), 7241–7244,
https://doi.org/10.1073/pnas.1200155109.
Jiang, Z., Li, W., Recio, G., Liu, Y., Luo, W., Zhang,D., . . . Sun, D. (2014). Time pressure inhibits
dynamic advantage in the classification of facial
expressions of emotion. PLoS One, 9(6), 1–7,
https://doi.org/10.1371/journal.pone.0100162.
Johnston, P., Mayes, A., Hughes, M., & Young,
A. W. (2013). Brain networks subserving
the evaluation of static and dynamic facial
expressions. Cortex, 49(9), 2462–2472,
https://doi.org/10.1016/j.cortex.2013.01.002.
Kamachi, M., Bruce, V., Mukaida, S., Gyoba,
J., Yoshikawa, S., & Akamatsu, S. (2001).
Dynamic properties influence the perception of
facial expressions. Perception, 30(7), 875–887,
https://doi.org/10.1068/p3131.
Kamachi, M., Bruce, V., Mukaida, S., Gyoba, J.,
Yoshikawa, S., & Akamatsu, S. (2013). Dynamic
properties influence the perception of facial
expressions. Perception, 42(11), 1266–1278,
https://doi.org/10.1068/p3131n.
Kan, Y., Mimura, M., Kamijima, K., & Kawamura,
M. (2004). Recognition of emotion frommoving facial and prosodic stimuli in depressed
patients. Journal of Neurology, Neurosurgery
and Psychiatry, 75(12), 1667–1671, https://doi.org/10.1136/jnnp.2004.036079.
Kätsyri, J. (2006). Human recognition of basic
emotions from posed and animated dynamic facial
expressions (Publication Number B59) [Doctoral
dissertation, Helsinki University of Technology].
Pennsylvania State University, https://citeseerx.ist.
psu.edu/document?repid=rep1&type=pdf&doi=
4a1526ce90532adcf64c7eb755a043a59f7012e2.
Kätsyri, J., & Sams, M. (2008). The effect ofdynamics on identifying basic emotions fromsynthetic and natural faces. International Journalof Human–Computer Studies, 66(4), 233–242,
https://doi.org/10.1016/j.ijhcs.2007.10.001.
Kessler, H., Doyen-Waldecker, C., Hofer, C.,
Hoffmann, H., Traue, H. C., & Abler, B.
(2011). Neural correlates of the perceptionof dynamic versus static facial expressionsof emotion. Psycho-Social Medicine, 8, 1–8,
https://doi.org/10.3205/psm000072.
Kilts, C. D., Egan, G., Gideon, D. A., Ely, T. D., &Hoffman, J. M. (2003). Dissociable neural pathways
are involved in the recognition of emotion in staticand dynamic facial expressions. NeuroImage, 18(1),156–168, https://doi.org/10.1006/nimg.2002.1323.
Kleiner, M., Brainard, D., Pelli, D., Ingling, A.,
Murray, R., & Broussard, C. (2007). What’s new in
Psychtoolbox-3. Perception, 36(14), 1–16.

Downloaded from jov.arvojournals.org on 01/11/2024

20

Korolkova, O. A. (2018). The role of temporal
inversion in the perception of realistic andmorphed dynamic transitions between facial
expressions. Vision Research, 143, 42–51,
https://doi.org/10.1016/j.visres.2017.10.007.
Krumhuber, E. G., Kappas, A., & Manstead, A.
S. (2013). Effects of dynamic aspects of facial
expressions: A review. Emotion Review, 5(1), 41–46,
https://doi.org/10.1177/1754073912451349.
LaBar, K. S., Crupain, M. J., Voyvodic, J. T., &McCarthy, G. (2003). Dynamic perceptionof facial affect and identity in the humanbrain. Cerebral Cortex, 13(10), 1023–1033,
https://doi.org/10.1093/cercor/13.10.1023.
Liang, Y., Liu, B., Xu, J., Zhang, G., Li, X., Wang, P.,. . . Wang, B. (2017). Decoding facial expressions
based on face-selective and motion-sensitive
areas. Human Brain Mapping, 38(6), 3113–3125,
https://doi.org/10.1002/hbm.23578.
Liu, M., Duan, Y., Ince, R. A., Chen, C.,
Garrod, O. G., Schyns, P. G., . . . Jack, R. E.
(2022). Facial expressions elicit multiplexed
perceptions of emotion categories anddimensions. Current Biology, 32(1), 200–209,
https://doi.org/10.1016/j.cub.2021.10.035.
MATLAB. (2014), 8.4 (R2014b). The MathWorks Inc.
https://ch.mathworks.com/products/matlab.html.
Namba, S., Kabir, R. S., Miyatani, M., & Nakao, T.
(2018). Dynamic displays enhance the ability todiscriminate genuine and posed facial expressionsof emotion. Frontiers in Psychology, 9(672), 1–12,
https://doi.org/10.3389/fpsyg.2018.00672.
O’Toole, A. J., Roark, D. A., & Abdi, H. (2002).
Recognizing moving faces: A psychological andneural synthesis. Trends in Cognitive Science, 6(6),261–266, https://doi.org/10.1016/s1364-6613(02)01908-3.
Paulmann, S., Jessen, S., & Kotz, S. A. (2009).
Investigating the multimodal nature ofhuman communication: An ERP study.
Journal of Psychophysiology, 23(2), 63–76,
http://dx.doi.org/10.1027/0269-8803.23.2.63.
Pitcher, D., Dilks, D. D., Saxe, R. R., Triantafyllou, C.,& Kanwisher, N. (2011). Differential selectivity for
dynamic versus static information in face-selective
cortical regions. NeuroImage, 56(4), 2356–2363,
https://doi.org/10.1016/j.neuroimage.2011.
03.067.
Plouffe-Demers, M.-P., Fiset, D., Saumure, C., Duncan,J., & Blais, C. (2019). Strategy shift toward lower
spatial frequencies in the recognition of dynamic
facial expressions of basic emotions: When it moves
it is different. Frontiers in Psychology, 10(1563),
1–13, https://doi.org/10.3389/fpsyg.2019.01563.

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

Poncet, F., Baudouin, J.-Y., Dzhelyova, M. P., Rossion,
B., & Leleu, A. (2019). Rapid and automatic
discrimination between facial expressions in thehuman brain. Neuropsychologia, 129, 47–55, https://doi.org/10.1016/j.neuropsychologia.2019.03.006.
Poncet, F., Soussignan, R., Jaffiol, M., Gaudelus,
B., Leleu, A., Demily, C., . . . Baudouin, J.-Y.
(2021). The spatial distribution of eye movements
predicts the (false) recognition of emotional
facial expressions. PLoS One, 16(1), 1–24,
https://doi.org/10.1371/journal.pone.0245777.
Quesque, F., Coutrot, A., Cox, S., de Souza, L. C.,
Baez, S., Cardona, J. F., . . . Bertoux, M. (2022).
Does culture shape our understanding of others’
thoughts and emotions? An investigation across
12 countries. Neuropsychology, 36(7), 664–682,
https://doi.org/10.1037/neu0000817.
Rainer, G., Augath, M., Trinath, T., & Logothetis,
N. K. (2001). Nonmonotonic noise tuningof BOLD fMRI signal to natural images
in the visual cortex of the anesthetizedmonkey. Current Biology, 11(11), 846–854,
https://doi.org/10.1016/S0960-9822(01)00242-1.
Recio, G., Schacht, A., & Sommer, W. (2013).
Classification of dynamic facial expressions ofemotion presented briefly. Cognition & Emotion,
27(8), 1486–1494, https://doi.org/10.1080/02699931.
2013.794128.
Richoz, A.-R., Jack, R. E., Garrod, O. G., Schyns,
P. G., & Caldara, R. (2015). Reconstructing
dynamic mental models of facial expressions in
prosopagnosia reveals distinct representations
for identity and expression. Cortex, 65, 50–64,
https://doi.org/10.1016/j.cortex.2014.11.015.
Richoz, A. R., Lao, J., Ceroni, M., Sacco, L., Pignatti,
R., & Caldara, R. (2018a). Investigating therecognition of static and dynamic facial expressionsof emotion in MCI patients. Journal of Vision,
18(10), 926–926, https://doi.org/10.1167/18.10.926.
Richoz, A.-R., Lao, J., Pascalis, O., & Caldara, R.
(2018b). Tracking the recognition of static anddynamic facial expressions of emotion acrossthe life span. Journal of Vision, 18(9), 1–27,
https://doi.org/10.1167/18.9.5.
Rodger, H., Lao, J., & Caldara, R. (2018).
Quantifying facial expression signal andintensity use during development. Journal of
Experimental Child Psychology, 174, 41–59,
https://doi.org/10.1016/j.jecp.2018.05.005.
Rodger, H., Lao, J., Stoll, C., Richoz, A.-R.,
Pascalis, O., Dye, M., . . . Caldara, R. (2021). The
recognition of facial expressions of emotion indeaf and hearing individuals. Heliyon, 7(5), 1–10,
https://doi.org/10.1016/j.heliyon.2021.e07018.

Downloaded from jov.arvojournals.org on 01/11/2024

21

Rodger, H., Vizioli, L., Ouyang, X., & Caldara,
R. (2015). Mapping the development of facial
expression recognition. Developmental Science,18(6), 926–939, https://doi.org/10.1111/desc.12281.
Rousselet, G. A., Pernet, C. R., Bennett, P. J., &Sekuler, A. B. (2008). Parametric study of
EEG sensitivity to phase noise during faceprocessing. BMC Neuroscience, 9(98), 1–22,
https://doi.org/10.1186/1471-2202-9-98.
Sato, W., Kochiyama, T., Yoshikawa, S., Naito,
E., & Matsumura, M. (2004). Enhanced
neural activity in response to dynamic facial
expressions of emotion: An fMRI study.
Cognitive Brain Research, 20(1), 81–91,
https://doi.org/10.1016/j.cogbrainres.2004.01.008.
Schaefer, K. L., Baumann, J., Rich, B. A., Luckenbaugh,
D. A., & Zarate, C. A., Jr. (2010). Perceptionof facial emotion in adults with bipolar or
unipolar depression and controls. Journalof Psychiatric Research, 44(16), 1229–1235,
https://doi.org/10.1016/j.jpsychires.2010.04.024.
Schultz, J., & Pilz, K. S. (2009). Natural facial
motion enhances cortical responses to faces.
Experimental Brain Research, 194(3), 465–475,
https://doi.org/10.1007/s00221-009-1721-9.
Schyns, P. G., Petro, L. S., & Smith, M. L. (2009).
Transmission of facial expressions of emotion
co-evolved with their efficient decoding in thebrain: Behavioral and brain evidence. PLoS One,
4(5), 1–16, https://doi.org/10.1371/journal.pone.
0005625.
Tian, J., Wang, J., Xia, T., Zhao, W., Xu, Q.,& He, W. (2018). The influence of spatial
frequency content on facial expression processing:
An ERP study using rapid serial visualpresentation. Scientific Reports, 8(2383), 1–8,
https://doi.org/10.1038/s41598-018-20467-1.
Trautmann, S. A., Fehr, T., & Herrmann, M.
(2009). Emotions in motion: Dynamic compared
to static facial expressions of disgust andhappiness reveal more widespread emotion-specificactivations. Brain Research, 1284, 100–115,
https://doi.org/10.1016/j.brainres.2009.05.075.
Vrticka, P., Lordier, L., Bediou, B., & Sander,
D. (2014). Human amygdala response to
dynamic facial expressions of positive andnegative surprise. Emotion, 14(1), 161,https://doi.org/10.1037/a0034619.
Wallraven, C., Breidt, M., Cunningham, D.
W., & Bülthoff, H. H. (2008). Evaluating
the perceptual realism of animated facial
expressions. ACM Transactions on Applied
Perception, 4, 1–20, https://doi.org/http://dx.doi.org/10.1145/1278760.1278764.

Journal of Vision (2024) 24(1):7, 1–22

Richoz et al.

Widen, S. C., & Russell, J. A. (2015). Do dynamic
facial expressions convey emotions to children
better than do static ones? Journal of Cognition andDevelopment, 16(5), 802–811.
Willenbockel, V., Sadr, J., Fiset, D., Horne, G. O.,
Gosselin, F., & Tanaka, J. W. (2010). Controlling
low-level image properties: The SHINE toolbox.
Behavior Research Methods, 42(3), 671–684,
https://doi.org/10.3758/BRM.42.3.671.
Wyssen, A., Lao, J., Rodger, H., Humbel, N.,
Lennertz, J., Schuck, K., . . . Whinyates, K.
(2019). Facial emotion recognition abilities
in women experiencing eating disorders.

Downloaded from jov.arvojournals.org on 01/11/2024

22

Psychosomatic Medicine, 81(2), 155–164,
https://doi.org/10.1097/psy.0000000000000664.
Yitzhak, N., Gilaie-Dotan, S., & Aviezer, H. (2018). The
contribution of facial dynamics to subtle expression
recognition in typical viewers and developmental
visual agnosia. Neuropsychologia, 117,
26–35, https://doi.org/10.1016/j.neuropsychologia.
2018.04.035.
Zloteanu, M., Krumhuber, E. G., & Richardson,
D. C. (2018). Detecting genuine and deliberate
displays of surprise in static and dynamic
faces. Frontiers in Psychology, 9(1184), 1–9,
https://doi.org/10.3389/fpsyg.2018.01184.