Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017rspb.royalsocietypublishing.org

Fearful but not happy expressions boost
face detection in human infants
Laurie Bayet1,2, Paul C. Quinn3, Rafael Laboissière1,2, Roberto Caldara4,
Kang Lee5 and Olivier Pascalis1,2

Research
Cite this article: Bayet L, Quinn PC,
Laboissière R, Caldara R, Lee K, Pascalis O.
2017 Fearful but not happy expressions boost
face detection in human infants. Proc. R. Soc. B
284: 20171054.
http://dx.doi.org/10.1098/rspb.2017.1054

Received: 12 May 2017
Accepted: 28 July 2017

Subject Category:
Behaviour
Subject Areas:
behaviour, cognition
Keywords:
infant, face, emotion, fear, detection, noise

Author for correspondence:
Laurie Bayete-mail: laurie.bayet@childrens.harvard.edu

Electronic supplementary material is available
online at https://dx.doi.org/10.6084/m9.
figshare.c.3852118.

1

LPNC, Université Grenoble-Alpes, 38000 Grenoble, France
LPNC, CNRS, 38000 Grenoble, France
3
Department of Psychological and Brain Sciences, University of Delaware, Newark, DE, USA
4
Eye and Brain Mapping Laboratory (iBMLab), Department of Psychology, University of Fribourg, Fribourg,
Switzerland
5
Dr. Eric Jackman Institute of Child Study, University of Toronto, Toronto, ON, Canada
2

LB, 0000-0002-0323-7949; RL, 0000-0002-2180-9250
Human adults show an attentional bias towards fearful faces, an adaptive
behaviour that relies on amygdala function. This attentional bias emerges
in infancy between 5 and 7 months, but the underlying developmental
mechanism is unknown. To examine possible precursors, we investigatedwhether 3.5-, 6- and 12-month-old infants show facilitated detection of fearfulfaces in noise, compared to happy faces. Happy or fearful faces, mixed withnoise, were presented to infants (N ¼ 192), paired with pure noise. We
applied multivariate pattern analyses to several measures of infant looking
behaviour to derive a criterion-free, continuous measure of face detection
evidence in each trial. Analyses of the resulting psychometric curves supported the hypothesis of a detection advantage for fearful faces comparedto happy faces, from 3.5 months of age and across all age groups. Overall,our data show a readiness to detect fearful faces (compared to happy
faces) in younger infants that developmentally precedes the previously
documented attentional bias to fearful faces in older infants and adults.

1. Introduction
Humans from all cultures display distinct facial expressions signalling different
emotions [1], a behaviour that engages complex facial musculature and iscommon to all mammals, particularly primates [2 –4]. Perceiving these
emotionally, socially and ecologically salient stimuli [5] engages a remarkably
large number of subcortical and cortical pathways in the human brain [6].
Numerous disorders and atypical experiences disrupt facial emotion perception, including autism [7], neglect or abuse [8], institutional rearing [9],callous-unemotional traits [10], and psychiatric disorders [11]. Perception of
fearful faces is of particular interest, characterized by enhanced attentional allocation [12], visual search [13], categorization [14] and contrast sensitivity [15],reflecting the upregulation of perceptual cortices by amygdala nuclei[6,16,17]. These crucial pathways, which underlie the perception of faces andtheir expressions, could be present from birth [17]. Tracking the developmental
trajectories leading to adaptive facial emotion perception is critical to determine
how disorders affect such capacity [18]. However, it is unknown whether
fearful faces are better detected in infancy.
The ability to categorize certain facial emotions emerges in infants between
5 and 7 months [18– 21], along with an attentional bias to fearful faces and eyes,
i.e. a tendency for these stimuli to hold attention [18,22 –27]. Infants as young as
5 months rapidly detect ecologically relevant stimuli such as angry faces or
snakes [28 –32], and fearful faces may be considered ecologically relevant[33]. We hypothesized that fearful faces could be preferentially detected ininfancy, possibly before the onset of an attentional bias to fear depending on
task demands and with visual inputs designed to parametrically probe the& 2017 The Author(s) Published by the Royal Society. All rights reserved.

Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017

olds, six 6-month-olds, eight 12-month-olds) were excluded
due to fussiness (nine 3.5-month-olds, two 6-month-olds, eight
12-month-olds), technical failure (one 3.5-month-old, one 6month-old), experimenter error (three 3.5-month-olds, three
6-month-olds), or side-bias on at least three of the six trials(three 3.5-month-olds). Side-bias in a given 10-s trial was definedas the infant looking to the same side of the screen more than
95% of the time. All caregivers provided written, informed consent before the experiment, which was approved by the local
ethics committee (Institutional Review Board).

Happy and fearful frontal view faces from the same 12 models(six males, six females) were selected from the Karolinska
Directed Emotional Faces database [51]. Stimuli were greyscaled, and external features were cropped. Luminance, contrast,
spatial frequencies, and eye placement were matched in SHINE[52] and Psychomorph [53]. Faces subtended a visual angle ofabout 18 (vertically) by 12 degrees (horizontally).
Weighted mean phase noise preserving global contrast andfrequency spectrum [34] was generated in MATLAB v. 7.9.0.529,
and stimuli were gamma-corrected ( g ¼ 1.7286). These stimuli
were similar to those previously validated with children in an
emotion labelling task [35]. Faces were mixed with 0, 20, 40, 50,
60 or 70% noise, i.e. they had 30, 40, 50, 60, 80 or 100% signal.
Stimuli were generated so that half had more visible eyes thanmouth (eyeþ), and half had less visible eyes than mouth (eye2)
as measured by peak signal to noise ratio (PSNR). Signal in theeye region was significantly greater in eyeþ than eye2 stimuli(PSNR, p , 0.001; Structural SIMilarity [SSIM], p ¼ 0.031; Wanget al. [54]; electronic supplementary material, table S1). Examples
are presented in figure 1a,b and electronic supplementary material,
figure S1; average stimuli are presented in electronic supplementary material, figure S2. There was no effect of emotion on signal
quantity in the stimuli (PSNR, or SSIM in the face or eye region,
as a main effect or in interaction with eye visibility, all ps . 0.5).

(c) Procedure
The infants sat on their caregiver’s lap about 60 cm from the screen.
The caregiver was instructed not to interact with the infant during
the experiment. Stimuli were presented using Psychtoolbox [55].
Each infant saw six trials consisting of a face paired with matchedvisual noise (figure 1a). Trials lasted 10 s from the first look. Faces
were happy or fearful, male or female, counterbalanced across participants. Each of the trials featured a different face model andsignal level (30, 40, 50, 60, 80 or 100%, figure 1b). Faces were on
the left (or right) for half of the trials. Models, signal levels andside were randomly ordered across trials. Faces were sampled sothat, across participants, half of the faces in a given conditionand signal level had more signal in the eye than mouth region(eyeþ) and vice versa (eye2).

(d) Data analysis(i) Pre-processing

2. Method(a) Participants
Sixty-four 3.5-month-olds (31 girls, mean age 116.9 + 0.6 days,
s.e.m.), 64 6-month-olds (31 girls, mean age 191.0 + 0.8 days),and 64 12-month-olds (32 girls, mean age 375.6 + 0.7 days)
from a predominantly Caucasian environment were includedin the study. All infants were born full term (38.8 + 0.1 weeks
of amenorrhoea). Thirty additional infants (sixteen 3.5-month-

Infant looking was recorded by a camera and coded offline with
40 ms precision. A subsample was coded by a second observerwith 0.98, 0.96 and 0.96 agreement in the 3.5-, 6- and 12month-old groups, respectively (Pearson’s r, 25% of thevideos). Percentages of total looking time (PTLTs) to the left
and right sides were derived relative to total time looking at
the screen during each trial. For each condition, age group, andsignal level, trials with PTLT further than 2 s.d. from the mean
were considered outliers and excluded (4.17%, 2.60% and4.43% of the trials for 3.5-, 6- and 12-month-olds, respectively).

Proc. R. Soc. B 284: 20171054(b) Stimuli

2

rspb.royalsocietypublishing.org

sensitivity of the face (emotional) system. To this end, we
designed a degraded face detection task with very low task
demands.
We presented fearful or happy faces mixed with random
noise [34,35] and paired them with pure noise in a faceversus-noise detection task to 192 infants at 3.5, 6 and 12
months of age. These three age groups represent ages
before, during and after the documented developmental
onset of an attentional bias to fearful faces between 5 and7 months. We hypothesized that fearful faces would be
more easily detected by infants than happy faces at thesame level of signal at 6 and 12 months. In addition, we
aimed to test whether 3.5-month-olds would also detect fearful faces preferentially, or conversely detect happy faces
preferentially given their familiarity [36] and visual preference
for this expression [37].
Global contrast and amplitude spectra were equated toprevent those cues from supporting face detection [38,39].
We used phasic noise, preserving these visual properties[34]. Eye visibility was included as a control variable, due
to the critical role of the eyes in perceiving fearful expressions[16,40] and faces in general [41]. Eyes may be especially
important for fearful face detection by infants who lackexperience with this expression [36].
Face detection was measured by preferential looking andmultivariate pattern analysis (MVPA). MVPA combines multiple variables (such as voxel activations) to classify trials
according to stimulus category, target location, or otherparameters [42]. MVPA is used with different data types
including fMRI [43], EEG [44], fNIRS [45] and behavioural[46] data. One previous study used MVPA with infant
fNIRS [45], and at least one used MVPA with infant EEG
data [47]. To our knowledge, the current findings represent
the first use of MVPA with infant behavioural data.
In the current study, we used MVPA to detect patterns oflooking behaviour that differentiate between face and noise,and derived multivariate measures of face versus noise discrimination for each trial. By doing so, we quantitatively
assessed how well the looking behaviour of infants differentiated between face and noise without requiring a humanobserver to guess the location of the face (as is the case forforced-choice preferential looking [48]), and without
making an assumption on which particular aspect of looking
behaviour would best capture the difference between faceand noise. Preferential looking and multivariate face versus
noise discrimination were used to fit psychometric curves
and estimate face detection thresholds. Psychometric curves
have been used in studies of children and adults to investigate perceptual development [49] or the effect of emotion
on perception [15], and there are some precedents to using
them in infant behavioural studies [50].

Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017(a)3

rspb.royalsocietypublishing.org
Proc. R. Soc. B 284: 20171054(b)

30% signal

40% signal

50% signal

60% signal

80% signal

100% signal

Figure 1. Stimuli. (a) Example trial featuring a fearful face at 100% signal (left) paired with matched pure noise (right). (b) Example stimulus at 30 – 100% signal
level. Original stimulus was distributed as part of the KDEF database [51], whose copyright holder is: Karolinska Institute, Department of Clinical Neuroscience,
Section of Psychology, Stockholm, Sweden.

(ii) Multivariate measure of face detection(iii) Modelling of psychometric curves

A multivariate measure of face detection was derived by classifying trials as ‘face is on the left’ or ‘face is on the right’. The
rationale for this metric is similar to the idea of ‘double psychophysics’ [48]: if one can reliably guess on which side of the screen
the face was presented by looking at the infant’s behaviour, thenit can be inferred that the infant is discriminating between thepresence of a face or noise. We implemented this idea computationally using MVPA [42] to locate the side of presentation of theface (left or right) on each trial based on (i) PTLT to the left,(ii) number of looks to the left, (iii) number of looks to the right,(iv) duration of first look to the left, (v) duration of first look to
the right, (vi) median duration of looks to the left, (vii) median
duration of looks to the right, and (viii) direction of first look(left or right). Durations were log-transformed [56]. Continuous
measures were z-scored within-subject. Measures were chosena priori given the visual preference of infants for faces [57].
PTLT to the right is equal to 100% minus PTLT to the left, andthus did not need to be included. Trials from all participants
were pooled to maximize the number of training examples, anda logistic regression algorithm (a common classifier for MVPA)
was repeatedly trained on all trials except one and tested on thetrial that was left-out (leave-one-out cross-validation). Forward
sequential feature selection was implemented inside each crossvalidation loop (see electronic supplementary material, table S2
for results on the full dataset). This procedure led to locating
the face side for each trial in a way that reflects generalization.
We used logistic regression because it provides log-odds, a
direct, criterion-free, continuous measure of evidence for each
response (‘face is on the left’ versus ‘face is on the right’)—as
opposed to accuracy, a binary measure dependent on a decision
criterion. Raw evidence (log-odds for the right versus left side)
was pooled to derive correct evidence (log-odds for the correct
versus incorrect side) as a multivariate measure of face versus
noise discrimination.

Infant psychometric curves exhibited a positive asymptote well
below the level of maximal response. For example, visual preferences for the face side presented an average of about 75% at the
maximal level of signal (100%), even though on a given trial
infants could theoretically look more than 90% of the time tothe face side. This characteristic violates an assumption ofusual psychometric models, where performance approaches
100% at the maximal level of signal. To obviate the need forthis assumption, we used a nonlinear mixed-effects model
approach with the following formula:f ðx, zÞ ¼ Y0 þyðzÞ  Y0
1 þ exp½aðx  uðzÞÞ

where f(x, z) is the fitted response (logit-transformed PTLT or correct evidence), x the signal level, a the slope, and z the experimental
condition (emotion and feature visibility). This corresponds to alogistic function with lower (0% signal) and upper (100% signal)
asymptotes whose values are Y0 and y(z), respectively. When x is
equal to the threshold value Q(z), f(x, z) is halfway between Y0
and y(z). Upper asymptote and threshold are assumed to belinearly dependent on the experimental conditions:yðzÞ ¼ YF þ dH ðzÞdYH

uðzÞ ¼ x0FE þ dF ðzÞdM ðzÞdx0FM þ dH ðzÞdE ðzÞdx0HE
þ dH ðzÞdM ðzÞdx0HM
where YF is the upper asymptote in the fearful face condition, dYH
the difference in upper asymptote for the happy face condition,
x0FE the perceptual threshold in the fearful eyeþ face condition,and dx0FM, dx0HE, and dx0HM the difference in threshold for the
fearful eye2, happy eyeþ, and happy eye2 face conditions,
respectively. dH(z), dF(z), dM(z) and dE(z) are binary variables
equal to 1 in the happy, fearful, eye2 and eyeþ conditions,
respectively, and equal to 0 otherwise. Because eye and mouthDownloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017(iv) Software

3. Results(a) Univariate and multivariate measuresof face detection
Visual preference for faces over juxtaposed noise has beenused in earlier studies [64] as a univariate proxy for facedetection because the visual preference for faces over noise
in infants should be strong [57]. As expected, infants in all
age groups showed a visual preference for the face side that
increased with face visibility (overall PTLT to the face side:
66.18 + 0.68; figure 2a).
If an infant’s looking behaviour can be used to locate theside of the face (left or right), that is strong evidence that the
infant detected the face [48], regardless of whether the infant
showed a reliable visual preference during the entire trial asmeasured by PTLT. We introduced a multivariate measureof face detection evidence by using a classifier to locate theside of the face (i.e. discriminating face from noise) basedon PTLT and other characteristics of looking behaviour (seeMethod). The side of the face was located from infant looking
behaviour at the level of single trials with 80.92% + 1.22
cross-validation accuracy. Accuracy and correct multivariate
detection evidence (correct log-odds) increased with face visibility as expected (figure 2b,c). Several aspects of looking
behaviour (duration of first look to both sides, median durationof looks to both sides) significantly contributed to accurately
locating face side besides PTLT (electronic supplementarymaterial, table S2), confirming that the multivariate approachprovides increased sensitivity compared to PTLT alone.
Linear mixed-effects models revealed no effect ofinfant gender on PTLT to the face side (logit-transformed,
x2 ½1 ¼ 0:34, p ¼ 0.559) or multivariate discrimination (correct
evidence, x2 ½1 ¼ 0:16, p ¼ 0.687). Data were pooled acrossthis variable in further analyses.

(b) Face detection thresholds
We fitted nonlinear mixed-effects models to infant responses( psychometric curves) to estimate face detection thresholds.
Due to sample-size based limitations in the number of parameters, the analysis was restricted to estimating the facedetection threshold for the Fearful eyeþ condition and differences in detection threshold between this and each of theother conditions (Fearful eye2, Happy eyeþ, Happy eye2),
across all age groups.

(c) Levels of face detection around threshold
A linear mixed-effects model analysis of the (logit-transformed)
PTLT around the detection thresholds (40–50% face signal;
figure 3b,c) revealed a significant main effect of age(x2 ½2 ¼ 9:62, p ¼ 0.008) and an interaction of age with facial
emotion and eye visibility (x2 ½2 ¼ 6:62, p ¼ 0.037) driven bya significant effect of facial emotion (x2 ½1 ¼ 6:33, p ¼ 0.012) at
3.5 months, a marginal interaction of facial emotion and eye visibility (x2 ½1 ¼ 3:1046, p ¼ 0.078) at 3.5 months, and a marginal
effect of eye visibility at 12 months (x2 ½1 ¼ 3:05, p ¼ 0.081).
Correct discrimination evidence increased with age
generally (main effect of age, x2 ½2 ¼ 14:35, p , 0.001;
figure 3e). Based on correct multivariate discrimination
evidence, fearful faces were detected preferentially relative
to happy faces around the threshold (40– 50% signal), regardless of age (linear mixed-effects model: main effect of faceemotion, x2 ½1 ¼ 6:03, p ¼ 0.014; no interaction with age andno other significant effects, all ps . 0.05; figure 3f ).
Overall, the results point to a fearful face detection advantage around threshold (40 –50% signal), compared to happy

4Proc. R. Soc. B 284: 20171054

Analyses were conducted in Matlab 7.9.0529 and R v. 3.2.0.
Mixed-effects model analyses [58] were run in R using nlme
v. 3.1.120 [59], car 2.0.25 [60] and lme4 1.1.7 [61]. A control analysis (electronic supplementary material, figure S3) was performed
with the Computer Vision System Toolbox in Matlab 8.2.0.701,
using its implementation of the Viola – Jones automated faceand eye detection algorithm that is pre-trained on a large
number of upright frontal faces and is sensitive to facial features
and their second-order relations [62]. Data are openly accessible
online at https://doi.org/10.6084/m9.figshare.3439349.v1 [63].

Psychometric curves of the PTLT to the face side revealed
a significantly lower threshold for the Fearful eyeþ face condition (44.41 + 1.98% face signal; electronic supplementarymaterial, table S3; figure 3a) than for the Happy eye2 facecondition (difference: 5.20 + 2.62% face signal, 95% CI[0.001 0.103]), but not the other conditions (electronic supplementary material, table S3; figure 3a), across all age
groups. Subject-dependent random effects were retained forx0FE and YF, and rejected for a based on BIC.
A similar result was found when applying psychometriccurve modelling to the correct multivariate face versus
noise discrimination evidence; the face detection thresholdfor the Fearful eyeþ condition (44.07 + 2.14% face signal;
electronic supplementary material, table S4; figure 3c) was
significantly lower than the detection threshold for the
Happy eye2 condition (increase in threshold: 7.90 + 2.52%
face signal, 95% CI [0.030 0.128]) but not the other conditions(Wald confidence intervals, a ¼ 5%; electronic supplementarymaterial, table S4; figure 3c).
Similar models were used to estimate the difference inthreshold between the Fearful eye2 condition and other conditions, or between the Happy eyeþ condition and otherconditions (electronic supplementary material, tables S5–S8).
Results are summarized in figure 4. Overall, psychometriccurve modelling of infant looking data revealed face detection thresholds at about 44% signal, with an increase ofabout 5% signal in threshold for Happy eye2 condition
compared to the Fearful eyeþ condition, and intermediate
thresholds for Happy eyeþ and Fearful eye2 conditions
depending on whether PTLT alone (figure 4a) or correct
multivariate discrimination evidence (figure 4b) was used
as a measure of face versus noise detection.
To clarify whether these differences in detection
thresholds reflected a main effect of facial expression, amain effect of eye visibility, or an interaction between thetwo, and to test for an effect of age, we conducted further
analyses focused on the linear portion of the psychometriccurves corresponding to trials around the fitted detection
thresholds (40 –50% signal). These trials correspond tolevels of signal where most variations in face detection
performance are expected to occur.

rspb.royalsocietypublishing.org

are equally visible at 100% signal, trials with 100% face signal werediscarded from the fit and the model did not include a difference
in asymptote between the eyeþ and eye2 conditions. We considered subject-dependent random effects for x0FE, a, and YF.
Their inclusion was based on Bayesian information criterion.
residuals were considered to be normally distributed.

Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017

6-month-olds

3.5-month-olds

12-month-olds

5rspb.royalsocietypublishing.org

mean PTLT ± s.e.m. (%)

(a)
90
80

*

70

*

*

*

*
*

*

*

*
*

60

*

*

*

*

to fa
f ce

o fa
f ce

to fa
f ce

to pure noise

to pure noise

to pure noise

30 40 50 60
80
100
face signal level (%)

30 40 50 60
80
100
face signal level (%)

30 40 50 60
80
100
face signal level (%)

3.5-month-olds

6-month-olds

12-month-olds
*
* *

50
40
30
20

mean evidence ± s.e.m.

3

*

2

*

1

*

*

*

*

*

*

73%

correct

correct

correct

incorrect

incorrect

incorrect

30 40 50 60
80
100
face signal level (%)

30 40 50 60
80
100
face signal level (%)

30 40 50 60
80
100
face signal level (%)

3.5-month-olds

6-month-olds

12-month-olds

–2

*

100
80

*
*

*

*

*
*

*

50%
predicted
probabilityof correct
choice

–1

(c)

95%
88%

*

*

*

0

–3

mean accuracy ± s.e.m. (%)

*

*

*

*

*

*

*
*

60
40
20
80
100
30 40 50 60
face signal level (%)

30 40 50 60
80
100
face signal level (%)

30 40 50 60
80
100
face signal level (%)

Figure 2. Face detection across signal levels and age groups. (a) Visual preference for the face side versus the noise side. (b,c) Locating the face side from looking
behaviour. (b) Discrimination evidence (log-odds). (c) Face side classification accuracy. Student’s t-tests against chance, a ¼ 0.05, Holm – Bonferroni corrected.
(Online version in colour.)faces. Results were mixed when considering PTLTs for theface side alone, as the effect of face emotion was restricted
to 3.5-month-olds on this measure. However, correct discrimination evidence, a more comprehensive multivariate
measure inclusive of PTLTs and other aspects of looking
behaviour (e.g. first look) revealed a detection advantage for
fearful faces compared to happy faces across all age groups(i.e. it did not significantly interact with age).

4. Discussion
We conducted a face versus noise detection task with 3.5-, 6and 12-month-old infants, i.e. before, during and after thepreviously described developmental emergence of the attentional bias for fearful faces [18], to test the hypothesis that

infants detect fearful faces better than happy faces. We derived
a multivariate measure of face detection inclusive of visual
preference, modelled the resulting psychometric curves, andexamined whether levels of detection around thresholdvaried according to facial emotion, eye visibility, and age.
Results were generally consistent with our main hypothesis.
There was evidence of a detection advantage for fearful faces
compared to happy faces, across all age groups when considering a multivariate measure, and in 3.5-month-olds only
when considering visual preference alone.

(a) Perceptual biases and threat sensitivity in infancy
An attentional bias to fearful faces emerges in infancy between
5–7 months for static faces [18,25] and 3.5–5 months

Proc. R. Soc. B 284: 20171054(b)

Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017

(b)
80

85

6
*

80
75

70

70

60

60

*

70
65
60
55
50

50

50
40

30 40 50 60
80
face signal (%)

100

40

3.5

eye+
eye+

6
age (months)

eye–
eye–
12

(e)
2.0

3.0

HE+ HE–

FE+

FE–

FE+

FE–

(f)
2.0
*

2.5

1.5

1.5

2.0
1.5

1.0

1.0

0.5

1.0

0.5

0.5

0

30 40 50 60
80
face signal (%)

100

–0.5

eye+
eye+

eye–
eye–

6
age (months)

12

happy
fearful

0
–0.5

40

3.5

0

HE+ HE–

Figure 3. Influence of facial emotion, eye visibility and infant age on face detection. Face detection was measured by (a –c) visual preference for the face side and(d – f ) multivariate face versus noise discrimination (correct evidence). (a,d ) Psychometric curve modelling and estimated face detection thresholds (grey boxes).
(b,c,e,f ) Levels of face detection around threshold, (b,e) at each age and (c,f ) across ages (c,f: Student’s t-tests for independent samples or linear mixed model,
as appropriate, a ¼ 0.05). Eþ, eyeþ, more visible eyes; E2, eye2, less visible eyes; F, fearful; H, happy. (Online version in colour.)for dynamic faces [65] (but see [66]). This effect is robust andhas been replicated over a variety of paradigms (e.g. [67]). Its
underlying mechanism is unclear: the hypothesis that it was
linked to direct experience with fearful faces after the onset ofcrawling was recently refuted [65]. Sensitivity to fearful faces
in younger infants also remains debated [66]; younger infants
sometimes exhibit a visual preference for happy over fearfulfaces [37,68], but some sensitivity to fear has been evidenced
in electrophysiological responses at 3.5 months [69]. Deeplimbic regions remain inaccessible to functional neuroimaging in awake infants [70], and data linking threat
processing in infancy to developmental mechanisms havereceived multiple interpretations. Two hypotheses havebeen proposed to explain the development of fear learning:
early sensitivity [28,71], and functional maturation at 5 – 7months [25]. According to the former hypothesis [71], earlysensitivity to threat-relevant stimuli (e.g. snake shapes)
facilitates fear learning even with limited direct or vicarious
fearful experiences with these stimuli [28]. A readinessto detect fearful faces could presumably scaffold earlyexperiences with these expressions, leading to increased
attention-holding by fearful faces later on. The present resultsalign with this notion, and to our knowledge represent
the first evidence of facilitated fear detection in infancy,
and the first behavioural evidence of fear sensitivity at
3.5 months. The latter hypothesis [22 – 27] suggests a link
between the onset of the attentional fear bias and that of functional connectivity between limbic, visual, and attentional

networks [18]; this hypothesis predicts an absence of fear sensitivity before 5 – 7 months [25]. The present results diverge
from this prediction, but it is also possible that the enhanced
detection of fear and attentional fear bias reflect different
functional connections from limbic to visual and attentional
networks, respectively.

(b) Features driving emotional face detection in infants
Previous studies in adults have uncovered a role for low-levelcues, such as global amplitude spectra, in face detection[38,39]. The present results cannot be driven by global contrast or global amplitude spectra, as these were equatedacross conditions and between face and noise. Applying anestablished computational model of face detection (Viola –Jones [38,62]) to all stimuli revealed comparable levels of
automatic face (or eye) detection by the algorithm for fearful
versus happy faces (electronic supplementary material, figureS3). Thus, the fear detection advantage evidenced herecannot be attributed to a greater objective resemblance ofnoisy fearful faces with a generic face template, at least
when it comes to representational aspects captured by thealgorithm. Further research is needed to determine which
other cues, such as local amplitude or contrast in the eye or
mouth region [38], drive fearful face detection in infants. As
the eyes are a critical part of the face template from
3.5 months onward [41,72], infant perception may be attuned
to low-level characteristics of fearful eyes. The diagnostic

Proc. R. Soc. B 284: 20171054

happy
fearful45(d)mean correct evidence ± s.e.m.

(c)80

rspb.royalsocietypublishing.org

mean face preference ± s.e.m. (%)(a)

Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017(a) visual preference (PTLT) alone
happy, eye+
44 ± 2%

fearful, eye–
46 ± 2%

happy, eye–3 ± 3%

1–2 ± 3%5 ± 3%

45
46
47
48
49
estimated detection threshold (signal %)(b) multivariate face versus noise discrimination evidence
fearful, eye–
44 ± 2%

8 ± 3%

4 ± 3%

happy, eye+
48 ± 2%

1 ± 3%

happy, eye–

4 ± 3%

4 ± 3%
fearful, eye+
44 ± 2%
44

8 ± 3%

45

46
47
48
49
50
estimated detection threshold (signal %)51

52

Figure 4. Summary of estimated detection thresholds, across all ages, for each condition. (a) Detection thresholds based on visual preference alone. Trials aroundthreshold (main text) revealed a detection advantage for fearful faces at 3.5 months. (b) Detection thresholds based on multivariate face versus noise discrimination(correct evidence). Trials around threshold (main text) revealed a detection advantage for fearful faces across all ages. (Online version in colour.)character of the eyes in fearful faces could cause infants
to direct their visual attention to fearful eyes, facilitating
detection of the face [14,73,74]. However, there was noclear evidence for an effect of eye visibility, either alone or
in interaction with facial emotion, although it is possiblethat any such effects may be revealed by a stronger experimental manipulation of eye visibility or larger sample size.
Parts of the data did suggest a possible role of eye visibility
in happy face detection by infants. To our knowledge,
this has not been reported before, but aligns with otherfindings [41].

(c) Revisiting ‘double psychophysics’ with multivariate
pattern analysis
Most behavioural studies in infancy do not present a specifichypothesis regarding which variable should distinguishbetween conditions—the hypothesis is on the difference
itself (but see [75]). Teller’s ‘double psychophysics’ method[48] has been proposed as a solution to this problem, andto our knowledge, the current study is the first to implement
it with MVPA [42,43]. Specifically, we introduced a measureof face versus noise discrimination based on the statistical
relationship between behavioural patterns (e.g. looking preference) and stimulus category (face or noise). This method
is not specific to looking times or to face detection and may
be applied more broadly to different behavioural tasks. For
example, eye-tracking provides richer descriptions of looking
behaviour than looking times (e.g. gaze shifts and scanning
patterns), making the results from MVPA potentially stronger
and more sensitive. Stimulus –response relationships areimportant to the study of cognitive development, but areoften difficult to study in infants [76]. We propose that multivariate analysis of behavioural data represents a significant
step in that direction, as it specifically tests for stimulus –response relationships and obviates the need for a priori
assumptions about which behavioural variable should be
the most informative.

(d) Limitations
Sample sizes in the current study were typical of infant
behavioural research. Numbers of near-threshold trials werelimited by the lack of prior reports of perceptual thresholds
for phasic noise in infancy. Future research should determine
whether the readiness to detect fearful faces (compared to
happy faces) in infancy generalizes to naturalistic settings
beyond the laboratory. Comparing detection of fearful
versus angry versus sad faces will also clarify whether theeffect applies just to threat-relevant stimuli (anger and fear)
or more generally to negative novel expressions (fear,
anger, and sadness) [28].
In conclusion, we used multivariate pattern analysis
applied to infant looking behaviour to estimate face detection
thresholds for fearful and happy faces in 3.5-, 6- and 12month-olds. Results supported the hypothesis of a superior
detection of fearful faces as early as 3.5 months, comparedto happy faces. Further research is needed to determine
which characteristics of fearful faces facilitated their detection, and whether this advantage represents an early
instance in which emotional salience scaffolds social fear
learning.
Authors’ contributions. L.B., R.C. and O.P. conceived and designed theexperiments. R.C. provided code to generate the stimuli. L.B. andProc. R. Soc. B 284: 20171054

44

rspb.royalsocietypublishing.org

6 ± 3%
2–3 ± 3%0–1 ± 3%fearful, eye+
44 ± 2%

7

Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017

Funding. This work was funded by NIH Grant R01 HD-46526 to K.L.,
O.P. and P.C.Q., and a PhD scholarship from the French MESR to L.B.
Acknowledgements. The authors are grateful to the families that took partin the study, and thank Marie Sarremejeanne for technical support.

References
1.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.
18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

categorizing facial expressions. Curr. Biol. 17,
1580 –1585. (doi:10.1016/j.cub.2007.08.048)
Phelps EA, Ling S, Carrasco M. 2006 Emotion
facilitates perception and potentiates the perceptual
benefits of attention. Psychol. Sci. 17, 292–299.
(doi:10.1111/j.1467-9280.2006.01701.x)
Morris JS, DeBonis M, Dolan RJ. 2002 Human
amygdala responses to fearful eyes. Neuroimage 17,
214 –222. (doi:10.1006/nimg.2002.1220)
Johnson MH. 2005 Subcortical face processing. Nat.
Rev. Neurosci. 6, 766– 774. (doi:10.1038/nrn1766)
Leppänen JM, Nelson CA. 2009 Tuning thedeveloping brain to social signals of emotions. Nat.
Rev. Neurosci. 10, 37 –47. (doi:10.1038/nrn2554)
Nelson CA. 1987 The recognition of facial
expressions in the first two years of life:mechanisms of development. Child Dev. 58,
889 –909. (doi:10.2307/1130530)
Quinn PC, Anzures G, Izard CE, Lee K, Pascalis O,
Slater AM, Tanaka JW. 2011 Looking across domains
to understand infant representation of emotion.
Emot. Rev. 3, 197 –206. (doi:10.1177/
1754073910387941)
Geangu E, Ichikaw H, Lao J, Kanazawa S, Yamaguchi
MR, Caldara R, Turati C. 2016 Culture shapes 7month-olds’ perceptual strategies in discriminating
facial expressions of emotion. Curr. Biol. 26, R663 –R664. (doi:10.1016/j.cub.2016.05.072)
Leppänen JM, Peltola MJ, Mantymaa M,
Koivuluoma M, Salminen A, Puura K. 2010 Cardiac
and behavioral evidence for emotional influences on
attention in 7-month-old infants. Int. J. Behav. Dev.
34, 547– 553. (doi:10.1177/0165025410365804)
Peltola MJ, Leppänen JM, Vogel-Farley VK, Hietanen
JK, Nelson CA. 2009 Fearful faces but not fearfuleyes alone delay attention disengagement in 7month-old infants. Emotion 9, 560 –565. (doi:10.
1037/a0015806)
Peltola MJ, Leppänen JM, Palokangas T, Hietanen
JK. 2008 Fearful faces modulate looking duration
and attention disengagement in 7-month-old
infants. Dev. Sci. 11, 60 –68. (doi:10.1111/j.14677687.2007.00659.x)
Leppänen JM, Nelson CA. 2012 Early developmentof fear processing. Curr. Dir. Psychol. Sci. 21,
200 –204. (doi:10.1177/0963721411435841)
Jessen S, Grossmann T. 2014 Unconscious
discrimination of social cues from eye whites in
infants. Proc. Natl Acad. Sci. USA 111, 16 208–
16 213. (doi:10.1073/pnas.1411333111)
Jessen S, Grossmann T. 2015 Neural signatures ofconscious and unconscious emotional faceprocessing in human infants. Cortex 64, 260–270.
(doi:10.1016/j.cortex.2014.11.007)

28. LoBue V, Rakison DH. 2013 What we fear most: adevelopmental advantage for threat-relevant
stimuli. Dev. Rev. 33, 285 –303. (doi:10.1016/j.dr.
2013.07.005)
29. Rakison DH, Derringer J. 2008 Do infants possessan evolved spider-detection mechanism? Cognition
107, 381–393. (doi:10.1016/j.cognition.2007.07.022)
30. LoBue V, Rakison DH, DeLoache JS. 2010 Threat
perception across the life span: evidence for multiple
converging pathways. Curr. Dir. Psychol. Sci. 19,
375–379. (doi:10.1177/0963721410388801)
31. DeLoache JS, LoBue V. 2009 The narrow fellow in
the grass: human infants associate snakes and fear.
Dev. Sci. 12, 201–207. (doi:10.1111/j.1467-7687.
2008.00753.x)
32. Lobue V, Deloache JS. 2010 Superior detection of
threat-relevant stimuli in infancy. Dev. Sci. 13,
221–228. (doi:10.1111/j.1467-7687.2009.00872.x)
33. Vuilleumier P. 2005 How brains beware: neuralmechanisms of emotional attention. Trends
Cogn. Sci. 9, 585–594. (doi:10.1016/j.tics.2005.
10.011)
34. Dakin SC, Hess RF, Ledgeway T, Achtman RL. 2002
What causes non-monotonic tuning of fMRI
response to noisy images? Curr. Biol. 12, 476 –477.
(doi:10.1016/S0960-9822(02)00960-0)
35. Rodger H, Vizioli L, Ouyang X, Caldara R. 2015
Mapping the development of facial expression
recognition. Dev. Sci. 18, 926–939. (doi:10.1111/
desc.12281)
36. Malatesta CZ, Haviland JM. 1982 Learning display
rules: the socialization of emotion expression in
infancy. Child Dev. 53, 991– 1003. (doi:10.2307/
1129139)
37. La Barbera JD, Izard CE, Vietze P, Parisi SA. 1976
Four- and six-month-old infants’ visual responses tojoy, anger, and neutral expressions. Child Dev. 47,
535–538. (doi:10.2307/1128816)
38. Crouzet SM, Thorpe SJ. 2011 Low-level cues andultra-fast face detection. Front. Psychol. 2, 342.
(doi:10.3389/fpsyg.2011.00342)
39. Torralba A, Oliva A. 2003 Statistics of natural image
categories. Netw. Comput. Neural Syst. 14,
391–412. (doi:10.1088/0954-898X/14/3/302)
40. Vanderwert RE, Westerlund A, Montoya L,
McCormick SA, Miguel HO, Nelson CA. 2015 Looking
to the eyes influences the processing of emotion on
face-sensitive event-related potentials in 7-monthold infants. Dev. Neurobiol. 75, 1154–1163.
(doi:10.1002/dneu.22204)
41. Gliga T, Csibra G. 2007 Seeing the face through theeyes: a developmental perspective on face expertise.
Prog. Brain Res. 164, 323– 339. (doi:10.1016/
S0079-6123(07)64018-7)

Proc. R. Soc. B 284: 20171054

2.

Ekman P, Oster H. 1979 Facial expressions ofemotion. Annu. Rev. Psychol. 30, 527– 554. (doi:10.
1146/annurev.ps.30.020179.002523)
Darwin C. 1872 The expression of the emotions
in man and animals. London, UK: John Murray.
Waller BM, Micheletta J. 2013 Facial expression in
nonhuman animals. Emot. Rev. 5, 54– 59. (doi:10.
1177/1754073912451503)
Burrows AM. 2008 The facial expression musculaturein primates and its evolutionary significance.
Bioessays 30, 212–225. (doi:10.1002/bies.20719)
Dezecache G, Mercier H, Scott-Phillips TC. 2013 An
evolutionary approach to emotional communication.
J. Pragmat. 59, 221–233. (doi:10.1016/j.pragma.
2013.06.007)
Pessoa L, Adolphs R. 2010 Emotion processing andthe amygdala: from a ‘low road’ to ‘many roads’ ofevaluating biological significance. Nat. Rev. Neurosci.
11, 773–783. (doi:10.1038/nrn2920)
Lozier LM, Vanmeter JW, Marsh AA. 2014
Impairments in facial affect recognition associatedwith autism spectrum disorders: a meta-analysis.
Dev. Psychopathol. 26, 933– 945. (doi:10.1017/
S0954579414000479)
Pollak SD, Kistler DJ. 2002 Early experience isassociated with the development of categoricalrepresentations for facial expressions of emotion.
Proc. Natl Acad. Sci. USA 99, 9072 –9076. (doi:10.
1073/pnas.142165999)
Moulson MC, Westerlund A. 2009 The effects ofearly experience on face recognition: an eventrelated potential study of institutionalized childrenin Romania. Child Dev. 80, 1039– 1056. (doi:10.
1111/j.1467-8624.2009.01315.x)
Muñoz LC. 2009 Callous-unemotional traits are
related to combined deficits in recognizing afraid
faces and body poses. J. Am. Acad. Child Adolesc.
Psychiatry 48, 554–562. (doi:10.1097/CHI.
0b013e31819c2419)
Brennan AM, Harris AWF, Williams LM. 2014 Neuralprocessing of facial expressions of emotion in first
onset psychosis. Psychiatry Res. 219, 477–485.
(doi:10.1016/j.psychres.2014.06.017)
Calder AJ, Young AW, Keane J, Dean M. 2000
Configural information in facial expression
perception. J. Exp. Psychol. Hum. Percept. Perform.
26, 527–551. (doi:10.1037/0096-1523.26.2.527)
Lobue V. 2009 More than just another face in thecrowd: superior detection of threatening facial
expressions in children and adults. Dev. Sci.
12, 305–313. (doi:10.1111/j.1467-7687.2008.
00767.x)
Schyns PG, Petro LS, Smith ML. 2007 Dynamics ofvisual information integration in the brain for

8rspb.royalsocietypublishing.org

O.P. performed the experiments. L.B. and R.L. analysed the data.
L.B., P.C.Q., R.L., R.C., K.L. and O.P. drafted the manuscript. All
authors gave final approval for publication.
Competing interests. We have no competing interests.

Downloaded from http://rspb.royalsocietypublishing.org/ on September 6, 2017

66. Grossmann T, Jessen S. 2017 When in infancy doesthe ‘fear bias’ develop? J. Exp. Child Psychol. 153,
149–154. (doi:10.1016/j.jecp.2016.06.018)
67. Jessen S, Grossmann T. 2016 The developmental
emergence of unconscious fear processing from eyesduring infancy. J. Exp. Child Psychol. 142, 334 –343.
(doi:10.1016/j.jecp.2015.09.009)
68. Farroni T, Menon E, Rigato S, Johnson MH. 2007
The perception of facial expressions in newborns.
Eur. J. Dev. Psychol. 4, 2 –13. (doi:10.1080/
17405620601046832)
69. Hoehl S, Wiese L, Striano T. 2008 Young infants’neural processing of objects is affected by eye
gaze direction and emotional expression.
PLoS ONE 3, e2389. (doi:10.1371/journal.pone.
0002389)
70. Minagawa-Kawai Y, Mori K, Hebden JC, Dupoux E.
2008 Optical imaging of infants’ neurocognitive
development: recent advances and perspectives.
Dev. Neurobiol. 68, 712–728. (doi:10.1002/dneu.
20618)
71. Ohman A, Mineka S. 2001 Fears, phobias, andpreparedness: toward an evolved module of fear
and fear learning. Psychol. Rev. 108, 483–522.
(doi:10.1037/0033-295X.108.3.483)
72. Dupierrix E, de Boisferon AH, Méary D, Lee K, Quinn
PC, Di Giorgio E, Simion F, Tomonaga M, Pascalis O.
2014 Preference for human eyes in human infants.
J. Exp. Child Psychol. 123, 138–146. (doi:10.1016/j.
jecp.2013.12.010)
73. Scheller E, Büchel C, Gamer M. 2012 Diagnostic
features of emotional expressions are processed
preferentially. PLoS ONE 7, e41792. (doi:10.1371/
journal.pone.0041792)
74. Smith ML, Cottrell GW, Gosselin F, Schyns PG. 2005
Transmitting and decoding facial expressions.
Psychol. Sci. 16, 184 –189. (doi:10.1111/j.09567976.2005.00801.x)
75. Cohen LB. 1972 Attention-getting and attentionholding processes of infant visual preferences. Child
Dev. 43, 869–879. (doi:10.2307/1127638)
76. Aslin RN, Fiser J. 2005 Methodological challenges
for understanding cognitive development in infants.
Trends Cogn. Sci. 9, 92 –98. (doi:10.1016/j.tics.2005.
01.003)9Proc. R. Soc. B 284: 20171054

53. Tiddeman B. 2005 Towards realism in facial imagetransformation: results of a wavelet MRF method.
Comput. Graph. Forum 24, 449– 456. (doi:10.1111/
j.1467-8659.2005.00870.x)
54. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. 2004
Image quality assessment: from error visibility to
structural similarity. IEEE Trans. Image Process. 13,
600 –612. (doi:10.1109/TIP.2003.819861)
55. Brainard DH. 1997 The psychophysics toolbox. Spat.
Vis. 10, 433 –436. (doi:10.1163/156856897X00357)
56. Csibra G, Hernik M, Mascaro O, Tatone D, Lengyel
M. 2016 Statistical treatment of looking-time data.
Dev. Psychol. 52, 521–536. (doi:10.1037/
dev0000083)
57. Fantz RL. 1961 The origin of form perception. Sci. Am.
204, 66–72. (doi:10.1038/scientificamerican0561-66)
58. Laird NM, Ware JH. 1982 Random-effect models forlongitudinal data. Biometrics 38, 963–974. (doi:10.
2307/2529876)
59. Pinheiro J, Bates D, DebRoy S, Sarkar D, R Core
Team. 2012 nlme: Linear and nonlinear mixed
effects models. R Package version 3.1.105.
60. Fox F, Weisberg S. 2011 An R companion to applied
regression, 2nd edn. Thousand Oaks CA: Sage. See
http://socserv.socsci.mcmaster.ca/jfox/Books/
Companion.
61. Bates D, Maechler M, Bolker B. 2013 lme4: Linear
mixed-effects models using S4 classes. R Packag.
version 1.0.4.
62. Viola P, Jones M. 2001 Rapid object detection
using a boosted cascade of simple features. In
Conference on Computer Vision and Pattern
Recognition (CVPR). Vol. 1, pp. I-511 –I-518.
Piscataway, NJ: IEEE.
63. Bayet L, Quinn PC, Laboissière R, Caldara R, Lee K,
Pascalis O. 2017 Data from: Fearful but not happy
expressions boost face detection in human infants.
Figshare (doi.org/10.6084/m9.figshare.3439349.v1)
64. Gelskov SV, Kouider S. 2010 Psychophysicalthresholds of face visibility during infancy. Cognition
114, 285–292. (doi:10.1016/j.cognition.2009.09.012)
65. Heck A, Hock A, White H, Jubran R, Bhatt RS. 2016
The development of attention to dynamic facial
emotions. J. Exp. Child Psychol. 147, 100–110.
(doi:10.1016/j.jecp.2016.03.005)

rspb.royalsocietypublishing.org

42. Haynes J-D, Rees G. 2006 Decoding mental statesfrom brain activity in humans. Nat. Rev. Neurosci. 7,
523–534. (doi:10.1038/nrn1931)
43. Norman KA, Polyn SM, Detre GJ, Haxby JV. 2006
Beyond mind-reading: multi-voxel pattern analysis
of fMRI data. Trends Cogn. Sci. 10, 424–430.
(doi:10.1016/j.tics.2006.07.005)
44. Cauchoix M, Barragan-Jason G, Serre T, Barbeau EJ.
2014 The neural dynamics of face detection in thewild revealed by MVPA. J. Neurosci. 34, 846–854.
(doi:10.1523/JNEUROSCI.3030-13.2014)45. Emberson LL, Zinszer BD, Raizada RD. S, Aslin RN.
2017 Decoding the infant mind: multichannel
pattern analysis (MVPA) using fNIRS. PLoS ONE 12,
e0172500. (doi:10.1371/journal.pone.0172500)
46. Tseng PH, Cameron IGM, Pari G, Reynolds JN,
Munoz DP, Itti L. 2013 High-throughput
classification of clinical populations from natural
viewing eye movements. J. Neurol. 260, 275–284.
(doi:10.1007/s00415-012-6631-2)
47. Stahl D, Pickles A, Elsabbagh M, Johnson MH. 2012
Novel machine learning methods for ERP analysis: avalidation from research on infants at risk forautism. Dev. Neuropsychol. 37, 274–298. (doi:10.
1080/87565641.2011.650808)
48. Teller DY. 1979 The forced-choice preferential
looking procedure: a psychophysical technique foruse with human infants. Infant Behav. Dev. 2,
135–153. (doi:10.1016/S0163-6383(79)80016-8)
49. Gori M, Del Viva M, Sandini G, Burr DC. 2008 Youngchildren do not integrate visual and haptic form
information. Curr. Biol. 18, 694–698. (doi:10.1016/
j.cub.2008.04.036)
50. Dobkins KR, Anderson CM, Lia B. 1999 Infant temporalcontrast sensitivity functions (tCSFs) mature earlier for
luminance than for chromatic stimuli: evidence forprecocious magnocellular development? Vision Res. 39,
3223–3239. (doi:10.1016/S0042-6989(99)00020-6)
51. Lundqvist D, Flykt A, Öhman A. 1998 The Karolinska
directed emotional faces. Stockholm, Sweden:
Karolinska Inst.
52. Willenbockel V, Sadr J, Fiset D, Horne GO, Gosselin
F, Tanaka JW. 2010 Controlling low-level image
properties: the SHINE toolbox. Behav. Res. Methods
42, 671–684. (doi:10.3758/BRM.42.3.671)