# An implicit and reliable neural measure quantifying impaired visual coding of facial expression: evidence from the 22q11.2 deletion syndrome

Arnaud Leleu, Emilie Favre, Alexandre Yailian, Hugo Fumat, Juliette Klamm, Isabelle Amado, Jean-Yves Baudouin, Nicolas Franck, Caroline Demily

► To cite this version:

Arnaud Leleu, Emilie Favre, Alexandre Yailian, Hugo Fumat, Juliette Klamm, et al.. An implicit and reliable neural measure quantifying impaired visual coding of facial expression: evidence from the 22q11.2 deletion syndrome. Translational Psychiatry, 2019, 9 (1), pp.67. 10.1038/s41398-019-0411-z. hal-02618950

HAL Id: hal-02618950

https://hal.inrae.fr/hal-02618950

Submitted on 25 May 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ARTICLE

Open Access

# An implicit and reliable neural measure quantifying impaired visual coding of facial expression: evidence from the 22q11.2 deletion syndrome

Arnaud Leleu<sup>1</sup>, Emilie Favre<sup>2</sup>, Alexandre Yailian<sup>3</sup>, Hugo Fumat<sup>2</sup>, Juliette Klamm<sup>4</sup>, Isabelle Amado<sup>5</sup>, Jean-Yves Baudouin<sup>1,6</sup>, Nicolas Franck<sup>4</sup> and Caroline Demily<sup>2</sup>

**Abstract**

Although various psychiatric disorders present with social-cognitive impairment, a measure assessing social-cognitive processes implicitly and reliably, with high selectivity and with enough signal-to-noise ratio (SNR) for individual evaluation of any population at any age, is lacking. Here we isolate a neural marker quantifying impaired visual coding of facial expression in individuals with 22q11.2 deletion syndrome (22q11DS) using frequency-tagging with electroencephalography (EEG). Twenty-two 22q11DS participants and 22 healthy controls were presented with changes of facial expression displayed at low, moderate, and high intensities every five cycles in a stream of one neutral face repeating 6 times per second (i.e., at a 6 Hz base rate). The brain response to expression changes tagged at the 1.2 Hz (i.e., 6 Hz/5) predefined frequency was isolated over occipito-temporal regions in both groups of participants for moderate- and high-intensity facial expressions. Neural sensitivity to facial expression was reduced by about 36% in 22q11DS, revealing impaired visual coding of emotional facial signals. The significance of the expression-change response was estimated for each single participant thanks to the high SNR of the approach. Further analyses revealed the high reliability of the response and its immunity from other neurocognitive skills. Interestingly, response magnitude was associated with the severity of positive symptoms, pointing to a potential endophenotype for psychosis risk. Overall, the present study reveals an objective, selective, reliable, and behavior-free signature of impaired visual coding of facial expression implicitly quantified from brain activity with high SNR. This novel tool opens avenues for clinical practice, providing a potential early biomarker for later psychosis onset and offering an alternative for individual assessment of social-cognitive functioning in even difficult-to-test participants.

**Introduction**

Social-cognitive impairment is a hallmark characteristic of various psychiatric disorders, such as schizophrenia<sup>1,2</sup>,

autism spectrum disorders³, and genetic syndromes associated with broad psychiatric phenotypes⁴. Precisely delineating the mechanisms underlying impaired social cognition in these populations and developing validated tools to assess their social-cognitive processes is a major challenge to better understand the neurocognitive determinants of inappropriate social skills and more generally to promote early clinical interventions. Our aim here was therefore to establish the proof-of-concept for a selective measure of the visual coding of facial expression—a key

Correspondence: Arnaud Leleu (arnaud.leleu@u-bourgogne.fr) or Caroline Demily (caroline.demily@ch-le-vinatier.fr)

Full list of author information is available at the end of the article.

<figure><img src="image_2.png" /></figure>

<footnote n="1">¹Developmental Ethology and Cognitive Psychology group, Centre des Sciences du Goût et de l’Alimentation, CNRS, Université Bourgogne Franche-Comté, Inra, AgroSup Dijon, F-21000 Dijon, France</footnote>

<footnote n="2">²Reference Center for Rare Diseases with Psychiatric Phenotype - GénoPsy, Centre Hospitalier le Vinatier, Marc Jeannerod Institute (CNRS & Claude Bernard Lyon 1 University), Bron, France</footnote>
component of social cognition—that could be usable in a
clinical context to assess any population at any age.

For this purpose, we focused on the 22q11.2 deletion syndrome (22q11DS), a genetic condition affecting up to 1 per 2000 individuals⁵ and one of the highest known risks for schizophrenia spectrum disorders (SSD) with approximately 40% of the affected people having developed psychosis in adulthood⁶. Reduced social functioning and social-cognitive impairment are well documented in 22q11DS⁷, both associated with psychotic symptoms⁸⁻¹⁴, and with neural underpinnings in the structural integrity of the “social brain network”¹⁵⁻¹⁷. In particular, a large body of research investigated the recognition of emotions from facial expression and reported that 22q11DS individuals exhibit difficulties to discriminate or identify facial emotional signals⁹,¹²⁻¹⁴,¹⁸⁻²⁶. Impaired detection of facial emotions expressed at variable intensities¹⁴,²²,²⁵,²⁷ and abnormal visual exploration of expressive faces¹⁸,²⁰,²³,²⁷ were also described.

However, discrepancies remain across studies about the
most relevant measure of impaired facial emotion recog-
nition (e.g., accuracy vs. reaction times¹³,¹⁴,²¹,²⁴,²⁷), the
differences between 22q11DS and control participants in
visual exploration patterns for expressive faces¹⁸,²⁰,²³,²⁷, or
the selectivity of the observed difficulties (refs. ¹²,²⁰,²²,²³,²⁷
and refs. ²⁵,²⁶), respectively, for the contribution of general
cognitive functioning and basic visual processing). These
inconsistent findings may be explained by the use of
explicit behavioral responses from different methodolo-
gies tapping various mechanisms with variable sensitivity.
Especially, response-related processes (e.g., working
memory, response inhibition) are conflated with the visual
coding of facial expression in explicit behavioral mea-
sures²⁸ and easily contaminated by comprehension,
motivation, and attention. Hence, the contribution of
such unspecific processes cannot be definitely ruled out
from explicit behavioral outputs in 22q11DS individuals
suffering from verbal difficulties²⁹⁻³², impaired executive
processes²⁴,³⁰,³³⁻³⁵, low attentional resources²⁹,³⁰, and
reduced motivation³⁶. Moreover, explicit measures also
require variable age-appropriate testing tools that hinder
the opportunity to develop life-span standardized assess-
ments in 22q11DS and more generally in any other neu-
rodevelopmental disorder.

Neural measures are ideally suited to provide alternative
biomarkers for unambiguous quantification of the visual
coding of facial expression in any population at any age.
Using orthogonal behavioral tasks (i.e., implicit processing
of facial expression), functional neuroimaging studies³⁷⁻³⁹
revealed abnormal sensitivity to facial expression in
22q11DS within visual brain regions from the core cor-
tical network mediating face perception⁴⁰ where struc-
tural alterations have been reported⁴¹⁻⁴⁴. Interestingly,
normal activation when viewing houses as well as lower

sensitivity to faces in psychotic vs. non-psychotic parti-
cipants³⁸ suggest a face-specific visual impairment eligible
as an endophenotype for psychosis risk in 22q11DS.

However, as for behavioral measures, findings lack consistency across studies (e.g., higher vs. equal vs. lower expression-selective activation in the fusiform gyrus for 22q11DS compared with healthy participants³⁷⁻³⁹). Unfortunately, functional neuroimaging exhibits technical limitations that hamper its potential translation into clinical practice. It requires long recording sessions to reach adequate signal-to-noise ratios (SNRs) that prevent from rapid testing and individual assessment. In addition, analysis routines are not standardized across studies, reducing objectivity and generalizability. When a behavioral task is performed during stimulation, even an orthogonal task, response-related processes can still contaminate the recorded neural response. Finally, standard post hoc subtraction contrasting expressive from neutral faces indirectly reflects their visual discrimination⁴⁵,⁴⁶. Hence, neuroimaging remains limited for developing an implicit and objective measure that rapidly isolates the visual coding of facial expression with high SNR and selectivity in any individual participant.

Here we circumvent these issues and provide a neural
marker of impaired visual coding of facial expression in
22q11DS individuals using a frequency-tagging
approach⁴⁷ with scalp electroencephalography (EEG).
Previous frequency-tagging studies in healthy individuals
quantified a neural response to facial expression change in
a few minutes in every participant with consistent
response patterns across studies⁴⁸,⁴⁹. Accordingly, fol-
lowing ref. ⁴⁹, 22q11DS and healthy participants per-
formed a non-periodic orthogonal task while presented
with a neutral face at a rapid 6 Hz rate and brief changes
of facial expression inserted every fifth cycle (i.e., at a 6/5
= 1.2 Hz rate) at low, moderate, and high intensities.
Using a frequency-domain analysis of EEG amplitude
spectra, the visual discrimination of facial expression was
objectively measured at the predefined 1.2 Hz frequency
with high SNR and directly dissociated from general
unspecific visual processes captured at the 6 Hz rate of
stimulus presentation within the same stimulation
sequence (i.e., without post hoc subtraction).

**Materials and methods**

Participants

Twenty-two individuals with 22q11.DS (10 females, mean age: 25.3 ± 8.7 (SD) years, range 16–44 years) and 22 healthy participants (7 females, mean age: 26.9 ± 6.7 years, range 19–42 years) participated in the study (sample size in adequacy with previous studies showing impaired recognition of facial expression in 22q11DS). The two groups did not differ according to age ($T_{42} = 0.72, p = .48$) nor sex ($X^2_1 = 0.86, p = .35$). Participants
with 22q11DS were diagnosed using comparative geno-
mic hybridization array analysis confirmed by fluores-
cence in situ hybridization analysis. Eight of them
presented concomitant diagnosis of schizophrenia (Diag-
nostic and Statistical Manual of Mental Disorders, Fifth
Edition (DSM-5) criteria) and were treated with anti-
psychotic medication. All were clinically stable at testing
time with unchanged medication during the month pre-
ceding the experiment. Detailed information about
22q11DS participants at their inclusion is available in
Supplementary Table S1. Healthy participants did not
have any relative with 22q11DS. Exclusion criteria were
pregnancy, substance abuse (DSM-5 criteria except for
tobacco and caffeine), neurodegenerative, neurovascular,
infectious disorders or head injury, uncorrected loss of
visual acuity, brain stimulation during the 2 previous
months, or inclusion in a social-cognitive remediation
program. Absence of premorbid intellectual disability was
confirmed using the French adaptation of the National
Adult Reading Test⁵⁰. The study was conducted according
to the Declaration of Helsinki and approved by the French
ethics committee (Comité de protection des personnes
Sud-Est II - ANSM Reference:151109B-31; ID RCB: 2015-
A01247-42). All participants (parents for minor partici-
pants) provided written informed consent before inclu-
sion. Participants with 22q11DS completed clinical and

neuropsychological evaluations. Clinical ratings were
obtained using the positive and negative symptoms
scale⁵¹, a reliable measurement of psychotic symptoms
largely used in psychiatry research, e.g., ref. ⁵². Neu-
ropsychological assessment evaluated various cognitive
processes detailed in Supplementary Methods. Results for
both clinical and neuropsychological assessments are
summarized in Supplementary Table S3 and discussed in
Supplementary Results.

Stimuli

Face stimuli were adapted from previous studies<sup>25,49</sup>
and comprised 24 color pictures from 4 individuals (2
females) with a neutral expression or expressing 5 basic
emotions (anger, disgust, fear, happiness, sadness) in full-
front view. Faces were cropped into medallion-shaped
windows displayed on a mid-level gray (i.e., 128/255 in
grayscale) background (Fig. 1 and Supplementary Movie).
For each individual face and emotional expression, 3
intensity levels (i.e., low = 20% of the full-blown expres-
sion, moderate = 60%, high = 100%) were designed using
a morphing procedure (Morpheus Photo Morpher 1.85,
Morpheus Software, USA) combining neutrality and the
expression. The final set of stimuli was thus composed of
64 pictures, 16 for each individual face (3 intensities × 5
emotions + neutrality).

<figure><img src="image_3.png" /></figure>
## Procedure

The procedure was adapted from previous studies using frequency-tagging with EEG to investigate facial expression perception<sup>48,49</sup>. Stimuli were displayed on a computer screen (60 Hz refresh rate) with a mid-level gray background (i.e., 128/255 in grayscale) through sinusoidal contrast modulation (from 0% to 100%) at a rapid rate of 6 Hz (Fig. 1 and Supplementary Movie) using a custom software written in Java. At this rate, each stimulus lasts ≈167 ms with full contrast reached around 83 ms. Neutral faces were presented at the 6 Hz base rate and an emotional expression was introduced every fifth stimulus. Thus a brief change of expression from neutrality to an emotion occurred at a rate of 6/5 = 1.2 Hz (i.e., ≈833 ms between two expressive faces). To reduce expression-change detection based on low-level visual cues, stimulus size was randomly varied between 95% and 105% at every stimulus onset (initial size was set to 6 × 4.8° of visual angle when displayed on the screen).

Several stimulation sequences were presented with one individual neutral face changing to one emotional expression throughout each sequence. Sequences started with a variable interval of 0.5–1 s of blank screen, followed by a 2.5-s fade-in of increasing contrast modulation depth with no expression change. Then stimulation lasted 75 s with intensity of the 1.2 Hz expression change increasing every 25 s (i.e., during the first 25 s, the emotion was expressed at low (20%) intensity, followed by the moderate (60%) intensity during the next 25 s, finally followed by the high (100%) intensity during the last 25 s). A 1-s fade-out of decreasing contrast modulation depth then followed and sequences ended with a variable interval of 0.5–1 s of blank screen. Each emotion was repeated 4 times (4 individual faces), resulting in 20 sequences throughout the experiment. They were divided into 4 blocks of 5 sequences, each block presenting one sequence per emotion. The presentation orders of blocks and sequences within blocks were randomized across participants.

Participants were seated in front of a computer screen at a distance of 47 cm. To maintain their attention on the screen, they performed an orthogonal behavioral task. During each sequence, a fixation circle (located just below the eyes when the face stimuli were displayed) briefly (i.e., 200 ms) changed into a square 9 random times with a minimum interval of 5 s between two changes. Participants were asked to detect these shape changes by pressing the space bar of a keyboard as fast as possible. Response times and accuracy are detailed in Supplementary Results.

## EEG acquisition and analysis

Throughout the experiment, EEG was continuously recorded from 128 Ag/AgCl electrodes mounted in an electrode cap (Waveguard, ANT, The Netherlands)

according to the 10–10 classification system (acquisition reference: AFFz, electrode impedance <15 kΩ, sampling rate: 1024 Hz). EEG data were then preprocessed and averaged across individual faces for each facial expression. Primary analyses in the frequency domain (Supplementary Methods) were performed to isolate and quantify both brain responses to the 6 Hz stimulation rate and to the 1.2 Hz expression-change rate and their harmonics (i.e., integer multiples). Note that frequency-domain analysis allows high SNR measurement since the signal is captured in tiny target frequencies (e.g., 1.2 Hz, 2.4 Hz, etc.) while the noise spreads to broad frequency ranges. Here frequency-domain resolution was 0.04 Hz, leading to 29 frequency bins between two target frequencies (e.g., between 1.2 Hz and 2.4 Hz). These surrounding frequency bins were used for noise level estimation. Statistical analyses were finally conducted using Z-scores (to estimate the significance of each brain response compared with surrounding noise level, see Supplementary Methods) and repeated-measures analyses of variance (ANOVAs) performed on EEG amplitudes corrected for surrounding noise level. ANOVAs included *Group* (22q11DS, controls) as a between-subject factor and *Emotion* (anger, disgust, fear, happiness, sadness) and *Intensity* (low, moderate, high) as within-subject factors. ANOVAs also included regions of interest (ROIs) as a within-subject factor. Four ROIs (left, medial, and right occipital [LO, mO, and rO, respectively], and medial parietal [mP]) were analyzed for the response elicited by the 6 Hz stimulation rate, and two ROIs (left and right occipito-temporal [IOT and rOT, respectively]) for the response elicited by the 1.2 Hz expression-change rate (see Supplementary Methods for definition of ROIs). Mauchly’s test for sphericity violation was computed and Greenhouse-Geisser correction for degrees of freedom was applied whenever sphericity was violated. For significant effects involving the *Group* factor, comparisons between groups of participants were conducted using *T* tests. Pearson’s correlation coefficients were calculated to explore the association between each brain response and both clinical and neuropsychological assessments, as well as the reliability of the expression-change response.

## Results

For both groups of participants, the 6 Hz base rate of rapid stimulation and the 1.2 Hz rate of brief expression change elicited periodic EEG responses identified in the frequency domain at the same frequencies and their harmonics. Detailed description of the EEG amplitude spectra obtained for both brain responses and their harmonics is available in Supplementary Results. Here we present summed corrected amplitudes for significant harmonics (Supplementary Methods), for they summarize each response in a single value expressed in microvolts
(µV). For sake of brevity and according to our main purpose, we report only the expression-change response as a direct signature of the visual system ability to process expressive faces (results for the general visual response elicited by the 6 Hz stimulation rate are detailed in Supplementary Results). Similarly, despite some brief general descriptions of the response, we focus on statistical effects involving the *Group* factor to characterize the visual coding of facial expression in 22q11DS participants as compared with healthy controls.

## Impaired visual coding of moderate- and high-intensity facial expressions

As previously observed⁴⁹, introducing brief changes of facial expression in a rapid sequence of neutral faces does not elicit identifiable brain activities at low expression intensity (M = 0.03 ± 0.02 (SEM) µV) while an occipito-temporal response progressively emerges as expression intensity increases (moderate intensity: M = 0.26 ± 0.03 µV, high intensity: M = 0.62 ± 0.06 µV, *Intensity* effect: F₁,₄,₅₆.₉ = 84.66, ε = 0.68, *p* < .001, ηₚ² = 0.67). This

response is larger in the right hemisphere (*ROI* effect: F₁,₄₂ = 8.49, *p* = .006, ηₚ² = 0.17) with a greater amplitude increase as a function of expression intensity (high minus low intensity: M = +0.64 ± 0.07 µV) than in the left hemisphere (M = +0.54 ± 0.06 µV, *Intensity* × *ROI*: F₂,₈₄ = 3.52, *p* = .034, ηₚ² = 0.08).

Most importantly for our purpose, a main effect of *Group* (F₁,₄₂ = 5.01, *p* = .031, ηₚ² = 0.11) qualified by a *Group* × *Intensity* interaction (F₁,₄,₅₆.₉ = 4.79, ε = 0.68, *p* = .023, ηₚ² = 0.10) revealed that while the expression-change response is almost absent at low expression intensity for both groups of participants (22q11DS: M = 0.03 ± 0.04 µV, controls: M = 0.02 ± 0.02 µV) with no difference between groups (*p* = .75), the increase of the response as a function of expression intensity is lower in 22q11DS individuals (M = +0.45 ± 0.06 µV) than healthy controls (M = +0.73 ± 0.09 µV) with significant differences between groups at both moderate (22q11DS: M = 0.20 ± 0.03 µV, controls: M = 0.32 ± 0.05 µV, *p* = .044) and high (22q11DS: M = 0.48 ± 0.06 µV, controls: M = 0.75 ± 0.09 µV, *p* = .021) intensities (Fig. 2). For both

<figure><img src="image_4.png" /><figcaption>**Fig. 2** The expression-change brain response in both healthy and 22q11DS participants. Three-dimensional topographical maps (posterior view) of summed corrected amplitudes of the expression-change response as a function of expression intensity averaged across emotions for both healthy controls (top) and 22q11DS individuals (bottom). Center: the same data displayed averaged across regions of interest (ROIs) showing the expression-change response (F) surrounded by ±0.5 Hz of summed corrected amplitude spectra. Shadowed areas represent standard errors of the mean. Z-scores displayed next to each map indicate the significance of the response for each group and each expression intensity (*p* < .05, **p** < .001). While the expression-change response is non-significant and practically absent for both groups of participants when emotions are expressed at low intensity, it progressively emerges as expression intensity increases with significant responses for both groups at moderate and high intensities. However, it is significantly lower in 22q11DS participants with a magnitude corresponding to 64% of the response observed for healthy controls</figcaption></figure>
groups of participants, Z-scores confirmed the lack of expression-change response when emotions are expressed at low intensity (22q11DS: Z = -0.04, controls: Z = -0.13, both ps > .51, one-tailed, signal > noise) while emerging at moderate (22q11DS: Z = 1.89, p = .029; controls: Z = 4.15, p < .001) and high (22q11DS: Z = 4.68, controls: Z = 9.61, both ps < .001) intensities (Fig. 2). In sum, the visual coding of periodic expression changes inserted in a rapid sequence of neutral faces is impaired in individuals with 22q11DS compared with healthy participants, leading to a reduction of about 36% of the “typical” expression-change response when emotions are expressed at moderate and high intensities. Note that a significant Group × Emotion interaction ($F_{4,168} = 2.65, p = 0.035, \eta_p^2 = 0.06$) revealed that amplitude differences between groups are not equivalent for every facial emotion, but this observation did not survive when controlling for the effects of antidepressant and antipsychotic medications (Supplementary Results).

## Individual brain responses to brief changes of facial expression

In order to determine whether the visual coding of facial expression is impaired in most 22q11DS individuals, significance of the expression-change response was also evaluated in every single participant (Fig. 3). No participants, both from the 22q11DS and control groups, exhibit a significant response for low-intensity facial expressions (all Zs < 1.41, all ps > .079) corroborating group-level observations. For emotions expressed at moderate intensity, 11 (i.e., 50%) participants with 22q11DS and 15 (i.e., 68%) healthy controls present with a significant expression-change response (Zs > 1.64, ps < .05). However, if we consider a higher significance threshold (i.e., Z > 2.32, p < .01), only 5 (i.e., 23%) individuals with 22q11DS still present with a significant response vs. 14 (i.e., 64%) healthy participants, thus demonstrating the reduced sensitivity of the visual system to brief expression changes in many 22q11DS individuals. An even higher threshold (i.e., Z > 3.09, p < .001) abolishes significant responses in the 22q11DS group compared with 10 (i.e., 45%) participants still remaining in the control group. For high-intensity facial expressions, 18 (i.e., 82%) participants with 22q11DS and all healthy controls show a significant expression-change response, but again Z-scores are lower in the 22q11DS group as clearly visible in Fig. 3.

## Reliability of the expression-change brain response

The reliability of the expression-change response averaged across emotions and expression intensities was estimated by separating and correlating the amplitudes obtained for stimulation sequences using female and male faces. A first estimate was calculated with individual amplitudes averaged across channels included in the

ROIs. It indicated that the strength of the response is highly reliable across measurements (Fig. 4a) for all participants ($r = .83, p < .001$) and separately for 22q11DS ($r = .83, p < .001$) and healthy ($r = .81, p < .001$) individuals. In addition, a second estimate was computed with the amplitude recorded for each channel over the scalp and averaged across participants. It revealed that the topography of the expression-change response is also highly reliable (Fig. 4b; all participants: $r = .96, p < .001$; 22q11DS: $r = .90, p < .001$; controls: $r = .97, p < .001$).

## Relationship to positive symptoms

Correlations between the expression-change response averaged across emotions, expression intensities, and ROIs, and both clinical and neuropsychological ratings revealed a significant inverse association between the amplitude of the response and positive symptoms ($r = -.59, p = .004$). This indicates that the higher the neural response to expression changes is in 22q11DS participants, the lower are their positive symptoms (Fig. 5). No other significant correlations were found (all ps > .082) with clinical symptoms and cognitive processes. Correlations are fully detailed in Supplementary Results.

## Discussion

By using frequency-tagging with EEG, a highly effective approach for evaluation of visual perception⁴⁷, the present study isolates a neural marker for impaired visual coding of facial expression in 22q11DS over occipito-temporal regions and quantifies this reduced brain response to brief changes of facial expression from a neutral face as about 64% of the “typical” response measured in healthy controls for emotions expressed at moderate and high intensities. This observation reinforces previous evidence about the difficulties of 22q11DS individuals for facial emotion processing⁹,¹²⁻¹⁴,¹⁸⁻²⁷, and additionally clearly reveals that the impairment is subtended, at least partly, by a reduced visual sensitivity to changes of facial expression, as previously although indirectly suggested by neuroimaging findings³⁷⁻³⁹.

Most importantly for our purpose, the present observation provides several benefits in terms of social-cognitive assessment with critical implications for both researchers and clinicians. First, frequency-tagging in EEG is an objective approach thanks to pre-experimental definition of target frequencies. Here two brain responses were respectively tagged at 6 Hz and 1.2 Hz and directly quantified at these specific frequencies and their harmonics in the EEG frequency spectrum. This objectivity avoids variable analyses across studies (e.g., definition of time windows for event-related potential analysis in EEG studies), thus increasing reliability and favoring the development of standardized assessments in a clinical context.
<figure><img src="image_5.png" /><figcaption>**Fig. 3** Individual brain responses to brief changes of facial expression. Z-scores estimating the significance of the expression-change response for every single participant from both the 22q11DS (left) and the control (right) groups and for low (top), moderate (center), and high (bottom) expression intensities averaged across emotions and regions of interest (ROIs). In each graph, horizontal gray bars indicate three significance thresholds ($Z > 1.64, p < .05$, one-tailed, signal > noise; $Z > 2.32, p < .01$; $Z > 3.09, p < .001$). Four brightness levels also indicate the significance of individual bins, from the lighter ($Z < 1.64, p > .05$) to the darker ($Z > 3.09, p < .001$) level. While the expression-change response does not reach significance for any participant when emotions are expressed at low intensity, it progressively emerges at moderate intensity for some individuals in each group and reaches higher significance thresholds in healthy controls. The same is observed at high expression intensity despite a significant response for almost every participant</figcaption></figure>

Second, periodic stimulation associated with frequency-domain analysis provide high SNR measurement of the EEG signal captured in tiny target frequencies compared with non-periodic noise spreading to broad frequency ranges. Thanks to this high sensitivity, the brain response to expression changes was isolated in a few minutes of recording (i.e., only 100 s of stimulation for one emotion expressed at one intensity level), this short recording time allowing rapid evaluation. Z-scores were calculated using

surrounding noise and provided direct significance estimation of the expression-change response in every individual participant. In addition, the expression-change response is highly reliable, both in terms of magnitude and topography. This key aspect of any approach aiming at individual testing would undeniably facilitate its translation into clinical practice. Admittedly, the approach must be consolidated for consistent individual measurements with high test-retest reliability determined
<figure><img src="image_6.png" /><figcaption>**Fig. 4 Reliability of the expression-change brain response.** a Scatter plot showing that the strength of the expression-change response calculated between summed corrected amplitudes obtained for female and male faces and averaged across emotions, expression intensities, and channels (from the regions of interest) is highly reliable. 22q11DS participants are depicted in red while healthy controls are depicted in blue. **b** Three-dimensional topographical maps (posterior view) revealing that the topography of the expression-change response obtained for female (left) and male (right) faces and averaged across emotions and intensities for healthy controls (top) and 22q11DS participants (bottom) is also highly reliable. Note that color scales have been adapted for each map.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Fig. 5 Association between the expression-change response and positive symptoms.** Top: Scatter plot of the negative relationship between individual assessments of positive symptoms (PANSS+) in 22q11DS participants and summed corrected amplitudes of the expression-change response averaged across emotions, expression intensities and regions of interest (ROIs). Eight participants with the lowest PANSS+ scores are indicated in blue while seven participants with the highest scores are depicted in red. Individuals with intermediate scores are displayed in black. Bottom: Three-dimensional topographical maps (posterior view) of the expression-change response averaged across emotions and intensities for 22q11DS participants with low (left) and high (right) PANSS+ scores, showing the larger amplitude of the response in individuals with less positive symptoms.</figcaption></figure>

at different testing times and high SNR responses in all individuals tested. Nonetheless, frequency-tagging in EEG is a promising tool to collect normative data directly from the brain in a large sample of healthy people, to define atypical visual coding of facial expression accordingly, and to apply individual assessment in any population.

Third, we provide a neural marker for automatic visual coding of facial expression that does not require any

explicit response, weakening the contribution of response-related processes that can vary considerably across individuals and across development, and are easily contaminated by difficulties observed in 22q11DS<sup>24,29-36</sup> and other disorders (e.g., verbal difficulties, reduced motivation and attention, impaired executive processes). As a consequence, contrary to previous studies that found a significant relationship between facial emotion recognition measured from behavior and other cognitive abilities<sup>12,23,25</sup>, the expression-change brain response measured here is unrelated to any cognitive ability evaluated during neuropsychological assessment. Moreover, frequency-tagging does not rely on post hoc subtraction<sup>45,46</sup> to isolate the neural processes underlying facial expression perception from other unspecific processes. The expression-change response is a direct differential measure of the brain activity elicited by a change of facial expression from a neutral face exempt from other visual processes common to both neutral and expressive faces (e.g., low-level contrast-change detection) captured by the
general visual response recorded at 6 Hz and harmonics. Accordingly, dissociated patterns were observed for the expression change vs. the general visual brain responses (see Supplementary Results for a detailed analysis of the latter and for the absence of significant correlation between the two responses). Altogether, these observations confirm that the expression-change response is a highly selective neural signature of automatic and implicit visual discrimination of facial expression giving the opportunity to test participants from various backgrounds without confounding perception with other cognitive skills. Hence, it might be used to differentiate various disorders that typically exhibit difficulties for facial expression recognition according to visual vs. non-visual impairments (e.g., post-perceptual difficulties in attributing emotions to clear-cut percepts; impaired explicit decision associated with reduced motivation, attention and executive functioning).

To a broader extent, frequency-tagging constitutes a unique approach for objective quantification of various cognitive functions in numerous populations whose condition precludes any classical neuropsychological assessment (e.g., high intellectual disability). Direct neural assessment might thus be used in a clinical context for diagnostic or even prognostic purpose. Similarly, the approach can easily explore a cognitive function across development. For instance, using the same design as in adults<sup>53,54</sup>, it was used for evaluating the development of reading abilities in preschool children<sup>55</sup> and identified a neural marker of face categorization (i.e., vs. non-face objects) in 4- to 6-month-old infants<sup>56</sup>. For 22q11DS and other neurodevelopmental disorders already diagnosed in infancy, early biomarkers for the onset of later cognitive difficulties could be detected using the approach, offering high predictive value and opportunities to develop adequate remediation strategies for optimizing outcomes.

Ultimately, frequency-tagging can also contribute to the search for early biomarkers of psychotic symptoms and SSD and enhance diagnosis and early treatment strategies. Indeed, the brain response to expression change in 22q11DS is associated with positive symptoms of psychosis (and the general visual response with negative symptoms, Supplementary Results). While further studies should obviously consolidate these observations with a larger sample of participants and determine the developmental course of impaired facial expression perception in relation to psychosis onset, it holds high promise for future clinical interventions. It could be used in association with verbal IQ<sup>31,32</sup> or executive processes<sup>57,58</sup> also related to clinical symptoms of psychosis. However, verbal skills and executive functioning are late-maturing cognitive abilities in comparison with facial expression perception<sup>59</sup>. Hence, according to the opportunity of using frequency-tagging in childhood or infancy, the present

signature of impaired visual processing of facial expression could serve as one of the most precocious biomarkers of later psychosis development.

In conclusion, frequency-tagging in EEG provides an implicit and reliable measure of the visual coding of facial expression that is obtained rapidly from brain activity and analyzed objectively (i.e., pre-experimentally defined analyses), with high SNR (i.e., signal captured in tiny target frequencies within high-resolution frequency spectrum), high selectivity (i.e., direct differential measure without post hoc subtraction), and large immunity from response-related processes (i.e., non-periodic orthogonal behavioral task). Thanks to this task-independent direct neural measurement of a cognitive process, the approach could be developed as a standardized tool for individual assessment of even difficult-to-test population at any age, opening avenues for both experimental exploration and clinical practice in the near future.

### Acknowledgements

The authors wish to thank all the participants and their families for their invaluable commitment. This work received support from "Le Vinatier" Psychiatric hospital (CSR grant), the "Conseil Régional Bourgogne Franche-Comté" (PARI grant to J.-Y.B. and A.L.), the FEDER (European Funding for Regional Economic Development), and the French "Investissements d'Avenir" program, project ISITE-BFC (contract ANR-15-IDEX-03 to J.-Y.B. and A.L.).

### Author details

¹Developmental Ethology and Cognitive Psychology group, Centre des Sciences du Goût et de l'Alimentation, CNRS, Université Bourgogne Franche-Comté, Inra, AgroSup Dijon, F-21000 Dijon, France. ²Reference Center for Rare Diseases with Psychiatric Phenotype - GénoPsy, Centre Hospitalier le Vinatier, Marc Jeannerod Institute (CNRS & Claude Bernard Lyon 1 University), Bron, France. ³Child and Adolescent Psychiatry, University Hospital of Montpellier, University Montpellier 1, Montpellier, France. ⁴Centre Ressource de Réhabilitation Psychosociale et de Remédiation Cognitive, Centre Hospitalier Le Vinatier & Université Lyon 1 (CNRS UMR 5229), Université de Lyon, Lyon, France. ⁵Centre Ressource Ile de France de Remédiation Cognitive et Réhabilitation Psychosociale, Groupe Hospitalier Universitaire, Institut de Psychiatrie et Neurosciences de Paris, Université Paris Descartes, Paris, France. ⁶Laboratoire Développement, Individu, Processus, Handicap, Education (DIPHE), Département Psychologie du Développement, de l'Éducation et des Vulnérabilités (PsyDEV), Institut de Psychologie, Université de Lyon (Lumière Lyon 2), 69676 Bron Cedex, France

### Conflict of interest

The authors declare that they have no conflict of interest.

### Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Supplementary Information** accompanies this paper at [https://doi.org/10.1038/s41398-019-0411-z](https://doi.org/10.1038/s41398-019-0411-z).

Received: 29 July 2018 Revised: 26 December 2018 Accepted: 17 January 2019

Published online: 04 February 2019

### References

1. Penn, D. L., Sanna, L. J. & Roberts, D. L. Social cognition in schizophrenia: an overview. *Schizophr. Bull.* **34**, 408–411 (2008).
2. Green, M. F., Horan, W. P. & Lee, J. Social cognition in schizophrenia. *Nat. Rev. Neurosci.* **16**, 620–631 (2015).

3. Leekam, S. Social cognitive impairment and autism: what are we trying to explain? *Philos. Trans. R. Soc. B* **371**, 20150082 (2016).

4. Morel, A. et al. Overview of social cognitive dysfunctions in rare developmental syndromes with psychiatric phenotype. *Front. Pediatr.* **6**, 120 (2018).

5. Shprintzen, R. J. Velo-cardio-facial syndrome: 30 years of study. *Dev. Disabil. Res. Rev.* **14**, 3–10 (2008).

6. Schneider, M. et al. Psychiatric disorders from childhood to adulthood in 22q11.2 deletion syndrome: results from the International Consortium on Brain and Behavior in 22q11.2 Deletion Syndrome. *Am. J. Psychiatry* **171**, 627–639 (2014).

7. Norkett, E. M., Lincoln, S. H., Gonzalez-Heydrich, J. & D'Angelo, E. J. Social cognitive impairment in 22q11 deletion syndrome: a review. *Psychiatry Res.* **253**, 99–106 (2017).

8. Debbané, M., Glaser, B., David, M. K., Feinstein, C. & Eliez, S. Psychotic symptoms in children and adolescents with 22q11.2 deletion syndrome: neuropsychological and behavioral implications. *Schizophr. Res.* **84**, 187–193 (2006).

9. Jalbrzikowski, M. et al. Social cognition in 22q11.2 microdeletion syndrome: relevance to psychosis? *Schizophr. Res.* **142**, 99–107 (2012).

10. Yuen, T., Chow, E. W. C., Silversides, C. K. & Bassett, A. S. Premorbid adjustment and schizophrenia in individuals with 22q11.2 deletion syndrome. *Schizophr. Res.* **151**, 221–225 (2013).

11. Biswas, A. B. & Furniss, F. Cognitive phenotype and psychiatric disorder in 22q11.2 deletion syndrome: a review. *Res. Dev. Disabil.* **53–54**, 242–257 (2016).

12. Vangkilde, A. et al. Associations between social cognition, skills, and function and subclinical negative and positive symptoms in 22q11.2 deletion syndrome. *J. Neurodev. Disord.* **8**, 42 (2016).

13. Weinberger, R. et al. Neurocognitive profile in psychotic versus nonpsychotic individuals with 22q11.2 deletion syndrome. *Eur. Neuropsychopharmacol.* **26**, 1610–1618 (2016).

14. Badoud, D. et al. Understanding others: a pilot investigation of cognitive and affective facets of social cognition in patients with 22q11.2 deletion syndrome (22q11DS). *J. Neurodev. Disord.* **9**, 35 (2017).

15. albrzikowski, M. et al. Altered white matter microstructure is associated with social cognition and psychotic symptoms in 22q11.2 microdeletion syndrome. *Front. Behav. Neurosci.* **8**, 393 (2014).

16. Berhanu, D. et al. Cortical-amygdala volumetric ratios predict onset of symptoms of psychosis in 22q11.2 deletion syndrome. *Psychiatry Res. Neuroimaging* **259**, 10–15 (2017).

17. Olszewski, A. K. et al. The social brain network in 22q11.2 deletion syndrome: a diffusion tensor imaging study. *Behav. Brain Funct.* **13**, 4 (2017).

18. Campbell, L. E. et al. Visual scanning of faces in 22q11.2 deletion syndrome: attention to the mouth or the eyes? *Psychiatry Res.* **177**, 211–215 (2010).

19. Campbell, L. E. et al. Is theory of mind related to social dysfunction and emotional problems in 22q11.2 deletion syndrome (velo-cardio-facial syndrome)? *J. Neurodev. Disord.* **3**, 152–161 (2011).

20. McCabe, K. L., Rich, D., Loughland, C. M., Schall, U. & Campbell, L. E. Visual scanpath abnormalities in 22q11.2 deletion syndrome: Is this a face specific deficit? *Psychiatry Res.* **189**, 292–298 (2011).

21. Goldenberg, P. C. et al. Computerized neurocognitive profile in young people with 22q11.2 deletion syndrome compared to youths with schizophrenia and at-risk for psychosis. *Am. J. Med. Genet. B Neuropsychiatr. Genet.* **159B**, 87–93 (2012).

22. Shashi, V. et al. Social skills and associated psychopathology in children with chromosome 22q11.2 deletion syndrome: implications for interventions. *J. Intellect. Disabil. Res.* **56**, 865–878 (2012).

23. McCabe, K. L. et al. Divergent patterns of social cognition performance in autism and 22q11.2 deletion syndrome (22q11DS). *J. Autism Dev. Disord.* **43**, 1926–1934 (2013).

24. Gur, R. E. et al. Neurocognitive development in 22q11.2 deletion syndrome: comparison with youth having developmental delay and medical comorbidities. *Mol. Psychiatry* **19**, 1205–1211 (2014).

25. Leleu, A. et al. Facial emotion perception by intensity in children and adolescents with 22q11.2 deletion syndrome. *Eur. Child Adolesc. Psychiatry* **25**, 297–310 (2016).

26. McCabe, K. L. et al. Visual perception and processing in children with 22q11.2 deletion syndrome: associations with social cognition measures of face identity and emotion recognition. *J. Neurodev. Disord.* **8**, 30 (2016).

27. Franchini, M. et al. Visual processing of emotional dynamic faces in 22q11.2 deletion syndrome. *J. Intellect. Disabil. Res.* **60**, 308–321 (2016).

28. Phillips, L. H., Channon, S., Tunstall, M., Hedenstrom, A. & Lyons, K. The role of working memory in decoding emotions. *Emotion* **8**, 184 (2008).

29. Chow, E. W. C., Watson, M., Young, D. A. & Bassett, A. S. Neurocognitive profile in 22q11 deletion syndrome and schizophrenia. *Schizophr. Res.* **87**, 270–278 (2006).

30. Lewandowski, K. E., Shashi, V., Berry, P. M. & Kwapil, T. R. Schizophrenic-like neurocognitive deficits in children and adolescents with 22q11 deletion syndrome. *Am. J. Med. Genet. B Neuropsychiatr. Genet.* **144**, 27–36 (2007).

31. Gothelf, D. et al. Risk factors for the emergence of psychotic disorders in adolescents with 22q11.2 deletion syndrome. *Am. J. Psychiatry* **164**, 663–669 (2007).

32. Green, T. et al. Psychiatric disorders and intellectual functioning throughout development in velocardiofacial (22q11.2 deletion) syndrome. *J. Am. Acad. Child Adolesc. Psychiatry* **48**, 1060–1068 (2009).

33. Henry, J. C. et al. An investigation of the neuropsychological profile in adults with velo-cardio-facial syndrome (VCFS). *Neuropsychologia* **40**, 471–478 (2002).

34. Shapiro, H. M., Tassone, F., Choudhary, N. S. & Simon, T. J. The development of cognitive control in children with chromosome 22q11.2 deletion syndrome. *Front. Psychol.* **5**, 566 (2014).

35. Maeder, J. et al. Developmental trajectories of executive functions in 22q11.2 deletion syndrome. *J. Neurodev. Disord.* **8**, 10 (2016).

36. Schneider, M. et al. Preliminary structure and predictive value of attenuated negative symptoms in 22q11.2 deletion syndrome. *Psychiatry Res.* **196**, 277–284 (2012).

37. Van Amelsvoort, T. et al. Processing facial emotions in adults with velo-cardio-facial syndrome: functional magnetic resonance imaging. *Br. J. Psychiatry* **189**, 560–561 (2006).

38. Andersson, F. et al. Impaired activation of face processing networks revealed by functional magnetic resonance imaging in 22q11.2 deletion syndrome. *Biol. Psychiatry* **63**, 49–57 (2008).

39. Azuma, R. et al. An fMRI study of facial emotion processing in children and adolescents with 22q11.2 deletion syndrome. *J. Neurodev. Disord.* **7**, 1 (2015).

40. Haxby, J. V., Hoffman, E. A. & Gobbini, M. I. The distributed human neural system for face perception. *Trends Cogn. Sci.* **4**, 223–233 (2000).

41. Eliez, S. et al. Velocardiofacial syndrome: are structural changes in the temporal and mesial temporal regions related to schizophrenia? *Am. J. Psychiatry* **158**, 447–453 (2001).

42. Van Amelsvoort, T. et al. Structural brain abnormalities associated with deletion at chromosome 22q11 Quantitative neuroimaging study of adults with velo-cardio-facial syndrome. *Br. J. Psychiatry* **178**, 412–419 (2001).

43. Campbell, L. E. et al. Brain and behaviour in children with 22q11.2 deletion syndrome: a volumetric and voxel-based morphometry MRI study. *Brain* **129**, 1218–1228 (2006).

44. Glaser, B. et al. Structural changes to the fusiform gyrus: a cerebral marker for social impairments in 22q11.2 deletion syndrome? *Schizophr. Res.* **96**, 82–86 (2007).

45. Friston, K. J. et al. The trouble with cognitive subtraction. *Neuroimage* **4**, 97–104 (1996).

46. D'Esposito, M. Why methods matter in the study of the biological basis of the mind: A behavioral neurologist's perspective. In *The Cognitive Neuroscience of Mind: A Tribute to Michael S. Gazzaniga* (eds Reuter-Lorenz, P. A., Baynes, K., Mangun, G. R. & Phelps, E. A.) 203–221 (MIT Press, Cambridge, MA, 2010).

47. Norcia, A. M., Appelbaum, L. G., Ales, J. M., Cottereau, B. R. & Rossion, B. The steady-state visual evoked potential in vision research: a review. *J. Vis.* **15**, 4–4 (2015).

48. Dzhelyova, M., Jacques, C. & Rossion, B. At a single glance: fast periodic visual stimulation uncovers the spatio-temporal dynamics of brief facial expression changes in the human brain. *Cereb. Cortex* **27**, 4106–4123 (2017).

49. Leleu, A. et al. Tuning functions for automatic detection of brief changes of facial expression in the human brain. *Neuroimage* **179**, 235–251 (2018).

50. Mackinnon, A. & Mulligan, R. Estimation de l'intelligence prémorbide chez les francophones. *L'Encéphale* **31**, 31–43 (2005).

51. Kay, S. R., Flszbein, A. & Opfer, L. A. The positive and negative syndrome scale (PANSS) for schizophrenia. *Schizophr. Bull.* **13**, 261 (1987).

52. Bell, M., Milstein, R., Beam-Goulet, J., Lysaker, P. & Cicchetti, D. The Positive and Negative Syndrome Scale and the Brief Psychiatric Rating Scale: reliability, comparability, and predictive validity. *J. Nerv. Ment. Dis.* **180**, 723–728 (1992).

53. Lochy, A., Van Belle, G. & Rossion, B. A robust index of lexical representation in the left occipito-temporal cortex as evidenced by EEG responses to fast periodic visual stimulation. *Neuropsychologia* **66**, 18–31 (2015).
54. Rossion, B., Torfs, K., Jacques, C. & Liu-Shuang, J. Fast periodic presentation of natural images reveals a robust face-selective electrophysiological response in the human brain. *J. Vis.* **15**, 18 (2015).

55. Lochy, A., Van Reybroeck, M. & Rossion, B. Left cortical specialization for visual letter strings predicts rudimentary knowledge of letter-sound association in preschoolers. *Proc. Natl Acad. Sci.* **113**, 8544–8549 (2016).

56. de Heering, A. & Rossion, B. Rapid categorization of natural face images in the infant right hemisphere. *eLife* **4**, e06564 (2015).

57. Antshel, K. M. et al. Cognitive and psychiatric predictors to psychosis in velocardiofacial syndrome: a 3-year follow-up study. *J. Am. Acad. Child Adolesc. Psychiatry* **49**, 333–344 (2010).

58. Schneider, M. et al. Predominant negative symptoms in 22q11.2 deletion syndrome and their associations with cognitive functioning and functional outcome. *J. Psychiatr. Res.* **48**, 86–93 (2014).

59. Leppänen, J. M. & Nelson, C. A. Tuning the developing brain to social signals of emotions. *Nat. Rev. Neurosci.* **10**, 37–47 (2009).