PNAS Nexus, 2024, 3, 1–12
https://doi.org/10.1093/pnasnexus/pgae095
Advance access publication 1 March 2024

Research Report

Simon Faghel-Soubeyrand a,b,*, Meike Ramon c, Eva Bamps
Roberto Caldara g, Frédéric Gosselinb and Ian Charest b,*d, Matteo Zoiae, Jessica Woodhamsb,f, Anne-Raphaelle Richoz

g

,

a

Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, UK
Département de psychologie, Université de Montréal, Montréal, Québec H2V 2S9, Canada
Institute of Psychology, University of Lausanne, Lausanne CH-1015, SwitzerlanddCenter for Contextual Psychiatry, Department of Neurosciences, KU Leuven, Leuven ON5, Belgiume
Department for Biomedical Research, University of Bern, Bern 3008, SwitzerlandfSchool of Psychology, University of Birmingham, Hills Building, Edgbaston Park Rd, Birmingham B15 2TT, UKg
Département de Psychology, Université de Fribourg, Fribourg CH-1700, Switzerland*To whom correspondence should be addressed: Email: ian.charest@umontreal.ca (I.C.); Email: simon.faghel-soubeyrand@psy.ox.ac.uk (S.F.-S.)
Edited By: Stephen Flemingbc

Abstract
Why are some individuals better at recognizing faces? Uncovering the neural mechanisms supporting face recognition ability has provenelusive. To tackle this challenge, we used a multimodal data-driven approach combining neuroimaging, computational modeling, andbehavioral tests. We recorded the high-density electroencephalographic brain activity of individuals with extraordinary facerecognition abilities—super-recognizers—and typical recognizers in response to diverse visual stimuli. Using multivariate patternanalyses, we decoded face recognition abilities from 1 s of brain activity with up to 80% accuracy. To better understand themechanisms subtending this decoding, we compared representations in the brains of our participants with those in artificial neuralnetwork models of vision and semantics, as well as with those involved in human judgments of shape and meaning similarity.
Compared to typical recognizers, we found stronger associations between early brain representations of super-recognizers andmidlevel representations of vision models as well as shape similarity judgments. Moreover, we found stronger associations between
late brain representations of super-recognizers and representations of the artificial semantic model as well as meaning similarity
judgments. Overall, these results indicate that important individual variations in brain processing, including neural computations
extending beyond purely visual processes, support differences in face recognition abilities. They provide the first empirical evidence
for an association between semantic computations and face recognition abilities. We believe that such multimodal data-driven
approaches will likely play a critical role in further revealing the complex nature of idiosyncratic face recognition in the human brain.

Significance Statement
The ability to robustly recognise faces is crucial to our success as social beings. Yet, we still know very little about the brain mecha­
nisms allowing some individuals to excel at face recognition. This study builds on a sizeable neural dataset measuring the brain ac­
tivity of individuals with extraordinary face recognition abilities—super-recognizers—to tackle this challenge. Using state-of-the-art
computational methods, we show robust prediction of face recognition abilities in single individuals from a mere second of brain ac­
tivity and reveal specific brain computations supporting individual differences in face recognition ability. Doing so, we provide direct
empirical evidence for an association between semantic computations and face recognition abilities in the human brain—a key com­ponent of prominent face recognition models.

Introduction
The ability to robustly recognize the faces of our colleagues, friends,
and family members is paramount to our success as social beings.
Our brains complete this feat with apparent ease and speed, in a ser­ies of computations unfolding within tens of milliseconds in a wide
brain network comprising the inferior occipital gyrus, the fusiform
gyrus, the superior temporal sulcus, and more anterior areas suchas the anterior temporal lobe (1–3). Accumulating neuropsychologic­
al and behavioral evidence indicates that not all individuals,
however, are equally competent at recognizing faces in their sur­roundings (4). Developmental prosopagnosics show a great difficultyat this task despite an absence of brain injury (5). In contrast, super-

recognizers exhibit remarkable abilities for processing facial identity
and can recognize individuals even after little exposure several years
before (6–8). The specific nature of the neural processes responsible
for these individual differences remains largely unknown. So far, in­dividual differences studies have used univariate techniques to in­vestigate face-specific aspects of brain processing. This revealed
that contrasts between responses to faces compared to nonfaces,
measured by the N170 event-related potential component or by
the blood oxygen level dependent signals in regions of interest, aremodulated by ability (9–15). However, univariate and contrast ap­
proaches are limited in their capacity to reveal the precise natureof the underlying brain computations (16–19).

Competing Interest: The authors declare no competing interest.
Received: June 19, 2023. Accepted: February 20, 2024
© The Author(s) 2024. Published by Oxford University Press on behalf of National Academy of Sciences. This is an Open Access article
distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits
unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

Decoding face recognition abilities in the human brain

2 | PNAS Nexus, 2024, Vol. 3, No. 3

Results
Behavioral results
All participants’ face recognition ability was assessed using theCambridge Face Memory Test long form (CFMT+ (8)). Scores on
the CFMT+ ranged from 50 to 85 in the typical recognizers group(MTRs = 70.00; SD = 9.09), and from 92 to 100 in the experimental
super-recognizers group (MSRs = 95.38, SD = 2.68; difference be­
tween groups: t(31) = 10.6958, P < 0.00001 see Fig. 1a). The main
experimental task was a one-back task (Fig. 1b). Accuracy was sig­
nificantly greater for the super-recognizers (MSRs = 0.8649, SD =0.0626) than for the typical recognizers (MTRs = 0.7591, SD = 0.096;
t(30) = 3.6131, P = 0.0011). This was also true when analyzing
separately face (MSRs = 0.8677, SD = 0.0590; MTRs = 0.7385,
SD = 0.1048; t(30) = 4.2180, P = 0.00020) and nonface trials(MSRs = 0.8619, SD = 0.0750; MTRs = 0.7798, SD = 0.1000; t(30) =2.6000, P = 0.0143). Furthermore, accuracy in the one-back task
was positively correlated with scores on the CFMT+ (r = 0.68, P <0.001; response time was marginally associated with CFMT+, r =0.37, P = 0.04). We observed a significant difference in response times
between the two groups for face stimuli (MSRs = 0.6222 ms, SD =0.1386 ms; MTRs = 0.6817 ms, SD = 0.0660 ms; P = 0.0258) but not for
nonface stimuli (MSRs = 0.6262 ms, SD = 0.1401 ms; MTRs =0.6739 ms, SD = 0.0643 ms; P = 0.0801).

Discriminating super-recognizers and typicalrecognizers from 1 s of brain activity
With this sizable and category-rich dataset, we first attempted toclassify a participant as either a super- or a typical recognizer
based solely on their brain activity. More specifically, we trained
Fisher linear discriminants to predict group membership fromsingle, 1-s trials of EEG patterns (in a moving searchlight of fiveneighboring electrodes; Fig. 2b). We observed up to ∼80% crossvalidated decoding performance, peaking over electrodes in
the right hemisphere. This performance is impressive given thatthe noise ceiling imposed on our classification by the test–retest
reliability of the CFMT+ (8, 27), the gold-standard test used to
identify super-recognizer individuals, is ∼93% (SD = 2.28%; see
methods). To reveal the time course of these functional differen­ces, we applied the same decoding procedure to each 4-ms inter­val of EEG recordings. Group-membership predictions attained
statistical significance (P < 0.001, permutation tests, Fig. 2a) fromabout 65 ms to at least 1 s after stimulus onset, peaking around
135 ms, within the N170 window (28, 29).
Notably, similar results were obtained following the presenta­tion of both face and nonface visual stimuli (Fig. 2a; see also

Fig. S1). This did not result from face representations stored in
short-term memory from one-back trials. Indeed, we repeatedour decoding analysis for nonface trials either preceded by facetrials or by nonface trials, and found significant decoding of groupmembership in both cases (Fig. S1). In addition, we successfully
cross-decoded group membership from a model trained on face
EEG activity and applied to nonface EEG activity (see Fig. S2).
The decoding of group membership could be based on various fea­tures. Some of these features are directly related to brain repre­
sentations for object and face recognition, and these aspects arefurther explored in subsequent sections of this article. On the oth­er hand, there could be additional contributing features that are
not directly linked to object and face recognition. For instance, dif­ferences in motor responses between the two subject groups
might explain these results to some extent. However, excluding
the 10% of trials with a motor response did not affect decoding ac­
curacy (Fig. S2). Additionally, the decoding model might have re­lied on potential noise differences between the two subject
groups. Nevertheless, our analysis did not reveal any evidence
supporting such differences in the cross-participant similarity of
the representational dissimilarity matrices (RDMs) for bothgroups (see the Linking neural representations with computation­
al models of vision section; Fig. S3).

Predicting individual recognition ability from1 s of brain activity
An ongoing debate in individual differences research is whether
the observed effects emerge from qualitative or quantitativechanges in the supporting brain mechanisms (30–38). The decod­ing results presented up to this point might give the impressionthat face recognition ability is supported by qualitative differen­
ces in brain mechanisms. However, these results were obtainedwith dichotomous classification models applied, by design, tothe brains of individuals from a bimodal distribution of ability
scores (35).
To better assess the nature of the relationship between neuralrepresentations and ability in the general population, we thus per­formed a decoding analysis on the typical recognizers only, using
a continuous regression model. Specifically, we used crossvalidated fractional ridge regression (39) to predict individual
CFMT+ face recognition ability scores from single-trial EEG data.
This showed essentially similar results to the previous dichotomic
decoding results: performance was above statistical threshold(P < 0.01, False Discovery Rate corrected) from about 80 ms to atleast 1 s, peaking around 135 ms following stimulus onset for
both face and nonface stimuli (Fig. 2c, peak-rhoface = 0.4149 at
133 ms, peak-rhononface = 0.4899 at 141 ms). This accurate decod­
ing of individual scores from EEG patterns is compatible with aquantitative account of variations in brain mechanisms across in­dividuals differing in face recognition abilities. Altogether, these
decoding results provide evidence for important, quantitative andtemporally extended variations in the brain activity supporting
face recognition abilities. This extended decoding suggests effects
of individual ability across multiple successive processing stages.

Linking neural representations
with computational models of vision
Decoding time courses, however, offer limited insights on the level ofbrain computations (40, 41). To better characterize the visual brain
computations covarying with face recognition ability, we compared,
using representational similarity analysis (20–22, 42), the brain rep­resentations of our participants to that of convolutional neural

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

Here, we tackled this challenge with a data-driven approach.
We examined the functional differences between the brains ofsuper-recognizers and typical recognizers using decoding and rep­resentational similarity analyses (RSAs (18, 20–23)) applied tohigh-density electrophysiological (EEG) signals and artificial neur­
al network models. We recruited 33 participants, including 16
super-recognizers, i.e. individuals better than the 98th percentile
on a battery of face recognition tests (8) (Fig. 1a). We measured
EEG in more than 100,000 trials while participants performed a
one-back task. The objects depicted in the stimuli belonged tomultiple visual categories including face images of different
sexes, emotions, and identities, as well as images of man-madeand nonface natural objects (e.g. a computer, a plant), animals(e.g. a giraffe, a monkey), and scenes (e.g. a city, a dining room;
Fig. 1b).

Faghel-Soubeyrand et al. |

a

3

b

networks (CNNs) trained to categorize objects (43–45). These CNNs
process visual features of gradually higher complexity and abstrac­
tion along their layers (45), from low-level (e.g. orientation, edges) tohigh-level features (e.g. objects and object parts).
The brain representations were characterized by computing
RDMs for each participant and for each 4-ms time interval.
These brain RDMs were derived using the cross-validated decod­
ing performance of a linear discriminant model, where brain ac­
tivity was decoded for every pair of stimuli at a given timeinterval (46, 47); see Fig. S4 for the group-average RDMs andtime course of key categorical distinctions. The visual model rep­resentations were characterized by computing RDMs from thelayers of the CNNs, using Pearson correlations of the unit activa­
tions across all pairs of stimuli. Compared to typical participants,
we found that the brain RDMs of super-recognizers showed larger
mutual information (48) with the layer RDMs of CNNs that re­present midlevel features (e.g. combinations of edges, contour,
shape, texture (45, 49)) between 133 and 165 ms (Fig. 3a, P < 0.05,
cluster-test; see also Fig. S5 for similar results with unconstrained
analyses; supplementary analyses on specific category conditionsin the RDMs are shown in Fig. S6). These results indicate that mid­level representations of an object-trained CNN matched the rep­resentations of super-recognizers more closely than those of
typical participants. We replicated these results using a facetrained CNN, e.g. (51–53), VGGface (54), which possess midlevel
representations similar to those of object-trained CNN (seeFig. S7; P < 0.05, cluster-test). The stronger association between
brain representational geometries of the super-recognizers withcomputational models of vision could be explained by a marked
difference in signal to noise between the two groups of partici­
pants. To control for this potential confound, we computed thecross-participant similarity of the RDMs in both groups (seeFig. S3). If the signal to noise was larger in the super-recognizers,
we would expect larger cross-participant similarity of the RDMs;
however, we observed no significant difference between the twogroups.

Linking neural representations withcomputational model of semantics
The finding that ability decoding was significant as late as 1 s after
stimulus onset hints that brain computations beyond what is typ­ically construed as pure visual processing also differ as a functionof face recognition ability. To test this hypothesis, we asked five

new participants to write captions describing the images pre­sented during our experiment (e.g. “A city seen through a forest.”),and used a deep averaging network (Google Universal Sentence
Encoder, GUSE (50)) to transform these captions into embeddings(points in a caption space). GUSE has been trained to predict se­
mantic textual similarity from human judgments, and its embed­
dings generalize to an array of other semantic judgment tasks (50).
We then compared the RDMs computed from this semantic modelto the brain RDMs of both typical- and super-recognizers.
Importantly, both this comparison, and the one comparing brain
and visual models, excluded the information shared between
the semantic and visual models (but see Fig. S8 for similar
results with unconstrained analyses). We found larger mutual in­formation with these semantic representations in the brains ofsuper-recognizers than in those of typical recognizers in alate window between 598 and 727 ms (Fig. 3c, P < 0.05,
cluster-test). Supplementary analyses on specific stimulus cat­egories of the RDMs (Fig. S9) suggest that these results emerged
mainly from the face vs. face and face vs. nonface stimuli pairconditions.

Linking neural representations with behavioral
representations for shape and semantic similarity
judgments
Our findings so far suggest that midlevel visual and semantic
brain processes both support individual differences in face recog­
nition abilities. We looked for further support for these conclu­
sions using RDMs derived from a behavioral experiment. A
group of 32 new human participants were submitted to two mul­
tiple arrangement tasks (55–57) in which they were asked to evalu­
ate the shape similarities of all pairs of the 49 visual stimuli usedin the main experiment, and the meaning similarities of all pairsof the 49 mean sentence captions produced by five human partic­
ipants to describe these images and used for the semantic model(see the Linking neural representations with computational mod­el of semantics section). More specifically, participants arranged
the images/sentences inside a white circular arena according to
the task instructions using simple drag and drop operations (seeFig. 4). We computed the mutual information between the mean
RDMs extracted from each of these tasks and the time-resolved
brain RDMs of super- and typical recognizers as well as the same
while excluding the information shared with the other task.
Results indicated only a trend for shape representations being

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

Fig. 1. Experimental procedure. a) The histogram shows the Cambridge Face Memory Test long form (CFMT+ (8)) scores of super-recognizers (yellowbars), typical recognizers (black bars), and an additional 332 neurotypical observers from three independent studies for comparison (24–26). b)
Participants engaged in a one-back task while their brain activity was recorded with high-density electroencephalography. The objects depicted in thestimuli belonged to various categories, such as faces, objects, and scenes. Note that the face drawings shown here are an anonymized substitute to theexperimental face stimuli presented to our participants.

4 | PNAS Nexus, 2024, Vol. 3, No. 3(area under the curve, AUC)

0.70

p<.001, permutations
faces
non-faces

0.65

0.60

0.55

0.50
0

0.2

0.4

0.6

0.8

1

135 ms

350 ms

560 ms775 mstime (s)b0-1100 ms
searchlight
80

70

B9

peak decoding

60

non-faces

5050

AUC (%)AUC (%)

faces

A26

0.60
p<.01, permutations(rho, fractional ridge regression)

prediction of individual ability scores

c

faces
non-faces

0.50
0.40
0.30
0.20

0.10

0
0

0.2

0.4

0.6

0.8

1

time (s)Fig. 2. Decoding interindividual recognition ability variations from EEG activity. a) Trial-by-trial group-membership predictions (super-recognizer ortypical recognizer) were computed from EEG patterns, for each 4-ms interval, while participants processed face (blue trace) or nonface stimuli (graytrace). Significant decoding performance occurred as early as 65 ms, peaked in the N170 window, and lasted for the remainder of the EEG epochs (P < 0.001). b)
Topographies were obtained using searchlight decoding analyses, either concatenating all time points (left topographies) or for selected time windows (right
topographies). Concatenating all time points resulted in peak classification performance of 77.3% over right occipitotemporal electrodes for face and 77.5%
over right occipitotemporal electrodes for nonface conditions. In the N170 window, we observed a peak classification performance of 74.8% over right-temporalelectrodes for face, and 72.1% over left-temporal electrodes for nonface conditions. c) We decoded the CFMT+ scores of the typical recognizers using fractionalridge regression. This yielded similar results with significant decoding as early as 75 ms, peaking around the N170 time window (peak-rhoface = 0.4149,
peak-rhononface = 0.4899), and lasted for the remainder of the EEG epochs (P < 0.01, 1K permutations, 10 repetitions).

enhanced around midlatencies in super-recognizers relative to typ­ical recognizers which did not survive cluster correction (P < 0.01,
uncorrected; PMI = 0.1259; PCMI = 0.2098; cluster-corrected; see
Fig. 4). Meaning representations were enhanced in late latencies insuper-recognizers compared to typical recognizers (sentence mean­
ing: 635–787 ms, P < 0.05, cluster-corrected; see Fig. 4). These results
confirm that semantic representations at relatively late latencies
and, to a lesser degree, shape representations at midlatencies are en­hanced in the brains of super-recognizers.

Discussion
Using a data-driven approach combining neuroimaging, compu­
tational models, and behavioral tests, we characterized the com­putations modulated by variations in face recognition ability inthe human brain. We recorded high-density electroencephalo­graphic (EEG) responses to face and nonface stimuli in superrecognizers and typical recognizers. Using multivariate analysis,
we reliably decoded group membership as well as recognition

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

"super" vs "typical" evidenceaFaghel-Soubeyrand et al. |5abilities of single individuals from a single second of brain activity.
We then characterized the neural computations underlying these
individual differences by comparing human brain activity
with representations from artificial neural network models of vi­
sion and semantics using representational similarity analysis.
Furthermore, we compared the brain activity with similarity

judgments derived from additional human participants engaged
in two tasks. In the first task, participants judged our visual stim­uli on the similarity of their shape, while in the second task, par­ticipants judged sentence captions describing these stimuli on thesimilarity of their meaning. These sets of comparisons revealed
two main findings. First, we found higher similarity between early

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

Fig. 3. Comparison of super- and typical-recognizer brain representations with those of artificial neural networks of visual and semantic processing. a)
RDMs were computed from CNNs (43, 44) of vision, human brain activity, and a deep neural network of caption classification and sentence semantics (50).
To characterize the CNN RDMs, we computed the pairwise similarity between unit activation patterns for each image independently in each CNN layer.
The caption-level RDMs were derived from human caption descriptions of the images transformed into sentence embeddings. Brain RDMs werecomputed using cross-validated decoding performance between the EEG topographies from each pair of stimuli at every 4 ms time point. Mutual
information (48) between the model RDMs and the brain RDMs was assessed, for every participant, at each 4 ms step from stimulus onset. b) Mutual
information between brain RDMs and AlexNet RDMs (removing shared mutual information between brain and semantic model) is shown for typical(gray solid curve) and super-recognizers (pink solid curve). We found greater similarity with midlevel visual representations (layers 4 and 5 shown, butsimilar results were found for midlayers of VGG16, another popular CNN model; see Fig. S5) in the brains of super-recognizers (black line indicatessignificant contrasts, P < 0.05, cluster-corrected) between 133 and 165 ms. Similar results were observed when comparing brains and CNN models
without removing the shared mutual information between brains and the semantic (caption-level) model (Fig. S5). c) Mutual information with thesemantic model (excluding shared mutual information between brain and AlexNet) differed for typical- and super-recognizers in a later time window
centered around 650 ms (cyan curve; super > typical, P < 0.05, cluster-corrected). Again, similar results were observed when comparing brains and thesemantic model without removing the shared mutual information between the brain and CNN model (see Fig. S8).

6 | PNAS Nexus, 2024, Vol. 3, No. 3

bFig. 4. Linking neural representations with behavioral representations for shape, function, and semantic similarity judgments. a) Mutual information
between brain RDMs and the mean RDM built from shape similarity judgments (first column) and sentence meaning similarity judgments (second
column) is shown for typical- (gray curves) and super-recognizers (coloured curves). Greater similarity with shape information in the brains ofsuper-recognizers (P < 0.01, uncorrected; P-correctedMI = 0.1259) and greater similarity with sentence meaning information for super-recognizers(P < 0.01, uncorrected; P-correctedMI = 0.0819) only reached significance before cluster corrections. The shaded areas of all curves represent the SE. b)
Conditional mutual information between brain RDMs and the mean RDM built from similarity judgments of shape and of sentence caption meaning(removing shared mutual information between brain and sentence caption meaning RDM for shape similarity, and vice versa) is shown in typical- andsuper-recognizers. We found greater similarity with sentence meaning in the brains of super-recognizers between 635 and 787 ms (black line indicatessignificant contrasts, P < 0.05, cluster-corrected), in agreement with our comparisons with the artificial semantic model (Fig. 3c). Greater similarity withshape information in the brains of super-recognizers only reached significance before cluster corrections (P < 01, uncorrected; P-correctedCMI = 0.2098).

brain representations of super-recognizers and midlevel re­presentations of vision models as well as, to a lesser degree,
shape similarity judgments. Second, this approach revealed
higher similarity between late brain representations of superrecognizers and representations of an artificial semantic model
as well as sentence caption similarity judgments. To our knowl­edge, this is the first demonstration of a link between face rec­ognition ability and brain computations beyond high-levelvision. Overall, these findings revealed specific computations

supporting our individual ability to recognize faces and suggest
widespread variations in brain processes related to this crucialability.
We achieved robust decoding of face recognition ability when
examining EEG responses to face and nonface stimuli. This is con­sistent with several neuropsychological (33, 58–62) and brain im­
aging findings (12, 32, 63, 64) showing face and nonface
processing effects in individuals across the spectrum of face rec­ognition ability (19, 65), but see Refs. (13, 66–68).

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024aFaghel-Soubeyrand et al. |Furthermore, while our decoding approach indicates that im­
portant differences in brain processing emerge from early (80 ms)
to late (∼1 s) processing windows, our RSA modeling approach
only explained part of these processing windows. Specifically,
while broad visual (∼150 ms) and semantic representations(∼600 ms) were found to differ in super-recognizers using thiscomputational approach, it still remains to be shown what specif­
ic representations are critical in differentiating the best facerecognizers during other windows of processing (e.g. mid-late pro­cessing around 400 ms).

Conclusion
Our results offer a stepping stone for a better understanding offace recognition idiosyncrasies in the human brain. Indeed, withthe development of novel and better artificial models simulating
an increasing variety of cognitive processes, and with the techno­
logical advances allowing the processing of increasingly larger
neuroimaging datasets, the approach described here provides a
new and promising way to tackle the link between individual dif­ferences in human behavior and specific computations in thebrain. In addition, this decoding approach may provide quick
and accurate alternatives to standardized behavioral tests assess­ing face recognition ability, for example in the context of security
settings that benefit from strong face processing skills among
their personnel (such as police agencies, border patrol, etc.). It
could also be used in a closed-loop training procedure designed
to improve face recognition ability (82).

Methods
Participants
A total of 33 participants were recruited for this study. The firstgroup consisted of 16 individuals with exceptional ability in facerecognition—super-recognizers. The second group was composedof 17 neurotypical controls. These sample sizes were chosen ac­
cording to the effect sizes described in previous multivariate ob­ject recognition studies (46, 47, 57). The data from one
super-recognizer were excluded due to faulty EEG recordings.
No participant had a history of psychiatric diagnostic or neuro­
logical disorder. All had normal or corrected to normal vision.
This study was approved by the Ethics and Research Committeeof the University of Birmingham, and informed consent was ob­
tained from all participants.
Sixteen previously known super-recognizers were tested in thecurrent study (30–44 years old, 10 female). Eight of these (SR1–SR8)
were identified by Prof. Josh P. Davis from the Universityof Greenwich using an online test battery comprising a total of
six face cognition tasks (6) and tested at the University ofBirmingham. The remaining eight (SR9 to SR16) were identified us­
ing three challenging face cognition tests (7) and were tested at the
University of Fribourg. The behavioral test scores for all partici­
pants are provided in Tables S1 and S2. Across SR cohorts, theCambridge Face Memory Test long form (CFMT+ (8)) was used as
the measure of face identity processing ability. A score greaterthan 90 (i.e. 2 SD above average) is typically considered the thresh­
old for super-recognition (8, 59, 83). Our 16 super-recognizers allscored above 92 (M = 95.31; SD = 2.68). A score of 92 corresponds
to the 99th percentile according to our estimation from a groupof 332 participants from the general population recruited in three
independent studies (24–26).
An additional 17 typical recognizers (20–37 years old, 11 fe­male) were recruited and tested on campus at the University of

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

The decoding we observed for face and nonface stimuli peaked
at right occipitotemporal electrodes, in the temporal window
around the N170 component (28). At that time, the representa­
tions in the brains of our participants differed most with respect
to the mid-layer representations of artificial models of vision.
These layers have been previously linked to processing in humaninferotemporal cortex (45, 69–71) and functionally to midlevel fea­
ture representations such as combinations of edges and parts ofobjects (45, 49). Such associations with the N170, however, do
not mean that this component is exclusively involved in thesemidlevel processes. Rather, it suggests that other visual computa­
tions, including the high-level visual computations usually asso­
ciated with the N170, do not differ substantially between
super-recognizers and typical recognizers. The fact that thesemidlevel features are mostly shared between face and nonface
stimuli could explain at least partly the high decoding perform­
ance observed for both classes of stimuli. They suggest that a mid­level visual processing is enhanced in super-recognizers leading
to improved processing of faces and objects.
Crucially, we found that face recognition ability is also associ­ated with semantic computations that extend beyond basic-levelvisual categorization in a late time window around the P600 com­ponent (72–74). Recent studies using computational techniques
have shown that word representations derived from models ofnatural language processing explain significant variance in thevisual ventral stream (18, 75–77). The current study goes beyond
this recent work in two ways. First, our use of human sentence de­scription and sentence encoders to characterise semantic(caption-level) computations provides a more abstract descriptionof brain representations. Second, and most importantly, our work
revealed a link between semantic brain computations and indi­vidual differences in face recognition ability. An association be­
tween semantic processes and face recognition ability had been
posited in models of face recognition (1, 78) but, to our knowledge,
it had never been shown empirically before.
Overall, thus, our findings suggest important differences in per­ceptual and semantic representations in individuals with out­
standing ability to recognize faces. The higher similarity withcomputational models of vision indicates that super-recognizers
have more efficient midlevel representations. These enhanced
representations suggest that part-based information about faces
and objects, putatively emerging from midlevel occipitotemporal
regions (79), is richer in individuals with strong face recognition
abilities (24). Furthermore, our findings show that the more simi­
lar the brain representations of an individual are to taskoptimized computational models of semantics, the better they
are at recognizing faces. These enhanced late semantic represen­tations, for example, might emerge from enhanced subordinatelevel information about objects and faces (80, 81).
Our approach of decoding group membership to reveal electro­
physiological differences between super-recognizers and typicalrecognizers, followed by representational similarity analysis
with computational models of vision and language, enabled re­vealing potential mechanisms underlying enhanced recognition
ability. Other possible differences between our participants mightalso contribute to our ability to decode their brain signals.
Differences in top-down (i.e. attention) mechanisms, better ability
to memorise images more generally, or both, could also lead to en­
hanced representations. Interestingly, we could decode from both
face and nonface categories, and across categories (training withface trials and testing on nonface trials) suggesting that the mech­anisms subtending the enhanced abilities of super-recognizers
are not restricted to faces (4, 59).

7

8 | PNAS Nexus, 2024, Vol. 3, No. 3

Tasks
CFMT+
All participants were administered the CFMT long form, or
CFMT + (8). In the CFMT+, participants are required to memorizea series of face identities, and to subsequently identify the newly
learned faces among three faces. It includes a total of 102 trials of
increasing difficulty. The duration of this test is about 15 min. EEG
was not recorded while participants completed this test.

One-back task
Stimuli
The stimuli used in this study consisted of 49 images of faces, an­imals (e.g. giraffe, monkey, puppy), plants, objects (e.g. car, com­puter monitor, flower, banana), and scenes (e.g. city landscape,kitchen, bedroom). The 24 faces (13 identities, 8 males, and 8 neu­
tral, 8 happy, 8 fearful expressions) were sampled from theRadboud Face dataset (84). The main facial features were aligned
across faces using Procrustes transformations. Each face image
was revealed through an ellipsoid mask that excluded nonfacialcues. The nonface images were sampled from the stimulus setof Kiani et al. (85). All stimuli were converted to 250 × 250 pixels(8 × 8° of visual angle) grayscale images. The mean luminanceand the luminance SD of these stimuli were equalized using the
SHINE toolbox (86).

Procedure
We measured high-density electroencephalographic (EEG; sam­pling rate = 1,024 Hz; 128-channel BioSemi ActiveTwo headset)
activity while participants performed ∼3,200 trials of a one-backtask in two recording sessions separated by at least one day andby a maximum of 2 weeks (Fig. 1b). Participants were asked topress a computer keyboard key on trials where the current image
was identical to the previous one. Repetitions occurred with a 0.1
probability. They were asked to respond as quickly and accurately
as possible. Feedback was done in the form of a change in color of
the fixation point (red or green) after a repetition trial (which hap­
pened on a 0.1 probability basis). This was done to help partici­
pants pay attention during the task. Target trials were notexcluded from the analyses. A trial unravelled as follows: a white
fixation point was presented on a gray background for 500 ms(with a jitter of ± 50 ms); followed by a stimulus presented on agray background for 600 ms; and, finally, by a white fixation point
on a gray background for 500 ms. Participants had a maximum of
1,100 ms following stimulus onset to respond. This interval, as
well as the 200 ms preceding stimulus onset, constituted theepoch selected for our EEG analyses. In total, our participants
completed 105,600 one-back trials which constituted ∼32 h of
EEG epochs.

Shape and sentence meaning multiple arrangement tasks
Thirty-two new neurotypical participants took part in two mul­
tiple arrangement tasks (22, 87) in counterbalanced orders. In
two of the tasks, they were asked to evaluate the shape or function
similarities of the 49 stimuli used in the main experiment while, inthe other task, they were instructed to judge the meaning similar­
ities of sentence captions describing these stimuli (see theSemantic caption-level deep averaging neural network RDM sec­tion for more information about these sentence captions).
More specifically, participants were asked to arrange stimuli or
sentence captions on a computer screen inside a white circulararena by using computer mouse drag and drop operations.
During the shape/function (vs. meaning) multiple arrangementtask, they were instructed to place the displayed visual stimuli(vs. sentence captions) in such a way that their pairwise distances
match their shape/function (vs. meaning) similarities as much as
possible (Fig. 4). On the first trial of each task, participants ar­ranged all 49 items. On subsequent trials, a subset of these items
was selected based on an adaptive procedure aimed at minimizing
uncertainty for all possible pairs of items (e.g. items that initiallywere placed very close to each other) and at better approximating
the high-dimensional perceptual representational space (87). This
procedure was repeated until the task timed out (20 min).
We computed one representational dissimilarity matrix (RDM)
per task per participant. Three participants were excluded fromthe final sample because their RDMs differed from the mean
RDMs by more than 2 SDs. Finally, we averaged the remaining in­dividual RDMs within each task.

Analyses
All reported analyses were performed independently for each EEG
recording session and then averaged. Analyses were completedusing custom code written in MATLAB (MathWorks) and Python.

EEG preprocessing
EEG data were preprocessed using FieldTrip (88): continuous raw
data were first rereferenced relative to Cz, filtered with a bandpass filter (0.01–80 Hz), segmented into trial epochs from −200 to
1,100 ms relative to stimulus onset, and down-sampled at 256 Hz.

Decoding analyses
Whole-brain analyses
To predict group membership from EEG brain activity, we trained
Fisher linear discriminant classifiers to predict participants’ groupmembership based on raw EEG topographies, using all 128 chan­nels of single-trial EEG data as features. Notably, here, the decod­
ing is made on an across-participants basis. This was done across alltrials of either face or nonface condition, for each of the two ses­
sions separately (∼26,000 observations per condition, per session,
5-fold cross-validation, 5 repetitions (89, 90)). The number of trials
was matched across participants. This process was repeated overall EEG time samples separately, starting from −200 ms and end­
ing to 1,100 ms after stimulus onset, creating decoding accuracy
time courses. The area under the curve (AUC) was used to assess
sensitivity. Decoding time courses were averaged across the two
EEG sessions. The resulting evidence indicates when superrecognizers can be categorized from brain activity when process­ing faces (blue) and nonface stimuli (gray), as shown in Fig. 2a.
Additional control decoding analyses investigating effects of oneback trials on the predictions are shown in Fig. S1. These trials re­quired that our participant compared their representations of thepresented image and the one stored in short-term memory. This
showed similar findings, with one notable difference being thatthe face–face discrimination condition was the one that obtained
peak decoding accuracy.

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

Fribourg (n = 10) and the University of Birmingham (n = 7). Their
CFMT+ scores ranged from 50 to 85 (M = 70.00; SD = 9.08).
Neither the average nor the distribution of this sample differedsignificantly from those of the 332 participants from the general
population mentioned above (see Fig. 1a; t(346) = 1.3065, P =0.1922; two-sample Kolmogorov–Smirnov test; D(346) = 0.2545,
P = 0.2372).

Faghel-Soubeyrand et al. |Searchlight analysis

Regression analysis
We used fractional ridge regression models (39) to predict individ­
ual face recognition ability scores (CFMT+) among the typical rec­ognizers from EEG patterns across time. We trained our model on
subsets of 60% of the EEG patterns. We chose the alpha hyper­
parameter with the best coefficient of determination among 20 al­
pha hyperparameters ranging linearly from 0.001 to 0.99 applied
on a 30% validation set. The decoding performance was assessedusing the Spearman correlation between the CFMT+ scores andpredictions from the overall best model (applied on the remaining
10% of EEG patterns). This process was repeated 10 times and theSpearman correlations were averaged. Significance was assessedusing a permutation test (see the Group comparison and inferen­tial statistics section).

Representational similarity analysis of brain
and computational models
We compared our participants’ brain representations to thosefrom visual and semantic (caption-level) artificial neural net­
works using RSA (20–22, 42).

distances) between the sentence embeddings across all pairs of
captions, resulting in a semantic caption-level RDM for each par­ticipant. The average RDM was used for further analyses.

Comparing brain representations withcomputational models
We compared our participants’ brain RDMs to those from the vi­
sion (Fig. 3b) and semantic (Fig. 3c) models described in the previ­
ous section using conditional mutual information (CMI) (48),which measures the statistical dependence between two variables(e.g. mutual information I(x; y)), removing the effect from a third
variable (i.e. I(x; y|z)). Additional comparisons using uncon­strained mutual information between brain RDMs and both mod­els are shown in Figs. S5 and S8.

Group comparison and inferential statistics
Comparison of CMITime courses of CMI were compared between the superrecognizers and typical recognizers using independent samples t
tests and a Monte Carlo procedure at a P-value of 0.05, as imple­
mented in the Fieldtrip Toolbox (88). Familywise errors were con­trolled for using cluster-based corrections, with maximum cluster
size as cluster-level statistic and an arbitrary t threshold for clus­
ter statistic of (−1.96, 1.96) for the comparison of brain and seman­tic (excluding CNN) and for the comparison of brain and CNN(excluding semantic) time courses. The SE is shown for all curves
as color-shaded areas (Fig. 3). Analyses with mutual information(MI) (brain; CNN) and MI (brain; semantic) were completed in an
identical manner.

Brain RDMs

Time course of group-membership decoding

For every participant, we trained a Fisher linear discriminant todistinguish pairs of stimuli from every 4-ms intervals of EEG re­sponse (on all 128 channels) to these stimuli from −200 to
1,100 ms after stimulus onset (91, 92). Cross-validated AUC served
as pairwise classification dissimilarity metric. By repeating thisprocess for all possible pairs (1,176 for our 49 stimuli), we obtained
an RDM. RDMs are shown for selected time points in Fig. S4.

Significance was assessed using nonparametric permutation
tests. We simulated the null hypothesis by training the linear clas­sifier to identify shuffled group-membership labels from the ex­perimental EEG patterns. This process was repeated 1,000 times
for each time point and each one of the two sessions. We thencompared the real, experimental decoding value at each time
point to its corresponding null distribution, and rejected the null
hypothesis if the decoding value was greater than the prescribed
critical value at a P < 0.001 level.

Visual convolutional neural network RDMs
We used a pretrained AlexNet (43) as one model of the visual com­putations along the ventral stream (45). Our 49 stimuli were input
to AlexNet. Layerwise RDMs were constructed comparing the unit
activation patterns for each pair of images using Pearson correla­tions. Similarly, we computed layerwise RDMs from another wellknown CNN, VGG-16 (see Fig. S5). Following previous studies
using this model (93, 94), we averaged the convolutional layer
RDMs situated between each max pooling layers and the layers’
input into five aggregated convolutional RDMs (e.g. conv1-1 andconv1-2 into RDM-conv1); this facilitated the comparison of ourresults with the five convolutional layers of AlexNet.

Semantic caption-level deep averaging neural network RDM
We asked five new participants to provide a sentence caption de­scribing each stimulus (e.g. “a city seen from the other side of theforest,” see Fig. 1d) using the Meadows online platform (www.
meadows-research.com). The sentence captions were input in
GUSE (50) resulting in 512-dimensional sentence embeddings foreach stimulus. We then computed the dissimilarities (cosine

Time course of individual ability decoding using ridge
regression
Significance was again assessed using nonparametric permutation
testing. The ridge regression analysis predicted cross-validated
CFMT+ scores from single-trial EEG patterns, and goodness of fit
is reported using Spearman’s correlation between the predicted
and observed CFMT+ scores. Under the null hypothesis that all par­ticipants elicited comparable EEG response patterns, irrespective of
their CFMT+ score, the face recognition ability scores are exchange­able. We simulated this null hypothesis by repeating the ridge re­gression model training using randomly shuffled CFMT+ scores.
The predicted CFMT+ scores were then correlated to the empirical,
observed CFMT+ scores using Spearman’s correlation, and this was
repeated 1,000 times for each time point. We finally compared thereal, experimental correlation value with its corresponding null
distribution at each time point, and rejected the null hypothesis
if the correlation value was greater than the prescribed criticalvalue at a P < 0.01 level.

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

We conducted a searchlight analysis decoding EEG signals fromall subsets of five neighboring channels to characterize the scalp
topographies of group-membership AUC. This searchlight ana­
lysis was done either using the entire EEG time series of a trial(0–1,100 ms; Fig. 2b, leftmost topographies) or using 60 ms tem­poral windows (centered on 135, 350, 560, and 775 ms; Fig. 2brightmost topographies). We ran additional control searchlightdecoding procedures investigating the effect of one-back trials(Fig. S1).

9

10 |

PNAS Nexus, 2024, Vol. 3, No. 3

Acknowledgments

Supplementary Material
Supplementary material is available at PNAS Nexus online.

Funding
Funding for this project was supported by an ERC Starting Grant(ERC-StG-759432) to I.C., an ERSC-IAA grant to J.W., I.C., and
S.F.-S., by a Swiss National Science Foundation PRIMA(Promoting Women in Academia) grant (PR00P1_179872) to M.R.,
and by IVADO (2021-6707598907), NSERC, and UNIQUE graduate
scholarships to S.F.-S.

7

work,
8

70

cases,

andguidelinesforfuturework.

Neuropsychologia. 158:107809.
Russell R, Duchaine B, Nakayama K. 2009. Super-recognizers:
people with extraordinary face recognition ability. Psychon Bull

9

Rev. 16:252–257.
Elbich DB, Scherf S. 2017. Beyond the FFA: brain-behavior corre­

spondences in face recognition abilities. Neuroimage. 147:
409–422.
10 Herzmann G, Kunina O, Sommer W. 2010. Individual differencesin face cognition: brain–behavior relationships. J Cogn Neurosci.
22:571–589.
11 Huang L, et al. 2014. Individual differences in cortical face select­ivity predict behavioral performance in face recognition. Front
Hum Neurosci. 8:483.
12 Kaltwasser L, Hildebrandt A, Recio G, Wilhelm O, Sommer W.
2014. Neurocognitive mechanisms of individual differences inface cognition: a replication and extension. Cogn Affect Behav
Neurosci. 14:861–878.
13 Lohse M, et al. 2016. Effective connectivity from early visual cor­tex to posterior occipitotemporal face areas supports face select­

Author Contributions
S.F.-S.: conceptualization, methodology, software, formal ana­
lysis, investigation, data curation, writing—original draft, visual­ization, supervision, project administration, funding acquisition.
M.R.: investigation, resources, project administration, writing—review and editing. E.B.: investigation, project administration.
M.Z. and A.-R.R.: investigation. J.W.: funding acquisition, writing—review and editing. R.C.: resources. F.G.: methodology, writing—original draft, supervision, funding acquisition. I.C.: supervi­
sion, methodology, software, resources, formal analysis, writing—original draft, project administration, funding acquisition.

Preprints
This manuscript was posted on a preprint: https://www.biorxiv.
org/content/10.1101/2022.03.19.484245v3.

Data Availability

ivity and predicts developmental prosopagnosia. J Neurosci. 36:
3821–3828.
14 Rossion B, Retter TL, Liu-Shuang J. 2020. Understanding humanindividuation of unfamiliar faces with oddball fast periodic vis­
ual stimulation and electroencephalography. Eur J Neurosci. 52:
4283–4344.
15 Nowparast Rostami H, Sommer W, Zhou C, Wilhelm O,
Hildebrandt A. 2017. Structural encoding processes contributeto individual differences in face and object cognition: inferencesfrom psychometric test performance and event-related brain po­
tentials. Cortex. 95:192–210.
16 Vinken K, Konkle T, Livingstone M. 2023. The neural code for“face cells” is not face specific. Sci Adv. 9:eadg1736.
17 Visconti di Oleggio Castello M, Haxby JV, Gobbini MI. 2021.
Shared neural codes for visual and semantic information aboutfamiliar faces in a common representational space. Proc Natl
Acad Sci U S A. 118:e2110474118.
18 Dwivedi K, Bonner MF, Cichy RM, Roig G. 2021. Unveiling func­
tions of the visual cortex using task-specific deep neural net­

High-density EEG data associated with this article are available on
The Open Science Framework (https://osf.io/pky28/).

works. PLoS Comput Biol. 17:e1009267.
19 Harel A, Kravitz D, Baker CI. 2013. Beyond perceptual expertise:

References

revisiting the neural substrates of expert object recognition.
Front Hum Neurosci. 7:885.
20 Kriegeskorte N, et al. 2008. Matching categorical object represen­

1
2

3

4

5
6

Duchaine B, Yovel G. 2015. A revised neural framework for faceprocessing. Annu Rev Vis Sci. 1:393–416.
Kanwisher N, McDermott J, Chun MM. 1997. The fusiform facearea: a module in human extrastriate cortex specialized forface perception. J Neurosci. 17:4302–4311.
Grill-Spector K, Weiner KS, Kay K, Gomez J. 2017. The functional
neuroanatomy of human face perception. Annu Rev Vis Sci. 3:
167–196.
White D, Mike Burton A. 2022. Individual differences and themultidimensional nature of face perception. Nat Rev Psychol. 1:
287–300. https://doi.org/10.1038/s44159-022-00041-3.
Susilo T, Duchaine B. 2013. Advances in developmental proso­
pagnosia research. Curr Opin Neurobiol. 23:423–429.
Noyes E, Davis JP, Petrov N, Gray KLH, Ritchie KL. 2021. The effectof face masks and sunglasses on identity and expression

tations in inferior temporal cortex of man and monkey. Neuron.
60:1126–1141.
21 Kriegeskorte N, Kievit RA. 2013. Representational geometry: inte­
grating cognition, computation, and the brain. Trends Cogn Sci. 17:
401–412.
22 Charest I, Kievit RA, Schmitz TW, Deca D, Kriegeskorte N. 2014.
Unique semantic space in the brain of each beholder predicts
perceived similarity. Proc Natl Acad Sci U S A. 111:14565–14570.
23 Kriegeskorte N, Diedrichsen J. 2019. Peeling The Onion of brain
representations. Annu Rev Neurosci. 42:407–432.
24 Tardif J, et al. 2019. Use of face information varies systematicallyfrom developmental prosopagnosics to super-recognizers.
Psychol Sci. 30:300–308.
25 Fysh MC, Stacchi L, Ramon M. 2020. Differences between andwithinindividuals,

and subprocessesof facecognition:

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

The authors thank Prof. Josh P. Davis for sharing behavioral scoresof super-recognizers and establishing first contact to theUK-based Super-Recognizers reported here. The authors also
thank Mick Neville, from Super-Recognisers Ltd, who helped
them to get in contact with some of their super-recognizer partic­
ipants. The authors thank Rose Jutras, who helped with dataacquisition.

recognition with super-recognizers and typical observers. R Soc
Open Sci. 8:201169.
Ramon M. 2021. Super-recognizers—a novel diagnostic frame­Faghel-Soubeyrand et al. | 11
implications for theory, research and personnel selection. R Soc
Open Sci. 7:200233.
26 Faghel-Soubeyrand S, et al. 2019. The two-faces of recognition
from left and right sides of face stimuli. J Vis. 19:136d.
27 Duchaine B, Nakayama K. 2006. The Cambridge face memory test:
results for neurologically intact individuals and an investigation of its
validity using inverted face stimuli and prosopagnosic participants.
Neuropsychologia. 44:576–585.
28 Bentin S, Allison T, Puce A, Perez E, McCarthy G. 1996.
Electrophysiological studies of face perception in humans.
J Cogn Neurosci. 8:551–565.
29 Rossion B, Jacques C. 2012. The N170: understanding the timecourse of face perception in the human brain. In: Kappenman ES,
Luck SJ, editors. The Oxford hand-book of event-related potential com­ponents. Vol. 641. Oxford, UK: Oxford University Press. p. 115–141.
30 Barton JJS, Corrow SL. 2016. The problem of being bad at faces.
Neuropsychologia. 89:119–124.
31 Bobak AK, Parris BA, Gregory NJ, Bennetts RJ, Bate S. 2017.
Eye-movement strategies in developmental prosopagnosia and“super” face recognition. Q J Exp Psychol. 70:201–217.
32 Rosenthal G, et al. 2017. Altered topology of neural circuits in con­genital prosopagnosia. Elife. 6:e25069.
33 Hendel RK, Starrfelt R, Gerlach C. 2019. The good, the bad, andthe average: characterizing the relationship between face andobject processing across the face recognition spectrum.
Neuropsychologia. 124:274–284.
34 Vogel EK, McCollough AW, Machizawa MG. 2005. Neural meas­ures reveal individual differences in controlling access to work­
ing memory. Nature. 438:500–503.
35 Maguire EA, Valentine ER, Wilding JM, Kapur N. 2003. Routes toremembering: the brains behind superior memory. Nat
Neurosci. 6:90–95.
36 Zadelaar JN, et al. 2019. Are individual differences quantitative or
qualitative? An integrated behavioral and fMRI MIMIC approach.
Neuroimage. 202:116058.
37 Price CJ, Friston KJ. 2002. Degeneracy and cognitive anatomy.
Trends Cogn Sci. 6:416–421.
38 Anderson AJ, et al. 2020. Decoding individual identity from brain
activity elicited in imagining common experiences. Nat Commun.
11:5916.
39 Rokem A, Kay K. 2020. Fractional ridge regression: a fast, inter­
pretable reparameterization of ridge regression. Gigascience. 9:
giaa133.
40 McDermott J, Schiller PH, Gallant JL. 2002. Spatial frequency andorientation tuning dynamics in area V1. Proc Natl Acad Sci U S A.
99:1645–1650.
41 Lamme VA, Roelfsema PR. 2000. The distinct modes of vision of­fered by feedforward and recurrent processing. Trends Neurosci.
23:571–579.
42 Kriegeskorte N, Mur M, Bandettini P. 2008. Representational
similarity analysis—connecting the branches of systems neuro­science. Front Syst Neurosci. 2:4.
43 Krizhevsky A, Sutskever I, Hinton GE. 2012. Imagenet classifica­tion with deep convolutional neural networks. In: Pereira F.
Burges CJC. Bottou L. Weinberger KQ, editors. Advances in neural
information processing systems 25. New York, NY: Curran
Associates, Inc. p. 1097–1105.
44 Simonyan K, Zisserman A. 2014. Very deep convolutional net­
works for large-scale image recognition, arXiv, arXiv:1409.1556.
https://doi.org/10.48550/arXiv.1409.1556,
reviewed.

preprint:

not

peer

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

ability: better face recognizers extract different physical content

45 Güçlü U, van Gerven MAJ. 2015. Deep neural networks reveal a
gradient in the complexity of neural representations across theventral stream. J Neurosci. 35:10005–10014.
46 Carlson TA, Tovar DA, Alink A, Kriegeskorte N. 2013.
Representational dynamics of object vision: the first 1000 ms. J
Vis. 13:1–1.
47 Cichy RM, Pantazis D, Oliva A. 2014. Resolving human object rec­ognition in space and time. Nat Neurosci. 17:455–462.
48 Ince RAA, et al. 2017. A statistical framework for neuroimaging
data analysis based on mutual information estimated via a
Gaussian copula. Hum Brain Mapp. 38:1541–1573.
49 Long B, Yu C-P, Konkle T. 2018. Mid-level visual features underlie
the high-level categorical organization of the ventral stream. Proc
Natl Acad Sci U S A. 115:E9015–E9024.
50 Cer D, et al. 2018. Universal sentence encoder, arXiv,
arXiv:1803.11175,
https://doi.org/10.48550/arXiv.1803.11175,
preprint: not peer reviewed.
51 Blauch NM, Behrmann M, Plaut DC. 2021. Computational in­sights into human perceptual expertise for familiar and unfamil­
iar face recognition. Cognition. 208:104341.
52 Abudarham N, Grosbard I, Yovel G. 2021. Face recognition de­pends on specialized mechanisms tuned to view-invariant facial
features: insights from deep neural networks optimized for faceor object recognition. Cogn Sci. 45:e13031.
53 O’Toole AJ, Castillo CD. 2021. Face recognition by humans andmachines: three fundamental advances from deep learning.
Annu Rev Vis Sci. 7:543–570.
54 Parkhi O, Vedaldi A, Zisserman A. 2015. Deep face recognition. In:
BMVC 2015—Proceedings of the British Machine Vision
Conference 2015; Swansea, UK, September 2, 2023.
55 Cichy RM, Kriegeskorte N, Jozwik KM, van den Bosch JJF, Charest
I. 2019. The spatiotemporal neural dynamics underlying per­ceived similarity for real-world objects. Neuroimage. 194:12–24.
56 Mur M, et al. 2013. Human object-similarity judgments reflect
and transcend the primate-IT object representation. Front
Psychol. 4:128.
57 Hebart MN, Bankson BB, Harel A, Baker CI, Cichy RM. 2018. The
representational dynamics of task and object processing in hu­
mans. eLife. 7:e32816.
58 Geskin J, Behrmann M. 2018. Congenital prosopagnosia without
object agnosia? A literature review. Cogn Neuropsychol. 35:4–54.
59 Bobak AK, Bennetts RJ, Parris BA, Jansari A, Bate S. 2016. An indepth cognitive examination of individuals with superior facerecognition skills. Cortex. 82:48–62.
60 Barton JJS, Albonico A, Susilo T, Duchaine B, Corrow SL. 2019.
Object recognition in acquired and developmental prosopagno­sia. Cogn Neuropsychol. 36:54–84.
61 Duchaine B, Germine L, Nakayama K. 2007. Family resemblance:
ten family members with prosopagnosia and within-class object
agnosia. Cogn Neuropsychol. 24:419–430.
62 Gabay Y, Dundas E, Plaut D, Behrmann M. 2017. Atypical percep­
tual processing of faces in developmental dyslexia. Brain Lang.
173:41–51.
63 Jiahui G, Yang H, Duchaine B. 2018. Developmental prosopagno­
sics have widespread selectivity reductions across categoryselective visual cortex. Proc Natl Acad Sci U S A. 115:E6418–E6427.
64 Avidan G, Hasson U, Malach R, Behrmann M. 2005. Detailed ex­
ploration of face-related processing in congenital prosopagnosia:
2. Functional neuroimaging findings. J Cogn Neurosci. 17:
1150–1167.
65 Behrmann M, Plaut DC. 2013. Distributed circuits, not circum­
scribed centers, mediate visual recognition. Trends Cogn Sci. 17:
210–219.

12 |PNAS Nexus, 2024, Vol. 3, No. 3

66 Duchaine BC, Yovel G, Butterworth EJ, Nakayama K. 2006.
Prosopagnosia as an impairment to face-specific mechanisms:

gyrus face selectivity relates to individual differences in facialrecognition ability. J Cogn Neurosci. 23:1723–1740.
68 Wilmer JB, et al. 2012. Capturing specific abilities as a window
into human individuality: the example of face recognition. Cogn
Neuropsychol. 29:360–392.
69 Khaligh-Razavi S-M, Kriegeskorte N. 2014. Deep supervised, but
not unsupervised, models may explain IT cortical representa­
tion. PLoS Comput Biol. 10:e1003915.
70 Jiahui G, et al. 2023. Modeling naturalistic face processing in hu­
mans with deep convolutional neural networks. Proc Natl Acad Sci
U S A. 120:e2304085120.
71 Grossman S, et al. 2019. Convergent evolution of face spaces
across human face-selective neuronal groups and deep convolu­
tional networks. Nat Commun. 10:4934.
72 van Herten M, Kolk HHJ, Chwilla DJ. 2005. An ERP study of P600
effects elicited by semantic anomalies. Brain Res Cogn Brain Res.
22:241–255.
73 Shen W, Fiori-Duharcourt N, Isel F. 2016. Functional significanceof the semantic P600: evidence from the event-related brain po­
tential source localization. Neuroreport. 27:548–558.
74 Eimer M, Gosling A, Duchaine B. 2012. Electrophysiologicalmarkers of covert face recognition in developmental prosopag­
nosia. Brain. 135:542–554.
75 Popham SF, et al. 2021. Visual and linguistic semantic representa­
tions are aligned at the border of human visual cortex. Nat
Neurosci. 24:1628–1636.
76 Fernandino L, Tong J-Q, Conant LL, Humphries CJ, Binder JR.
2022. Decoding the information structure underlying the neuralrepresentation of concepts. Proc Natl Acad Sci U S A. 119:e2108091119.
77 Frisby SL, Halai AD, Cox CR, Lambon Ralph MA, Rogers TT. 2023.
Decoding semantic representations in mind and brain. Trends
Cogn Sci. 27:258–281.
78 Bruce V, Young A. 1986. Understanding face recognition. Br J
Psychol. 77(Pt 3):305–327.
79 Pitcher D, Walsh V, Duchaine B. 2011. The role of the occipital
face area in the cortical face perception network. Exp Brain Res.
209:481–493.

Downloaded from https://academic.oup.com/pnasnexus/article/3/3/pgae095/7616957 by University of Fribourg - Bibliothèque cantonale et universitaire user on 26 March 2024

elimination of the alternative hypotheses in a developmental
case. Cogn Neuropsychol. 23:714–747.
67 Furl N, Garrido L, Dolan RJ, Driver J, Duchaine B. 2011. Fusiform

80 Anaki D, Bentin S. 2009. Familiarity effects on categorization lev­els of faces and objects. Cognition. 111:144–149.
81 Gauthier I, Anderson AW, Tarr MJ, Skudlarski P, Gore JC. 1997.
Levels of categorization in visual recognition studied using func­
tional magnetic resonance imaging. Curr Biol. 7:645–651.
82 Faghel-Soubeyrand S, Dupuis-Roy N, Gosselin F. 2019. Inducing
the use of right eye enhances face-sex categorization perform­
ance. J Exp Psychol Gen. 148:1834–1841.
83 Davis JP, Lander K, Evans R, Jansari A. 2016. Investigating predic­
tors of superior face recognition ability in police superrecognisers: superior face recognisers. Appl Cogn Psychol. 30:
827–840.
84 Langner O, et al. 2010. Presentation and validation of the radboud
faces database. Cogn Emot. 24:1377–1388.
85 Kiani R, Esteky H, Mirpour K, Tanaka K. 2007. Object category
structure in response patterns of neuronal population in monkey
inferior temporal cortex. J Neurophysiol. 97:4296–4309.
86 Willenbockel V, et al. 2010. Controlling low-level image proper­
ties: the SHINE toolbox. Behav Res Methods. 42:671–684.
87 Kriegeskorte N, Mur M, Kriegeskorte N, Kreiman G. 2012.
Representational similarity analysis of object population codesin humans, monkeys, and models. In: Kriegeskorte N, Kreiman G,
editors. Visual population codes: towards a common multivariate
framework for cell recording and functional imaging. Cambridge, MA.
88 Oostenveld R, Fries P, Maris E, Schoffelen J-M. 2011. FieldTrip:
open source software for advanced analysis of MEG, EEG, and in­vasive electrophysiological data. Comput Intell Neurosci. 2011:156869.
89 Treder MS. 2020. MVPA-light: a classification and regression tool­
box for multi-dimensional data. Front Neurosci. 14:289.
90 Grootswagers T, Wardle SG, Carlson TA. 2017. Decoding dynamic
brain patterns from evoked responses: a tutorial on multivariate
pattern analysis applied to time series neuroimaging data. J Cogn
Neurosci. 29:677–697.
91 Cichy RM, Oliva A. 2020. A M/EEG-fMRI fusion primer: resolving
human brain responses in space and time. Neuron. 107:772–781.
92 Graumann M, Ciuffi C, Dwivedi K, Roig G, Cichy RM. 2022. The
spatiotemporal neural dynamics of object location representa­
tions in the human brain. Nat Hum Behav. 6:796–811.
93 Liu J, et al. 2021. Transformative neural representations support
long-term episodic memory. Sci Adv. 7:eabg9715.
94 Xie S, Kaiser D, Cichy RM. 2020. Visual imagery and perception
share neural representations in the alpha frequency band. Curr
Biol. 30:3062.