# Infants' Preference for ID Speech in Face and Voice Extends to a Non-Native Language

Joan Birulés, David Méary, Mathilde Fort, Kim Hojin, Scott Johnson, Olivier

Pascalis

► To cite this version:

Joan Birulés, David Méary, Mathilde Fort, Kim Hojin, Scott Johnson, et al. Infants' Preference for ID Speech in Face and Voice Extends to a Non-Native Language. Infancy, 2024, 30 (1), pp.e12639. 10.1111/infa.12639. hal-04801751

HAL Id: hal-04801751

https://hal.science/hal-04801751v1

Submitted on 3 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Infants' Preference for ID Speech in Face and Voice Extends to a Non-Native Language

Joan Birulés<fnref n="1" /> ID | David Méary<fnref n="1" /> ID | Mathilde Fort<fnref n="1" />,² ID | Kim Hojin³ | Scott P. Johnson⁴ ID | Olivier Pascalis<fnref n="1" /> ID

**Correspondence:** Joan Birulés (joanbirules@gmail.com)

**Received:** 31 March 2024 | **Revised:** 22 July 2024 | **Accepted:** 25 October 2024

**Funding:** This work was supported by Agence Nationale de la Recherche (ANR), projects BabyFaceSpeech (ANR-17-FRAL-0014) and ControLearn (ANR-22-CE28-0004-01) and by the Fondation de France (Grant number: 00112028).

**Keywords:** audiovisual speech | face perception | face-voice association | infant-directed speech | talking faces

## ABSTRACT

Infants prefer infant-directed (ID) speech. Concerning talking faces, previous research showed that 3- and 5-month-olds prefer faces that produce native ID than native adult-directed (AD) speech, regardless of background speech being ID, AD or silent. Here, we explored whether infants also show a preference for non-native ID speech. We presented 3- and 6-month-old infants with pairs of talking faces, one producing non-native ID speech and the other non-native AD speech, either in silence (Experiment 1) or accompanied by non-native ID or AD background speech (Experiment 2). Results from Experiment 1 showed an overall preference for the silent ID talking faces across both age groups, suggesting a reliance on cross-linguistic, potentially universal cues for this preference. However, Experiment 2 showed that preference for ID faces was disrupted at 3 months when auditory speech was present (ID or AD). At 6 months, infants maintained a preference for ID talking faces, but only when accompanied by ID speech. These findings show that auditory non-native speech interferes with infants' processing of ID talking faces. They also suggest that by 6 months, infants start associating ID features from faces and voices irrespective of language familiarity, suggesting that infants' ID preference may be universal and amodal.

## 1 | Introduction

Infants experience human speech before birth and are especially attracted by auditory infant-directed (ID) speech, the speech signal that is specifically addressed to them. When adults talk to infants, they adjust their speech by using simpler sentences with more repetitions, a slower speech rate, longer pauses, higher pitch and greater pitch excursions (Cooper and Aslin 1990; Fernald et al. 1989). These linguistic and prosodic modifications that characterize ID speech are often assumed to facilitate infants' speech processing and, in turn, language learning (see Nencheva and Lew-Williams 2022 for a review). Indeed, infants

present a strong preference for auditory ID speech over adult-directed (AD) speech (Cooper and Aslin 1994, 1990; Cris-tia 2013; Fernald 1985; Soderstrom et al. 2003; Werker and Mcleod 1989), a preference that has been robustly replicated by a large multi-lab study (Frank et al. 2020).

Crucially, although ID speech has been mostly studied as an auditory event, infant-directedness is also reflected in a talker's face. The face of a talker exhibits articulatory movements of the mouth that are spatiotemporally congruent with the auditory speech cues (Chandrasekaran et al. 2009; Yehia, Rubin, and Vatikiotis-Bateson 1998), and lip movements are larger in

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly
cited.

© 2024 The Author(s). *Infancy* published by Wiley Periodicals LLC on behalf of International Congress of Infant Studies.

<footnote n="1">¹Laboratoire de Psychologie et NeuroCognition, UMR 5105, University Grenoble Alpes, Grenoble, France | ²Centre de Recherche en Neurosciences de Lyon, UMR 5292, Université Lyon 1, Lyon, France | ³Department of Quantitative Theory and Methods, Emory University, Atlanta, Georgia, USA | ⁴Department of Psychology, University of California, Los Angeles, California, USA</footnote>
hyperarticulated ID speech (Green et al. 2010). In addition, talking faces also provide eye gaze and facial expressions (Birmingham and Kingstone 2009). When adults produce ID speech, faces reflect more exaggerated smiles and expressions of soothing, comfort, and amazement (Chong et al. 2003), as well as prosodically aligned head movements (Smith and Strader 2014). As most interactions occur face-to-face, the study of infants' perception of ID speech should therefore include not only the voice but also the face of the talker.

Indeed, infants typically perceive speech as an audiovisual redundant signal. Prior studies show that before 6 months of age, infants are sensitive to the audiovisual correspondences in vowels or syllables (Kuhl and Meltzoff 1982; Patterson and Werker 2003; Pons, Lewkowicz, et al. 2009), and that their perception of speech is influenced by the visual information provided by the talker's face (i.e., McGurk effect; Kushnerenko et al. 2008; Rosenblum, Schmuckler, and Johnson 1997). Moreover, when presented with videos of audiovisual speech (i.e., talking faces), infants preferentially look at the eyes of the speaker during the first months of postnatal life, and slowly shift their attention to the talker's mouth from around 6 months of age (Jones and Klin 2013; Lewkowicz and Hansen-Tift 2012) as they start producing babbling sounds (Oller 2001), likely to enhance the processing of the audiovisual speech cues originated in a talker's mouth.

More recent studies have explored the impact of ID speech on infants' face scanning patterns, and they have shown that infants deploy more of their attention to the mouth when the talker produces ID speech as compared to AD speech, during the first and second year after birth (Hillairet de Boisferon et al. 2018; Lense et al. 2022; Roth, Clayton, and Reynolds 2022). Similarly, Alviar et al. (2023) showed that from 3 months of age, such preference for a talker's mouth becomes even greater if the talker is singing (i.e., ID song), especially after 6 months of age (Alviar et al. 2023).

These studies reveal that infants prefer listening and orienting their attention to auditory ID versus AD speech, and that when perceiving talking faces, ID speech (and ID song) increases their attention to the speaker's mouth, suggesting that, from 6 months of age, infants somewhat form coherent audiovisual percepts for ID speech.

Given the audiovisual nature of ID speech, Kim and Johnson (2014) explored whether such preference for the acoustic features of ID speech could also extend to the visual features of ID speech. To explore this question, they presented 3- and 5-month-old infants with two side-by-side videos of talking faces. One video depicted a face speaking in ID speech, while the other one presented the face of the same speaker but talking in AD speech. These videos were either played in silence or accompanied by an ID or an AD auditory speech stream. Crucially, the auditory speech stream was always from a different excerpt than the videos, so that infants would only perform audiovisual associations based on the type of speech (AD vs. ID) rather than on temporal audiovisual synchrony cues and/or similar semantic content. Their results showed that infants preferred looking at the ID talking faces than the AD

talking faces, regardless of the auditory condition. As a consequence, these results show that infants are visually more attracted by faces talking in ID than in AD speech.

Crucially, whether the visual cues that trigger this preference are universal or language-specific remains an open question. Infants could rely on universal non-speech specific cues: first, the audiovisual ID speech generally involves happier expressions as compared to AD speech, so infants' preference for ID talking faces could in fact be due to the general attraction infants exhibit for smiling faces (Farroni et al. 2007; LaBarbera et al. 1976). Such preference could also be due to the fact that visual ID speech is generally more salient than visual AD speech, given that it involves more head movements, more hyper-articulation, etc. (Green et al. 2010; Smith and Strader 2014). Last, it could also be that visible language-specific articulatory cues (i.e., visemes, prosodic cues) are more salient in visual ID speech than in visual AD speech, and thus attract infants' attention more. This would show that at those ages, infants prefer looking at a face talking in their native language, as it is the case for listening to auditory speech (Bosch and Sebastián-Gallés 2001), but only when the speech signal is in ID.

One way to disentangle this language-specific interpretation from the cross-linguistic, potentially universal one is to present the same stimuli and task (i.e., spoken American English) to same-age infants growing up in a different language environment (e.g., French), so that the language of the stimuli becomes non-native to them. If infants are attracted to ID speech cues, independently of their native-language, they should prefer the ID talking faces. On the other hand, if the non-native aspect of the language (as reflected in the acoustic and the visual speech signals) is detected by infants and modulates their attention and processing of the talking faces, then it is possible that the previously reported ID face preference may be disrupted.

Indeed, it has been shown that perceiving a non-native language modulates infants' perception of faces. Non-native languages elicit greater attention to a talker's mouth in infants at 12 months of age (Lewkowicz and Hansen-Tift 2012; Pons, Bosch, and Lewkowicz 2015) and above (Birulés, Bosch, Brieke, et al. 2018; Birulés, Bosch, Lewkowicz, et al. 2024), even when the talking faces are presented in silence (Kubicek et al. 2013). Moreover, 4- and 6-month-olds can discriminate a non-native language from their native one by just seeing a silently talking face (Weikum et al. 2007). Last, perceiving a non-native language has also been found to disrupt infants' face recognition processes (Hillairet de Boisferon et al. 2021). It emerges from these studies that perceiving a non-native language interacts with infants' audiovisual processing of faces, and thus, it is reasonable that a non-native language may affect infants' visual preference for ID talking faces.

Here, we are interested to determine whether a non-native language may disrupt or modulate the previously reported preference for ID talking faces (Kim and Johnson 2014). To answer this question, we used the original materials of English-speaking ID and AD talking faces from Kim and Johnson (2014), and we tested whether 3- and 6-month-old French-learning infants would show a preference for the ID faces
producing non-native speech. In Experiment 1, we presented the same two talking faces side-by-side for two 20-sec trials, producing speech in English, now presumably perceived as non-native language. The two videos were presented in silence. If infants' preference for ID talking faces is not constrained to their native language, we would expect infants to disregard the fact that the ID talking faces are not talking in their native language and orient toward the ID faces. On the contrary, it is also possible that watching a face talking in a non-native language disrupts the ID face preference, leading to a null preference.

# 2 | Experiment 1

## 2.1 | Method

### 2.1.1 | Participants

A total of 50 infants were recruited and tested in Experiment 1, divided into a group of 3-month-old infants (N = 22; mean age = 114.3 days, range 101–130 days, 13 girls) and a group of 6-month-old infants (N = 28; mean age = 192.6 days, range 182–217 days, 15 girls). Infants were healthy, full-term participants, recruited from the maternity ward of the Centre Hospitalier Universitaire (CHU) of Grenoble. Experimenters observed that infants looked White European (by North American standards), though direct questions regarding race or ethnicity were not permitted. All infants were growing up in Grenoble, in a monolingual French-speaking environment (i.e., according to parental report, infants were exposed to French more than 95% of the time and had no to little exposure to other-race faces). Ten additional infants were tested, but excluded from the analyses because they did not complete the procedure due to crying or fussiness. We set a target sample size of 24 infants per subgroup, based on previous work using this task, design and age groups (Hillairet de Boisferon et al. 2015; Kubicek et al. 2014). Testing concluded once the target sample size was achieved, with minor deviations due to exclusions and internal testing dynamics (e.g., 6-month-olds were easier to recruit). All parents gave their written informed consent prior to their infants' participation in the study. This paradigm was conducted according to guidelines laid down in the Declaration of Helsinki and approved by the Ethical Committee (CERGA-Avis-2024-14) at the University of Grenoble Alpes (France).

### 2.1.2 | Stimuli

We used the original videos from Kim and Johnson (2014). These consisted of 14 side-by-side video tapes, showing a White European woman's face as she engaged in a live face-to-face interaction with a member of her own family—either her husband or her 18-month-old infant. One woman served as the model for all stimuli. The recording sessions yielded segments of video and audio records that were either AD (i.e., to husband) or ID (i.e., to infant). In Experiment 1, the videos were muted to show the talking faces in silence. For more details see Kim and Johnson (2014).

### 2.1.3 | Procedure

Infants were tested in a quiet chamber at the Grenoble Babylab, University of Grenoble Alpes. Infants sat on their parents' lap approximately 60 cm away from a screen, which displayed the videos. The experimenter was out of sight during testing, and both the parents and the experimenter remained quiet. As in Kim and Johnson (2014), we used a paired visual preference task to test infants' spontaneous visual preference for ID or AD faces. Each infant was presented with two trials, to counterbalance the left-right positioning of the videos (see Supporting Information S1: Section 1). Each pair of videos was presented for a total duration of 20 s. In Experiment 1, infants viewed the two trials in silence.

We recorded infant looking behavior to each of the two talking faces with a video camera. These recordings were analyzed offline frame by frame on a computer using specialized software. An independent observer recoded 20% of the data for reliability. Both observers were blind to condition. The average level of inter-observer agreement across the four age groups was very high (Pearson *r* > 0.95). The code and data that support the findings of this study are openly available in OSF https://osf.io/dqm92/?view_only=a447f5f425 d54c1eaa681b3355867e51.

## 2.2 | Results and Discussion

To evaluate infants' visual preference for ID versus AD talking faces, total looking times (TLT) to the two faces were extracted from the manual coding and then submitted to an ANOVA with age (3, 6) and face (ID face, AD face). Results showed a main effect of face (ID face TLT mean = 8.93 s, SD = 3.07; AD face TLT mean = 6.75, SD = 2.37; F(1, 48) = 11.83, *p* = 0.001, η² = 0.20, see Figure 1), and no significant main effect of age, nor interaction of Face and age (*ps* > 0.050; see Supporting Information S1: Section 2 for full results).

<figure><img src="image_3.png" /><figcaption>**FIGURE 1** | Total looking times (TLT, in sec) to the ID Face (red) and AD face (blue). Dots represent each child's mean TLT score and bars and crosses with error bars represent mean TLT scores and standard errors of the mean (SE) for each group.</figcaption></figure>
These results show that when seeing two silently talking faces side-by-side, one producing ID speech and the other AD speech, both 3- and 6-month-old infants showed a visual preference for the ID face. These results extend those from Kim and Johnson (2014) and show that regardless of whether the face is silently speaking in their native language or in a non-native language, infants prefer looking at the ID talking faces.

# 3 | Experiment 2

Experiment 1 showed that infants from both age groups prefer looking at faces silently talking in a non-native ID speech, as compared to non-native AD speech. These results indicate that infants can visually discriminate ID speech from AD speech, even when the speech is in a novel and foreign language.

As earlier noted, it is known that infants are attracted to smiling faces over neutral faces (Malatesta and Haviland 1982). Therefore, the present results could be interpreted as a preference for the facial expression that is most familiar to them, or to the fact that ID talking faces and/or visual ID speech provides an overall more salient signal that attracts infants' attention.

However, regardless of either interpretation, Experiment 1 results do suggest that infants' preference for visual ID speech is not constrained to their native language, but may rely on cross-linguistic, potentially universal cues. This result is novel and noteworthy as infants in our study are supposed to be able to visually discriminate their native language (i.e., French) from the non-native one used in the present stimuli (English), as long as they belong to different rhythmic classes (Weikum et al. 2007). Additionally, a non-native language is also supposed to induce changes in their face processing and selective attention patterns (Kubicek et al. 2013). It thus seems unlikely that the preference for ID talking faces in the present study was due to the fact that infants did not notice that the face was talking in a non-native language.

Another related question regarding the processing of visual ID speech is whether infants can associate it with auditory ID speech. Associating ID speech across the visual and auditory modalities would suggest that infants' processing of “infant-directedness” relies on amodal cues. Kim and Johnson (2014) explored this question using infants' native language, and they found that infants preferred the ID talking faces regardless of the auditory background. In Experiment 2, we ran the same design as Experiment 1 but in the presence of background speech, either AD or ID. Importantly, to avoid potential carry-over effects, we explored the two conditions between-subjects; that is, half the sample was presented with the visual speech signals in ID Speech, and the other half in AD Speech.

As earlier mentioned, if the ID speech features that lead infants' ID face preference are amodal, then infants should be able to associate the visual ID speech signals with the auditory ones. On the other hand, it could also be that hearing a non-native speech (rather than just watching it) renders the non-nativeness of the speech signal even more salient, inducing a greater cognitive load and thus disrupting infants' processing of the talking faces

(as in Hillairet de Boisferon et al. 2021). If so, we might expect the auditory non-native speech (both ID and AD) to interrupt the preference for ID talking faces and elicit a null preference.

## 3.1 | Participants

Ninety-eight infants were recruited and tested in Experiment 2, divided into 3-month-old infants (N = 41; mean age = 113.42 days, range = 89–158 days, 19 girls) and a group of 6-month-old infants (N = 57; mean age = 192.07 days, range = 173–218 days, 23 girls). Infants' characteristics were equivalent to Experiment 1. Thirteen additional infants were tested, but excluded from the analyses because they did not complete the procedure due to crying or fussiness.

## 3.2 | Stimuli and Procedure

Stimuli and procedure were equivalent to Experiment 1, with the exception that in Experiment 2 the videos contained a background audio speech excerpt, either in AD speech (audio recording of the actress talking to her husband) or in ID speech (the actress talking to her 18-month-old infant). As in Experiment 1, the audio clips (taken from Kim and Johnson 2014) were in English, a non-native language to the French-speaking participants. Moreover, the audio clips were always completely different to either the ID or AD faces, to avoid a potential matching on the base of audiovisual synchrony. For more details see Kim and Johnson (2014).

## 3.3 | Results and Discussion

To evaluate whether infants' visual preference for ID versus AD talking faces is modulated by the type of acoustic signal they are hearing (ID vs. AD), in Experiment 2 we submitted infants' total looking times (TLT)¹ to the faces to an ANOVA with age (3, 6), face (ID face, AD face) and sound (ID speech, AD speech). The results of the ANOVA showed a significant main effect of face ($F(1, 94) = 4.24, p = 0.042, \eta_p^2 > = 0.04$), a double interaction between age and face ($F(1, 94) = 10.20, p = 0.002, \eta_p^2 > = 0.10$), and a significant triple interaction (age × sound × face; $F(1, 94) = 4.22, p = 0.043, \eta_p^2 = 0.04$).

To explore the triple interaction, depicted in Figure 2, we run two new ANOVAs, one per age group. Three-month-old infants' ANOVA showed no significant effects (ps > 0.5). Contrarily, the ANOVA in 6-month-olds showed a significant main effect of face ($F(1, 55) = 24.33, p < 0.001, \eta_p^2 = 0.31$), reflecting an overall higher attention for ID faces than for AD faces, and crucially, a significant interaction of sound and face ($F(1, 55) = 7.24, p = 0.009, \eta_p^2 = 0.12$). Follow up t-tests explored this interaction and showed that 6-month-old infants looked longer at the visual ID speech when another excerpt of the auditory ID speech played ($t(25) = 5.20, p < 0.001, d = 1.02$), but showed no visual preference when the auditory AD speech played ($t(30) = 165, p = 0.109, d = 0.30$). See Supporting Information S1: Section 3 for full results.
<figure><img src="image_4.png" /><figcaption>**FIGURE 2** | Total looking times (TLT, in sec) to the ID Face (red) and AD face (blue), as a function of sound (IDS, ADS) and age (3 m, left; 6 m right). Dots represent each child's mean TLT score and bars and crosses with error bars represent mean TLT scores and standard errors of the mean (SE) for each group.</figcaption></figure>

These results show that the acoustic signal modulated the preference observed in Experiment 1, when the videos were played in silence.² At 3 months of age, the presence of the auditory non-native speech stream impeded the preference for both talking faces (ID and AD). This is in contrast with Kim and Johnson (2014) who found a preference for the ID face, regardless of the auditory condition. Importantly, different from Kim and Johnson's (2014), our participants are now hearing a non-native language. Therefore, it could be that at the age of 3 months, hearing a novel and foreign language induces a greater cognitive load than just watching it, and thus disrupts infants' processing of the talking faces, as previously seen in a face recognition experiment (Hillairet de Boisferon et al. 2021).

Differently, 6-month-old infants successfully associated the ID talking faces with the auditory ID speech. However, they could not do so for AD speech. For the AD speech condition, the null preference could reflect the fact that infants are attracted to the ID face, but that the AD speech drove them to look at the AD face. Therefore, it could be that the behavior of looking more to the matching face (ID face on ID speech) cancels out the general attraction to the more familiar and smiling face, producing an overall null preference.

Last, the fact that the main effect of Speech was not significant reflects that, different from previous studies showing overall higher looking times to ID versus AD speech (e.g., Frank et al. 2020), here the visual stimuli of faces likely attract infants' attention, hiding such differences in looking times.

4 | General Discussion

ID speech provides infants with a simplified, slower paced and articulatorily exaggerated speech signal (e.g., Cooper & Aslin, 1990) that is thought to enhance infant processing of speech and, more generally, early language acquisition. Therefore, attending preferentially to ID speech has been interpreted

as an adaptive behavior to enhance infants' processing and learning of language (see Nencheva and Lew-Williams 2022).

While prior studies in infants have shown a robust preference for auditory ID speech (see Frank et al. 2020, for a multi-lab study), less is known about infants' perception and preference for infant-directedness in talking faces. Kim and Johnson (2014) showed that, like in ID voices, 3- and 5-month-old infants preferred looking at talking faces that produce native ID rather than AD speech, and importantly, that they did so regardless of the background sound being silent, in AD or in ID speech. It remained then to be known whether such preference for ID talking faces was (native) language specific, and if that is not the case, whether infants may be able to associate the infant-directed features of a speech signal across the visual and auditory modalities. To answer these questions, in the present study we explored 3- and 6-month-old infants' preference for ID versus AD talking faces but, in this case, using an unfamiliar, non-native language. We did so in two experiments; first in silence (Experiment 1) and then with a background sound, either AD or ID speech (Experiment 2). The results showed that as for native speech, infants preferred looking at the ID faces talking in a non-native language when presented in silence (Experiment 1). When the non-native auditory speech stream was played in the background (Experiment 2), such preference for the ID talking face was generally disrupted at both 3 and 6 months of age, regardless of the auditory speech being in ID or AD. Interestingly, however, the 6-month-old group that heard the ID auditory speech stream did exhibit a preference for the ID talking faces, suggesting that they associate the ID features of face and voice.

We were first interested in exploring whether infants' reported preference for silent ID faces (Kim and Johnson 2014) was (native) language specific, or rather based on cross-linguistic ID speech cues. Given that very young infants can use visual speech cues only (i.e., silently talking faces) to discriminate between a non-native language and their native one (Weikum et al. 2007), we expected that if infants' preference for ID talking faces was specific to their native language, the visual speech cues of a non-native language would interact and potentially hinder such preference. On the contrary, the results of Experiment 1 showed that when 3- and 6-month-olds watched silent videos of non-native AD and ID speech, infants still looked longer to the ID than the AD faces in all conditions. Importantly, the fact that infants detected and preferred to attend to ID faces when they silently spoke in a non-native language suggests that infants' preference for ID silently talking faces is not based on language familiarity or native speech-specific cues, but rather on cross-linguistic, potentially universal cues. These universal ID cues may entail the emotional features of talking faces (Chong et al. 2003), the hyperarticulation (Green et al. 2010) and the greater head movements (Smith and Strader 2014). Future studies should explore the weight of each of these features attracting infants' attention to silent ID speech, unaffected by language familiarity. Moreover, given the proximity between the cultures and communicative styles here explored (French infants perceiving English ID speech) the universality of ID cues beyond western cultures and languages should also be investigated by future studies, exploring whether infants' ID preference is also present in other more distant languages, cultures and faces (i.e., other-race faces).
In Experiment 2 we were interested in exploring whether the
presence of background AD or ID speech would disrupt such
preference for ID talking faces and show evidence of a cross-
modal association between the ID features of the visual and
auditory speech cues. Given that the auditory speech was
completely asynchronous to the visual signal, a preference for
the audiovisual correspondence would reflect that infants have
formed an amodal representation of ID speech.

The fact that 3-month-olds and 6-month-old infants (except in
the ID audio condition) showed no preference for either AD or
ID talking face indicates that, while visual non-native speech
did not modulate infants' preference for ID faces, auditory non-
native speech did. Like prior research showing that listening to
a non-native language can hinder infants' face recognition
processes (Hillairet de Boisferon et al. 2021), the present results
showed that auditory non-native speech interfered with infants'
preference for ID talking faces. In the case of the 3-month-old
group, the fact that they showed no preference for either face
in Experiment 2 when the ID or AD speech played in the
background is likely due to the fact that processing a non-native
language increases infants' cognitive load, thus leaving fewer
cognitive resources available to process the visual features and
detect the infant-directedness in faces, or even the emotional
status of the faces. Thus, infants may be focusing on processing
the unfamiliar, non-native speech sounds, while overlooking the
visual signal and thus hampering the previously reported pref-
erence for ID faces, as found in Experiment 1.

Crucially, however, the fact that 6-month-old infants do show a
preference for ID faces only when ID speech sound is presented
suggests that there are higher-level association processes at play
at this age, and that beyond a general interference of the non-
native speech sounds, infants start having an expectation of
ID faces to speak ID speech. Regarding the asymmetry of the
results—that is, the congruency effect is only found in the ID
but not the AD speech condition—it is likely caused by an
interference with infants' overall preference for ID talking faces.
A similar case was previously reported by Hillairet de Boisferon
et al. (2021), showing that 9-month-olds would match female
voices with female faces but not male voices with male faces.
This was interpreted as reflecting an accumulation (here,
neutralization) of two different effects, the overall preference for
female faces and the preference for the congruent face, leading
to a 50% preference in the male voice condition. We may have
observed the same phenomenon in the 6-month-old group
listening to AD speech.

These results contrast with Kim and Johnson (2014), who found
that infants' preference for ID talking faces was not modulated
by the auditory speech signal, be it native ID or AD speech, and
thus reported no association effects. It is possible that the fa-
miliarity of native speech renders the acoustic signal less salient
or less costly to process for infants, and thus the preference for
the more smiling ID faces continues to appear in a native lan-
guage but is disrupted when a non-native language is presented.
On the other hand, given that Kim and Johnson's (2014) con-
ditions were presented within-subjects and always after the si-
lent condition, it is also possible that the experimental design
was not sensitive enough to show the emerging association

effects. Contrarily, the present between-subjects design allowed
to explore the effect of AD and ID speech sound independently.

Indeed, the finding that infants may start forming an amodal
representation of ID speech and performing a multisensory as-
sociation between the ID features of faces and voices around
6 months of age is in accordance with prior work exploring early
face-voice multisensory associations. These studies show that 5-
to 6-month-olds can associate a face with their native language
(Kinzler et al. 2007), and unfamiliar speech with unfamiliar
faces (Uttley et al. 2013). Moreover, the mapping of novel voices
into novel faces emerges between 4 and 8 months of age (Orena
and Werker 2021), and that memorizing the face-voice relations
is only possible from 6 months of age (Bahrick, Hernandez-Reif,
and Flom 2005; Orena and Werker 2021). In this context, it
seems possible that 6-month-olds are starting to extract the
infant-directed features from both the face and the voice and to
create an amodal, multisensory representation of ID speech, as
suggested by Experiment 2 results.

It is worth noting that the observed preference for IDS in both
face and voice could also be attributed to infants' learned as-
sociation that IDS is typically directed toward them. Regardless
of whether they have associated the infant directedness in a
speaker's face and voice, listening to AD speech could disrupt
their preference for ID faces, as it may be perceived as signaling
that communication is not directed to them, whereas Silent or
IDS conditions would maintain this preference. Future studies
could explore the interpretation of infants' looking time pref-
erences as a reflection of social engagement, or rather more low-
level saliency-related attentional processes.

Overall, the present article shows that infants' detection and
preference for ID speech is not only present in auditory speech,
but it is also present in visual speech, and that such preference is
not language-specific but rather caused by cross-linguistic,
potentially universal cues of ID speech that are also present in
an unfamiliar, non-native language. Crucially, these results also
provide the first evidence to suggest that infants from 6 months
of age have formed a multimodal representation of the infant-
directed features of speech, that encompasses both the acous-
tic and the visual characteristics of a face speaking in ID speech,
beyond low-level temporal cues such as audiovisual synchrony.

Author Contributions

**Joan Birulés:** data curation, formal analysis, visualization, writing—original draft, writing—review and editing. **David Méary:** investigation, methodology, writing—review and editing. **Mathilde Fort:** investigation, writing—review and editing. **Kim Hojin:** methodology, writing—review and editing. **Scott P. Johnson:** methodology, writing—review and editing. **Olivier Pascalis:** conceptualization, funding acquisition, investigation, project administration, resources, supervision, writing—original draft, writing—review and editing.

Acknowledgments

We thank Marie Sarremejeanne and Charlotte Chauvet for their assis-
tance during the recruitment and testing of participants, and all the
families and infants that participated in this study.
## Ethics Statement

This paradigm was approved by the Ethical Committee at the University Grenoble Alpes in France (CERGA-Avis-2024-14).

## Conflicts of Interest

The authors declare no conflicts of interest.

## Data Availability Statement

The code and data can be openly accessed at OSF https://osf.io/dqm92/?view_only=e3900b614b3547018d34b35335e2f6fb.

## Endnotes

¹ Using total looking times (TLT) instead of proportion of total looking times (PTLT) facilitates the examination of overall gaze duration toward both faces depending on Age or Sound condition. Notably, our findings remain consistent when employing PTLT scores and comparing PTLT to one of two faces against chance.

² To ensure statistical robustness, data from both Experiment 1 and Experiment 2 were analyzed together using a mixed ANOVA and Linear Mixed Models (LMM). Both analyses yielded convergent results. For the complete analysis report, see Supporting Information S1: Sections 4 and 5.

## References

Alviar, C., M. Sahoo, L. Edwards, W. R. Jones, A. Klin, and M. D. Lense. 2023. "Infant-Directed Song Potentiates Infants' Selective Attention to Adults' Mouths Over the First Year of Life." *Developmental Science* 26, no. 5: e13359. https://doi.org/10.1111/desc.13359.

Bahrick, L. E., M. Hernandez-Reif, and R. Flom. 2005. "The Development of Infant Learning About Specific Face-Voice Relations." *Developmental Psychology* 41, no. 3: 541–552. https://doi.org/10.1037/0012-1649.41.3.541.

Birmingham, E., and A. Kingstone. 2009. "Human Social Attention: A New Look at Past, Present, and Future Investigations." *Annals of the New York Academy of Sciences* 1156, no. 1: 118–140. https://doi.org/10.1111/j.1749-6632.2009.04468.x.

Birulés, J., L. Bosch, R. Brieke, F. Pons, and D. J. Lewkowicz. 2018. "Inside Bilingualism: Language Background Modulates Selective Attention to a Talker's Mouth." *Developmental Science* 22, no. 3: 1–11. https://doi.org/10.1111/desc.12755.

Birulés, J., L. Bosch, D. J. Lewkowicz, and F. Pons. 2024. "Time Course of Attention to a Talker's Mouth in Monolingual and Close-Language Bilingual Children." *Developmental Psychology* 60, no. 1: 135–143. https://doi.org/10.1037/dev0001659.

Bosch, L., and N. Sebastián-Gallés. 2001. "Evidence of Early Language Discrimination Abilities in Infants From Bilingual Environments." *Infancy* 2, no. 1: 29–49. https://doi.org/10.1207/S15327078IN0201_3.

Chandrasekaran, C., A. Trubanova, S. Stillittano, A. Caplier, and A. A. Ghazanfar. 2009. "The Natural Statistics of Audiovisual Speech." *PLoS Computational Biology* 5, no. 7: e1000436. https://doi.org/10.1371/journal.pcbi.1000436.

Chong, S. C. F., J. F. Werker, J. A. Russell, and J. M. Carroll. 2003. "Three Facial Expressions Mothers Direct to Their Infants." *Infant and Child Development* 12, no. 3: 211–232. https://doi.org/10.1002/icd.286.

Cooper, R. P., and R. N. Aslin. 1994. "Developmental Differences in Infant Attention to the Spectral Properties of Infant-Directed Speech." *Child Development*, 65, no 6: 1663–1677. https://doi.org/10.2307/1131286.

Cooper, R. P., and R. N. Aslin. 1990. "Preference for Infant-Directed Speech in the First Month After Birth." *Source: Child Development* 61, no. 5: 1584. https://doi.org/10.2307/1130766.

Cristia, A. 2013. "Input to Language: The Phonetics and Perception of Infant-Directed Speech." *Linguistics and Language Compass* 7, no. 3: 157–170. https://doi.org/10.1111/lnc3.12015.

Farroni, T., E. Menon, S. Rigato, and M. H. Johnson. 2007. "The Perception of Facial Expressions in Newborns." *European Journal of Developmental Psychology* 4, no. 1: 2–13. https://doi.org/10.1080/17405620601046832.

Fernald, A. 1985. "Four-Month-Old Infants Prefer to Listen to Motherese." *Infant Behavior and Development* 8, no. 2: 181–195. https://doi.org/10.1016/s0163-6383(85)80005-9.

Fernald, A., T. Taeschner, J. Dunn, M. Papousek, B. De Boysson-Bardies, and I. Fukui. 1989. "A Cross-Language Study of Prosodic Modifications in Mothers' and Fathers' Speech to Preverbal Infants." *Journal of Child Language* 16, no. 3: 477–501. https://doi.org/10.1017/S0305000900010679.

Frank, M. C., K. J. Alcock, N. Arias-Trejo, et al. 2020. "Quantifying Sources of Variability in Infancy Research Using the Infant-Directed-Speech Preference." *Advances in Methods and Practices in Psychological Science* 3, no. 1: 24–52. https://doi.org/10.1177/2515245919900809.

Green, J. R., I. S. B. Nip, E. M. Wilson, A. S. Mefferd, and Y. Yunusova. 2010. "Lip Movement Exaggerations During Infant-Directed Speech." *Journal of Speech, Language, and Hearing Research* 53, no. 6: 1529–1542. https://doi.org/10.1044/1092-4388(2010/09-0005).

Hillairet de Boisferon, A., E. Dupierrix, P. C. Quinn, et al. 2015. "Perception of Multisensory Gender Coherence in 6- and 9-Month-Old Infants." *Infancy* 20, no. 6: 661–674. https://doi.org/10.1111/infa.12088.

Hillairet de Boisferon, A., C. Kubicek, J. Gervain, and M. Fort. 2021. "Language Familiarity Influences Own-Race Face Recognition in 9- and 12-Old Infants." *Infancy* April: 1–13. https://doi.org/10.1111/infa.12404.

Hillairet de Boisferon, A., A. H. Tift, N. J. Minar, and D. J. Lewkowicz. 2018. "The Redeployement of Attention to the Mouth of a Talking Face During the Second Year of Life." *Journal of Experimental Child Psychology* 172, no. April: 189–200. https://doi.org/10.1016/j.jecp.2018.03.009.

Jones, W., and A. Klin. 2013. "Attention to Eyes is Present But in Decline in 2-6-Month-Old Infants Later Diagnosed With Autism." *Nature* 504, no. 7480: 427–431. https://doi.org/10.1038/nature12715.

Kim, H. I., and S. P. Johnson. 2014. "Detecting 'Infant-Directednes' Face and Voice." *Developmental Science* 17, no. 4: 621–627. https://doi.org/10.1111/desc.12146.

Kinzler, K. D., E. Dupoux, and E. S. Spelke. 2007. "The Native Language of Social Cognition." *Proceedings of the National Academy of Sciences of the United States of America* 104, no. 30: 12577–12580. https://doi.org/10.1073/pnas.0705345104.

Kubicek, C., A. H. de Boisferon, E. Dupierrix, H. Loevenbruck, J. Gervain, and G. Schwarzer. 2013. "Face-Scanning Behavior to Silently Talking Faces in 12-Month-Old Infants: The Impact of Pre-Exposed Auditory Speech." *International Journal of Behavioral Development* 37, no. 2: 106–110. https://doi.org/10.1177/0165025412473016.

Kubicek, C., J. Gervain, A. Hillairet de Boisferon, O. Pascalis, H. Loevenbruck, and G. Schwarzer. 2014. "The Influence of Infant-Directed Speech on 12-Month-Olds' Intersensory Perception of Fluent Speech." *Infant Behavior and Development* 37, no. 4: 644–651. https://doi.org/10.1016/j.infbeh.2014.08.010.

Kuhl, P. K., and A. N. Meltzoff. 1982. "The Bimodal Perception of Speech in Infancy." *Science* 218, no. 4577: 1138–1141. https://doi.org/10.1126/science.7146899.

Kushnerenko, E., T. Teinonen, A. Volein, and G. Csibra. 2008. "Electrophysiological Evidence of Illusory Audiovisual Speech Percept in Human Infants." *Proceedings of the National Academy of Sciences of the United States of America* 105, no. 32: 11442–11445. https://doi.org/10.1073/pnas.0804275105.
LaBarbera, J. D., C. E. Izard, P. Vietze, and S. A. Parisi. 1976. "Four- and Six-Month-Old Infants' Visual Responses to Joy, Anger, and Neutral Expressions." *Child Development* 47, no. 2: 535. https://doi.org/10.2307/1128816.

Lense, M. D., S. Shultz, C. Astésano, and W. Jones. 2022. "Music of Infant-Directed Singing Entrains Infants' Social Visual Behavior." *Proceedings of the National Academy of Sciences of the United States of America* 119, no. 45: 1-9. https://doi.org/10.1073/pnas.2116967119.

Lewkowicz, D. J., and A. M. Hansen-Tift. 2012. "Infants Deploy Selective Attention to the Mouth of a Talking Face When Learning Speech." *Proceedings of the National Academy of Sciences of the United States of America* 109, no. 5: 1431-1436. https://doi.org/10.1073/pnas.1114783109.

Malatesta, C. Z., and J. M. Haviland. 1982. "Learning Display Rules: The Socialization of Emotion Expression in Infancy." *Child Development* 53, no. 4: 991. https://doi.org/10.2307/1129139.

Nencheva, M. L., and C. Lew-Williams. 2022. "Understanding Why Infant-Directed Speech Supports Learning: A Dynamic Attention Perspective." *Developmental Review* 66: 101047. https://doi.org/10.1016/j.dr.2022.101047.

Olleir, D. K. 2001. "The Emergence of the Speech Capacity." *Journal of Child Language* 30, no. 3: 731-734. https://doi.org/10.1121/1.1388001.

Orena, A. J., and J. F. Werker. 2021. "Infants' Mapping of New Faces to New Voices." *Child Development* 92, no. 5: e1048-e1060. https://doi.org/10.1111/cdev.13616.

Patterson, M. L., and J. F. Werker. 2003. "Two-Month-Old Infants Match Phonetic Information in Lips and Voice." *Developmental Science* 6, no. 2: 191-196. https://doi.org/10.1111/1467-7687.00271.

Pons, F., L. Bosch, and D. J. Lewkowicz. 2015. "Bilingualism Modulates Infants' Selective Attention to the Mouth of a Talking Face." *Psychological Science* 26, no. 4: 490-498. https://doi.org/10.1177/0956797614568320.

Pons, F., D. J. Lewkowicz, S. Soto-Faraco, and N. Sebastián-Gallés. 2009. "Narrowing of Intersensory Speech Perception in Infancy." *Proceedings of the National Academy of Sciences of the United States of America* 106, no. 26: 10598-10602. https://doi.org/10.1073/pnas.0904134106.

Rosenblum, L. D., M. A. Schmuckler, and J. A. Johnson. 1997. "The McGurk Effect in Infants." *Perception & Psychophysics* 59, no. 3: 347-357. https://doi.org/10.3758/BF03211902.

Roth, K. C., K. R. H. Clayton, and G. D. Reynolds. 2022. "Infant Selective Attention to Native and Non-Native Audiovisual Speech." *Scientific Reports* 12, no. 1: 15781. https://doi.org/10.1038/s41598-022-19704-5.

Smith, N. A., and H. L. Strader. 2014. "Infant-Directed Visual Prosody." *Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems* 15, no. 1: 38-54. https://doi.org/10.1075/is.15.1.02smi.

Soderstrom, M., A. Seidl, D. G. Kemler Nelson, and P. W. Jusczyk. 2003. "The Prosodic Bootstrapping of Phrases: Evidence From Prelinguistic Infants." *Journal of Memory and Language* 49, no. 2: 249-267. https://doi.org/10.1016/S0749-596X(03)00024-X.

Uttley, L., A. Hillairet de Boisferon, E. Dupierrix, et al. 2013. "Six-Month-Old Infants Match Other-Race Faces With a Non-Native Language." *International Journal of Behavioral Development* 37, no. 2: 84-89. https://doi.org/10.1177/0165025412467583.

Weikum, W. M., A. Vouloumanos, J. Navarra, S. Soto-Faraco, N. Sebastián-Gallés, and J. F. Werker. 2007. "Visual Language Discrimination in Infancy." *Science (New York, N.Y.)* 316, no. 5828: 1159. https://doi.org/10.1126/science.1137686.

Werker, J. F., and P. J. Mcleod. 1989. "Infant Preference for Both Male and Female Infant-Directed Talk: A Developmental Study of Attentional and Affective Responsiveness." *Canadian Journal of Psychology* 43, no. 2: 230-246. https://doi.org/10.1037/h0084224.

Yehia, H., P. Rubin, and E. Vatikiotis-Bateson. 1998. "Quantitative Association of Vocal-Tract and Facial Behavior." *Speech Communication* 26, no. 1-2: 23-43. https://doi.org/10.1016/S0167-6393(98)00048-X.

## Supporting Information

Additional supporting information can be found online in the Supporting Information section.